Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper introduces a hybrid-action neural controller (HANC) for cyber-physical systems requiring both continuous and discrete control decisions, overcoming the non-differentiability challenge of discrete actions. HANC employs a differentiable assembly layer with a straight-through Gumbel estimator to jointly train continuous and categorical branches via backpropagation. Deployed on a district heating network, HANC achieved a 30% reduction in operating costs under dynamic electricity pricing compared to a rule-based baseline.
A single neural network now learns to seamlessly switch between discrete operational modes and set continuous parameters in complex systems, slashing district heating costs by 30%.
Many cyber-physical systems require control policies that combine continuous setpoints with discrete operational de- cisions, such as equipment switching, mode selection, or resource scheduling. Discrete actions are not differentiable, which ob- structs gradient-based policy training, while conventional mixed- integer formulations remain costly to solve online. This paper proposes a hybrid-action neural controller (HANC) in which a continuous branch, a categorical branch and a differentiable assembly layer jointly generate commands that satisfy complex actuator constraints by construction. Categorical decisions are handled using a straight-through Gumbel estimator, enabling the policy to be trained by backpropagation through time over full closed-loop rollouts. The proposed framework is deployed on a district heating network (DHN) featuring multiple heat generation units and stratified thermal energy storage. Its performance is evaluated on a simulation of a real DHN located at RSE SpA in Italy. The resulting policy jointly learns switching decisions and continu- ous operating setpoints. Under dynamic electricity pricing, the learned controller reduces operating cost by 30% compared to a rule-based industrial baseline. We also show that, compared with a deterministic straight-through relaxation, injecting noise during training achieves similar cost while reducing hard switching by an order of magnitude, and attribute this difference to the wider decision margins of the resulting policy.