The paper introduces NOMAD‑RL, a reinforcement learning controller for HVAC systems that learns to adapt across diverse thermal zones via a universal thermostat interface. It employs an adaptive domain randomization scheme using physics‑informed normalizing flows to generate realistic, multimodal training data, enabling the recurrent policy to handle partial observability. Experiments show NOMAD‑RL outperforms constant‑setpoint PID and non‑randomized RL, and rivals well‑tuned model predictive control, especially in multi‑zone scenarios.
arXiv:2607. 16534v1 Announce Type: cross Abstract: Reinforcement learning (RL) has achieved strong results in control, yet learned policies remain brittle to changes in dynamics, action spaces, observation spaces, or goals, a critical limitation for real-world deployment.
Buildings account for roughly one-third of global energy consumption and CO$_2$ emissions. Optimizing indoor climate systems plays a critical role for urban climate mitigation aligned with UN Sustainable Development Goals 11 and 13.
arXiv:2608. 19804v1 Announce Type: new Abstract: Buildings account for roughly one-third of global energy consumption and CO$_2$ emissions.
arXiv:2510. 01475v2 Announce Type: replace-cross Abstract: Model Predictive Control (MPC) has demonstrated significant performance improvements over today's control methods for residential Heating, Ventilation, and Air Conditioning (HVAC), but deploying MPC often requires substantial engineering effort.
arXiv:2608. 11324v1 Announce Type: cross Abstract: This paper proposes a contextual quality-diversity evolutionary reinforcement-learning controller, CQD-ERL, for the supervisory control of a tropical, water-cooled chiller plant and its associated air side.
arXiv:2606. 02852v1 Announce Type: new Abstract: Accurate short-term forecasting of residential energy load and indoor temperature is essential for home energy management systems, grid-level demand response, and community energy efficiency efforts.
The paper presents a zero‑shot model predictive control (MPC) approach for buildings that uses excitation‑based generalized transfer learning models. By pretraining on purposefully probed operational data from multiple source buildings, the authors demonstrate that these models achieve superior control performance in 32 simulated target buildings, outperforming both an online linear MPC and a PI controller. This method eliminates the need for target‑specific data, reducing setup cost and facilitating broader deployment of energy‑efficient MPC in the building sector.
arXiv:2409. 19716v2 Announce Type: replace-cross Abstract: Constrained Reinforcement Learning (RL) has emerged as a significant research area within RL, where integrating constraints with rewards is crucial for enhancing safety and performance across diverse control tasks.
The paper presents a reinforcement‑learning framework that dynamically selects between full‑order models (FOMs) and reduced‑order models (ROMs) in hybrid domain decomposition simulations using the overlapping Schwarz alternating method (O‑SAM). Offline‑trained Deep Q‑networks choose subdomain‑local FOMs or pre‑trained Operator Inference ROMs based on a reward that balances accuracy, computational cost, and model‑switching frequency, and the learned policies are deployed on unseen problem instances without needing a reference FOM solution. Experiments on a 1D advection‑diffusion problem and a 3D elastic wave propagation benchmark show that the RL‑guided policies adaptively allocate high‑fidelity resolution as features propagate, outperforming static FOM/ROM assignments and demonstrating the feasibility of predictive online fidelity adaptation in Schwarz‑based hybrid simulations.
arXiv:2607. 27914v1 Announce Type: new Abstract: Multi-zone variable-air-volume control must balance thermal comfort, indoor air quality, and electricity use across several continuous actuators.
arXiv:2610.18511v1 Announce Type: new Abstract: Buildings cause nearly 40\% of carbon emissions in the US. Of total building emissions, HVAC systems are responsible for approximately 40-60\% and acco...
The paper introduces Multi‑Fidelity Policy Gradient (MFPG) methods that combine scarce, high‑fidelity (HF) reinforcement‑learning data with abundant, low‑fidelity (LF) data to stabilize training. Unlike prior approaches that directly optimize biased LF objectives, MFPG uses LF data only to construct a control variate that reduces variance without biasing the policy gradient. The authors extend MFPG to modern actor‑critic algorithms, notably a budget‑aware MFPG‑PPO that preserves cross‑fidelity correlation, monitors estimator uncertainty, and allocates a fixed sampling budget between HF and LF sources. Experiments on simulated robot locomotion and a physical Franka arm show that MFPG‑PPO consistently outperforms standard PPO trained on HF data alone and matches performance achieved with 16× more HF data on the hardest tasks, while enabling stable learning with as few as four real‑robot episodes per update.