Multi-Agent Optimal Control for Central Chiller Plants Using Reinforcement Learning and Game Theory

Author:

Qiu Shunian1ORCID,Li Zhenhai2,Pang Zhihong3ORCID,Li Zhengwei2,Tao Yinying4ORCID

Affiliation:

1. School of Civil Engineering and Architecture, Zhejiang University of Science and Technology, Hangzhou 310023, China

2. School of Mechanical Engineering, Tongji University, Shanghai 200092, China

3. Department of Construction Management, Louisiana State University, Patrick F. Taylor Hall 3315-D, Baton Rouge, LA 70803, USA

4. School of Design and Fashion, Zhejiang University of Science and Technology, Hangzhou 310023, China

Abstract

To conserve building energy, optimal operation of a building’s energy systems, especially heating, ventilation and air-conditioning (HVAC) systems, is important. This study focuses on the optimization of the central chiller plant, which accounts for a large portion of the HVAC system’s energy consumption. Classic optimal control methods for central chiller plants are mostly based on system performance models which takes much effort and cost to establish. In addition, inevitable model error could cause control risk to the applied system. To mitigate the model dependency of HVAC optimal control, reinforcement learning (RL) algorithms have been drawing attention in the HVAC control domain due to its model-free feature. Currently, the RL-based optimization of central chiller plants faces several challenges: (1) existing model-free control methods based on RL typically adopt single-agent scheme, which brings high training cost and long training period when optimizing multiple controllable variables for large-scaled systems; (2) multi-agent scheme could overcome the former problem, but it also requires a proper coordination mechanism to harmonize the potential conflicts among all involved RL agents; (3) previous agent coordination frameworks (identified by distributed control or decentralized control) are mainly designed for model-based control methods instead of model-free controllers. To tackle the problems above, this article proposes a multi-agent, model-free optimal control approach for central chiller plants. This approach utilizes game theory and the RL algorithm SARSA for agent coordination and learning, respectively. A data-driven system model is set up using measured field data of a real HVAC system for simulation. The simulation case study results suggest that the energy saving performance (both short- and long-term) of the proposed approach (over 10% in a cooling season compared to the rule-based baseline controller) is close to the classic multi-agent reinforcement learning (MARL) algorithm WoLF-PHC; moreover, the proposed approach’s nature of few pending parameters makes it more feasible and robust for engineering practices than the WoLF-PHC algorithm.

Publisher

MDPI AG

Subject

Information Systems and Management,Computer Networks and Communications,Modeling and Simulation,Control and Systems Engineering,Software

Reference54 articles.

1. Delmastro, C., De Bienassis, T., Goodson, T., Lane, K., Le Marois, J.-B., Martinez-Gordon, R., and Husek, M. (2022). Buildings: Tracking Progress 2022, International Energy Agency.

2. Supervisory and Optimal Control of Building HVAC Systems: A Review;Wang;Hvac R Res.,2008

3. Commercial Buildings Energy Consumption Survey (CBECS) (2012). 2012 CBECS Survey Data.

4. Taylor, S.T. (2017). Fundamentals of Design and Control of Central Chilled-Water Plants, ASHRAE Learning Institute.

5. Qiu, S., Li, Z., Li, Z., and Wu, Q. (2022). Comparative Evaluation of Different Multi-Agent Reinforcement Learning Mechanisms in Condenser Water System Control. Buildings, 12.

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3