Chinese Journal of Intelligent Science and Technology ›› 2022, Vol. 4 ›› Issue (3): 426-444.doi: 10.11959/j.issn.2096-6652.202208

• Papers and Reports • Previous Articles     Next Articles

HVAC model-free optimal control method based on double-pools DQN

Shuai MA1,2,3, Qiming FU1,2,3, Jianping CHEN1,2,3, Fan FENG4, You LU1,2,3, Zhengwei LI5,6, Shunian QIU5,6   

  1. 1 School of Electronics ＆Information Engineering, Suzhou University of Science and Technology, Suzhou 215009, China
    2 Jiangsu Province Key Laboratory Intelligent Building Energy Efficiency,Suzhou University of Science and Technology, Suzhou 215009, China
    3 Suzhou Key Laboratory of Mobile Networking and Applied Technology, Suzhou 215009, China
    4 Texas A＆M University, College Station TX 77843, USA
    5 School of Mechanical Engineering, Tongji University, Shanghai 200092, China
    6 Key Laboratory of Performance Evolution and Control for Engineering Structures of Ministry of Education, Tongji University, Shanghai 200092, China
  • Revised:2021-08-28 Online:2022-09-15 Published:2022-09-01
  • Supported by:
    The National Key Research and Development Program of China(2020YFC2006602);The National Natural Science Foundation of China(62072324);The National Natural Science Foundation of China(61876217);The National Natural Science Foundation of China(61876121);The National Natural Science Foundation of China(61772357);The Research and Development Program of Jiangsu Province(BE2020026)

RichHTML

51

PDF下载

397

Abstract:

In the field of HVAC (heating, ventilation and air conditioning) control, the model-based optimal control method has been extensively studied and verified by scholars, but this method highly depends on the accuracy of the model, the collection of a large amount of historical data, and the deployment of sensors.In response to the above problems,combined with EnergyPlus, actual system parameters and historical data, the HVAC optimized control model was constructed, and an improved double pools-based DQN (DPs-DQN) algorithm was proposed.Finally, it was applied to the load distribution of different types of chillers, the combined optimal control of cooling tower fan frequency and cooling water pump frequency in HVAC system.Based on the constructed problem model, aiming at the problem of sample imbalance in the decision-making optimization process, the algorithm established two independent experience pools on the basis of DQN to store load distribution and non load distribution samples respectively.During the training process, followed a certain ratio to sample from the experience pool to speed up the algorithm convergence.The proposed method was compared with the model-based control method and the baseline method.The experimental results show that compared with the baseline method, the model-based HVAC controller can save 11.5% (optimal energy-saving efficiency), while the DPs-DQN can save energy by 7.5% in the first year.At the same time, as the system runs, the controller can obtain results close to the optimal energy saving efficiency in the eighth year.In addition, compared with the model-based HVAC controller, the controller does not depend on the system model, and requires less prior knowledge and sensors in the online control process, which is more valuable in actual engineering applications.

Key words: deep reinforcement learning, model-free optimal control, HVAC system, building energy saving

CLC Number: 

No Suggested Reading articles found!