Optimization of Multi Agent System for Distribution Control of Distinct Heating System Using Improved Q-Learning Controller

Abstract

The aim of this research is the use of multi-agent systems for the optimization of a distributed control for district heating systems. A district heating system comprises of production units, a distribution network, and a host of consumer substations. The operations of district heating system usually involves conflicting goals, e.g., to satisfy customers and to minimize production costs. Hence the agent must be capable of optimizing between maximizing supply to substations and minimizing production cost. Current substations employ purely reactive devices, making local decisions without taking into account the global state. Moreover the substations determine the flow in all parts of the district heating system. The optimal operation of the district heating system is therefore limited to providing sufficiently high temperature and pressure to all customers by taking local measurement to achieve this goal without considering other factors such as cost of production and time. The approach studied in this research is to equip substations with software agents to form a multi-agent system using Q-learning. The study also shows that it is possible to control the trade-off between quality of service and degree of surplus production as well as the possibility of extending the system with new consumers without increasing production capacity. In another study, an experiment was conducted in a controlled physical environment, where two agent-based approaches were evaluated and compared to existing technologies. The experiment shows that it is possible to automatically load balance a small district heating network using agent technology.. Finally, a generalized formal characterization of the problem space under investigation is provided, i.e., production and logistics network management, together with a preliminary evaluation of the applicability of the suggested multi-agent system approach for this general problem area.