Reinforcement Learning for Demand Response of Domestic Household Appliances

With today’s electricity grid being penetrated with more and more intermittent energy sources, the need arises to match demand for electricity with electricity generation, in order to maintain the stability of the grid. Demand response has been proposed as a mechanism that aims to solve this problem, by stimulating the shift of electricity demand towards production peaks. In previous work individual devices were trained using fitted-Q iteration to shift the consumption of electrical energy towards low-price periods, while still guaranteeing user comfort. This paper shows that controlling multiple devices with one independent agent per device is more cost-effective compared to a centralized agent. The paper expands the setting with an energy consumption constraint on the level of the household. The objective is to spread the energy consumption of the devices during low price periods as much as possible in order to avoid overloading the grid. Preliminary results for both centralized and independent learners show that these agents are capable of reducing the amount of constraint violations minimally but fail to reduce violations to zero.