Scheduling Electric Vehicle Charging for Participation in the Belgian Imbalance Market Using Model-Free Reinforcement Learning

This study explores the potential of using a model-free Reinforcement Learning (RL) approach to optimize Electric Vehicle (EV) charging scheduling for participation in the Belgian imbalance market, on the use case of an office parking. Motivated by changing regulations enabling smaller assets like EVs to participate in ancillary markets, the study aims to develop a smart charging strategy that minimizes charging costs by leveraging imbalance price volatility. The proposed problem was formulated as a Markov Decision Process (MDP), and an RL agent was subsequently trained to optimize the charging schedule. The proposed approach balances economic gains with considering the target SOC, offering promising benefits for grid flexibility and revenue generation. Results demonstrate that the proposed RL strategy outperforms uncoordinated and smart charging in the day-ahead market with respect to charging cost, achieving negative average charging costs by leveraging price fluctuations.