P2P Energy Trading through Prospect Theory, Differential Evolution, and Reinforcement Learning

Abstract

Peer-to-peer (P2P) energy tradingis a decentralized energy market where local energyprosumersact as peers, trading energy among each other. Existing works in this area largely overlook the importance of user behavioral modeling and assume users’ sustained active participation and full compliance in the decision-making process. To overcome these unrealistic assumptions, and their deleterious consequences, in this article, we propose anautomatedP2P energy-trading framework that specifically considers the users’ perception by exploitingprospect theory. We formalize an optimization problem that maximizes the buyers’ perceived utility while matching energy production and demand. We prove that the problem is NP-hard and we propose a Differential Evolution-based Algorithm for Trading Energy (DEbATE) heuristic. Additionally, we propose two automated pricing solutions to improve the sellers’ profit based on reinforcement learning. The first solution, named Pricing mechanism with Q-learning and Risk-sensitivity (PQR), is based on Q-learning. Additionally, given the scalability issues ofPQR, we propose a Deep Q-Network-based algorithm calledProDQNthat exploits deep learning and a novel loss function rooted in prospect theory. Results based on real traces of energy consumption and production, as well as realistic prospect theory functions, show that our approaches achieve 26% higher perceived value for buyers and generate 7% more reward for sellers, compared to recent state-of-the-art approaches.