Value Creation in Sustainable Energy Transition Using Reinforcement Learning

Abstract

Abstract The global challenge of climate change has prompted significant steps to reduce CO2 emissions, guided by international agreements like the Paris Agreement, which set targets for the transition from fossil fuels to renewable sources. Acting too slowly could result in future losses and reputational issues, while moving too swiftly could jeopardize shareholder value due to the marginal profitability or potential losses resulting from technology challenges and the immaturity of many renewable projects. To navigate this complex landscape wisely, energy companies will benefit from developing Sequential Decision Making (SDM) policies to maximize value creation from decision flexibility under uncertainties. However, achieving this goal is not trivial. This necessitates the use of decision-analysis and optimization methods that deliver robust decision insights without prohibitive computational costs or delays. In this research, we propose a multi-objective SDM framework to model the dynamic energy landscape up to 2050, allowing exploration of various decision strategies related to different portfolios for allocating funds across three sectors: hydrocarbon, renewables, and CO2 reduction investments. This approach aims to maximize value during the transformation while accounting for uncertainties in hydrocarbon and renewable energy productions, energy prices, and production costs. The framework targets three main objectives: maximizing profit, minimizing CO2 emission, and enhancing competitive advantage. Our research evaluates the benefits of using Reinforcement Learning (RL) to solve for the optimal investment decision policy for the SDM context. The sequential decisions of the energy company shape the virtual dynamic environment by influencing various variables. Subsequently, the environment responds with new variable states and immediate value feedback, reflecting the three defined objectives. Through repeated interactions, the RL algorithm navigates the extensive state space, learning the optimal decision policy amidst uncertainties.