Tacit algorithmic collusion in deep reinforcement learning guided price competition: A study using EV charge pricing game

Abstract

Players in pricing games with complex structures are increasingly adopting artificial intelligence (AI) aided learning algorithms to make pricing decisions for maximizing profits. This is raising concern for the antitrust agencies as the practice of using AI may promote tacit algorithmic collusion among otherwise independent players. Recent studies of games in canonical forms have shown contrasting claims ranging from none to a high level of tacit collusion among AI-guided players. In this paper, we examine the concern for tacit collusion by considering a practical game where EV charging hubs compete by dynamically varying their prices. Such a game is likely to be commonplace in the near future as EV adoption grows in all sectors of transportation. The hubs source power from the day-ahead (DA) and real-time (RT) electricity markets as well as from in-house battery storage systems. Their goal is to maximize profits via pricing and efficiently managing the cost of power usage. To aid our examination, we develop a two-step data-driven methodology. The first step obtains the DA commitment by solving a stochastic model. The second step generates the pricing strategies by solving a competitive Markov decision process model using a multi-agent deep reinforcement learning (MADRL) framework. We evaluate the resulting pricing strategies using an index for the level of tacit algorithmic collusion. An index value of zero indicates no collusion (perfect competition) and one indicates full collusion (monopolistic behavior). Results from our numerical case study yield collusion index values between 0.14 and 0.45, suggesting a low to moderate level of collusion.