Department of Industrial Engineering & Decision Analytics [Joint IEDA/MATH seminar] - Collusion or Compete: A Two Timescale Evolutionary Game Approach to Algorithmic Collusion Study

We propose a two-time scale evolutionary game approach to address multi-agent reinforcement learning (MARL) problems. The algorithm is built on three key components. First, we employ a perturbed best response to update agents’ policies. Second, we utilize a fictitious play rule to refine agents’ beliefs of their opponents. Third, policies and beliefs are updated at different learning rates compared to those used for Q-value updates. This novel approach provably converges to є-Nash equilibria in general-sum MARL problems without imposing the restrictive assumptions commonly found in the literature.

AI-powered algorithms are increasingly adopted in marketplaces for pricing goods and services. However, regulators and academics have raised serious concerns about the potential for these algorithms to learn collusive behavior through their strategic interactions. While researchers predominantly rely on Q-learning to model the behavior of pricing algorithms, this method lacks convergence guarantees in multi-agent settings. Our approach offers an innovative framework for studying algorithmic collusion. Numerical experiments indicate that the sophistication of algorithms is a significant driving force behind the emergence of collusion among algorithmic agents

 This talk is based on joint work with Ruixun Zhang and Yumin Xu (Peking University) and Mingyue Zhong (CUHK).