Beyond Supra-Competitive Outcomes: Collusive Behaviour in Deep Reinforcement Learning for Optimal Execution Games

RESUMO

The paper extends prior research on supra-competitive outcomes in optimal-execution games by identifying a learned punitive mechanism that deters deviations and provides behavioural evidence of collusion, using two independent proximal policy optimization agents in an Almgren-Chriss liquidation game. When agents achieve costs below the Nash benchmark, they develop punitive responses to deviations: when one agent implements a profitable deviation, the opponent accelerates liquidation, offsetting the deviator's gains while maintaining the punisher's average payoff. Punishment outweighs the gain from deviating, and behavioural changes are large enough to explain imposed losses, providing behavioural and economic evidence supporting a collusive interpretation of the learned outcomes.