A multi-agent reinforcement learning method with learning of other agents for competitive game

摘要

This report proposes a reinforcement learning (RL) method based on the Actor-Critic architecture, which can be applied to partially-observable multi-agent competitive games. As an example, we consider a card game "Hearts". The RL then becomes a partially-observable Markov decision process (POMDP). In our method, a single Hearts game is divided into three stages, and three actors are prepared so that one of them plays and learns separately in each stage. In particular, the actor for the middle stage plays so as to enlarge the expected temporal-difference error, which is calculated using the evaluation function approximated by the critic and the estimated state transition. Computer experiments with heuristic players show that our RL method works well.