Monte Carlo Tree Search based algorithms for Dynamic Difficulty Adjustment

Maintaining player immersion is a crucial step in making an enjoyable video game. One aspect of player immersion is the level of challenge the game presents to the player. To avoid a mismatch between a player's skill and the challenge of a game, which can result from traditional manual difficulty selection mechanisms (e.g. easy, medium, hard), Dynamic Difficulty Adjustment (DDA) has previously been proposed as a means of automatically detecting a player's skill and adjusting the level of challenge the game presents accordingly. This work contributes to the field of DDA by proposing a novel approach to artificially intelligent agents for opponent control. Specifically, we propose four new DDA Artificially Intelligent (AI) agents: Reactive Outcome Sensitive Action Selection (Reactive OSAS), Proactive OSAS, and their 'True' variants. These agents provide the player with an level of difficulty tailored to their skill in real-time by altering the action selection policy and the heuristic playout evaluation of Monte Carlo Tree Search. The DDA AI agents are tested within the FightingICE engine, which has been used in the past as an environment for AI agent competitions. The results of the experiments against other AI agents and human players show that these novel DDA AI agents can adjust the level of difficulty in real-time, by targeting a zero health difference as the outcome of the fighting game. This work also demonstrates the trade-off existing between targeting the outcome exactly (Reactive OSAS) and introducing proactive behaviour (i.e., the DDA AI agent fights even if the health difference is zero) to increase the agents believability (Proactive OSAS).