Monte Carlo Tree Search based algorithms for Dynamic Difficulty Adjustment

Simon Demediuk, Marco Tamassia, William L. Raffe, Fabio Zambetta, Xiaodong Li, Florian 'Floyd' Mueller

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

26 Citations (Scopus)

Abstract

Maintaining player immersion is a crucial step in making an enjoyable video game. One aspect of player immersion is the level of challenge the game presents to the player. To avoid a mismatch between a player's skill and the challenge of a game, which can result from traditional manual difficulty selection mechanisms (e.g. easy, medium, hard), Dynamic Difficulty Adjustment (DDA) has previously been proposed as a means of automatically detecting a player's skill and adjusting the level of challenge the game presents accordingly. This work contributes to the field of DDA by proposing a novel approach to artificially intelligent agents for opponent control. Specifically, we propose four new DDA Artificially Intelligent (AI) agents: Reactive Outcome Sensitive Action Selection (Reactive OSAS), Proactive OSAS, and their 'True' variants. These agents provide the player with an level of difficulty tailored to their skill in real-time by altering the action selection policy and the heuristic playout evaluation of Monte Carlo Tree Search. The DDA AI agents are tested within the FightingICE engine, which has been used in the past as an environment for AI agent competitions. The results of the experiments against other AI agents and human players show that these novel DDA AI agents can adjust the level of difficulty in real-time, by targeting a zero health difference as the outcome of the fighting game. This work also demonstrates the trade-off existing between targeting the outcome exactly (Reactive OSAS) and introducing proactive behaviour (i.e., the DDA AI agent fights even if the health difference is zero) to increase the agents believability (Proactive OSAS).

Original languageEnglish
Title of host publication2017 IEEE Conference on Computational Intelligence and Games (CIG)
EditorsClare Bates Congdon, Michael Buro
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages53-59
Number of pages7
ISBN (Electronic)9781538632338, 9781538632321
ISBN (Print)9781538632345
DOIs
Publication statusPublished - 2017
Externally publishedYes
EventIEEE Symposium on Computational Intelligence and Games 2017 - New York, United States of America
Duration: 22 Aug 201725 Aug 2017
http://www.cig2017.com

Conference

ConferenceIEEE Symposium on Computational Intelligence and Games 2017
Abbreviated titleCIG 2017
Country/TerritoryUnited States of America
CityNew York
Period22/08/1725/08/17
Internet address

Cite this