Demand-driven transparency for monitoring intelligent agents

In autonomous multiagent or multirobotic systems, the ability to quickly and accurately respond to threats and uncertainties is important for both mission outcomes and survivability. Such systems are never truly autonomous, often operating as part of a human-agent team. Artificial intelligent agents (IAs) have been proposed as tools to help manage such teams; e.g., proposing potential courses of action to human operators. However, they are often underutilized due to a lack of trust. Designing transparent agents, who can convey at least some information regarding their internal reasoning processes, is considered an effective method of increasing trust. How people interact with such transparency information to gain situation awareness while avoiding information overload is currently an unexplored topic. In this article, we go part way to answering this question, by investigating two forms of transparency: sequential transparency, which requires people to step through the IA's explanation in a fixed order; and demand-driven transparency, which allows people to request information as needed. In an experiment using a multivehicle simulation, our results show that demand-driven interaction improves the operators' trust in the system while maintaining, and at times improving, performance and usability. 118268743 1 0000-0001-5286-6509 Vered, M. Mor Vered Mor Mor Vered Vered 1 215742308 Monash University Faculty of Information Technology Clayton Australia 3800 Author [email protected] Faculty of Information Technology, Monash University, Clayton, VIC, Australia 118268744 2 0000-0001-6171-1381 Howe, P. Piers Howe Piers Piers Howe Howe 2 215742309 University of Melbourne Melbourne School of Psychological Sciences Melbourne Australia 3010 Author [email protected] Melbourne School of Psychological Sciences, University of Melbourne, Melbourne, VIC, Australia 118268745 3 Miller, T. Tim Miller Tim Tim Miller Miller 3 215742310 University of Melbourne School of Computing and Information Systems Melbourne Australia 3010 Author [email protected] School of Computing and Information Systems, University of Melbourne, Melbourne, VIC, Australia 118268746 4 0000-0002-8653-2555 Sonenberg, L. Liz Sonenberg Liz Liz Sonenberg Sonenberg 4 215742311 University of Melbourne School of Computing and Information Systems Melbourne Australia 3010 Author [email protected] School of Computing and Information Systems, University of Melbourne, Melbourne, VIC, Australia 118268747 5 0000-0003-4414-2249 Velloso, E. Eduardo Velloso Eduardo Eduardo Velloso Velloso 5 215742312 University of Melbourne School of Computing and Information Systems Melbourne Australia 3010 Author [email protected] School of Computing and Information Systems, University of Melbourne, Melbourne, VIC, Australia 2020 June 2020 5 20 2020 5 21 1355631 09097427.pdf