What information is required for explainable AI? a provenance-based research agenda and future challenges

Fariha Tasmin Jaigirdar, Carsten Rudolph, Gillian Oliver, David Watts, Chris Bain

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

1 Citation (Scopus)


Deriving explanations of an Artificial Intelligence-based system's decision making is becoming increasingly essential to address requirements that meet quality standards and operate in a transparent, comprehensive, understandable, and explainable manner. Furthermore, more security issues as well as concerns from human perspectives emerge in describing the explainability properties of AI. A full system view is required to enable humans to properly estimate risks when dealing with such systems. This paper introduces open issues in this research area to present the overall picture of explainability and the required information needed for the explanation to make a decision-oriented AI system transparent to humans. It illustrates the potential contribution of proper provenance data to AI-based systems by describing a provenance graph-based design. This paper proposes a six-Ws framework to demonstrate how a security-aware provenance graph-based design can build the basis for providing end-users with sufficient meta-information on AI-based decision systems. An example scenario is then presented that highlights the required information for better explainability both from human and security-aware aspects. Finally, associated challenges are discussed to provoke further research and commentary.

Original languageEnglish
Title of host publicationProceedings - 2020 IEEE 6th International Conference on Collaboration and Internet Computing, CIC 2020
EditorsSongqing Chen, Wei-Shinn Ku, Christin Seifert
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages7
ISBN (Electronic)9781728141466
ISBN (Print)9781728185422
Publication statusPublished - 2020
EventInternational Conference on Collaboration and Internet Computing 2020 - Virtual, Atlanta, United States of America
Duration: 1 Dec 20203 Dec 2020
Conference number: 6th
https://ieeexplore.ieee.org/xpl/conhome/9318988/proceeding (Proceedings)
http://www.sis.pitt.edu/lersais/conference/cic/2020/ (Website)


ConferenceInternational Conference on Collaboration and Internet Computing 2020
Abbreviated titleCIC 2020
Country/TerritoryUnited States of America
Internet address


  • artificial intelligence
  • cybersecurity
  • data provenance
  • decision-oriented systems
  • explainable AI
  • human-centric policy

Cite this