An information-theoretic predictive model for the accuracy of AI agents adapted from psychometrics

Nader Chmait, David L. Dowe, Yuan-Fang Li, David G. Green

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

    5 Citations (Scopus)


    We propose a new model to quantitatively estimate the accuracy of artificial agents over cognitive tasks of approximable complexities. The model is derived by introducing notions from algorithmic information theory into a well-known (psychometric) measurement paradigm called Item Response Theory (IRT). A lower bound on accuracy can be guaranteed with respect to task complexity and the breadth of its solution space using our model. This in turn permits formulating the relationship between agent selection cost, task difficulty and accuracy as optimisation problems. Further results indicate some of the settings over which a group of cooperative agents can be more or less accurate than individual agents or other groups.

    Original languageEnglish
    Title of host publicationArtificial General Intelligence
    Subtitle of host publication10th International Conference, AGI 2017, Melbourne, VIC, Australia, August 15–18, 2017, Proceedings
    EditorsTom Everitt, Alexey Potapov, Ben Goertzel
    Place of PublicationCham, Switzerland
    Number of pages12
    ISBN (Electronic)9783319637037
    ISBN (Print)9783319637020
    Publication statusPublished - 2017
    EventConference on Artificial General Intelligence (AGI) 2017 - Melbourne, Australia
    Duration: 15 Aug 201718 Aug 2017
    Conference number: 10th (Proceedings)

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349


    ConferenceConference on Artificial General Intelligence (AGI) 2017
    Abbreviated titleAGI 2017
    Internet address

    Cite this