An information-theoretic predictive model for the accuracy of AI agents adapted from psychometrics

Nader Chmait; David L. Dowe; Yuan-Fang Li; David G. Green

doi:10.1007/978-3-319-63703-7_21

An information-theoretic predictive model for the accuracy of AI agents adapted from psychometrics

Nader Chmait, David L. Dowe, Yuan-Fang Li, David G. Green

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

4 Citations (Scopus)

Abstract

We propose a new model to quantitatively estimate the accuracy of artificial agents over cognitive tasks of approximable complexities. The model is derived by introducing notions from algorithmic information theory into a well-known (psychometric) measurement paradigm called Item Response Theory (IRT). A lower bound on accuracy can be guaranteed with respect to task complexity and the breadth of its solution space using our model. This in turn permits formulating the relationship between agent selection cost, task difficulty and accuracy as optimisation problems. Further results indicate some of the settings over which a group of cooperative agents can be more or less accurate than individual agents or other groups.

Original language	English
Title of host publication	Artificial General Intelligence
Subtitle of host publication	10th International Conference, AGI 2017, Melbourne, VIC, Australia, August 15–18, 2017, Proceedings
Editors	Tom Everitt, Alexey Potapov, Ben Goertzel
Place of Publication	Cham, Switzerland
Publisher	Springer
Pages	225-236
Number of pages	12
ISBN (Electronic)	9783319637037
ISBN (Print)	9783319637020
DOIs	https://doi.org/10.1007/978-3-319-63703-7_21
Publication status	Published - 2017
Event	Conference on Artificial General Intelligence (AGI) 2017 - Melbourne, Australia Duration: 15 Aug 2017 → 18 Aug 2017 Conference number: 10th https://link.springer.com/book/10.1007/978-3-319-63703-7 (Proceedings)

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Publisher	Springer
Volume	10414
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	Conference on Artificial General Intelligence (AGI) 2017
Abbreviated title	AGI 2017
Country	Australia
City	Melbourne
Period	15/08/17 → 18/08/17
Internet address	https://link.springer.com/book/10.1007/978-3-319-63703-7 (Proceedings)

Access to Document

10.1007/978-3-319-63703-7_21

Link to publication in Scopus

Kurzweil Best Paper Prize
Li, Yuan-Fang (Recipient), 2017
Prize: Prize (including medals and awards)

Cite this

Chmait, N., Dowe, D. L., Li, Y-F., & Green, D. G. (2017). An information-theoretic predictive model for the accuracy of AI agents adapted from psychometrics. In T. Everitt, A. Potapov, & B. Goertzel (Eds.), Artificial General Intelligence : 10th International Conference, AGI 2017, Melbourne, VIC, Australia, August 15–18, 2017, Proceedings (pp. 225-236). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10414 ). Springer. https://doi.org/10.1007/978-3-319-63703-7_21

Chmait, Nader ; Dowe, David L. ; Li, Yuan-Fang ; Green, David G. / An information-theoretic predictive model for the accuracy of AI agents adapted from psychometrics. Artificial General Intelligence : 10th International Conference, AGI 2017, Melbourne, VIC, Australia, August 15–18, 2017, Proceedings. editor / Tom Everitt ; Alexey Potapov ; Ben Goertzel. Cham, Switzerland : Springer, 2017. pp. 225-236 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{82e4b28b707c4d87aad772fab3898d42,

title = "An information-theoretic predictive model for the accuracy of AI agents adapted from psychometrics",

abstract = "We propose a new model to quantitatively estimate the accuracy of artificial agents over cognitive tasks of approximable complexities. The model is derived by introducing notions from algorithmic information theory into a well-known (psychometric) measurement paradigm called Item Response Theory (IRT). A lower bound on accuracy can be guaranteed with respect to task complexity and the breadth of its solution space using our model. This in turn permits formulating the relationship between agent selection cost, task difficulty and accuracy as optimisation problems. Further results indicate some of the settings over which a group of cooperative agents can be more or less accurate than individual agents or other groups.",

author = "Nader Chmait and Dowe, {David L.} and Yuan-Fang Li and Green, {David G.}",

year = "2017",

doi = "10.1007/978-3-319-63703-7_21",

language = "English",

isbn = "9783319637020",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "225--236",

editor = "Everitt, {Tom } and Potapov, {Alexey } and Goertzel, {Ben }",

booktitle = "Artificial General Intelligence",

note = "Conference on Artificial General Intelligence (AGI) 2017, AGI 2017 ; Conference date: 15-08-2017 Through 18-08-2017",

url = "https://link.springer.com/book/10.1007/978-3-319-63703-7",

}

Chmait, N, Dowe, DL , Li, Y-F & Green, DG 2017, An information-theoretic predictive model for the accuracy of AI agents adapted from psychometrics. in T Everitt, A Potapov & B Goertzel (eds), Artificial General Intelligence : 10th International Conference, AGI 2017, Melbourne, VIC, Australia, August 15–18, 2017, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10414 , Springer, Cham, Switzerland, pp. 225-236, Conference on Artificial General Intelligence (AGI) 2017, Melbourne, Australia, 15/08/17. https://doi.org/10.1007/978-3-319-63703-7_21

An information-theoretic predictive model for the accuracy of AI agents adapted from psychometrics. / Chmait, Nader; Dowe, David L.; Li, Yuan-Fang ; Green, David G.

Artificial General Intelligence : 10th International Conference, AGI 2017, Melbourne, VIC, Australia, August 15–18, 2017, Proceedings. ed. / Tom Everitt; Alexey Potapov; Ben Goertzel. Cham, Switzerland : Springer, 2017. p. 225-236 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10414 ).

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - An information-theoretic predictive model for the accuracy of AI agents adapted from psychometrics

AU - Chmait, Nader

AU - Dowe, David L.

AU - Li, Yuan-Fang

AU - Green, David G.

N1 - Conference code: 10th

PY - 2017

Y1 - 2017

N2 - We propose a new model to quantitatively estimate the accuracy of artificial agents over cognitive tasks of approximable complexities. The model is derived by introducing notions from algorithmic information theory into a well-known (psychometric) measurement paradigm called Item Response Theory (IRT). A lower bound on accuracy can be guaranteed with respect to task complexity and the breadth of its solution space using our model. This in turn permits formulating the relationship between agent selection cost, task difficulty and accuracy as optimisation problems. Further results indicate some of the settings over which a group of cooperative agents can be more or less accurate than individual agents or other groups.

AB - We propose a new model to quantitatively estimate the accuracy of artificial agents over cognitive tasks of approximable complexities. The model is derived by introducing notions from algorithmic information theory into a well-known (psychometric) measurement paradigm called Item Response Theory (IRT). A lower bound on accuracy can be guaranteed with respect to task complexity and the breadth of its solution space using our model. This in turn permits formulating the relationship between agent selection cost, task difficulty and accuracy as optimisation problems. Further results indicate some of the settings over which a group of cooperative agents can be more or less accurate than individual agents or other groups.

UR - http://www.scopus.com/inward/record.url?scp=85028472248&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-63703-7_21

DO - 10.1007/978-3-319-63703-7_21

M3 - Conference Paper

AN - SCOPUS:85028472248

SN - 9783319637020

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 225

EP - 236

BT - Artificial General Intelligence

A2 - Everitt, Tom

A2 - Potapov, Alexey

A2 - Goertzel, Ben

PB - Springer

CY - Cham, Switzerland

T2 - Conference on Artificial General Intelligence (AGI) 2017

Y2 - 15 August 2017 through 18 August 2017

ER -

Chmait N, Dowe DL , Li Y-F , Green DG. An information-theoretic predictive model for the accuracy of AI agents adapted from psychometrics. In Everitt T, Potapov A, Goertzel B, editors, Artificial General Intelligence : 10th International Conference, AGI 2017, Melbourne, VIC, Australia, August 15–18, 2017, Proceedings. Cham, Switzerland: Springer. 2017. p. 225-236. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-63703-7_21

An information-theoretic predictive model for the accuracy of AI agents adapted from psychometrics

Abstract

Publication series

Conference

Access to Document

Kurzweil Best Paper Prize

Cite this