Abstract
We propose a new model to quantitatively estimate the accuracy of artificial agents over cognitive tasks of approximable complexities. The model is derived by introducing notions from algorithmic information theory into a well-known (psychometric) measurement paradigm called Item Response Theory (IRT). A lower bound on accuracy can be guaranteed with respect to task complexity and the breadth of its solution space using our model. This in turn permits formulating the relationship between agent selection cost, task difficulty and accuracy as optimisation problems. Further results indicate some of the settings over which a group of cooperative agents can be more or less accurate than individual agents or other groups.
| Language | English |
|---|---|
| Title of host publication | Artificial General Intelligence |
| Subtitle of host publication | 10th International Conference, AGI 2017, Melbourne, VIC, Australia, August 15–18, 2017, Proceedings |
| Editors | Tom Everitt, Alexey Potapov, Ben Goertzel |
| Place of Publication | Cham, Switzerland |
| Publisher | Springer-Verlag London Ltd. |
| Pages | 225-236 |
| Number of pages | 12 |
| ISBN (Electronic) | 9783319637037 |
| ISBN (Print) | 9783319637020 |
| DOIs | |
| Publication status | Published - 2017 |
| Event | Conference on Artificial General Intelligence (AGI) 2017 - Melbourne, Melbourne, Australia Duration: 15 Aug 2017 → 18 Aug 2017 Conference number: 10th |
Publication series
| Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
|---|---|
| Publisher | Springer |
| Volume | 10414 |
| ISSN (Print) | 0302-9743 |
| ISSN (Electronic) | 1611-3349 |
Conference
| Conference | Conference on Artificial General Intelligence (AGI) 2017 |
|---|---|
| Abbreviated title | AGI 2017 |
| Country | Australia |
| City | Melbourne |
| Period | 15/08/17 → 18/08/17 |
Cite this
}
An information-theoretic predictive model for the accuracy of AI agents adapted from psychometrics. / Chmait, Nader; Dowe, David L.; Li, Yuan-Fang; Green, David G.
Artificial General Intelligence : 10th International Conference, AGI 2017, Melbourne, VIC, Australia, August 15–18, 2017, Proceedings. ed. / Tom Everitt; Alexey Potapov; Ben Goertzel. Cham, Switzerland : Springer-Verlag London Ltd., 2017. p. 225-236 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10414 ).Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review
TY - GEN
T1 - An information-theoretic predictive model for the accuracy of AI agents adapted from psychometrics
AU - Chmait, Nader
AU - Dowe, David L.
AU - Li, Yuan-Fang
AU - Green, David G.
PY - 2017
Y1 - 2017
N2 - We propose a new model to quantitatively estimate the accuracy of artificial agents over cognitive tasks of approximable complexities. The model is derived by introducing notions from algorithmic information theory into a well-known (psychometric) measurement paradigm called Item Response Theory (IRT). A lower bound on accuracy can be guaranteed with respect to task complexity and the breadth of its solution space using our model. This in turn permits formulating the relationship between agent selection cost, task difficulty and accuracy as optimisation problems. Further results indicate some of the settings over which a group of cooperative agents can be more or less accurate than individual agents or other groups.
AB - We propose a new model to quantitatively estimate the accuracy of artificial agents over cognitive tasks of approximable complexities. The model is derived by introducing notions from algorithmic information theory into a well-known (psychometric) measurement paradigm called Item Response Theory (IRT). A lower bound on accuracy can be guaranteed with respect to task complexity and the breadth of its solution space using our model. This in turn permits formulating the relationship between agent selection cost, task difficulty and accuracy as optimisation problems. Further results indicate some of the settings over which a group of cooperative agents can be more or less accurate than individual agents or other groups.
UR - http://www.scopus.com/inward/record.url?scp=85028472248&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-63703-7_21
DO - 10.1007/978-3-319-63703-7_21
M3 - Conference Paper
SN - 9783319637020
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 225
EP - 236
BT - Artificial General Intelligence
A2 - Everitt, Tom
A2 - Potapov, Alexey
A2 - Goertzel, Ben
PB - Springer-Verlag London Ltd.
CY - Cham, Switzerland
ER -