TY - JOUR
T1 - Effects of a comprehensive brain computed tomography deep learning model on radiologist detection accuracy
AU - Buchlak, Quinlan D.
AU - Tang, Cyril H.M.
AU - Seah, Jarrel C.Y.
AU - Johnson, Andrew
AU - Holt, Xavier
AU - Bottrell, Georgina M.
AU - Wardman, Jeffrey B.
AU - Samarasinghe, Gihan
AU - Dos Santos Pinheiro, Leonardo
AU - Xia, Hongze
AU - Ahmad, Hassan K.
AU - Pham, Hung
AU - Chiang, Jason I.
AU - Ektas, Nalan
AU - Milne, Michael R.
AU - Chiu, Christopher H.Y.
AU - Hachey, Ben
AU - Ryan, Melissa K.
AU - Johnston, Benjamin P.
AU - Esmaili, Nazanin
AU - Bennett, Christine
AU - Goldschlager, Tony
AU - Hall, Jonathan
AU - Vo, Duc Tan
AU - Oakden-Rayner, Lauren
AU - Leveque, Jean Christophe
AU - Farrokhi, Farrokh
AU - Abramson, Richard G.
AU - Jones, Catherine M.
AU - Edelstein, Simon
AU - Brotchie, Peter
N1 - Funding Information:
Open Access funding enabled and organized by CAUL and its Member Institutions. This study has received funding from annalise.ai.
Funding Information:
This study was funded by annalise.ai. QB, CT, JS, AJ, XH, GB, JW, GS, LDSP, HX, HA, HP, JC, NE, MM, CC, BH, MR, BJ, JH, CJ, SE, and PB were employed by or seconded to annalise.ai. CJ and SE were employed by I-MED. The authors listed above report personal fees from annalise.ai during the conduct of the study and personal fees from annalise.ai outside the submitted work. Remaining authors have no competing interests. The annalise.ai CTB deep learning model is available commercially.
Publisher Copyright:
© 2023, The Author(s).
PY - 2024/8/22
Y1 - 2024/8/22
N2 - Objectives: Non-contrast computed tomography of the brain (NCCTB) is commonly used to detect intracranial pathology but is subject to interpretation errors. Machine learning can augment clinical decision-making and improve NCCTB scan interpretation. This retrospective detection accuracy study assessed the performance of radiologists assisted by a deep learning model and compared the standalone performance of the model with that of unassisted radiologists. Methods: A deep learning model was trained on 212,484 NCCTB scans drawn from a private radiology group in Australia. Scans from inpatient, outpatient, and emergency settings were included. Scan inclusion criteria were age ≥ 18 years and series slice thickness ≤ 1.5 mm. Thirty-two radiologists reviewed 2848 scans with and without the assistance of the deep learning system and rated their confidence in the presence of each finding using a 7-point scale. Differences in AUC and Matthews correlation coefficient (MCC) were calculated using a ground-truth gold standard. Results: The model demonstrated an average area under the receiver operating characteristic curve (AUC) of 0.93 across 144 NCCTB findings and significantly improved radiologist interpretation performance. Assisted and unassisted radiologists demonstrated an average AUC of 0.79 and 0.73 across 22 grouped parent findings and 0.72 and 0.68 across 189 child findings, respectively. When assisted by the model, radiologist AUC was significantly improved for 91 findings (158 findings were non-inferior), and reading time was significantly reduced. Conclusions: The assistance of a comprehensive deep learning model significantly improved radiologist detection accuracy across a wide range of clinical findings and demonstrated the potential to improve NCCTB interpretation. Clinical relevance statement: This study evaluated a comprehensive CT brain deep learning model, which performed strongly, improved the performance of radiologists, and reduced interpretation time. The model may reduce errors, improve efficiency, facilitate triage, and better enable the delivery of timely patient care. Key Points: • This study demonstrated that the use of a comprehensive deep learning system assisted radiologists in the detection of a wide range of abnormalities on non-contrast brain computed tomography scans. • The deep learning model demonstrated an average area under the receiver operating characteristic curve of 0.93 across 144 findings and significantly improved radiologist interpretation performance. • The assistance of the comprehensive deep learning model significantly reduced the time required for radiologists to interpret computed tomography scans of the brain.
AB - Objectives: Non-contrast computed tomography of the brain (NCCTB) is commonly used to detect intracranial pathology but is subject to interpretation errors. Machine learning can augment clinical decision-making and improve NCCTB scan interpretation. This retrospective detection accuracy study assessed the performance of radiologists assisted by a deep learning model and compared the standalone performance of the model with that of unassisted radiologists. Methods: A deep learning model was trained on 212,484 NCCTB scans drawn from a private radiology group in Australia. Scans from inpatient, outpatient, and emergency settings were included. Scan inclusion criteria were age ≥ 18 years and series slice thickness ≤ 1.5 mm. Thirty-two radiologists reviewed 2848 scans with and without the assistance of the deep learning system and rated their confidence in the presence of each finding using a 7-point scale. Differences in AUC and Matthews correlation coefficient (MCC) were calculated using a ground-truth gold standard. Results: The model demonstrated an average area under the receiver operating characteristic curve (AUC) of 0.93 across 144 NCCTB findings and significantly improved radiologist interpretation performance. Assisted and unassisted radiologists demonstrated an average AUC of 0.79 and 0.73 across 22 grouped parent findings and 0.72 and 0.68 across 189 child findings, respectively. When assisted by the model, radiologist AUC was significantly improved for 91 findings (158 findings were non-inferior), and reading time was significantly reduced. Conclusions: The assistance of a comprehensive deep learning model significantly improved radiologist detection accuracy across a wide range of clinical findings and demonstrated the potential to improve NCCTB interpretation. Clinical relevance statement: This study evaluated a comprehensive CT brain deep learning model, which performed strongly, improved the performance of radiologists, and reduced interpretation time. The model may reduce errors, improve efficiency, facilitate triage, and better enable the delivery of timely patient care. Key Points: • This study demonstrated that the use of a comprehensive deep learning system assisted radiologists in the detection of a wide range of abnormalities on non-contrast brain computed tomography scans. • The deep learning model demonstrated an average area under the receiver operating characteristic curve of 0.93 across 144 findings and significantly improved radiologist interpretation performance. • The assistance of the comprehensive deep learning model significantly reduced the time required for radiologists to interpret computed tomography scans of the brain.
KW - Artificial intelligence
KW - Brain
KW - Machine learning
KW - Supervised machine learning
KW - Tomography, x-ray computed
UR - http://www.scopus.com/inward/record.url?scp=85168576669&partnerID=8YFLogxK
U2 - 10.1007/s00330-023-10074-8
DO - 10.1007/s00330-023-10074-8
M3 - Article
C2 - 37606663
AN - SCOPUS:85168576669
SN - 0938-7994
VL - 34
SP - 810
EP - 822
JO - European Radiology
JF - European Radiology
ER -