Abstract
Manual scoring of polysomnography data is labor-intensive and time-consuming, and most existing software does not account for subjective differences and user variability. Therefore, we evaluated a supervised machine learning algorithm, SomnivoreTM, for automated wake–sleep stage classification. We designed an algorithm that extracts features from various input channels, following a brief session of manual scoring, and provides automated wake-sleep stage classification for each recording. For algorithm validation, polysomnography data was obtained from independent laboratories, and include normal, cognitively-impaired, and alcohol-treated human subjects (total n = 52), narcoleptic mice and drug-treated rats (total n = 56), and pigeons (n = 5). Training and testing sets for validation were previously scored manually by 1–2 trained sleep technologists from each laboratory. F-measure was used to assess precision and sensitivity for statistical analysis of classifier output and human scorer agreement. The algorithm gave high concordance with manual visual scoring across all human data (wake 0.91 ± 0.01; N1 0.57 ± 0.01; N2 0.81 ± 0.01; N3 0.86 ± 0.01; REM 0.87 ± 0.01), which was comparable to manual inter-scorer agreement on all stages. Similarly, high concordance was observed across all rodent (wake 0.95 ± 0.01; NREM 0.94 ± 0.01; REM 0.91 ± 0.01) and pigeon (wake 0.96 ± 0.006; NREM 0.97 ± 0.01; REM 0.86 ± 0.02) data. Effects of classifier learning from single signal inputs, simple stage reclassification, automated removal of transition epochs, and training set size were also examined. In summary, we have developed a polysomnography analysis program for automated sleep-stage classification of data from diverse species. Somnivore enables flexible, accurate, and high-throughput analysis of experimental and clinical sleep studies.
| Original language | English |
|---|---|
| Article number | 207 |
| Pages (from-to) | 1-18 |
| Number of pages | 18 |
| Journal | Frontiers in Neuroscience |
| Volume | 13 |
| DOIs | |
| Publication status | Published - 2019 |
| Externally published | Yes |
Keywords
- Machine learning algorithms
- Polysomnography
- Signal processing algorithms
- Sleep stage classification
- Sleep stage scoring
- Wake
Cite this
}
Validation of ‘Somnivore’, a Machine Learning Algorithm for Automated Scoring and Analysis of Polysomnography Data. / Allocca, Giancarlo; Ma, Sherie; Martelli, Davide; Cerri, Matteo; Del Vecchio, Flavia; Bastianini, Stefano; Zoccoli, Giovanna; Amici, Roberto; Morairty, Stephen R.; Aulsebrook, Anne E.; Blackburn, Shaun; Lesku, John A.; Rattenborg, Niels C.; Vyssotski, Alexei L.; Wams, Emma; Porcheret, Kate; Wulff, Katharina; Foster, Russell; Chan, Julia K.M.; Nicholas, Christian L.; Freestone, Dean R.; Johnston, Leigh A.; Gundlach, Andrew L.
In: Frontiers in Neuroscience, Vol. 13, 207, 2019, p. 1-18.Research output: Contribution to journal › Article › Research › peer-review
TY - JOUR
T1 - Validation of ‘Somnivore’, a Machine Learning Algorithm for Automated Scoring and Analysis of Polysomnography Data
AU - Allocca, Giancarlo
AU - Ma, Sherie
AU - Martelli, Davide
AU - Cerri, Matteo
AU - Del Vecchio, Flavia
AU - Bastianini, Stefano
AU - Zoccoli, Giovanna
AU - Amici, Roberto
AU - Morairty, Stephen R.
AU - Aulsebrook, Anne E.
AU - Blackburn, Shaun
AU - Lesku, John A.
AU - Rattenborg, Niels C.
AU - Vyssotski, Alexei L.
AU - Wams, Emma
AU - Porcheret, Kate
AU - Wulff, Katharina
AU - Foster, Russell
AU - Chan, Julia K.M.
AU - Nicholas, Christian L.
AU - Freestone, Dean R.
AU - Johnston, Leigh A.
AU - Gundlach, Andrew L.
PY - 2019
Y1 - 2019
N2 - Manual scoring of polysomnography data is labor-intensive and time-consuming, and most existing software does not account for subjective differences and user variability. Therefore, we evaluated a supervised machine learning algorithm, SomnivoreTM, for automated wake–sleep stage classification. We designed an algorithm that extracts features from various input channels, following a brief session of manual scoring, and provides automated wake-sleep stage classification for each recording. For algorithm validation, polysomnography data was obtained from independent laboratories, and include normal, cognitively-impaired, and alcohol-treated human subjects (total n = 52), narcoleptic mice and drug-treated rats (total n = 56), and pigeons (n = 5). Training and testing sets for validation were previously scored manually by 1–2 trained sleep technologists from each laboratory. F-measure was used to assess precision and sensitivity for statistical analysis of classifier output and human scorer agreement. The algorithm gave high concordance with manual visual scoring across all human data (wake 0.91 ± 0.01; N1 0.57 ± 0.01; N2 0.81 ± 0.01; N3 0.86 ± 0.01; REM 0.87 ± 0.01), which was comparable to manual inter-scorer agreement on all stages. Similarly, high concordance was observed across all rodent (wake 0.95 ± 0.01; NREM 0.94 ± 0.01; REM 0.91 ± 0.01) and pigeon (wake 0.96 ± 0.006; NREM 0.97 ± 0.01; REM 0.86 ± 0.02) data. Effects of classifier learning from single signal inputs, simple stage reclassification, automated removal of transition epochs, and training set size were also examined. In summary, we have developed a polysomnography analysis program for automated sleep-stage classification of data from diverse species. Somnivore enables flexible, accurate, and high-throughput analysis of experimental and clinical sleep studies.
AB - Manual scoring of polysomnography data is labor-intensive and time-consuming, and most existing software does not account for subjective differences and user variability. Therefore, we evaluated a supervised machine learning algorithm, SomnivoreTM, for automated wake–sleep stage classification. We designed an algorithm that extracts features from various input channels, following a brief session of manual scoring, and provides automated wake-sleep stage classification for each recording. For algorithm validation, polysomnography data was obtained from independent laboratories, and include normal, cognitively-impaired, and alcohol-treated human subjects (total n = 52), narcoleptic mice and drug-treated rats (total n = 56), and pigeons (n = 5). Training and testing sets for validation were previously scored manually by 1–2 trained sleep technologists from each laboratory. F-measure was used to assess precision and sensitivity for statistical analysis of classifier output and human scorer agreement. The algorithm gave high concordance with manual visual scoring across all human data (wake 0.91 ± 0.01; N1 0.57 ± 0.01; N2 0.81 ± 0.01; N3 0.86 ± 0.01; REM 0.87 ± 0.01), which was comparable to manual inter-scorer agreement on all stages. Similarly, high concordance was observed across all rodent (wake 0.95 ± 0.01; NREM 0.94 ± 0.01; REM 0.91 ± 0.01) and pigeon (wake 0.96 ± 0.006; NREM 0.97 ± 0.01; REM 0.86 ± 0.02) data. Effects of classifier learning from single signal inputs, simple stage reclassification, automated removal of transition epochs, and training set size were also examined. In summary, we have developed a polysomnography analysis program for automated sleep-stage classification of data from diverse species. Somnivore enables flexible, accurate, and high-throughput analysis of experimental and clinical sleep studies.
KW - Machine learning algorithms
KW - Polysomnography
KW - Signal processing algorithms
KW - Sleep stage classification
KW - Sleep stage scoring
KW - Wake
UR - http://www.scopus.com/inward/record.url?scp=85065910116&partnerID=8YFLogxK
U2 - 10.3389/fnins.2019.00207
DO - 10.3389/fnins.2019.00207
M3 - Article
VL - 13
SP - 1
EP - 18
JO - Frontiers in Neuroscience
JF - Frontiers in Neuroscience
SN - 1662-453X
M1 - 207
ER -