Abstract
Collaboration is built on trust, and establishing trust with a creative Artificial Intelligence is difficult when the decision process or internal state driving its behaviour isn’t exposed. When human musicians improvise together, a number of extra-musical cues are used to augment musical communication and expose mental or emotional states which affect musical decisions and the effectiveness of the collaboration. We developed a collaborative improvising AI drummer that communicates its confidence through an emoticon-based visualisation. The AI was trained on musical performance data, as well as real-time skin conductance, of musicians improvising with professional drummers, exposing both musical and extra-musical cues to inform its generative process. Uni- and bi-directional extra-musical communication with real and false values were tested by experienced improvising musicians. Each condition was evaluated using the FSS-2 questionnaire, as a proxy for musical engagement. The results show a positive correlation between extra-musical communication of machine internal state and human musical engagement.
| Original language | English |
|---|---|
| Title of host publication | CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems |
| Subtitle of host publication | May 4–9, 2019 Glasgow, Scotland UK |
| Place of Publication | New York NY USA |
| Publisher | Association for Computing Machinery (ACM) |
| Number of pages | 11 |
| ISBN (Electronic) | 9781450359702 |
| DOIs | |
| Publication status | Published - 2019 |
| Event | International Conference on Human Factors in Computing Systems 2019 - Glasgow, United Kingdom Duration: 4 May 2019 → 9 May 2019 https://chi2019.acm.org |
Conference
| Conference | International Conference on Human Factors in Computing Systems 2019 |
|---|---|
| Abbreviated title | CHI 2019 |
| Country | United Kingdom |
| City | Glasgow |
| Period | 4/05/19 → 9/05/19 |
| Internet address |
Keywords
- AI systems
- Extra-musical communication
- Improvisation
Cite this
}
In a silent way : communication between AI and improvising musicians beyond sound. / McCormack, Jon; Teresa Llano Rodriguez, Maria; Gifford, Toby; Yee-King, Matthew; Hutchings, Patrick; D’Inverno, Mark.
CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems: May 4–9, 2019 Glasgow, Scotland UK. New York NY USA : Association for Computing Machinery (ACM), 2019. 38.Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review
TY - GEN
T1 - In a silent way
T2 - communication between AI and improvising musicians beyond sound
AU - McCormack, Jon
AU - Teresa Llano Rodriguez, Maria
AU - Gifford, Toby
AU - Yee-King, Matthew
AU - Hutchings, Patrick
AU - D’Inverno, Mark
PY - 2019
Y1 - 2019
N2 - Collaboration is built on trust, and establishing trust with a creative Artificial Intelligence is difficult when the decision process or internal state driving its behaviour isn’t exposed. When human musicians improvise together, a number of extra-musical cues are used to augment musical communication and expose mental or emotional states which affect musical decisions and the effectiveness of the collaboration. We developed a collaborative improvising AI drummer that communicates its confidence through an emoticon-based visualisation. The AI was trained on musical performance data, as well as real-time skin conductance, of musicians improvising with professional drummers, exposing both musical and extra-musical cues to inform its generative process. Uni- and bi-directional extra-musical communication with real and false values were tested by experienced improvising musicians. Each condition was evaluated using the FSS-2 questionnaire, as a proxy for musical engagement. The results show a positive correlation between extra-musical communication of machine internal state and human musical engagement.
AB - Collaboration is built on trust, and establishing trust with a creative Artificial Intelligence is difficult when the decision process or internal state driving its behaviour isn’t exposed. When human musicians improvise together, a number of extra-musical cues are used to augment musical communication and expose mental or emotional states which affect musical decisions and the effectiveness of the collaboration. We developed a collaborative improvising AI drummer that communicates its confidence through an emoticon-based visualisation. The AI was trained on musical performance data, as well as real-time skin conductance, of musicians improvising with professional drummers, exposing both musical and extra-musical cues to inform its generative process. Uni- and bi-directional extra-musical communication with real and false values were tested by experienced improvising musicians. Each condition was evaluated using the FSS-2 questionnaire, as a proxy for musical engagement. The results show a positive correlation between extra-musical communication of machine internal state and human musical engagement.
KW - AI systems
KW - Extra-musical communication
KW - Improvisation
UR - http://www.scopus.com/inward/record.url?scp=85067600532&partnerID=8YFLogxK
U2 - 10.1145/3290605.3300268
DO - 10.1145/3290605.3300268
M3 - Conference Paper
BT - CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
PB - Association for Computing Machinery (ACM)
CY - New York NY USA
ER -