In a silent way: communication between AI and improvising musicians beyond sound

Jon McCormack, Maria Teresa Llano Rodriguez, Toby Gifford, Matthew Yee-King, Patrick Hutchings, Mark D’Inverno

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

1 Citation (Scopus)

Abstract

Collaboration is built on trust, and establishing trust with a creative Artificial Intelligence is difficult when the decision process or internal state driving its behaviour isn’t exposed. When human musicians improvise together, a number of extra-musical cues are used to augment musical communication and expose mental or emotional states which affect musical decisions and the effectiveness of the collaboration. We developed a collaborative improvising AI drummer that communicates its confidence through an emoticon-based visualisation. The AI was trained on musical performance data, as well as real-time skin conductance, of musicians improvising with professional drummers, exposing both musical and extra-musical cues to inform its generative process. Uni- and bi-directional extra-musical communication with real and false values were tested by experienced improvising musicians. Each condition was evaluated using the FSS-2 questionnaire, as a proxy for musical engagement. The results show a positive correlation between extra-musical communication of machine internal state and human musical engagement.

Original languageEnglish
Title of host publicationProceedings of the 2019 CHI Conference on Human Factors in Computing Systems
EditorsAnna Cox, Vassilis Kostakos
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Number of pages11
ISBN (Electronic)9781450359702
DOIs
Publication statusPublished - 2019
EventInternational Conference on Human Factors in Computing Systems 2019 - Glasgow, United Kingdom
Duration: 4 May 20199 May 2019
Conference number: 37th
https://chi2019.acm.org

Conference

ConferenceInternational Conference on Human Factors in Computing Systems 2019
Abbreviated titleCHI 2019
CountryUnited Kingdom
CityGlasgow
Period4/05/199/05/19
Internet address

Keywords

  • AI systems
  • Extra-musical communication
  • Improvisation

Cite this

McCormack, J., Teresa Llano Rodriguez, M., Gifford, T., Yee-King, M., Hutchings, P., & D’Inverno, M. (2019). In a silent way: communication between AI and improvising musicians beyond sound. In A. Cox, & V. Kostakos (Eds.), Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems [38] Association for Computing Machinery (ACM). https://doi.org/10.1145/3290605.3300268