Seen music: ambient music data visualization for children with hearing impairments

Jeeeun Kim, Swamy Ananthanarayan, Tom Yeh

Research output: Chapter in Book/Report/Conference proceedingConference PaperOtherpeer-review

2 Citations (Scopus)


In this paper, we propose a prototype of music visualization system that captures and records the music component into digital data form, and then displays the data in visual form for children with hearing impairments. The analog sound data of music played physically is scaled into a binary matrix and scalar values that is then used as data structures for transcribing the output. We designed a system that detects tune and speed from a physical violin, and demonstrated three tangible music visualizations that children see in their daily lives, employing a flowerpot, plants and a picture of frame. We describe how the data captured from physical musical instruments can be seen through these objects, and suggest future possibilities for interactive sound visualization in music education for children with hearing impairments. Copyright is held by the owner/author(s).

Original languageEnglish
Title of host publicationProceedings of the IDC 2015, The 14th International Conference on Interaction Design and Children
EditorsMelissa Lee, Taylor Strelevitz
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Number of pages4
ISBN (Electronic)9781450335904
Publication statusPublished - 2015
Externally publishedYes
EventInteraction Design and Children (ACM) 2015 - Boston, United States of America
Duration: 21 Jun 201524 Jun 2015
Conference number: 14th (Proceedings)


ConferenceInteraction Design and Children (ACM) 2015
Abbreviated titleIDC 2015
CountryUnited States of America
Internet address


  • Assistive technology
  • Music education
  • Music visualization

Cite this