Using autonomous agents to improvise music compositions in Real-Time

Patrick Hutchings, Jon McCormack

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

    9 Citations (Scopus)


    This paper outlines an approach to real-time music generation using melody and harmony focused agents in a process inspired by jazz improvisation. A harmony agent employs a Long Short-Term Memory (LSTM) artificial neural network trained on the chord progressions of 2986 jazz ‘standard’ compositions using a network structure novel to chord sequence analysis. The melody agent uses a rule-based system of manipulating provided, pre-composed melodies to improvise new themes and variations. The agents take turns in leading the direction of the composition based on a rating system that rewards harmonic consistency and melodic flow. In developing the multi-agent system it was found that implementing embedded spaces in the LSTM encoding process resulted in significant improvements to chord sequence learning.

    Original languageEnglish
    Title of host publicationComputational Intelligence in Music, Sound, Art and Design
    Subtitle of host publication6th International Conference, EvoMUSART 2017, Proceedings
    EditorsJoao Correia, Vic Ciesielski, Antonios Liapis
    Place of PublicationCham Switzerland
    Number of pages14
    ISBN (Electronic)9783319557502
    ISBN (Print)9783319557496
    Publication statusPublished - 2017
    EventInternational Conference on Evolutionary and Biologically Inspired Music, Sound, Art and Design 2017 - Amsterdam, Netherlands
    Duration: 19 Apr 201721 Apr 2017
    Conference number: 6th (Proceedings)

    Publication series

    NameLecture Notes in Computer Science
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349


    ConferenceInternational Conference on Evolutionary and Biologically Inspired Music, Sound, Art and Design 2017
    Abbreviated titleEvoMUSART 2017
    Internet address


    • Artificial neural networks
    • Multi-agent systems
    • Music composition

    Cite this