MMLD inference of multilayer perceptrons

Enes Makalic, Lloyd Allison

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

    1 Citation (Scopus)

    Abstract

    A multilayer perceptron comprising a single hidden layer of neurons with sigmoidal transfer functions can approximate any computable function to arbitrary accuracy. The size of the hidden layer dictates the approximation capability of the multilayer perceptron and automatically determining a suitable network size for a given data set is an interesting question. This paper considers the problem of inferring the size of multilayer perceptron networks with the MMLD model selection criterion which is based on the minimum message length principle. The two main contributions of the paper are: (1) a new model selection criterion for inference of fully-connected multilayer perceptrons in regression problems, and (2) an efficient algorithm for computing MMLD-type codelengths in mathematically challenging model classes. Empirical performance of the new algorithm is demonstrated on artificially generated and real data sets.

    Original languageEnglish
    Title of host publicationAlgorithmic Probability and Friends: Bayesian Prediction and Artificial Intelligence
    Subtitle of host publicationPapers from the Ray Solomonoff 85th Memorial Conference, Melbourne, VIC, Australia, November 30 – December 2, 2011
    EditorsDavid L. Dowe
    PublisherSpringer-Verlag London Ltd.
    Pages261-272
    Number of pages12
    ISBN (Electronic)9783642449581
    ISBN (Print)9783642449574
    DOIs
    Publication statusPublished - 2013
    EventRay Solomonoff Memorial Conference 2011: Bayesian Prediction and Artificial Intelligence - Melbourne, Australia
    Duration: 30 Nov 20112 Dec 2011
    Conference number: 85

    Publication series

    NameLecture Notes in Computer Science
    PublisherSpringer
    Volume7070 LNAI
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Conference

    ConferenceRay Solomonoff Memorial Conference 2011
    CountryAustralia
    CityMelbourne
    Period30/11/112/12/11
    OtherRay Solomonoff 85th Memorial Conference on Algorithmic Probability and Friends: Bayesian Prediction and Artificial Intelligence

    Cite this