Artificial neural network: deep or broad? an empirical study

Nian Liu, Nayyar A. Zaidi

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

    Abstract

    Advent of Deep Learning and the emergence of Big Data has led to renewed interests in the study of Artificial Neural Networks (ANN). An ANN is a highly effective classifier that is capable of learning both linear and non-linear boundaries. The number of hidden layers and the number of nodes in each hidden layer (along with many other parameters) in an ANN, is considered to be a model selection problem. With success of deep learning especially on big datasets, there is a prevalent belief in machine learning community that a deep model (that is a model with many number of hidden layers) is preferable. However, this belies earlier theorems proved for ANN that only a single hidden layer (with multiple nodes) is capable of learning any arbitrary function, i.e., a shallow broad ANN. This raises the question of whether one should build a deep network or go for a broad network. In this paper, we do a systematic study of depth and breadth of an ANN in terms of its accuracy (0–1 Loss), bias, variance and convergence performance on 72 standard UCI datasets and we argue that broad ANN has better overall performance than deep ANN.

    Original languageEnglish
    Title of host publicationAI 2016: Advances in Artificial Intelligence
    Subtitle of host publication29th Australasian Joint Conference Hobart, TAS, Australia, December 5–8, 2016 Proceedings
    EditorsByeong Ho Kang, Quan Bai
    Place of PublicationCham Switzerland
    PublisherSpringer
    Pages535-541
    Number of pages7
    ISBN (Electronic)9783319501277
    ISBN (Print)9783319501260
    DOIs
    Publication statusPublished - 2016
    EventAustralasian Joint Conference on Artificial Intelligence 2016 - Hobart, Australia
    Duration: 5 Dec 20168 Dec 2016
    Conference number: 29th
    https://ai2016.net/
    https://link.springer.com/book/10.1007/978-3-319-50127-7 (Proceedings)

    Publication series

    NameLecture Notes in Computer Science
    PublisherSpringer
    Volume9992
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Conference

    ConferenceAustralasian Joint Conference on Artificial Intelligence 2016
    Abbreviated titleAI 2016
    CountryAustralia
    CityHobart
    Period5/12/168/12/16
    Internet address

    Keywords

    • Artificial Neural Network
    • Hide Layer
    • Deep Learning
    • Convolutional Neural Network

    Cite this

    Liu, N., & Zaidi, N. A. (2016). Artificial neural network: deep or broad? an empirical study. In B. Ho Kang, & Q. Bai (Eds.), AI 2016: Advances in Artificial Intelligence: 29th Australasian Joint Conference Hobart, TAS, Australia, December 5–8, 2016 Proceedings (pp. 535-541). (Lecture Notes in Computer Science; Vol. 9992 ). Springer. https://doi.org/10.1007/978-3-319-50127-7_46