Self-organising neural network hierarchy

Satya Borgohain, Gideon Kowadlo, David Rawlinson, Christoph Bergmeir, Kok Loo, Harivallabha Rangarajan, Levin Kuhlmann

Research output: Chapter in Book/Report/Conference proceedingConference PaperOtherpeer-review

Abstract

Mammalian brains exhibit functional self-organisation between different neocortical regions to form virtual hierarchies from a physical 2D sheet. We propose a biologically-inspired self-organizing neural network architecture emulating the same. The network is composed of autoencoder units and driven by a meta-learning rule based on maximizing the Shannon entropy of latent representations of the input, which optimizes the receptive field placement of each unit within a feature map. Unlike Neural Architecture Search, here both the network parameters and the architecture are learned simultaneously. In a case study on image datasets, we observe that the meta-learning rule causes a functional hierarchy to form, and leads to learning progressively better topological configurations and higher classification performance overall, starting from randomly initialized architectures. In particular, our approach yields competitive performance in terms of classification accuracy compared to optimal handcrafted architecture(s) with desirable topological features for this network type, on both MNIST and CIFAR-10 datasets, even though it is not as significant for the latter.

Original languageEnglish
Title of host publicationAI 2020
Subtitle of host publicationAdvances in Artificial Intelligence - 33rd Australasian Joint Conference, AI 2020, Proceedings
EditorsMarcus Gallagher, Nour Moustafa, Erandi Lakshika
Place of PublicationCham Switzerland
PublisherSpringer
Pages359-370
Number of pages12
Edition1st
ISBN (Electronic)9783030649845
ISBN (Print)9783030649838
DOIs
Publication statusPublished - 2020
EventAustralasian Joint Conference on Artificial Intelligence 2020 - Canberra, Australia
Duration: 29 Nov 202030 Nov 2020
Conference number: 33rd
https://link.springer.com/book/10.1007/978-3-030-64984-5 (Proceedings)

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12576 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceAustralasian Joint Conference on Artificial Intelligence 2020
Abbreviated titleAI 2020
CountryAustralia
CityCanberra
Period29/11/2030/11/20
Internet address

Keywords

  • Biologically plausible networks
  • Greedy training
  • Self-supervised learning

Cite this