Continuous function optimisation via gradient descent on a neural network approximation function

Kate A Smith, Jatinder N D Gupta

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

    3 Citations (Scopus)

    Abstract

    Existing neural network approaches to optimisation problems are quite limited in the types of optimisation problems that can be solved. Convergence theorems that utilise Liapunov functions limit the applicability of these techniques to minimising usually quadratic functions only. This paper proposes a new neural network approach that can be used to solve a broad variety of continuous optimisation problems since it makes no assumptions about the nature of the objective function. The approach comprises two stages: first a feedforward neural network is used to approximate the optimisation function based on a sample of evaluated data points; then a feedback neural network is used to perform gradient descent on this approximation function. The final solution is a local minima of the approximated function, which should coincide with true local minima if the learning has been accurate. The proposed method is evaluated on the De Jong test suite: a collection of continuous optimisation problems featuring various characteristics such as saddlepoints, discontinuities, and noise.
    Original languageEnglish
    Title of host publicationConnectionist Models of Neurons, Learning Processes, and Artificial Intelligence
    Subtitle of host publication6th International Work-Conference on Artificial and Natural Neural Networks, IWANN 2001 Granada, Spain, June 13-15, 2001 Proceedings, Part I
    EditorsJose Mira, Alberto Prieto
    Place of PublicationBerlin Germany
    PublisherSpringer
    Pages741-748
    Number of pages8
    ISBN (Print)3540422358
    DOIs
    Publication statusPublished - 2001
    EventInternational Work-Conference on Artificial and Natural Neural Networks 2001 - Granada, Spain
    Duration: 13 Jun 200115 Jun 2001
    Conference number: 6th
    https://link.springer.com/book/10.1007/3-540-45720-8 (Proceedings)

    Publication series

    NameLecture Notes in Computer Science
    PublisherSpringer
    Volume2084
    ISSN (Print)0302-9743

    Conference

    ConferenceInternational Work-Conference on Artificial and Natural Neural Networks 2001
    Abbreviated titleIWANN 2001
    CountrySpain
    CityGranada
    Period13/06/0115/06/01
    Internet address

    Cite this