Performance of Deep and Shallow Neural Networks, the Universal Approximation Theorem, Activity Cliffs, and QSAR

David Winkler, Tu C Le

Research output: Contribution to journalArticleResearchpeer-review

27 Citations (Scopus)

Abstract

Neural networks have generated valuable Quantitative Structure-Activity/Property Relationships (QSAR/QSPR) models for a wide variety of small molecules and materials properties. They have grown in sophistication and many of their initial problems have been overcome by modern mathematical techniques. QSAR studies have almost always used so-called “shallow” neural networks in which there is a single hidden layer between the input and output layers. Recently, a new and potentially paradigm-shifting type of neural network based on Deep Learning has appeared. Deep learning methods have generated impressive improvements in image and voice recognition, and are now being applied to QSAR and QSAR modelling. This paper describes the differences in approach between deep and shallow neural networks, compares their abilities to predict the properties of test sets for 15 large drug data sets (the kaggle set), discusses the results in terms of the Universal Approximation theorem for neural networks, and describes how DNN may ameliorate or remove troublesome “activity cliffs” in QSAR data sets.

Original languageEnglish
Article number1600118
Number of pages6
JournalMolecular Informatics
Volume36
Issue number1-2
DOIs
Publication statusPublished - 1 Jan 2017

Keywords

  • deep learning
  • neural network
  • Bayesian regularisation
  • universal approximation theorem
  • QSAR
  • activity cliff
  • prediction

Prizes

Herman Skolnik award from the American Chemical Society

Winkler, David (Recipient), 22 Aug 2017

Prize: National/international honour

Cite this