Smoothing supervised learning of neural networks for function approximation

Two popular hazards in supervised learning of neural networks are local minima and overfitting. Application of the momentum technique dealing with the local optima has proved efficient but it is vulnerable to overfitting. In contrast, deployment of the early stopping technique might overcome the overfitting phenomena but it sometimes terminates into the local minima. This paper proposes a hybrid approach, which is a combination of two processing neurons: momentum and early stopping, to tackle these hazards, aiming at improving the performance of neural networks in terms of both accuracy and processing time in function approximation. Experimental results conducted on various kinds of non-linear functions have demonstrated that the proposed approach is dominant compared with conventional learning approaches.