Analytical Incremental Learning: Fast Constructive Learning Method for Neural Network

Abstract

Extreme learning machine (ELM) is a fast learning algorithm for single hidden layer feed-forward neural network (SLFN) based on random input weights which usually requires large number of hidden nodes. Recently, novel constructive and destructive parsimonious (CP and DP)-ELM which provide the effectiveness generalization and compact hidden nodes have been proposed. However, the performance might be unstable due to the randomization either in ordinary ELM or CP and DP-ELM. In this study, analytical incremental learning (AIL) algorithm is proposed in which all weights of neural network are calculated analytically without any randomization. The hidden nodes of AIL are incrementally generated based on residual error using least square (LS) method. The results show the effectiveness of AIL which has not only smallest number of hidden nodes and more stable but also good generalization than those of ELM, CP and DP-ELM based on seven benchmark data sets evaluation.