Adaptive Learning Rate for Neural Network Classification Model

Abstract

Imbalanced data cause prediction inaccuracy of the classification model. Two types of techniques have been devised to address this problem: pre-processing data before training a classification model and adjusting the classification algorithm. This study, which introduced the adaptive learning rate into a backpropagation neural network algorithm, is of the latter type. The learning rate was adjusted in each iterative learning cycle: the learning rate is increased for the data class with fewer samples and decreased for the data class with more samples. K-fold cross-validation was used to test the effectiveness of the prediction model on 10 datasets. The results showed that the proposed ZMP algorithm outperformed the original backpropagation neural network on 6 datasets; the improvement ranged from 2.24% to 20.22%. Moreover, on the other 4 datasets, even though the proposed technique provided less accurate predictions, the differences were very slight.