A bayesian approach for constructing ensemble neural network

Abstract

Ensemble neural networks (ENNs) are commonly used in many engineering applications due to its better generalization properties compared with a single neural network (NN). As the NN architecture has a significant influence on the generalization ability of an NN, it is crucial to develop a proper algorithm to design the NN architecture. In this paper, an ENN which combines the component networks by using the Bayesian approach and stochastic modelling is proposed. The cross validation data set is used not only to stop the network training, but also to determine the weights of the component networks. The proposed ENN searches the best structure of each component network first and employs the Bayesian approach as an automating design tool to determine the best combining weights of the ENN. Peak function is used to assess the accuracy of the proposed ensemble approach. The results show that the proposed ENN outperforms ENN obtained by simple averaging and the single NN. Copyright © 2012 SciTePress - Science and Technology Publications.