A review of artificial neural network learning rule based on multiple variant of conjugate gradient approaches

Abstract

Abstract The evolution of Artificial Neural Network (ANN) begins in1940s when McCulloch and Pitts published research articles in 1943 discussing about the idea of neural network in general. Basically, the concept of ANN has been inspired by biological human brain model. Then, this concept is transformed into a mathematical formulation and lastly become a machine learning used to solve many problems in this world. Mathematic formulations, design concept, algorithm and computer program can be constructed from ANN. Artificial neural network had undergone many changes on its algorithm and its execution. Otherwise, areas of applications are numerous involving different techniques and approaches of algorithms. ANN algorithm use optimization techniques as a way to find the best outcome based on the problem to be solved. Conjugate Gradient (CG) is one of the popular optimization practices used in ANN to improve learning algorithm nowadays. Therefore, this paper is intended to find out the function and role of modified conjugate gradient method in neural networks and potential related approaches along the age of its advancement. This paper also projected to give an overview approach of ANN with CG method especially on modified CG and overalls performances of those selected models.