Pareto Genetic Design of GMDH-type Neural Networks for Nonlinear Systems. N. Nariman-Zadeh, A. Jamali. IWIM, Prague, 2007.

 Article (in pdf)

Abstract. In this paper, Genetic Algorithms (GAs) are deployed for multi-objective Pareto optimal design of Group Method of Data Handling (GMDH)-type neural networks that have been used for modelling of a nonlinear system. In this way, GAs with a specific encoding scheme is firstly presented to evolutionary design of the generalized GMDH-type neural networks in which the connectivity configurations in such networks are not limited to adjacent layers. Multi-objective GAs with a new diversity preserving mechanism are secondly used for Pareto optimization of such GMDH-type neural networks. The important conflicting objectives of GMDH-type neural networks that are considered in this work are, namely, Training Error (TE), Prediction Error (PE) and Number of Neurons (N) of such neural networks. It is shown that the obtained non-dominated Pareto points are inclusive of those which can be found using Akaike’s Information Criterion (AIC) for both training and prediction errors .Moreover, an important trade-off can be discovered by such Pareto optimum approach to the design of GMDH-type neural networks which helps a designer to select a network compromisingly.

Keywords. Multi-objective optimization, Genetic algorithms, GMDH, Pareto.

References.

1. Ivakhnenko, A. G.,: Polynomial Theory of Complex Systems. IEEE Trans. Syst. Man & Cybern, SMC-1, 364-378, 1971.
2. Farlow, S. J.,: Self-organizing Method in Modeling: GMDH type algorithm. Marcel Dekker Inc., 1984.
3. Iba, H., deGaris, H. and Sato, T.,: A numerical Approach to Genetic Programming for System Identification. Evolutionary Computation 3(4):417-452, 1996.
4. Nariman-Zadeh, N., Darvizeh, A., Felezi, M. E. and Gharababaei, H.,: Polynomial modelling of explosive compaction process of metallic powders using GMDH-type neural networks and singular value decomposition. Modelling and Simulation in Materials Science and Engineering, Vol. 10, no. 6, pp. 727-744(18), 2002.
5. Nariman-Zadeh, N., Darvizeh, A. and Ahmad-Zadeh, G. R.,: Hybrid Genetic Design of GMDHType Neural Networks Using Singular Value Decomposition for Modelling and Prediction of the Explosive Cutting Process. Proceedings of the I MECH E Part B Journal of Engineering Manufacture, Volume: 217, Page: 779 – 790, 2003.
6. Yao, X.,: Evolving Artificial Neural Networks. Proceedings of IEEE 87(9):1423-1447, 1999.
7. Kondo, T. and Ueno, J.,: Revised GMDH-type Neural Network Algorithm with a Feedback Loop Identifying Sigmoid Function Neural Network. Int. J. of Innovative Computing , Information and Control, Vol. 2 No.5, 2006.
8. Deb, K., Agrawal, S., Pratap, A. and Meyarivan, T.,: A fast and elitist multi-objective genetic algorithm: NSGA-II. IEEE Trans. On Evolutionary Computation 6(2):182-197, 2002.
9. Coello Coello, C. A.,: A comprehensive survey of evolutionary based multiobjective optimization techniques. Knowledge and Information Systems: An Int. Journal, (3), pp 269-308, 1999.
10. Nariman-zadeh, N., Atashkari, K., Jamali, A., Pilechi, A. and Yao, X.,: Inverse Modelling of Multi-objective Thermodynamically Optimized Turbo Engines using GMDH-type Neural Networks and Evolutionary Algorithms. Engineering Optimization, Vol. 37, No. 5, 437-462, 2005.
11. Press, W. H., Teukolsky, S. A., Vetterling, W. T. and Flannery B. P.,: Numerical Recipes in FORTRAN: The Art of Scientific Computing, 2nd Edition, Cambridge University Press, 1992.
12. Nariman-zadeh, N., Darvizeh, A., Jamali, A. and Moeini,: A. Evolutionary Design of Generalized Polynomial Neural Networks for Modelling and Prediction of Explosive Forming Process. Journal of Material Processing and Technology, Vol 164-165, pp 1561-1571, Elsevier, 2005.
13. Akaike, H.,: A new look at the statistical model identification, IEEE Trans. Automatic Control, vol.AC-19, no.6, pp.716-723, 1974.