Behaviour of FeRaNGA method for Feature Ranking during learning process using Inductive Modelling. Ales Pilny, Pavel Kordik, Miroslav Snorek

Abstract. Nowadays a Feature Ranking (FR) is commonly used method for obtaining information about a large data sets with various dimensionality. This knowledge can be used in a next step of data processing. Accuracy and a speed of experiments can be improved by this. Our approach is based on Artificial Neural Networks (ANN) instead of classical statistical methods. We obtain the knowledge as a by-product of Niching Genetic Algorithm (NGA) used for creation of a feedforward hybrid neural network called GAME. In this paper we present a behaviour of FeRaNGA (Feature Ranking method using Niching Genetic Algorithm (NGA)) during a learning process, especially in every layer of generated GAME network. We want to answer how important is NGA configuration and processing procedure for FR results because behaviour of GA is nondeterministic and thereby were results of FeRaNGA also indefinitive. This method ranks features depending on a percentage of processing elements that survived a selection process. Processing elements transforms parent input features to an output. The selection process is realized by means of NGA where units connected to the least significant features starve and fade from population. To obtain the best results and to find optimal configuration is behaviour of the FeRaNGA algortithm tested using various parameters of NGA and number of ensemble GAME models on well known artificial data sets.

Keywords. Inductive modelling, Feature Ranking, Artificial Neural Networks, FeRaNGA, Nitching Genetic Algorithm, FAKE-GAME

References.
1. P. Kordik. Fully Automated Knowledge Extraction using Group of Adaptive Models Evolution. PhD thesis, Czech Technical University in Prague, FEE, Dep. of Comp. Sci. and Computers, FEE, CTU Prague, Czech Republic, September 2006.
2. H. Madala and A. Ivakhnenko. Inductive Learning Algorithm for Complex System Modelling. CRC Press, 1994. Boca Raton. -  http://articles.gmdh.net
3. A. Pilny, P. Kordik, and M. Snorek. Feature ranking derived from data mining process. pages 889.898, Berlin, 2008. ICANN, Springer-Verlag Berlin Heidelberg 2008.
4. S. Piramuthu. Evaluating feature selection methods for learning in data mining applications. European Journal of Operational Research, 156:483.494, 2004.
5. M. Tesmer and P. Estevez. Amifs: adaptive feature selection by using mutual information. In Proceedings of the 2004 IEEE International Joint Conference on Neural Networks, volume 1, page 308, Dept. of Electr. Eng., Chile Univ., Santiago, Chile, July 2004.