RPNN: Structural modeling robust to outliers in input and output variables. Alessandro Villa, Vladyslav Shaposhnyk, Tetiana Aksenova

Abstract. The robust regression analysis works on data affected by deviations from a general assumption of normality. There are number of stable and robust methods in the field of linear regression analysis. In contrast the robust structural modeling is still under active development.

This paper describes a novel algorithm designed to solve a task of optimal polynomial model selection on multivariate data sets in presence of outliers in both input and output variables. On one side it is based on GMDH-type Polynomial Neural Network (PNN), which gives an universal model structure identification thanks to the evolving adaptively synthesized bounded network. From the other side the algorithm is based on application of MM-estimator, which allows achieving robustness to outliers in both input and output data sets. Previous version of Robust PNN was addressed to the modeling of the data with outliers in output variables only.

Enhanced RPNN was developed and tested on the artificial data set resulted from the simulation polynomials up to third degree. The Gaussian noise as well as outliers was added to the data. RPNN demonstrated robustness to outliers in both input and output variables (20% of outliers) and good accuracy of the automatic structure syntheses as well as of the parameters estimation.

Keywords. Polynomial neural network, robust regression, nonlinear regression, gm-estimators, structure selection.

References.

1. H. Akaike. A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19:716723, 1974. 2. T. I. Aksenova, V. Volkovich, and A.E.P. Villa. Robust structural modeling and outlier detection with gmdh-type polynomial neural networks. LNCS, pages 881–886, 2005.

3. K.P. Burnham and D.R. Anderson. Multimodel inference: Understanding aic and bic in model selection. Sociological Methods Research, 33(2):261–304, November 2004.

4. D. L. Donoho and P. J. Huber. The notion of breakdown point. Festschr. for Erich L. Lehmann, pages 157–184, 1983.

5. F. R. Hampel. A general qualitative definition of robustness. Ann. Math. Stat., 42:1887–1896, 1971.

6. A. G. Ivakhnenko. Polynomial theory of complex systems. IEEE Transactions on Systems, Man, and Cybernetics, SMC-1(4):364–378, 1971.

7. E. Ronchetti. Robustness aspects of model choice. Statistica Sinica, 7:327–338, 1997.

8. P. Rousseeuw and V. Yohai. Robust regression by means of s-estimators. Robust and nonlinear time series analysis, SMC-1(26):256–272, 1983.

9. P.J. Rousseeuw. Least median of squares regression. Journal of the American Statistical Association, 79:871–880, 1984.

10. V. Yohai. High breakdown-point and high efficiency robust estimates for regression. Ann. Stat., 15:642–656, 1987.