H.R. Madala, A.G. Ivakhnenko. Inductive Learning Algorithms for Complex Systems Modeling. - CRC Press, Boca Raton, 1994.

Download  .zip (11MB)

A professional monograph that surveys new types of learning algorithms for modelling complex scientific systems in science and engineering. The book features:

• Discussions of algorithms structure, noise immunity and behavior;
• Presents comprehensive coverage of all types of algorithms useful for this subject;
• Applications of various modelling activities (e.g. environmental systems, stock market, economic systems, noise immunity, decision trees, data mining and neural networks);
• Includes recent studies on clusterization and recognition problems;
• Provides listings of algorithms in FORTRAN that can be run directly on PCs.

It is a valuable reference for graduate students and scientists in applied mathematics, statistics, computer science. The book will also benefit engineers and research workers from applied fields such as stock market, weather forecasting, air and water pollution studies, economics, finances, hydrology, agriculture and time series evaluations.

Contents

1. Introduction . . . .1

1. SYSTEMS AND CYBERNETICS . . . . 1

1.1 Definitions . . . . 2
1.2 Model and simulation . . . . 4
1.3 Concept of black box . . . . 5

2. SELF-ORGANIZATION MODELING . . . . 6

2.1 Neural approach . . . . 6
2.2 Inductive approach . . . . 7

3. INDUCTIVE LEARNING METHODS . . . . 9

3.1 Principal shortcoming in model development . . . . 10
3.2 Principle of self-organization . . . . 11
3.3 Basic technique . . . . 11
3.4 Selection criteria or objective functions . . . . 12
3.5 Heuristics used in problem-solving . . . . 17

2. Inductive Learning Algorithms . . . . 27

1. SELF-ORGANIZATION METHOD . . . . 27

1.1 Basic iterative algorithm . . . . 28

2. NETWORK STRUCTURES . . . . 30

2.1 Multilayer algorithm . . . . 30
2.2 Combinatorial algorithm . . . . 32
2.3 Recursive scheme for faster combinatorial sorting . . . . 35
2.4 Multilayered structures using combinatorial setup . . . . 38
2.5 Selectional-combinatorial multilayer algorithm . . . . 38
2.6 Multilayer algorithm with propagating residuals (front propagation algorithm) . . . . 41
2.7 Harmonic Algorithm . . . . 42
2.8 New algorithms . . . . 44

3. LONG-TERM QUANTITATIVE PREDICTIONS . . . . 51

3.1 Autocorrelation functions . . . . 51
3.2 Correlation interval as a measure of predictability . . . . 53
3.3 Principal characteristics for predictions . . . . 60

4. DIALOGUE LANGUAGE GENERALIZATION . . . . 63

4.1 Regular (subjective) system analysis . . . . 64
4.2 Multilevel (objective) analysis . . . . 65
4.3 Multilevel algorithm . . . . 65

3. Noise Immunity and Convergence . . . . 75

1. ANALOGY WITH INFORMATION THEORY . . . . 75

1.1 Basic concepts of information and self-organization theories . . . . 77
1.2 Shannon's second theorem . . . . 79
1.3 Law of conservation of redundancy . . . . 81
1.4 Model complexity versus transmission band . . . . 82

2. CLASSIFICATION AND ANALYSIS OF CRITERIA . . . . 83

2.1 Accuracy criteria . . . . 84
2.2 Consistent criteria . . . . 85
2.3 Combined criteria . . . . 86
2.4 Correlational criteria . . . . 86
2.5 Relationships among the criteria . . . . 87

3. IMPROVEMENT OF NOISE IMMUNITY . . . . 89

3.1 Minimum-bias criterion as a special case . . . . 90
3.2 Single and multicriterion analysis . . . . 93

4. ASYMPTOTIC PROPERTIES OF CRITERIA . . . . 98

4.1 Noise immunity of modeling on a finite sample . . . . 99
4.2 Asymptotic properties of the external criteria . . . . 102
4.3 Calculation of locus of the minima . . . . 105

5. BALANCE CRITERION OF PREDICTIONS . . . . 108

5.1 Noise immunity of the balance criterion . . . . Ill

6. CONVERGENCE OF ALGORITHMS . . . . 118

6.1 Canonical formulation . . . . 118
6.2 Internal convergence . . . . 120

4. Physical Fields and Modeling . . . . 125

1. FINITE-DIFFERENCE PATTERN SCHEMES . . . . 126

1.1 Ecosystem modeling . . . . 128

2. COMPARATIVE STUDIES . . . . 133

2.1 Double sorting . . . . 135
2.2 Example - pollution studies . . . . 137

3. CYCLIC PROCESSES . . . . 143

3.1 Model formulations . . . . 146
3.2 Realization of prediction balance . . . . 151
3.3 Example - Modeling of tea crop productions . . . . 153
3.4 Example - Modeling of maximum applicable frequency (MAP) . . . . 159

5. Clusterization and Recognition . . . . 165

1. SELF-ORGANIZATION MODELING AND CLUSTERING . . . . 165
2. METHODS OF SELF-ORGANIZATION CLUSTERING . . . . 177

2.1 Objective clustering - case of unsupervised learning . . . . 178
2.2 Objective clustering - case of supervised learning . . . . 180
2.3 Unimodality - "criterion-clustering complexity" . . . . 188

3. OBJECTIVE COMPUTER CLUSTERING ALGORITHM . . . . 194
4. LEVELS OF DISCRETIZATION AND BALANCE CRITERION . . . . 202
5. FORECASTING METHODS OF ANALOGUES . . . . 207

5.1 Group analogues for process forecasting . . . . 211
5.2 Group analogues for event forecasting . . . . 217

6. Applications . . . . 223

1. FIELD OF APPLICATION . . . . 225
2. WEATHER MODELING . . . . 227

2.1 Prediction balance with time- and space-averaging . . . . 227
2.2 Finite difference schemes . . . . 230
2.3 Two fundamental inductive algorithms . . . . 233
2.4 Problem of long-range forecasting . . . . 234
2.5 Improving the limit of predictability . . . . 235
2.6 Alternate approaches to weather modeling . . . . 238

3. ECOLOGICAL SYSTEM STUDIES . . . . 247

3.1 Example - ecosystem modeling . . . . 248
3.2 Example - ecosystem modeling using rank correlations . . . . 253

4. MODELING OF ECONOMICAL SYSTEM . . . . 256

4.1 Examples - modeling of British and US economies . . . . 257

5. AGRICULTURAL SYSTEM STUDIES . . . . 270

5.1 Winter wheat modeling using partial summation functions . . . . 272

6. MODELING OF SOLAR ACTIVITY . . . . 279

7. Inductive and Deductive Networks . . . . 285

1. SELF-ORGANIZATION MECHANISM IN THE NETWORKS . . . . 285

1.1 Some concepts, definitions, and tools . . . . 287

2. NETWORK TECHNIQUES . . . . 291

2.1 Inductive technique . . . . 291
2.2 Adaline . . . . 292
2.3 Back Propogation . . . . 293
2.4 Self-organization boolean logic . . . . 295

3. GENERALIZATION . . . . 296

3.1 Bounded with transformations . . . . 297
3.2 Bounded with objective functions . . . . 298

4. COMPARISON AND SIMULATION RESULTS . . . . 300

8. Basic Algorithms and Program Listings . . . . 311

1. COMPUTATIONAL ASPECTS OF MULTILAYERED ALGORITHM . . . . 311

1.1 Program listing . . . . 313
1.2 Sample output . . . . 323

2. COMPUTATIONAL ASPECTS OF COMBINATORIAL ALGORITHM . . . . 326

2.1 Program listing . . . . 327
2.2 Sample outputs . . . . 336

3. COMPUTATIONAL ASPECTS OF HARMONICAL ALGORITHM . . . . 339

3.1 Program listing . . . . 341
3.2 Sample output . . . . 353

Epilogue . . . . 357
Bibliography . . . . 359
Index . . . . 365