Network Architecture - GMDH


Network Architecture - GMDH

GMDH works by building successive layers with links that are simple polynomial terms.  These polynomial terms are created by using linear and non-linear regression.  The initial layer is simply the input layer.  The first layer created is made by computing regressions of the input variables and then choosing the best ones.  The second layer is created by computing regressions of the values in the first layer along with the input variables.  Again, only the best are chosen by the algorithm.  These are called survivors.  This process continues until the net stops getting better (according to a prespecified selection criterion).


The resulting network can be represented as a complex polynomial (i.e., a familiar formula) description of the model.  You may view the formula, which contains the most significant input variables.  In some respects, it is very much like using regression analysis, but it is far more powerful than regression analysis.  GMDH can build very complex models while avoiding overfitting problems.


GMDH contains several criteria, called selection criteria, to determine when it should stop training.  One of these, called Regularity, is similar to Calibration in that the net uses the constructed architecture that works the best on the test set.


The other selection criteria do not need a test set because the network automatically penalizes models that become too complex in order to prevent overtraining.  The advantage of this is that you can use all available data to train the network.  If you are not using Regularity, don't extract a test set so GMDH will use the entire pattern file.  If you already have extracted a test (.TST) file, erase it and the .TRN file.


A by-product of GMDH is that it recognizes the best variables as it trains, and will display a list of them.


The technique called Group Method of Data Handling (GMDH) was invented by A. G. Ivakhnenko in the former Soviet Union, but enhanced by others, including A. R. Barron.  The software was programmed for Ward Systems Group by NeuroP Ltd.  This technique has also been called "polynomial nets".


For a detailed description of how GMDH works, refer to GMDH Overview.