Kohonen Architecture

Top  Previous  Next

 

The Kohonen Self Organizing Map network used in NeuroShell 2 is a type of unsupervised network, which has the ability to learn without being shown correct outputs in sample patterns.  These networks are able to separate data into a specified number of categories.

 

There are only two layers:  an input layer and an output layer which has one neuron for each possible output category.  You specify the maximum number of output categories when you select Network Architecture Options by entering that number of neurons in the output layer.

 

The training patterns are presented to the input layer, then propagated to the output layer and evaluated.  One output neuron is the "winner".  The network weights are adjusted during training.  This process is repeated for all patterns for a number of epochs chosen in advance.

 

This network is very sensitive to learning rate.  It is lowered slightly but steadily by NeuroShell 2 as the training progresses, causing smaller and smaller weight changes.  This causes the network to stabilize.

 

The network adjusts the weights for the neurons in a neighborhood around the winning neuron.  The neighborhood size is variable, starting off fairly large (sometimes even close to the number of categories) and decreasing with learning until during the last training events the neighborhood is zero, meaning by then only the winning neuron's weights are changed.  By that time the learning rate is very small, and the clusters have been defined.  NeuroShell 2 automatically adjusts learning rate and neighborhood size for you, but you have to specify the initial values as well as the total number of epochs that learning will continue.

 

Slabs

Click on each Slab to select or inspect the number of neurons.  Change the default settings by typing in a new value in the text box.  Because the purpose of a Kohonen Self Organizing map network is to separate outputs into different categories, two or more outputs are required.

 

Connection Arrows (Links)

You can click on the Connection Arrows to set or inspect parameters such  as learning rate, initial weights, neighborhood size, and number of epochs.

 

Kohonen networks are trained for a fixed number of epochs, which you specify in the text box (the number may be changed at learning time).  Small problems may train in as little as 50 epochs, but you can set the number to 10,000 for really large problems.  

 

The learning rate and neighborhood size are automatically reduced as training progresses, honing in on the answer.  The neighborhood size should begin with a relatively high number, such as 90 percent of the number of neurons in the output layer.  The learning rate should begin with a relatively high number such as .5 or .1.

 

The initial weight, e.g., .3,  is actually a range that goes from + .3 to - .3.  The network's initial weight for that link is a random value within that range.

 

How well a Kohonen network classifies data is dependent upon how well you set these parameters.  Learning may not be successful if you don't have a large enough neighborhood or train the network for enough epochs, or by having too small of a learning rate when you first start.