Advanced Neural Network and Genetic Algorithm Software

  
NeuroShell 2

NeuroShell 2 is our legacy neural network product targeted towards computer science instructors and students.  It contains classic algorithms and architectures popular with graduate school professors and computer science students.

 

 

NeuroShell 2 combines powerful neural network architectures, a Microsoft® Windows icon driven user interface, sophisticated utilities, and popular options to give users the ultimate neural network experimental environment.  It is recommended for academic users only, or those users who are concerned with classic neural network paradigms like backpropagation.  Users interested in solving real problems should consider the NeuroShell Predictor, NeuroShell Classifier, or the NeuroShell Trader.

Beginner's Neural Networks

NeuroShell 2 includes both Beginner's and Advanced Systems in addition to Runtime Facilities.

NeuroShell 2 includes interfaces for beginners and experts alike. The Beginner's System in NeuroShell 2 is designed for novices and first-time users. It includes a simplified set of procedures for building and executing a complete, powerful neural network application. The Beginner's System uses our powerful enhanced backpropagation paradigm, and defaults network parameters such as learning rate, momentum, and number of hidden neurons. To use the system you enter data, specify the inputs and outputs, and train the network. You are then able to apply the trained network to new data and export the results to other programs.

Advanced Neural Networks

The NeuroShell 2 Advanced Options screen displays the independent modules that may be used to create a neural network application.

 

The Advanced System in NeuroShell 2 gives experienced neural network users the ability to create and execute 16 different neural network architectures, with more user control compared to the Beginner's System.

Spreadsheet Format

Users can choose to enter their data in their familiar spreadsheet program or work in the NeuroShell 2 Datagrid. The program uses spreadsheet files as its internal format so you can view or edit them yourself. In fact, NeuroShell 2 will call your favorite Windows spreadsheet program.

File Import and Export

NeuroShell 2 imports ASCII, binary, and spreadsheet files.

The program imports and exports ASCII and binary file formats.  The File Export Module allows you to convert NeuroShell 2 files to ASCII or binary files. It will also merge data into an existing spreadsheet or print a file.

Data Preprocessing

The Rules Module allows you to create If/Then/Else rules to pre-process data.

The Symbol Translate module converts alphanumeric data or strings into numbers which can be processed by the network. For example, you may want to convert "cold", "warm", and "hot" to 1, 2, and 3 respectively.

The Rules Module allows you to create If/Then/Else type rules to preprocess data prior to feeding it to the network. For example, you can use two input variables to create a third. You may believe that if you combine the price of gold and IBM stock and the total is greater than 400, the economy is improving. You can translate this information into a new network input as follows:

If gold + IBM > 400,
then economy (the new variable) = 1,
else economy =0.

You may also use the Rules Module to post-process the network's predictions and classifications.

Test Set Extract

NeuroShell 2 gives you great flexibility in choosing which columns of your spreadsheet are inputs and which are outputs.

 

NeuroShell 2 makes it easy to pull out test and production data sets from the training data. The Test Set Extract Module offers four different methods for selecting data:

  • N percent randomly chosen.

  • Every Nth pattern.

  • Block designation.

  • By row marker in the data.

  • Combination block designation and random.


Neural Network Architectures

The Design Module offers a palette of 16 different network architectures/paradigms for different types of data. (Users may customize each network architecture by setting parameters.)

 

The NeuroShell 2 Design Module offers 16 "icon selectable" architectures, each with several learning methods. You only need to point and click on "slabs" of neurons or weight links to set parameters. Slabs are subsets of layers.


Each network architecture allows you to click on slabs of neurons and links to set parameters.

 


User Control

Users can specify their own learning rate, momentum, activation functions, and initial weight ranges on a layer basis in the Design Module. Elect rotational or random pattern selection. Choose multiple criteria for stopping training. Select different methods for handling missing data. Users can view weight values during training, including weights displayed in modified Hinton diagrams. Weights may be modified during learning.


Backpropagation Training

Twelve of the architectures include the Ward Systems Group version of backpropagation, which has been enhanced for speed and accuracy.

Standard Connections

This is the standard type of backpropagation network in which every layer is connected or linked only to the previous layer.

Jump Connections

This is the type of backpropagation network in which every layer is connected or linked to every previous layer.

Recurrent Networks

This type of backpropagation network is often used in predicting financial markets because recurrent networks can learn sequences. Therefore, they are excellent for time series data.

A regular feed forward network responds to a given input pattern with exactly the same output every time the given input pattern is presented. A recurrent network may respond to the same input pattern differently at different times, depending upon the input patterns which have been presented previously. Recurrent networks build a long-term memory of internal neurons.

Ward Nets

This type of backpropagation network is able to detect different features in the data with the use of multiple slabs of neurons in the hidden layer, each with a different activation function. Activation functions are functions used internally in neurons to "fire" the neurons. When you apply a different activation function to each slab in the hidden layer, the network can discover novel features in a single pattern processed through the network.


All backpropagation algorithms come with a choice of Vanilla, Momentum, or TurboProp weight updates. With Vanilla, a learning rate is applied to the weight updates but a momentum term is not. Momentum means that the weight updates not only include the change dictated by learning rate, but include a portion of the last weight change as well.

All of the backpropagation networks include the Calibration feature, which prevents overtraining (thereby greatly reducing training time), and increases the network's ability to generalize well on new data.


TurboProp

This is a training method for feed forward networks that operates much faster than backpropagation networks. TurboProp offers users the additional advantage of not requiring learning rate and momentum to be set.


Kohonen Architecture

The Kohonen Self Organizing Map network used in NeuroShell 2 is a type of unsupervised network, which means it has the ability to learn without being shown correct outputs in sample patterns. The networks are able to separate data into a specified number of categories.


PNN Architecture

Probabilistic Neural Networks (PNN) are known for their ability to train on sparse data sets and they train in only one pass of the training set! PNN separates data into a specified number of output categories. PNN networks are often able to function as soon as two training patterns are available, so training can be incremental.


GRNN Architecture

Genetic Adaptive nets optimize architectures and select inputs.

 

Like PNN networks, General Regression Neural Networks (GRNN) are known for their ability to train in only one pass of the training set using sparse data sets. Rather than categorizing data like PNN, however, GRNN applications are able to produce continuous valued outputs.

GRNN is especially useful for continuous function approximation, and can fit multidimensional surfaces through data. In our tests we found that GRNN responds much better than Backpropagation to many types of problems. Note: GRNN is not the same as regression analysis!


Genetic Adaptive Nets

Our Genetic Adaptive feature uses a Genetic Algorithm to optimize the network structure of our GRNN and PNN nets. At the same time, the genetic algorithm eliminates bad inputs and gives you a sensitivity factor for the ones it keeps. This feature makes our Genetic Adaptive nets among the most powerful nets available today.


GMDH Architecture

The GMDH Learning window.

 


The GMDH Advanced Setup window. This is for more mathematically sophisticated users.

 

NeuroShell 2 includes a very powerful architecture which is called Group Method of Data Handling (GMDH) or polynomial nets. GMDH nets derive a mathematical formula which is a nonlinear polynomial expression relating the values of the most important inputs to predict the output variable. In fact, the GMDH network is not like regular feedforward networks and was not originally represented as a network. The GMDH network is implemented with polynomial terms in the links and a genetic-like component to decide how many layers are built. The result of training at the output layer can be represented as a polynomial function of all or some of the inputs.

GMDH works by building successive layers with complex links (or connections) that are the individual terms of a polynomial. These polynomial terms are created using linear and non-linear regression. The initial layer is simply the input layer. The first layer created is made by computing regressions of the input variables and then choosing the best ones. The second layer is created by computing regressions of the values in the first layer along with the input variables. (Note that we are essentially building polynomials of polynomials.)

Again, only the best are chosen by the algorithm. These are called survivors. This process continues until the net stops improving (according to a pre-specified selection criterion).

The resulting network can be represented as a complex polynomial (i.e., a familiar formula) description of the model just like the ones statisticians use. You may view the formula, which contains the most significant input variables. In some respects, it is very much like using regression analysis, but it is far more powerful than regression analysis. GMDH can build very complex models while avoiding overfitting problems.


Calibration

Calibration solves one of the most difficult problems for neural networks -- knowing when to stop training. The longer the network learns the training set, the closer the network gets to "memorizing" the training set. If you present the network with a pattern that was in the training set, the network will be able to make a very accurate prediction. If, however, you present the network with a pattern that wasn't in the training set, the network may not be able to "generalize" well on data it hasn't "seen" before if you have let the problem learn too long. Calibration corrects this.

For backpropagation networks, Calibration saves the network at the point where it gives the most accurate answers for patterns outside the training set.

For GRNN and PNN networks, Calibration finds the optimum smoothing factor, a parameter that is used when you apply the network. The Genetic Adaptive nets calibrate with the genetic algorithm.


Additional Features

Although not nearly as complete and powerful as the NeuroShell Trader, NeuroShell 2 has standard features and options that make it easier to import and preprocess market and futures data.

ASCII and Spreadsheet File Convert

ASCII and spreadsheet file importing will allow you to import data from some of the popular price retrieval services.


Recurrent Networks and Ward Nets

NeuroShell 2 includes three types of fully recurrent networks, which are excellent for time series, stock market, futures, and commodities prediction. These architectures have long-term memories with adjustable dampening factors. Ward Nets capture different views of the same data which result in more accurate predictions.


Market Indicator Package (See Included Features Page)

The Market Indicator Package provides 150 popular and powerful technical indicators for financial or other time series data.

 

The Market Indicator package is a NeuroShell 2 option which provides 150 popular and powerful technical indictors you can apply to your financial or other time series data. With this incredible option you eliminate much of the need for purchasing other trading software. You can apply indicators to other indicators to the depth you desire! When you add the Optimizer to the Market Indicator Package, you can automatically find the most appropriate indicators to apply to your raw data or other indicators. It will also find the most appropriate parameters for each indicator, e.g., how many days to use in a moving average.  These, together with our Graphics and Rules modules, provide you with features of powerful trading systems built into your neural network system.

 


Graphics Aid Data Analysis

Graphics facilities enable you to look for patterns in your data.

NeuroShell® 2 includes extensive graphics utilities such as line charts, bar charts, scatter plots and high- low-close graphs. Options allow you to add 3-D effects and change colors or types of graphs.

The graphs may be printed on any Windows compatible printer, copied to the clipboard, or saved in a file. Before and after network processing, you may use the Variable Graphs module to create four different types of graphs:


Graph Variable(s) Across All Patterns

This graph type lets you graph different types of variables, such as advertising expenditures and cost of goods sold, across all patterns in a file. This graph is also useful for graphing time series data.


Graph Variable Sets in a Pattern

Use this graph to examine your data if all of the variables in a pattern are of the same type, e.g., 100 points in a physiological signal such as an electrocardiogram.


Correlation Scatter Plot

Use this graph type to make a scatter plot of one variable against another through all patterns. At the same time the linear correlation coefficient is computed.


High-Low-Close Graph

This graph allows you to select variables from your data file that are displayed as the high, low, and close values of a stock price. The graph manifests trends in your data.


Training Graphs

NeuroShell 2 allows you to display graphs of training set/test set errors while training backpropagation networks. When training Kohonen networks, you can display a graph of category distributions in either a bar or pie chart. The Probabilistic Neural Networks (PNN) and General Regression Neural Networks (GRNN) Learning Modules allow you to display a smoothing factor optimization graph. GMDH nets graph the criterion value for the created formula against the layer number.

 


Built-in Capabilities

Statistics can be viewed when a trained network is applied.

 

NeuroShell 2 lets you analyze how well your trained network is doing in a variety of ways.

For Backpropagation networks, the Contribution Factor Module produces a number for each input variable. This number is a rough measure of the importance of that variable in predicting the network’s output, relative to other input variables in the same network. A similar sensitivity factor is available with our Genetic Adaptive Networks.

When you apply a trained Backpropagation network, you can choose to compute several different statistics including R Squared, the statistical coefficient of multiple determination. You can also compute the mean squared error, the mean absolute error, and the minimum and maximum absolute error of the actual answers compared to the network’s predicted answers. Also displayed for Backpropagation networks is the correlation coefficient (r), a statistical measure of the strength of the relationship between the actual vs. predicted outputs.

The computed statistics for General Regression Neural Networks are the same as those for Backpropagation networks.

For Probabilistic Neural networks, which categorize data, the computed statistics are related to the numbers of correctly classified patterns, including the numbers of true positives, false positives, true negatives, and false negatives.

The Group Method of Data Handling (GMDH) gives you a non-linear formula for your model. Since it uses regression analysis to build the models, the formula is a familiar one to statisticians. At the same time, GMDH removes inputs that are not relevant to the model.

 


Use Network Outside of NeuroShell

NeuroShell 2's Runtime System allows you to call a DLL to fire your net or generate source code.

The Runtime System, included in the package, allows you to convert your neural network into a form callable by a Dynamic Link Library (DLL) that may be called from other programs or Microsoft Excel. The Runtime Module will also generate Visual Basic or C source code for your trained net that may be called from non-Windows programs. NeuroShell 2 includes royalty free, unlimited distribution rights for any networks that you create using NeuroShell 2.


The Dynamic Link Library (DLL) Primer

Once a network has been trained, you can process files of input data in NeuroShell 2. However, you can also build interactive programs or Microsoft Excel spreadsheets with your network in them. The NeuroShell 2 DLL Primer can save your network so that it can be reloaded later by a NeuroShell 2 DLL.

For example, if you later want to execute this network from a program you write in C, Pascal, Microsoft Visual Basic, Visual C++TM, Access Basic, Delphi, etc., you would use the DLL function FireNet to execute it. Your program passes network inputs to FireNet, and FireNet gives back network outputs. In Excel, you place the function Predict into a cell, passing it network inputs. A network output will be placed into that cell for you.


Source Code Generator

No matter which network architecture you choose, the Source Code Generator makes it possible to port trained networks from NeuroShell 2 to non-Windows platforms by generating C source code (also Basic and plain formulas). If you're working with Windows platforms, however, the DLL module is more efficient to use.

 


Specifications

Icon-Driven User Interface

  • Beginner's System for novice and first-time users.
  • Advanced System with more power and flexibility.
  • Runtime System included, allows you to convert your neural network into a form callable by a Dynamic Link Library (DLL) that may be called from other programs or Microsoft Excel. You can also generate source code in C, Visual Basic, or formulas.

Network Architectures

  • WSG enhanced backpropagation with standard connections.
  • WSG enhanced backpropagation with jump connections.
  • WSG enhanced backpropagation recurrent networks with feedback from input, hidden, or output layer can learn sequences in patterns.
  • WSG enhanced backpropagation Ward Nets use multiple slabs of neurons in the hidden layer with different activation functions to extract different features from the same data pattern, with the end result being more precise predictions.
  • Kohonen Self Organizing Map Network can learn to categorize data without being shown correct outputs in sample patterns.
  • Probabilistic Neural Networks and General Regression Neural Networks are our most powerful networks which, for many problems, are far superior to backpropagation.
  • Group Method of Data Handling (Polynomial Nets) compute a mathematical formula which is a non-linear polynomial expression relating the values of the most important inputs to predict the output variable.

Genetic Adaptive Networks

PNN and GRNN nets that utilize a genetic algorithm to find the best inputs and network architecture.

TurboPropTM 1 (Not to be confused with TurboPropTM 2 in our newer products)

Enables users to create a network without setting learning rate and momentum. It is much faster than backpropagation while still generalizing very well.

Calibration

Prevents overtraining (overfitting) and increases generalization.

User Control

Every layer (or even sublayers called slabs) can have their own learning rate, momentum, activation functions, and initial weight ranges. Pattern selection may be rotational or random. Select multiple criteria for stopping network training. Select different methods for handling missing data. Weights may be modified during training. Text description file attached to each net.

Limits

NeuroShell 2 has theoretical limits of 65,535 rows, 32,768 columns, and 16,000 neurons. However, neural nets will not train effectively with much over 2,000 inputs, and some network types lose effectiveness at about 400 inputs and 5,000 to 10,000 rows (patterns), and may not function at all with higher limits. Actually 65,535 rows are possible, but many older spreadsheet programs will display only 16,000 rows. Our own Datagrid displays 32,000.

Input Sensitivity

NeuroShell 2 has three different ways of determining sensitivity of inputs depending on network type.

Rules Module

The Rules Module lets you apply If/Then/Else rules to your pattern files.

Symbol Translate Module

The Symbol Translate Module will translate strings to numerics and vice versa.

On-Line Tutorial

NeuroShell 2's on-line tutorial directs you through the steps required to use NeuroShell 2 and create a working application.

Two tutorial example programs, including one which uses stock market data, are included in the on-line manual and on the distribution diskettes. Sample programs are included for backpropagation, Kohonen, PNN, and GRNN networks. Several other examples are also provided.

Graphics Capabilities

  • Graphs variables across all patterns.
  • Graphs variable sets in a pattern.
  • Correlation scatter plot displays a linear correlation coefficient between the two graphed variables.
  • High-low-close graph manifests trends in market data.
  • Training graphics included for each learning paradigm.

File Import/Export

Import and export files in the following formats: ASCII, binary, and spreadsheet.

 


Included Features

Market Indicator Package

The Market Indicator Package provides 150 popular and powerful technical indicators for financial or other time series data.

 

This feature is also called the Time Series Indicator Package. The Market Indicator Package allows you to build a complex set of indicators from raw stock market or futures data such as high, low, close, and volume. When doing financial or other time series predictions, it is usually necessary to enhance the time series data by showing the network specific features from the past. This enhancement makes it easier for the net to determine trends. The Indicator Package has 150 technical indicators that market analysts have been using for years for forecasting. The Indicator Package calculates these indicators from your raw data and puts them into your spreadsheet so they can be used as additional inputs to the network.


Market Indicator Package with Optimizer

The Optimizer can automatically select the best indicators and corresponding parameters for the indictors.

 

The Optimizer included with Market Indicator Package allows you to select the best indicators and corresponding parameters for the indicators. For example, suppose you choose a simple indicator like a lagged moving average. How long should you lag it and how long should you average it? The Optimizer will tell you these things based upon your data. It will also run through all indicators and tell you the best ones to use for your data.


Batch Processor Option

The Batch Processor Option automates running multiple problems or variations of a single problem.

 

NeuroShell 2 has an intuitive graphical user interface that lets you configure your nets and run them interactively. But you may have a need to set up dozens, or even hundreds, of nets and run them one after another automatically without being near the computer (for example, overnight). The Batch Processor does this. Basically, you load each problem into a row of a matrix. The first row is a base problem, and the subsequent rows are variations of the base problem (usually). The columns contain the training sets, the configuration information, and the data sets the nets will be applied to after training. There are multiple tabs, so you can bring up multiple matrices, each with its own base problem.

You may also use the Batch Processor to apply previously trained nets to different data files.

When all runs are completed, a log file will show which nets give the best results. You get a statistical summary of each net and the name(s) of the apply files.


Race Handicapping Option

The Race Handicapping Option is useful for predicting race outcomes and solving other ranking problems.

 

This Package is also called the Data Ranking Package. Neural networks are often used successfully to predict the outcome of a horse race or dog race or for other ranking problems. Input data may include variables about each of the field of horses with the network ranking the horses in terms of probable finish places. Neural nets, however, do much better at ranking two horses at a time rather than ranking as many as eight or more at a time. So the handicapping package “explodes” the training patterns into all permutations of eight horses taken two at a time. Then the net uses that file to learn to rank two horses. The pairs’ rankings are then automatically analyzed to give you the ranking for the whole field of eight.


Graphics Add-On

The Graphics Add-On creates a three- dimensional plot of a trained network's output.

 

The Graphics Add-On creates a three-dimensional plot of a trained network’s output value as it is calculated by varying two selected input variables. The inputs are displayed as the X and Y dimensions; the output is displayed as dimension Z.

The graphs are useful in determining how two input variables influence the net’s output. The program selects the most significant inputs as the default values for the X and Y dimensions.

For backpropagation networks, the most significant input variables are determined by the Contribution Factors Module. For Genetic Adaptive General Regression nets, the genetic algorithm determines the most significant inputs. You also have the package of selecting which inputs to graph.

The program includes a Graphics Wizard which allows you to change picture properties.

Ward Systems Group, Inc.
Executive Park West
5 Hillcrest Drive
Frederick, MD 21703

Phone: (301) 662-7950
Fax: (301) 663-9920
Email: sales@wardsystems.com
Skype (Sales Only): wardsystems
Copyright © 1997-2007 Ward Systems Group, Inc. All rights reserved.
Copyright Information
| Privacy Statement