This repository and the associated neural2d.net website are no longer supported and will be going away soon. Neural2d started as a fun weekend project, then gained some momentum from excellent code contributions from a number of programmers. But I’m at a place in my life where I need to turn my attention elsewhere. I would refer users and educators to the many excellent neural net resources that are more pedagogically effective and more technologically current than this project.

The most recent neural2d user manual is shown below for historical reference.

A sincere thank-you to everyone who particpiated in neural2d, and best wishes in your neural-netting.

— Dave

Neural2d - Neural Net Simulator User Manual

Features

• Optimized for 2D input data
• Neuron layers can be abstracted as 1D or 2D arrangements of neurons
• Input data can binary or text
• Network topology is defined in a text file
• Neurons in layers can be fully or sparsely connected
• Selectable transfer function per layer
• Adjustable or automatic training rate (eta)
• Optional momentum (alpha) and regularization (lambda)
• Convolution filtering and convolution networking
• Standalone console program
• Simple, heavily-commented code, suitable for prototyping, learning, and experimentation
• Optional web-browser-based GUI controller
• Graphic visualizations of hidden-layer data
• No dependencies! Just C++11 (and POSIX networking for the optional webserver interface)

Document Contents

Overview
Requirements
Compiling the source
How to run the digits demo
How to run the XOR example
GUI interface
How to use your own data
The 2D in neural2d
Convolution filtering
Convolution networking and pooling
Layer depth
Topology config file format
Topology config file examples
How-do-I X?

• How do I run the command-line program?
• How do I run the GUI interface?
• How do I disable the GUI interface?
• How do I use my own data instead of the digits images?
• How do I use a trained net on new data?
• How do I train on the MNIST handwritten digits data set?
• How do I change the learning rate parameter?
• Are the output neurons binary or floating point?
• How do I use a different transfer function?
• How do I define a convolution filter?
• How do I define convolution networking and pooling?
• How do the color image pixels get converted to floating point for the input layer?
• How can I use .jpg and .png images as inputs to the net?
• How do I find my way through the source code?
• Why does the net error rate stay high? Why doesn't my net learn?
• What other parameters do I need to know about?

Overview<a name="Overview"></a>

Neural2d is a standalone console program with optional HTTP web interface written in C++. It's a backpropagation neural net simulator, with features that make it easy to think of your input data as either one-dimensional or two-dimensional. You specify a network topology in a text file (topology.txt). The input data to the neural is specified in a text file (inputData.txt). The inputData.txt file can contain the actual input values for all the input samples, or it can contain a list of .bmp or .dat files that contain the input data in binary form.

Neural2d is for educational purposes. It's not production-quality code, and it's not burdened with a lot of bling. It's just heavily-commented neural net code that you can take apart and see how it works. Or modify it and experiment with new ideas. Or extract the functions you need and embed them into your own project.

If you're not using the optional GUI interface, then neural2d has no dependencies other than a conforming C++11 compiler. If using the GUI interface, you'll need to link with a standard POSIX sockets networking library.

Requirements<a name="Requirements"></a>

• C++-11 compiler
  • e.g., g++ 4.7 on Linux or Cygwin, Visual Studio 2013 on Windows
• POSIX sockets (only needed if compiling the optional GUI)
  • e.g., Cygwin on Windows
• CMake 2.8.12 or later
• Compiles and runs on Linux, Windows, and probably Mac

Compiling the source<a name="Compiling"></a>

We use CMake to configure the build system. First get the source code from the Gitub repository. If using the command line, the command is:

 git clone https://github.com/davidrmiller/neural2d

That will put the source code tree into a directory named neural2d.

Compiling with CMake graphical interface

If you are using the CMake graphical interface, run it and set the "source" directory to the neural2d top directory, and set the binary output directory to a build directory under that (you must create the build directory), then click Configure and Generate. Uncheck WEBSERVER if you don't want to compile the optional GUI. Here is what it looks like:

Compiling with CMake command line interface

If you are using CMake from the command line, cd to the neural2d top level directory, make a build directory, then run cmake from there:

git clone https://github.com/davidrmiller/neural2d
cd neural2d
mkdir build
cd build
cmake ..
make

There is no "install" step. After the neural2d program is compiled, you can execute it or open the project file from the build directory.

On Windows, by default CMake generates a Microsoft Visual Studio project file in the build directory. On Linux and Cygwin, CMake generates a Makefile that you can use to compile neural2d. You can specify a different CMake generator with the -G option, for example:

 cmake -G "Visual Studio 14 2015" ..

To get a list of available CMake generators:

 cmake --help

If you get errors when compiling the integrated webserver, you can build neural2d without webserver support by running CMake with the -DWEBSERVER=OFF option, like this:

 cmake -DWEBSERVER=OFF ..

How to run the digits demo<a name="Demo"></a>

On systems using Makefiles, in the build directory, execute:

make test

This will do several things: it will compile the neural2d program if necessary; it will expand the archive named image/digits/digits.zip into 5000 individual images; and it will then train the neural net to classify those digit images.

The input data, or "training set," consists of images of numeric digits. The first 50 look like these:

The images are 32x32 pixels each, stored in .bmp format. In this demo, the neural net is configured to have 32x32 input neurons, and 10 output neurons. The net is trained to classify the digits in the images and to indicate the answer by driving the corresponding output neuron to a high level.

Once the net is sufficiently trained, all the connection weights are saved in a file named "weights.txt".

If you are not using Makefiles, you will need to expand the archive in images/digits, then invoke the neural2d program like this:

 neural2d ../images/digits/topology.txt ../images/digits/inputData.txt weights.txt

How to run the XOR example<a name="XorExample"></a>

On systems using Makefiles, in the build directory, execute:

 make test-xor

For more information about the XOR example, see this wiki page.

GUI interface (optional)<a name="GUI"></a>

First, launch the neural2d console program in a command window with the -p option:

 ./neural2d topology.txt inputData.txt weights.txt -p

The -p option causes the neural2d program to wait for a command before starting the training. The screen will look something like this:

At this point, the neural2d console program is paused and waiting for a command to continue. Using any web browser, open:

 http://localhost:24080

A GUI interface will appear that looks like:

Press Resume to start the neural net training. It will automatically pause when the average error rate falls below a certain threshold (or when you press Pause). You now have a trained net. You can press Save Weights to save the weights for later use.

See the neural2d wiki for design notes on the web interface.

Visualizations

At the bottom of the GUI window, a drop-down box shows the visualization options that are available for your network topology, as shown below. There will be options to display the activation (the outputs) of any 2D layer of neurons 3x3 or larger, and convolution kernels of size 3x3 or larger. Visualization images appear at the bottom of the GUI. You can mouse-over the images to zoom in.

How to use your own data<a name="YourOwnData"></a>

Input data to the neural net can be specified in text or binary format. If you want to specify input values in text format, just list the input values in the inputData.txt file. If you're inputting from .bmp or .dat binary files, then list those filenames in inputData.txt. Details are ex