What are neural nets? bottom of page The learning process

jump directly to

Types of neural nets
Perceptron
Multi-Layer-Perceptron
Backpropagation Net
Hopfield Net
Kohonen Feature Map



Types of Neural Nets

As mentioned before, several types of neural nets exist.
They can be distinguished by
their type (feedforward or feedback),
their structure
and the learning algorithm they use.
The type of a neural net indicates, if the neurons of one of the net's layers may be connected among each other. Feedforward neural nets allow only neuron connections between two different layers, while nets of the feedback type have also connections between neurons of the same layer.

In this section, a selection of neural nets will be described.


Perceptron

The Perceptron was first introduced by F. Rosenblatt in 1958.
It is a very simple neural net type with two neuron layers that accepts only binary input and output values (0 or 1). The learning process is supervised and the net is able to solve basic logical operations like AND or OR. It is also used for pattern classification purposes.
More complicated logical operations (like the XOR problem) cannot be solved by a Perceptron.

Perceptron characteristics
sample structure sample structure of a Perceptron
type feedforward
neuron layers 1 input layer
1 output layer
input value types binary
activation function hard limiter
learning method supervised
learning algorithm Hebb learning rule
mainly used in simple logical operations
pattern classification
top of page


Multi-Layer-Perceptron

The Multi-Layer-Perceptron was first introduced by M. Minsky and S. Papert in 1969.
It is an extended Perceptron and has one ore more hidden neuron layers between its input and output layers.
Due to its extended structure, a Multi-Layer-Perceptron is able to solve every logical operation, including the XOR problem.

Multi-Layer-Perceptron characteristics
sample structure sample structure of a Multi-Layer-Perceptron
type feedforward
neuron layers 1 input layer
1 or more hidden layers
1 output layer
input value types binary
activation function hard limiter / sigmoid
learning method supervised
learning algorithm delta learning rule
backpropagation (mostly used)
mainly used in complex logical operations
pattern classification
top of page


Backpropagation Net

The Backpropagation Net was first introduced by G.E. Hinton, E. Rumelhart and R.J. Williams in 1986
and is one of the most powerful neural net types.
It has the same structure as the Multi-Layer-Perceptron and uses the backpropagation learning algorithm.

Backpropagation Net characteristics
sample structure sample structure of a Backpropagation Net
type feedforward
neuron layers 1 input layer
1 or more hidden layers
1 output layer
input value types binary
activation function sigmoid
learning method supervised
learning algorithm backpropagation
mainly used in complex logical operations
pattern classification
speech analysis
top of page


Hopfield Net

The Hopfield Net was first introduced by physicist J.J. Hopfield in 1982 and belongs to neural net types which are called "thermodynamical models".
It consists of a set of neurons, where each neuron is connected to each other neuron. There is no differentiation between input and output neurons.
The main application of a Hopfield Net is the storage and recognition of patterns, e.g. image files.

Hopfield Net characteristics
sample structure sample structure of a Hopfield Net
type feedback
neuron layers 1 matrix
input value types binary
activation function signum / hard limiter
learning method unsupervised
learning algorithm delta learning rule
simulated annealing (mostly used)
mainly used in pattern association
optimization problems
top of page


Kohonen Feature Map

The Kohonen Feature Map was first introduced by finnish professor Teuvo Kohonen (University of Helsinki) in 1982.
It is probably the most useful neural net type, if the learning process of the human brain shall be simulated. The "heart" of this type is the feature map, a neuron layer where neurons are organizing themselves according to certain input values.
The type of this neural net is both feedforward (input layer to feature map) and feedback (feature map).
(A Kohonen Feature Map is used in the sample applet)

Kohonen Feature Map characteristics
sample structure sample structure of a Kohonen Feature Map
type feedforward / feedback
neuron layers 1 input layer
1 map layer
input value types binary, real
activation function sigmoid
learning method unsupervised
learning algorithm selforganization
mainly used in pattern classification
optimization problems
simulation



What are neural nets? top of page The learning process


navigation
[main page] [content] [neural net overview]
[class structure] [using the classes] [sample applet]
[glossary] [literature] [about the author] [what do you think ?]
Copyright 1996-97 Jochen Fröhlich. All rights reserved. A new version is available.