WHAT IS A NEURAL NET
 

 

 

 

 

 

 

 

 

 

 

 

 

 

Neural networks consist of basic units that mimic, in a simplified fashion, the behavior of biological neurons found in nature, whether comprising the brain of a human or of a frog. It has been claimed, for example, that there is a unit within the visual system of a frog that fires in response to fly-like movements, and that there is another unit that fires in response to things about the size of a fly. These two units are connected to a neuron that fires when the combined value of these two inputs is high. This neuron is an input into yet another which triggers tongue-flicking behavior. The basic idea is that each neural unit, whether in a frog or a computer, has many inputs that the unit combines into a single output value. In brains, these units may be connected to specialized nerves.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The unit combines its inputs into a single value, which it then transforms to produce the output; these together are called the activation function. The most common acti­ vation functions are based on the biological model where the output remains very low until the combined inputs reach a threshold value. When the com­ bined inputs reach the threshold, the unit is activated and the output is high. Like its biological counterpart, the unit in a neural network has the property that small changes in the inputs, when the combined values are within some middle range, can have relatively large effects on the output. Conversely, large changes in the inputs may have little effect on the output, when the combined inputs are far from the middle range. This property, where sometimes small changes matter and sometimes they do not, is an example of nonlinear behavior. The power and complexity of neural networks arise from their nonlinear behavior, which in turn arises from the particular activation function used by the constituent neural units.