HISTORY OF THE NEURAL NETWORK
 

 

 

 

 

 

 

 

 

 

 

 

 

 

Neural networks have an interesting history in the annals of computer science. The original work on the functioning of neurons—biological neurons—took place in the 1930s and 1940s, before digital computers really even existed. In 1943, Warren McCulloch, a neurophysiologist at Yale University, and Walter Pitts, a logician, postulated a simple model to explain how biological neurons work and published it in a paper called “A Logical Calculus Immanent in Nervous Activity.” While their focus was on understanding the anatomy of the brain, it turned out that this model provided inspiration for the field of artifi­ cial intelligence and would eventually provide a new approach to solving cer­ tain problems outside the realm of neurobiology. In the 1950s, when digital computers first became available, computer scientists implemented models called perceptrons based on the work of McCulloch and Pitts.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

An example of a problem solved by these early networks was how to balance a broom standing upright on a moving cart by controlling the motions of the cart back and forth. As the broom starts falling to the left, the cart learns to move to the left to keep it upright. Although there were some limited successes with perceptrons in the laboratory, the results were disap­ pointing as a general method for solving problems. One reason for the limited usefulness of early neural networks is that most powerful computers of that era were less powerful than inexpensive desktop computers today. Another reason was that these simple networks had theoreti­ cal deficiencies, as shown by Seymour Papert and Marvin Minsky (two profes­ sors at the Massachusetts Institute of Technology) in 1968. Because of these deficiencies, the study of neural network implementations on computers slowed down drastically during the 1970s. Then, in 1982, John Hopfield of the California Institute of Technology invented back propagation, a way of training neural networks that sidestepped the theoretical pitfalls of earlier approaches.