![]() |
Fusion of Neural Networks, Fuzzy Systems and Genetic Algorithms: Industrial Applications
by Lakhmi C. Jain; N.M. Martin CRC Press, CRC Press LLC ISBN: 0849398045 Pub Date: 11/01/98 |
Previous | Table of Contents | Next |
Artificial neural networks are being applied to a wide variety of automation problems including adaptive control, optimization, medical diagnosis, decision making, as well as information and signal processing, including speech processing. ANNs have proven to be very suitable for processing sensor data, in particular, feature extraction and automated recognition of signals and multi-dimensional objects. Pattern recognition has, however, emerged as a major application because the network structure is suited to tasks that biological systems perform well, and pattern recognition is a good example where biological systems out-perform traditional rule-based artificial intelligence approaches.
The name artificial neural network given to the study of these mathematical processes is, in a sense, unfortunate in that it creates a false impression which leads to the formation of unwarranted expectations. Despite some efforts to change to a less spectacular name such as connectionist systems, it seems that the title Artificial Neural Networks is destined to remain. At this time the performance of the best ANN is trivial when compared with even the simplest biological system.
The first significant paper on artificial neural networks is generally considered to be that of McCullock and Pitts [2] in 1943. This paper outlined some concepts concerning how biological neurons could be expected to operate. The neuron models proposed were modeled by simple arrangements of hardware that attempted to mimic the performance of the single neural cell. In 1949 Hebb [3] formed the basis of Hebbian learning which is now regarded as an important part of ANN theory. The basic concept underlying Hebbian learning is the principle that every time a neural connection is used, the pathway is strengthened. About this time of neural network development, the digital computer became more widely available and its availability proved to be of great practical value in the further investigation of ANN performance. In 1958 Neumann proposed modeling the brain performance using items of computer hardware available at that time. Rosenblatt [4] constructed neuron models in hardware during 1957. These models ultimately resulted in the concept of the Perceptron. This was an important development and the underlying concept is still in wide use today. Widrow and Hoff [5] were responsible for simplified artificial neuron development. First the ADALINE and then the MADALINE networks. The name ADALINE comes from ADAptive LInear NEuron, and the name MADALINE comes from Multiple ADALINE.
In 1969 Minsky and Pappert published [6] an influential book Perceptrons which showed that the Perceptron developed by Rosenblatt had serious limitations. He further contended that the Perceptron, at the time, suffered from severe limitations. The essence of the book Perceptrons was the assumption that the inability of the perception to be able to handle the exclusive or function was a common feature shared by all neural networks. As a result of this assumption, interest in neural networks greatly reduced. The overall effect of the book was to reduce the amount of research work on neural networks for the next 10 years. The book served to dampen the unrealistically high expectations previously held for ANNs. Despite the reduction in ANN research funding, a number of people still persisted in ANN research work.
John Hopfield [7] produced a paper in 1982 that showed that the ANN had potential for successful operation, and proposed how it could be developed. This paper was timely as it marked a second beginning for the ANN. While Hopfield is the name frequently associated with the resurgence of interest in ANN it probably represented the culmination of the work of many people in the field. From this time onward the field of neural computing began to expand and now there is world-wide enthusiasm as well as a growing number of important practical applications.
Today there are two classes of ANN paradigm, supervised and unsupervised. The multilayer back-propagation network (MLBPN) is the most popular example of a supervised network. It results from work carried out in the mid-eighties largely by David Rumelhart [8] and David Parker [9]. It is a very powerful technique for constructing nonlinear transfer functions between several continuous valued inputs and one or more continuously valued outputs. The network basically uses a multilayer perceptron architecture and gets its name from the manner in which it processes errors during training.
Adaptive Resonance Theory (ART) is an example of an unsupervised or self-organizing network and was proposed by Carpenter and Grossberg [10]. Its architecture is highly adaptive and evolved from the simpler adaptive pattern recognition networks known as the competitive learning models. Kohonens Learning vector quantiser [11] is another popular unsupervised neural network that learns to form activity bubbles through the actions of competition and cooperation when the feature vectors are presented to the network. A feature of biological neurons, such as those in the central nervous system, is their rich interconnections and abundance of recurrent signal paths. The collective behavior of such networks is highly dependent upon the activity of each individual component. This is in contrast to feed forward networks where each neuron essentially operates independent of other neurons in the network.
The underlying reason for using an artificial neural network in preference to other likely methods of solution is that there is an expectation that it will be able to provide a rapid solution to a non-trivial problem. Depending on the type of problem being considered, there are often satisfactory alternative proven methods capable of providing a fast assessment of the situation.
Artificial Neural Networks are not universal panaceas to all problems. They are really just an alternative mathematical device for rapidly processing information and data. It can be argued that animal and human intelligence is only a huge extension of this process. Biological systems learn and then interpolate and extrapolate using slowly propagated (100 m/s) information when compared to the propagation speed (3 108 m/s) of a signal in an electronic system. Despite this low signal propagation speed the brain is able to perform splendid feats of computation in everyday tasks. The reason for this enigmatic feat is the degree of parallelism that exists within the biological brain.
Previous | Table of Contents | Next |