Fusion of Neural Networks, Fuzzy Systems and Genetic Algorithms: Industrial Applications Fusion of Neural Networks, Fuzzy Systems and Genetic Algorithms: Industrial Applications
by Lakhmi C. Jain; N.M. Martin
CRC Press, CRC Press LLC
ISBN: 0849398045   Pub Date: 11/01/98
  

Previous Table of Contents Next


Neural Net Learning

The equivalent error, dkout, at the output layer is

where ok is the output of the output neuron k, tk is the desired output of the output neuron k, and f ′(netk) is the derivative which is unity for layers 2 and 3 neurons as mentioned above.

The weight modification equation for the weights between the hidden layer and output layer, Wjk, is

where ε is the learning rate, Wjk is the weight between hidden layer neuron j and output layer neuron k, and oj is the output from the hidden layer neuron j.

The general equation for the equivalent error in the hidden layer is

However, for the fuzzification layer in Figure 4, the equivalent error is different, as for the middle layer, the netpj is

where oi is the output of the input layer neuron i.

Thus, for the input layer (Figure 3), the equivalent error expression becomes

where both i and k are indices in the input layer and j is the index in the hidden layer; the summation is over j and product is over k.

5.1.3 Fuzzy Logic Processing

The generated fuzzy rules and membership functions are processed using the neural net based nonheuristic fuzzy logic algorithm as described below.

Antecedent Processing

We are using multiplication of the membership functions, as opposed to minimum of the membership functions, in processing the antecedents. Thus the equation to combine two antecedents is

where υc is the membership function of the combinations of the membership functions υa and υb. Use of multiplication as dictated by the neural net significantly improves the results [9].

Rule Format

The rule format is

IF Input 1 is Low AND Input 2 is Low THEN the output is X

where X is a number. Thus, instead of using an output membership function, we are using singletons.

Defuzzification / Rule Evaluation

Consider the following equation for the defuzzification:

where Output is the final defuzzified output which includes the contribution from all the rules as represented by layer 2 and y represents the output of layer 2 neurons.

Clearly, we get a defuzzification which exactly matches the neural net behavior. This defuzzification is also simpler, as it does not use the division operation used in the conventional COG (Center of Gravity) defuzzification algorithm. Another point to note is that the proposed defuzzification is actually a rule evaluation. Since the output of the rule is singleton, we actually do not need a defuzzification.

5.2 Recurrent Neural Fuzzy System (RNFS)

Figure 5 shows a recurrent neural network based neural fuzzy system. This is based on the NeuFuz architecture (Figure 3) by adding recurrent connections in the hidden layer (i.e., rule base).


Figure 5a  Recurrent neural net using the architecture of NeuFuz shown in Figure 3. Recurrent connections are added in the hidden layers. D represents a unit delay.

5.2.1 Recurrent Neural Net

The recurrent connections in the hidden layer (Figure 5) use multiple delays. These recurrent connections are used both during learning and recall phases. In calculating the output of a neuron in layer 2, outputs of layer 1 neurons connected to layer 2 neurons are multiplied with the feedback signals of layer 2 neuron with appropriate delays. Recurrency can also be added to the input neurons to exploit more information about context.


Figure 5b  Hidden layer of Figure 5a is redrawn showing all possible recurrent path delays and weights. Only one neuron is shown for simplicity. D represents a unit delay. Weight WRj,n represents feedback weight of j-th neuron in layer 2 with n delays. The recurrent weights are modified during the learning process using back propagation as described in Section 5.2.1. Recurrent weights are also used in the recall mode calculations.

Using similar analysis as we did for NeuFuz in Section 5.1.2, we derive the following equations:

The netpjhidden(t), at the hidden layer, is

where netpjhidden(t - m) = netpjhidden with delay m with respect to current time “t,”
netpjhidden(t) = layer 2 (hidden layer) output at time t, and
Oi = output of the i-th neuron in layer 1.

We have used linear neurons with a slope of unity for the hidden (layer 2) and output (layer 3) layer neurons. Thus, the equivalent error at the output layer is

where ok is the output of the output neuron k

The general equation for the equivalent error at the hidden layer neurons using Back Propagation model is

However, for the fuzzification layer in Figure 5a, the equivalent error is different, as for the hidden layer we have product (instead of sum) neuron.

Thus, for the input layer in Figure 5a, the equivalent error expression (after incorporating the effect of the recurrent paths) becomes ([11, 12, 13])

where both i and k are indices in the input layer and j and m are the indices in the hidden layer. Also, the value of k is different from the value of i.

The weight update equation for any layer (except recurrent path in layer 2) is

For the feedback connections (layer 2), oi is denoted by oirecurrent and is the netpjhidden(t-m) multiplied by WRj,m (which corresponds to the weight in the path with “m” delays). We add a bias term of unity to oirecurrent to help convergence.

Thus, oirecurrent is given by

and dj is the equivalent error in the hidden layer.


Previous Table of Contents Next

Copyright © CRC Press LLC