0 Speeding up Backpropagation Algorithms
Table 2. Results of the best runs for the Geneeproblem. CENNBPROP yields both the lowest training and the lowest testing error. Training set Testing set Algorithm SSE Class. rate SSE Class. achieved the lowest SSE although RPROP normally outperforms BPROP. F or the Thyroiddproblem CENNBPROP proved to be comparable with standard RPROP. CENNRPROP always yields the best results SSE=0.0 and beats the standard RPROP which still left some residual errors. In the beginning of training, CENNBPROP has worked best for both benchmark problems. This can be explained by the fact that RPROP needs some time to adapt its step widths. The CENNoptimization improves BPROP training with respect to convergence speed and the nal quality of the trained networks. Especially, t h i s is exploited in the case of the Thyroiddproblem. Since CENNBPROP updates the weights after each pattern presentation online training, it should be well suited for applications with feedback, e.g. neural controllers. In summary, CENNoptimization has proven to outperform all the investigated BP algorithms and we guess that it is the fastest training algorithm at the time being. Of course, this claim needs to be veriied by using much more benchmarks and comparisons to other training algorithms. A direct adaptive method for faster backpropagation learning: The rprop algorithm. Table 1. Results of the best runs for the Thyroiddproblem. CENNRPROP yields both the lowest training error and the lowest testing error. training set testing set Algorithmus SSE Class. rate SSE Class. rate BPROP 246.0 For each problem 10 networks were created which are then used for the four different training algorithms. Before training, the networks were initialized by random numbers drawn from a normal distribution with zero mean and a standard deviation of 0.01. Thus, all algorithms had equal initial conditions. BPROP and CENNBPROP used a learning rate of 0.01 and a momentum of 0.9. RPROP and CENNRPROP used an initial step width of 0.001, uppdown factors of 0.551.2 and a minimallmaximal step width of 10 ,6 50.0, repectively. The results are displayed in gure 1 for the Thyroiddproblem and in gure 2 for the Geneeproblem. All the 10 runs were plotted simultaneously. Note that the standard and the CENNoptimized versions can only be compared if the SSE is plotted over the numberofepochs. In table 1 and in table 2, the results of the best runs are summarized. The generalization performance is estimated by the error with respect to …