ReservoirRiddles:Suggestions forEcho State NetworkResearch(Extended Abstract)
Echostate networks (ESNs) offer asimple learning algorithm fordynamical systems. Itworksbytraining linear readout neurons thatcombinethesignals froma random, fixed, excitable "dynamical reservoir" network. Oftenthemethod worksbeautifully, sometimes itworkspoorly - andwe donot really understand why.Thiscontribution discusses phenomena related topoorlearning performance andsuggests research directions. Thecommonthemeistounderstand thereservoir dynamics intermsofadynamical representation ofthetask's input signals. I.RESERVOIR RIDDLES... Echostate networks (ESNs), aswellastheclosely related "liquid state machines" (LSM)(1), present arecurrent neural network (RNN)learning architecture whichischaracterized by . alarge, randomly connected, recurrent "reservoir" net- workthat ispassively excited bythetask's input signal, and . trainable readout neurons that combine thedesired output fromtheexcited reservoir state. Training anESNonasupervised learning taskboils downto compute theoutput weights. Fromacomputational perspective thisisjust a linear regression, forwhichnumerous batch andadaptive online algorithms areavailable. Thissimple methodyields models that inmanyengineering tasks surpass inaccuracy othermodelling methods (2).TheESN/LSM principle - combine atarget signal fromrandom, dynamic input variations - mayalsobeeffective inbiological brains