Learning of depth two neural networks with constant fan-in at the hidden nodes (extended abstract)
We present algorithms for learning neural networks where the hidden depth two nodes are threshold gates with constant fan-in. The transfer function of the output node might be more general: we have results for the cases when the threshold function, the logistic function or the identity function is used as the transfer function at the output node. We give batch and on-line learning algorithms for these classes of neural networks and prove bounds on the performance of our algorithms, The batch algorithms work for real valued inputs whereas the on-line algorithms assume that the inputs are discretized. The hypotheses of our algorithms are essentially also neural networks of depth two. However, their number of hidden nodes might be much larger than the number of hidden nodes of the neural network that haa to be learned. Our algorithms can handle such a large number of hidden nodes since they rely on multiplicative weight updates at the output node, and the performance of these algorithms scales only logarithmically with the number of hidden nodes used. *Address: Department of Computer Science, University of California, Santa Cruz, CA 95064. E-mail: pauer@cse.ucsc.edu ‘Address: Department of Computer Science, University of Illinois, Urbana, IL 61801. E-mail: kwek@cs.uiuc.edu. i Address: Institute of Theoretical Computer Science, Technische Universitat Graz, Klosterwiesgasse 32/2, A-8olo Graz, Austria. E-mail: maass@igi.tu-graz. ac.at SAddress: Department of Computer Science, University of California, Santa Cruz, CA 95064. E-mail: manfred@cis.ucsc .edu. Permission to make dlgitallhard copies of all or part of tids material for peraoml or classroom use is granted without fee provided that the copies are not made or dk%ributed for profit or commercial advantage, the. copyright notice, the title of the publication and its date appear, and notice IS given tha} copyright is by permission of the.ACM, Inc. To co~>yotherwise, to repubhsh, to post on servers or to distribute to Ms, requires specitic permission and/or fee. COLT’96, Desenzano del Garda, Italy @1996 ACM 0-89791-8 1143/9fj/ofj ..$3 .5t)