Scalingin a Hierar chical UnsupervisedNetwork
A persistent worry with computational modelsof unsupervised learningis thatlearningwill becomemoredifficult astheproblem is scaled.We examinethis issuein thecontext of a novel hierarchical,generati ve model that can be viewed as a nonlineargeneralizationof factoranalysisandcanbeimplemented in a neuralnetwork. Themodelperformsperceptual inference in a probabilistically consistentmannerby using top-down, bottom-upandlateralconnections.Theseconnectionscanbe learnedusingsimplerules that requireonly locally available information. We first demonstratethat the modelcanextract a sparse,distributed,hierarchicalrepresentationof globaldisparity from simplified random-dotstereograms.We then investigatesomeof the scalingpropertiesof the algorithm on this problemandfind that: (1) Increasingtheimagesizeleads to fasterandmorereliablelearning;(2) Increasingthedepthof thenetwork from oneto two hiddenlayersleadsto betterrepresentationsat thefirst hiddenlayer, and(3) Onceonepartof thenetwork hasdiscoveredhow to representdisparity, it “supervises”otherpartsof thenetwork, greatlyspeedingup their learning.