Learning Domain Structures
Learning Domain Structures Charles Kemp, Amy Perfors & Joshua B. Tenenbaum {ckemp, perfors, jbt}@mit.edu Department of Brain and Cognitive Sciences Massachusetts Institute of Technology structure, although with repeated training, its hidden unit representations may implicitly come to approximate the taxonomic relations between biological species. This paper proposes an alternative approach – struc- ture learning – that combines important insights from both of these traditions. Our key contribution is to show how structured domain representations can be ac- quired within a domain-general framework for Bayesian inference. Like nativists, we suggest that different do- mains are represented with qualitatively different struc- tures, and we show how these structured representations serve as critical constraints on inductive generalization. Like empiricists, though, we emphasize the importance of learning, and attempt to show how domain structures can be acquired through domain-general statistical in- ference. This is not only more parsimonious than the nativist position, but allows us to explain the origin of structured representations in novel domains, where the prior existence of domain-specific innate structure is highly implausible. After describing our structure learning framework, we present two empirical tests of its performance. First, we show that it chooses the appropriate domain structure for both synthetic and real-world data sets. It correctly chooses a tree structure for a biological domain (animal feature judgments), and a linear structure for a politi- cal domain (US Supreme Court decisions). Second, we model two classic data sets of inductive judgments in bi- ology [13] and show that our framework performs better than an unstructured connectionist approach. Abstract How do people acquire and use knowledge about do- main structures, such as the tree-structured taxonomy of folk biology? These structures are typically seen ei- ther as consequences of innate domain-speci c knowl- edge or as epiphenomena of domain-general associative learning. We present an alternative: a framework for statistical inference that discovers the structural princi- ples that best account for di erent domains of objects and their properties. Our approach infers that a tree structure is best for a biological dataset, and a linear structure (“left”–“right”) is best for a dataset of peo- ple and their political views. We compare our proposal with unstructured associative learning and argue that our structured approach gives the better account of in- ductive generalization in the domain of folk biology. Psychologists have argued that cognition in differ- ent domains draws on qualitatively different mental representations. Tree structures appear well-suited to representing relationships between animal species [1, 2, 10], while a one-dimensional structure (the liberal- conservative spectrum) seems better for representing people’s political views. The possibility of different structures raises a fundamental question: how do peo- ple learn what kind of structure is appropriate in each domain? The standard approach to this question is to reject one of its assumptions. Nativists deny that core struc- tures are learned, at least for evolutionarily important domains like folkbiology. Instead, infants come equipped with innate knowledge about which structures are appro- priate for which domains. Atran [1], for example, argues that folkbiology is a core domain of human knowledge, and that the tendency to group living kinds into hier- archies re ects an “innately determined cognitive struc- ture.” More generally, Keil [8] has argued that ontologi- cal knowledge obeys an innate “M-constraint”, requiring the extensions of predicates to conform to rigidly tree- structured hierarchies of objects. Alternatively, empiricists generally deny that struc- tured representations are present at all. Domain-specific representations are merely emergent properties of un- structured, domain-general associative learning architec- tures. McClelland and Rogers [12], for example, have re- cently suggested that the acquisition of semantic knowl- edge in domains such as intuitive biology can be ex- plained as learning in a generic connectionist network. Their architecture never explicitly represents any tree Bayesian structure learning Our proposal takes the form of a rational analysis. We aim to demonstrate the computational plausibility and explanatory value of Bayesian structure learning, but leave for future work the question of how these com- putations might be implemented or approximated by cognitive processes. Assume the learner’s data consist of a binary-valued object-feature matrix D specifying the features of each object in a given domain. In bi- ology, for instance, the rows of D might correspond to species, and the columns to anatomical and behavioral attributes. The entry in row i and column j would then specify the value of feature j for species i. Structure- learning includes computational problems at two levels. First, which structure class is most appropriate for the domain? Second, given a structure class, which structure