Topics in semantic association
Learning and using language requires retrieving concepts from memory in response to an ongoing stream of information. The human memory system solves this prob lem by using the gist of a sentence, conversation, or document to predict lated concepts and disambiguate words. Two approaches to representing gist h ave dominated research on semantic representation: semantic networks and semantic spaces. We take a step back from these approaches, and analyze the abstract c ompu ational problem underlying the extraction and use of gist, formulating this problem in s tatistical terms. This analysis allows us to explore a novel approach to semantic representation, in which words are represented using a set of probab ilistic topics. The topic model performs well in predicting word association, free recall, a nd the senses of words, and provides a foundation for developing richer sta tistic l models of language. Learning, speaking, and understanding language all require solving a challenging computational problem: retrieving a variety of concepts from memory in response to a n ongoing stream of information. The human memory system solves this problem by using the semantic c ontext – the gistof a sentence, conversation, or document – to predict related concepts and disambiguate words. Online processing of sentences can be facilitated by predicting which conc epts are likely to be relevant before they are needed. For example, if the word BANK appears in a sentence, it might become more likely that words likeFEDERAL and RESERVEwould also appear in that sentence, and this information could be used to initiate retrieval of the information related to these wo rds. This preThis work was supported by a grant from the NTT Communication Science s Laboratory. While completing this work, TLG was supported by a Stanford Graduate Fellowship, and JBT by the Pa ul E. Newton chair. We thank Touchstone Applied Sciences, Tom Landauer, and Darrell Laham for making the TA SA corpus available, and for their thoughts on these topics. TOPICS IN SEMANTIC ASSOCIATION 2