Subsymbolic Natural Language Processing: An Integrated Model of Scripts, Lexicon, and Memory

Risto Miikkulainen draws on recent connectionist work in language comprehension to create a model that can understand natural language. Using the DISCERN system as an example, he describes a general approach to building high-level cognitive models from distributed neural networks and shows how the special properties of such networks are useful in modeling human performance. In this approach connectionist networks are not only plausible models of isolated cognitive phenomena, but also sufficient constituents for complete artificial intelligence systems.Distributed neural networks have been very successful in modeling isolated cognitive phenomena, but complex high-level behavior has been tractable only with symbolic artificial intelligence techniques. Aiming to bridge this gap, Miikkulainen describes DISCERN, a complete natural language processing system implemented entirely at the subsymbolic level. In DISCERN, distributed neural network models of parsing, generating, reasoning, lexical processing, and episodic memory are integrated into a single system that learns to read, paraphrase, and answer questions about stereotypical narratives.Miikkulainen's work, which includes a comprehensive survey of the connectionist literature related to natural language processing, will prove especially valuable to researchers interested in practical techniques for high-level representation, inferencing, memory modeling, and modular connectionist architectures.Risto Miikkulainen is an Assistant Professor in the Department of Computer Sciences at The University of Texas at Austin.