Machine Learning and Friends Lunch
DYNAMIC BAYESIAN NETS FOR LANGUAGE MODELING
Dr. Leon Peshkin
Harvard
Abstract
Dynamic Bayesian networks (DBNs) offer an elegant way to integrate various aspects of language in one model. Many existing algorithms developed for learning and inference in DBNs are applicable to probabilistic language modeling. To demonstrate the potential of DBNs for natural language processing, we employ a DBN for information extraction and part-of-speech tagging tasks. Our methods outperform previously published results on an established benchmark domain. This talk will overview the following papers, avilable from http://www.ai.mit.edu/~pesha/Public/papers.html
"Why Build Another Part-Of-Speech Tagger ?" (in review) 8p.
- How simple a PoS tagger could we make ?
- How could it be trained independently, then integrated into a system?
- Is the key to PoS tagging in the features or in the model after all ?
- Do linguistic features really help ?
"Bayesian Nets in Syntactic Categorization of Novel Words" (in review) 3p.
- Our PoS tagger fares well on novel data, trained on WSJ, tested on Brown corpus, email corpus and even "Jabberwocky".
"Bayesian Information Extraction Network" accepted IJCAI - 2003 8p.
- We assemble wealth of emerging linguistic instruments for shallow parsing, syntactic and semantic tagging, morphological decomposition, named entity recognition etc. in order to incrementally build a robust information extraction system. (camera ready due Apr 10th - feedback and comments appreciated)
"Integrated probabilistic reasoning about text" (in preparation) 8p.
- This paper presents an alternative architecture for probabilistic inference about text which performs integrated reasoning about syntactic and semantic categories.
- Demonstrates the power of approximate inference algorithms for NLP.