UMass Amherst
UMass Amherst | Library | Umail | Spire | People Finder 

Search

Match case Regex

Current issue

Categories

WHISC Back Issues

Other Ling Newsletters

Subscribe to this blog's feed
[What is this?]
Creative Commons License
This weblog is licensed under a Creative Commons License.

« HUMDRUM on the Horizon | Main | Amherst College Lecture: Timothy Williamson»

Computer Science Talk: Noah Smith

Noah Smith
Language Technologies Institute and Machine Learning Department
Carnegie Mellon

Hidden Grammar: Advances in Data-Driven Models of Language

Monday, March 9, 1:30 pm Computer Science Building, Room 151

Abstract

With the field of computational linguistics' empirical revolution of the 1990s came the realization that human intuitions about language are insufficient for accurate and robust natural language technologies. The move from hand-written, rule-based models to data-driven techniques led to huge advances, yet we still leaned on human intuition for constructing annotated linguistic datasets. We now know that, in the wild world of real and diverse linguistic data, natural language technology raised on expert-made annotations remains insufficient for real, robust applications.

In this talk we adopt the premise that unsupervised learning will, in the long run, be the way forward for learning computational models of language cheaply. We focus on dependency syntax learning without trees, beginning with the classic EM algorithm and presenting several ways to alter EM for drastically improved performance using crudely represented "knowledge" of linguistic universals. We then present more recent work in the empirical Bayesian paradigm, where we encode our background knowledge as a prior over grammars, applying inference to obtain hidden structure. Of course, "background knowledge" is still human intuition. We argue, however, that by representing this knowledge compactly in a prior distribution--far more compactly than the many decisions made in building treebanks--we can experimentally explore the connection between proposed linguistic universals and unsupervised learning.

Time permitting, we close with discussion of some recent advances in more traditional supervised learning of syntax models with rich non-local features that break old independence assumptions, as well as recent work incorporating syntax models in question answering and paraphrase modeling.

This talk includes discussion of joint work with Shay Cohen, Dipanjan Das, Jason Eisner, Kevin Gimpel, Andre Martins, and Eric Xing.

March 5, 2009 12:08 AM | 07:07 (2009-03-05) , computation , events