OCRonym: Entity Extraction and Retrieval for Scanned Books

FAIN: HD-50794-09

University of Massachusetts, Amherst (Amherst, MA 01003-9242)
James Allan (Project Director: April 2009 to January 2011)
David Smith (Co Project Director: April 2009 to January 2011)

Development of an extraction and retrieval system for named entities-people, places, and organizations-located across a large number of documents in order to use the system to track Optical Character Recognition (OCR) error rates in an effort to improve "noisy" OCR.

In the past five years, massive book-scanning projects have produced an explosion in the number of sources for the humanities, available on-line to the broadest possible audiences. Transcribing page images by optical character recognition makes many searching and browsing tasks practical for scholars. But even low OCR error rates compound into high probability of error in a given sentence, and the error rate is even higher for names. We propose to build a prototype system for information extraction and retrieval of noisy OCR. In particular, we will optimize the extraction and retrieval of names, which are highly informative features for detecting topics and events in documents. We will build statistical models of characters and words from scanned books to improve lexical coverage, and we will improve name categorization and disambiguation by linking document contexts to external sources such as Wikipedia. Our testbed comes from over one million scanned books from the Internet Archive.