(Linguistic) Science Through Web Collaboration in the ANAWIKI project

Abstract

Perhaps the greatest obstacle to progress towards systems able to extract semantic information from text is the lack of semantically annotated corpora large enough to be used to train and evaluate semantic interpretation methods. The community is beginning to realize that even the 1M word annotated corpora created in substantial efforts such as PropBank and OntoNotes are likely to be too small; but unfortunately, the creation of 100M-plus corpora via hand annotation is likely to be prohibitively expensive. Yet initiatives such as Wikipedia and, in the AI community, OpenMind CommonSense show that it is possible to get thousands of people to participate in science-through-the-Web initiatives. And the ESP game showed that the game format is a promising way to address the motivation issue. The goal of the ANAWIKI project is to experiment with Web collaboration as a solution to the problem of creating large-scale annotated corpora, both by developing tools through which members of our scientific community can participate in corpus creation (using annotation tools with a Web interface such as Serengeti and through the use of game-like interfaces. We will present ongoing work on Phrase Detectives, a game designed to collect judgments about anaphoric annotations.