Resolving Complex Cases of Definite Pronouns: The Winograd Schema Challenge

Altaf Rahman and Vincent Ng.
Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 777-789, 2012.

Click here for the PostScript or PDF version.

Abstract

We examine the task of resolving complex cases of definite pronouns, specifically those for which traditional linguistic constraints on coreference (e.g., Binding Constraints, gender and number agreement) as well as commonly-used resolution heuristics (e.g., string-matching facilities, syntactic salience) are not useful. Being able to solve this task has broader implications in artificial intelligence: a restricted version of it, sometimes referred to as the Winograd Schema Challenge, has been suggested as a conceptually and practically appealing alternative to the Turing Test. We employ a knowledge-rich approach to this task, which yields a pronoun resolver that outperforms state-of-the-art resolvers by nearly 18 points in accuracy on our dataset.

Datasets

The data used in this paper are available from this page.

BibTeX entry

@InProceedings{Rahman+Ng:12c,
  author = {Altaf Rahman and Vincent Ng},
  title = {Resolving Complex Cases of Definite Pronouns: The Winograd Schema Challenge},
  booktitle = {Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning},
  pages = {777--789}, 
  year = 2012
}

The city councilmen refused the demonstrators a permit because they advocated violence. The city councilmen refused the demonstrators a permit because they feared violence.
James asked Robert for a favor, but he refused. James asked Robert for a favor, but he was refused.
Keith fired Blaine but he did not regret. Keith fired Blaine although he is diligent.
Emma did not pass the ball to Janie, although she was open. Emma did not pass the ball to Janie, although she should have.
Medvedev will cede the presidency to Putin because he is more popular. Medvedev will cede the presidency to Putin because he is less popular.