HyperLex

HyperLex is a gold standard resource for measuring and evaluating how well semantic models capture graded or soft lexical entailment (also known as the type-of, is-a, or hypernymy-hyponymy relation) rather than semantic similarity or relatedness. It quantifies the extent of the semantic category membership and lexical entailment (LE) relation.

We provide 2616 word pairs (2163 noun pairs and 453 verb pairs) with ratings on a scale 0-10, annotated according to the question: "To what degree is X a type of Y?". Here are some examples:

Pair Rating
girl / person 9.85
citizen / person 8.63
person / citizen 5.17
idol / person 4.28
plant / person 0.42
to talk / to communicate 9.25
to pray / to communicate 4.83

HyperLex covers plenty of normed word types from the USF free-association database, and provides annotated examples of different WordNet-based lexical relations (i.e., hyponymy-hypernymy at different levels, co-hyponymy, synonymy, antonymy, meronymy-holonymy, no-relation). it also contains examples of different concreteness levels.

Download

Download HyperLex by clicking here.

All design details are outlined in the following paper. Please cite it if you use HyperLex in your own work:

HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment
Ivan Vulić, Daniela Gerz, Douwe Kiela, Felix Hill, and Anna Korhonen. arXiv 2016.
[pdf][bibtex]

The provided archive includes the full HyperLex dataset, noun and verb subsets, as well as two different data splits (random and lexical) into training, development and test data. Please see the accompanying readme file for the file formats and further details.

Please contact Ivan Vulić if you have any questions not addressed in the paper.