Astrid van Aggelen
Centrum Wiskunde Informatica, Amsterdam, The Netherlands
Antske Fokkens
Vrije Universiteit Amsterdam, The Netherlands
Laura Hollink
Centrum Wiskunde Informatica, Amsterdam, The Netherlands
Jacco van Ossenbruggen
Centrum Wiskunde Informatica, Amsterdam, The Netherlands
Download article
Published in: Proceedings of the 22nd Nordic Conference on Computational Linguistics (NoDaLiDa), September 30 - October 2, Turku, Finland
Linköping Electronic Conference Proceedings 167:5, p. 44--54
NEALT Proceedings Series 42:5, p. 44--54
Published: 2019-10-02
ISBN: 978-91-7929-995-8
ISSN: 1650-3686 (print), 1650-3740 (online)
Determining how words have changed their meaning is an important topic in Natural Language Processing. However, evaluations of methods to characterise such change have been limited to small, handcrafted resources. We introduce an English evaluation set which is larger, more varied, and more realistic than seen to date, with terms derived from a historical thesaurus. Moreover, the dataset is unique in that it represents change as a shift from the term of interest to a WordNet synset. Using the synset lemmas, we can use this set to evaluate (standard) methods that detect change between word pairs, as well as (adapted) methods that detect the change between a term and a sense overall. We show that performance on the new data set is much lower than earlier reported findings, setting a new standard.
