Capturing Evolution in Word Usage: Just Add More Clusters?

Abstract

The way the words are used evolves through time, mirroring cultural or\ntechnological evolution of society. Semantic change detection is the task of\ndetecting and analysing word evolution in textual data, even in short periods\nof time. In this paper we focus on a new set of methods relying on\ncontextualised embeddings, a type of semantic modelling that revolutionised the\nNLP field recently. We leverage the ability of the transformer-based BERT model\nto generate contextualised embeddings capable of detecting semantic change of\nwords across time. Several approaches are compared in a common setting in order\nto establish strengths and weaknesses for each of them. We also propose several\nideas for improvements, managing to drastically improve the performance of\nexisting approaches.\n