Twitter event detection: combining wavelet analysis and topic inference summarization

Abstract

Today streaming text mining plays an important role within real-time social media mining. Given the amount and cadence of the data generated by those platforms, classical text mining techniques are not suitable to deal with such new mining challenges. Event detection is no exception, available algorithms rely on text mining techniques applied to pre-known datasets processed with no restrictions about computational complexity and required execution time per document analysis. This work presents a lightweight event detection using wavelet signal analysis of hashtag occurrences in the twitter public stream. It also pro- poses a strategy to describe detected events using a Latent Dirichlet Allocation topic inference model based on Gibbs Sampling. Peak detec- tion using Continuous Wavelet Transformation achieved good results in the identification of abrupt increases on the mentions of specific hash- tags. The combination of this method with the extraction of topics from tweets with hashtag mentions proved to be a viable option to summarize detected twitter events in streaming environments.