Complementing Travel Diary Surveys with Twitter Data: Application of Text Mining Techniques on Activity Location, Type and Time

Abstract

A growing body of literature in social science has been devoted to extracting new information from social media to assist authorities in manage crowd projects. In this paper geolocation (or spatial) based information provided in social media is investigated to utilize intelligent transportation services. Further, the general trend of travel activities during weekdays is studied. For this purpose, a dataset consisting of more than 40,000 tweets in south and west part of the Sydney metropolitan area is utilized. After a data processing effort, the tweets are clustered into seven main categories using text mining techniques, where each category represents a type of activity including shopping, recreation, and work. Unlike the previous studies in this area, the focus of this work is on the content of the tweets rather than only using geotagged data or sentiment analysis. Beside activity type, temporal and spatial distributions of activities are used in the classification exercise. Categories are mapped to the identified regions within the city of Sydney across four time slots (two peak periods and two off-peak periods). Each time slot is used to construct a network with nodes representing people, activities and locations and edges reflecting the association between the nodes. The constructed networks are used to study the trend of activities/locations in a typical working day.