Proceedings of the 31st ACM International Conference on Information & Knowledge Management · 2022 · 40 citations · 21 references
ComputingComputer ScienceComputational LinguisticsMachine LearningNatural Language ProcessingDeep LearningLanguage ModelingPattern RecognitionRepresentation LearningSequence LearningTrajectory ClassificationTrajectory RepresentationTrajectory Representation LearningTransformer Language ModelsTransformer ModelsTransformer-based Classification
Transformers have been an efficient alternative to recurrent neural networks in many sequential learning tasks. When adapting transformers to modeling trajectories, we encounter two major issues. First, being originally designed for language modeling, transformers assume regular intervals between input tokens, which contradicts the irregularity of trajectories. Second, transformers often suffer high computational costs, especially for long trajectories. In this paper, we address these challenges by presenting a novel transformer architecture entitled TrajFormer. Our model first generates continuous point embeddings by jointly considering the input features and the information of spatio-temporal intervals, and then adopts a squeeze function to speed up the representation learning. Moreover, we introduce an auxiliary loss to ease the training of transformers using the supervision signals provided by all output tokens. Extensive experiments verify that our TrajFormer achieves a preferable speed-accuracy balance compared to existing approaches.
21
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio et al. · Proceedings of the IEEE · 1998 · 58.2K citations · Full text
Advances in neural information processing systems 7
Computers & Mathematics with Applications · 1996 · 14.4K citations
Understanding mobility based on GPS data
Yu Zheng, Quannan Li, Yukun Chen et al. · 2008 · 1.1K citations
Computing, Geoinformatics, Geographic Information Systems +16
Indexing the positions of continuously moving objects
Simonas Šaltenis, Christian S. Jensen, Scott T. Leutenegger et al. · ACM SIGMOD Record · 2000 · 916 citations