Machine Learning · 2021 · 20 citations · 33 references
ComputingComputer ScienceData MiningMachine LearningNatural Language ProcessingActive LearningStream ProcessingActive Learning For ClassificationLabelsSliding Window AlgorithmsStream-based Active LearningStreaming LearningStreaming Machine Learning
Abstract Stream-based active learning (AL) strategies minimize the labeling effort by querying labels that improve the classifier’s performance the most. So far, these strategies neglect the fact that an oracle or expert requires time to provide a queried label. We show that existing AL methods deteriorate or even fail under the influence of such verification latency. The problem with these methods is that they estimate a label’s utility on the currently available labeled data. However, when this label would arrive, some of the current data may have gotten outdated and new labels have arrived. In this article, we propose to simulate the available data at the time when the label would arrive. Therefore, our method Forgetting and Simulating (FS) forgets outdated information and simulates the delayed labels to get more realistic utility estimates. We assume to know the label’s arrival date a priori and the classifier’s training data to be bounded by a sliding window. Our extensive experiments show that FS improves stream-based AL strategies in settings with both, constant and variable verification latency.
33
Models and issues in data stream systems
Brian Babcock, Shivnath Babu, Mayur Datar et al. · 2002 · 2.5K citations
Burr Settles · Synthesis lectures on artificial intelligence and machine learning · 2012 · 598 citations
Education, Instruction, Teaching +7