Stream mining of frequent sets with limited memory

Abstract

With advances in technology, streams of data are produced in many applications. Efficient techniques for extracting implicit, previously unknown, and potentially useful information (e.g., in the form frequent sets) from data streams are in demand. Many existing stream mining algorithms capture important streaming data and assume that the captured data can fit into main memory. However, problem arose when the available memory is so limited that such an assumption does not hold. In this paper, we propose a novel data structure called DSTable to capture important data from the streams onto the disk. The DSTable can be easily maintained; it can be applicable for mining frequent sets from datasets, especially in limited memory environments.