International Journal of Intelligent Systems · 2019 · 20 citations · 21 references
ComputingComputer ScienceData ScienceBig Data AnalyticsData MiningMachine LearningData PreparationData PreprocessingData QualityData ReductionEnsemble LearningEnsemble MethodsBig Data ClassificationData FilteringLabel Noise MitigationLarge-scale Data ProcessingNoise Filtering
The quality of the data is directly related to the quality of the models drawn from that data. For that reason, many research is devoted to improve the quality of the data and to amend errors that it may contain. One of the most common problems is the presence of noise in classification tasks, where noise refers to the incorrect labeling of training instances. This problem is very disruptive, as it changes the decision boundaries of the problem. Big Data problems pose a new challenge in terms of quality data due to the massive and unsupervised accumulation of data. This Big Data scenario also brings new problems to classic data preprocessing algorithms, as they are not prepared for working with such amounts of data, and these algorithms are key to move from Big to Smart Data. In this paper, an iterative ensemble filter for removing noisy instances in Big Data scenarios is proposed. Experiments carried out in six Big Data datasets have shown that our noise filter outperforms the current state-of-the-art noise filter in Big Data domains. It has also proved to be an effective solution for transforming raw Big Data into Smart Data.
21
UCI Machine Learning Repository
Arthur Asuncion · Medical Entomology and Zoology · 2007 · 24.4K citations
Sanjay Ghemawat · Communications of the ACM · 2008 · 18.5K citations · Full text
Challenges of Big Data analysis
Jianqing Fan, Fang Han, Han Liu · National Science Review · 2014 · 1.5K citations · Full text