Deep learning applications and challenges in big data analytics

TLDR

Big Data Analytics and Deep Learning are central to data science, with organizations collecting massive domain‑specific data for applications such as intelligence, security, fraud detection, marketing, and medicine, and Deep Learning extracting hierarchical high‑level abstractions from largely unlabeled data, making it a valuable tool for analyzing large volumes. The study investigates how Deep Learning can address key Big Data Analytics challenges such as extracting complex patterns, semantic indexing, data tagging, rapid information retrieval, and simplifying discriminative tasks. The authors examine additional research aspects required for Deep Learning in Big Data, namely streaming data, high‑dimensional data, model scalability, and distributed computing. The authors conclude with future research directions, posing questions on data sampling, domain adaptation, abstraction criteria, semantic indexing, semi‑supervised and active learning.

Abstract

Abstract Big Data Analytics and Deep Learning are two high-focus of data science. Big Data has become important as many organizations both public and private have been collecting massive amounts of domain-specific information, which can contain useful information about problems such as national intelligence, cyber security, fraud detection, marketing, and medical informatics. Companies such as Google and Microsoft are analyzing large volumes of data for business analysis and decisions, impacting existing and future technology. Deep Learning algorithms extract high-level, complex abstractions as data representations through a hierarchical learning process. Complex abstractions are learnt at a given level based on relatively simpler abstractions formulated in the preceding level in the hierarchy. A key benefit of Deep Learning is the analysis and learning of massive amounts of unsupervised data, making it a valuable tool for Big Data Analytics where raw data is largely unlabeled and un-categorized. In the present study, we explore how Deep Learning can be utilized for addressing some important problems in Big Data Analytics, including extracting complex patterns from massive volumes of data, semantic indexing, data tagging, fast information retrieval, and simplifying discriminative tasks. We also investigate some aspects of Deep Learning research that need further exploration to incorporate specific challenges introduced by Big Data Analytics, including streaming data, high-dimensional data, scalability of models, and distributed computing. We conclude by presenting insights into relevant future works by posing some questions, including defining data sampling criteria, domain adaptation modeling, defining criteria for obtaining useful data abstractions, improving semantic indexing, semi-supervised learning, and active learning.