2012 · 215 citations · 0 references
ComputingComputer ScienceInformation TechnologyBig Data AnalyticsCluster ComputingData ManagementData Storage SystemsDatabase ManagementDistributed Data ProcessingDistributed StorageScalable Data StorageStorage EnginesBig Data Analytics ArchitectureBig Data ArchitectureDistributed File System ArchitectureLarge-scale Data ProcessingMapreduceMapreduce AlgorithmsScalable Data ManagementUnstructured Data Extraction
The volume of data worldwide is exploding, with unstructured data such as imagery, sensors, and documents growing fastest, creating a challenge to store, manage, and extract value from massive, heterogeneous datasets. The paper focuses on analyzing unstructured data. The authors employ MapReduce with HDFS and HBase within the Apache Hadoop ecosystem, using scale‑out, elastic, high‑availability clusters to process massive volumes efficiently.
The amount of data in our industry and the world is exploding. Data is being collected and stored at unprecedented rates. The challenge is not only to store and manage the vast volume of data (“big data”), but also to analyze and extract meaningful value from it. There are several approaches to collecting, storing, processing, and analyzing big data. The main focus of the paper is on unstructured data analysis. Unstructured data refers to information that either does not have a pre-defined data model or does not fit well into relational tables. Unstructured data is the fastest growing type of data, some example could be imagery, sensors, telemetry, video, documents, log files, and email data files. There are several techniques to address this problem space of unstructured analytics. The techniques share a common characteristics of scale-out, elasticity and high availability. MapReduce, in conjunction with the Hadoop Distributed File System (HDFS) and HBase database, as part of the Apache Hadoop project is a modern approach to analyze unstructured data. Hadoop clusters are an effective means of processing massive volumes of data, and can be improved with the right architectural approach.