A Scalable, Research Oriented, Generic, Sensor Data Platform

Abstract

Research interests spanning numerous domains increasingly rely upon computational systems which can store and process a large volume of variable data that is stored at high velocity-representing a big data problem. This is particularly notable within the domain of ubiquitous and pervasive computing. This domain increasingly relies on storage and retrieval of sensor data to enable outcomes such as predictive analytics and activity recognition. Several current big data platforms exist; however, they have a range of deficiencies including lack of generic interoperability with agnostic sensors and an absence of features supporting academic research. Due to these deficiencies a custom, research oriented, high performance, and big data platform was devised and implemented. This platform is called SensorCentral and is presented within this paper. SensorCentral provides a framework which enables interoperability with a large range of agnostic sensor devices whilst simultaneously providing features which support research. Research supporting features include; facility to define experiments, ability to annotate experimental instances via purpose-built mobile applications, integrated machine learning functionality, facility to export data sets, rule-based classification and an extensible platform. The flagship implementation of this platform has been in operation for over 28 months within a University research group and has been successfully integrated with a range of sensors from a variety of manufacturers. This implementation currently stores over 850 million records and has been central to several research and industrial projects. Future work will integrate this platform into the open data initiative enabling collaboration with the international community of researchers.