IEEE Transactions on Systems Man and Cybernetics Systems · 2021 · 20 citations · 32 references
ComputingComputer SystemsMachine LearningMarkov Decision ProcessesSensorsIot Energy ManagementReinforcement LearningSensor NetworksSmart SensorsTime Series PredictionWireless Sensor NetworksAccuracyEvent-driven Iot SystemsInternet Of Things SensingLong Short-term Memory NetworksMachine Learning For IotMarkov Decision ModelsSequence ModelingState Sequence Modeling
Prediction of sensor readings in event-based Internet-of-Things (IoT) applications is considered. A new approach is proposed, which allows turning off sensors in periods when their readings can be predicted, thus preserving energy that would be consumed for sensing and communications. The proposed approach uses a long short-term memory (LSTM) model that learns spatiotemporal patterns in sequences of sensorial data for future predictions. The LSTM model and the sensors collaboratively monitor the environment. They are controlled by a reinforcement learning (RL) agent that dynamically decides about using the LSTM prediction versus physical sensing in a way that maximizes energy saving while maintaining prediction accuracy. Two approaches are used for the RL: 1) the Markov decision process (MDP) model-based for low scale applications and 2) deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula> -Network-based for larger scales. Compared to the current literature, the proposed solution is unique in predicting all sensor readings for real-time event detection and providing a model capable of learning long-term spatiotemporal correlations, enabling power conservation and detection accuracy balance. We compare the proposed solutions to the most relevant state-of-the-art approaches using a large real dataset collected in a dynamic space by measuring the accuracy, consumed energy, network lifetime, latency, and missed events’ ratio. To investigate the scalability of the solutions, these parameters are calculated for different network sizes. The results show that the system achieves 50% accuracy with 32% of activation time and 75% accuracy with 60% activation time.
32
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
Reinforcement Learning: An Introduction
Richard S. Sutton, Andy Barto · IEEE Transactions on Neural Networks · 1998 · 27.1K citations
Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates
Shixiang Gu, Ethan Holly, Timothy Lillicrap et al. · 2017 · 1.5K citations