Semisupervised Deep Reinforcement Learning in Support of IoT and Smart City Services

TLDR

Smart services in IoT and smart cities rely on sensory data, yet large labeled datasets are often unavailable, making semi‑supervised deep reinforcement learning—an approach that has succeeded in many domains—an attractive solution for leveraging partially labeled user feedback. The authors propose a semi‑supervised deep reinforcement learning model that consumes both labeled and unlabeled data to enhance learning‑agent performance in smart city applications. The model employs a Variational Autoencoder as an inference engine and is applied to indoor localization in smart buildings using BLE signal strength. This is the first study extending DRL to a semi‑supervised paradigm, achieving a 23 % reduction in localization error and at least 67 % higher rewards compared to a supervised DRL baseline.

Abstract

Smart services are an important element of the smart cities and the Internet of Things (IoT) ecosystems where the intelligence behind the services is obtained and improved through the sensory data. Providing a large amount of training data is not always feasible; therefore, we need to consider alternative ways that incorporate unlabeled data as well. In recent years, Deep reinforcement learning (DRL) has gained great success in several application domains. It is an applicable method for IoT and smart city scenarios where auto-generated data can be partially labeled by users' feedback for training purposes. In this paper, we propose a semi-supervised deep reinforcement learning model that fits smart city applications as it consumes both labeled and unlabeled data to improve the performance and accuracy of the learning agent. The model utilizes Variational Autoencoders (VAE) as the inference engine for generalizing optimal policies. To the best of our knowledge, the proposed model is the first investigation that extends deep reinforcement learning to the semi-supervised paradigm. As a case study of smart city applications, we focus on smart buildings and apply the proposed model to the problem of indoor localization based on BLE signal strength. Indoor localization is the main component of smart city services since people spend significant time in indoor environments. Our model learns the best action policies that lead to a close estimation of the target locations with an improvement of 23% in terms of distance to the target and at least 67% more received rewards compared to the supervised DRL model.