CSI-based Device-Free Joint Activity Recognition and Localization using Siamese Networks

Abstract

Wireless sensing plays a vital role in a connected ubiquitous environment for Internet of Things (IoT) applications. Two important applications include Human Activity Recognition (HAR) and location estimation. Most existing works perform either HAR or localization only using features such as Received Signal Strength (RSS) or Channel State Information (CSI). This work proposes a hardware for easy capturing of CSI and performing joint HAR and localization using Siamese networks. The joint task is crucial for smart home applications where gesture commands can be used to operate appliances/devices from a specific location. The designed hardware is cost-efficient using only ESP32 microcontroller suitable for edge computing. The Siamese networks provide embeddings that increase the HAR and localization accuracy in an unseen environment compared to other machine learning (ML) or deep learning (DL) methods. Additionally, the user need not carry or wear any device for this task, and no use of cameras helps maintain the user's privacy.