MARS: Multiagent Reinforcement Learning for Spatial—Spectral and Temporal Feature Selection in EEG-Based BCI

Abstract

In recent years, deep learning methods have shown promising capabilities for extracting informative and discriminative features from electroencephalography (EEG) data. However, several studies have reported that the feature selection process followed by feature extraction can be beneficial to achieve further performance improvement. Even though a recent work achieved promising results by using the single-agent reinforcement learning (RL)-based framework to select task-relevant features in the temporal domain, it still failed to consider other significant features in the spatial–spectral domain. To overcome such limitations, we propose a cooperative multiagent RL-based framework (MARS) that performs feature selection in both the spatial–spectral and temporal domains simultaneously for a motor imagery (MI)-EEG classification task. In this framework, we enable our RL agents to collaborate with each other as a team to solve a complex multiobjective feature selection problem. Furthermore, we adopt a counterfactual advantage function to overcome the free-rider problem, which is associated with the credit assignment issue in multiagent cases. To assess the MARS framework, we conduct extensive experiments with two public MI datasets under subject-dependent and subject-independent scenarios and we apply the MARS to different backbone networks. The experimental results demonstrate that our MARS outperforms other competing methods in terms of mean accuracy and achieves statistically significant improvements.