2018 · 16 citations · 30 references
ComputingComputer ScienceDecision TheoryMachine LearningOptimizationGradient-based OptimizationReinforcement LearningRobust OptimizationBandit AlgorithmsContextual BanditsContextual Bandits In HealthcareDecision-making AlgorithmsExploration - Exploitation Trade-offExploration-exploitation Trade-offObjective Function ValueThreshold Parameter Estimation
We consider the actor-critic contextual bandit for the mobile health (mHealth) intervention. State-of-the-art decision-making algorithms generally ignore the outliers in the data-set. In this paper, we propose a novel robust contextual bandit method for the mHealth. It can achieve the conflicting goal of reducing the influence of outliers, while seeking for a similar solution compared with the state-of-the-art contextual bandit methods on the datasets without outliers. Such performance relies on two technologies: (1) the capped-L2 norm; (2) a reliable method to set the threshold hyper-parameter, which is inspired by one of the most fundamental techniques in the statistics. Although the model is non-convex and non-differentiable, we propose an effective reweighted algorithm and provide solid theoretical analyses. We prove that the proposed algorithm can sufficiently decrease the objective function value at each iteration and will converge after a finite number of iterations. Extensive experiment results on two datasets demonstrate that our method can achieve almost identical results compared with state-of-the-art contextual bandit methods on the dataset without outliers, and significantly outperform those state-of-the-art methods on the badly noised dataset with outliers in a variety of parameter settings.
30
Reinforcement Learning: An Introduction
IEEE Transactions on Neural Networks · 2005 · 25.8K citations
A contextual-bandit approach to personalized news article recommendation
Lihong Li, Wei Chu, John Langford et al. · 2010 · 2.5K citations · Full text
Efficient and Robust Feature Selection via Joint ℓ2,1-Norms Minimization
Feiping Nie, Heng Huang, Xiao Cai et al. · 2010 · 1.6K citations
A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients
I. Grondman, Lucian Buşoniu, Gabriel A. D. Lopes et al. · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · 2012 · 1K citations · Full text
Computing, Computer Science, Finance +13
A Smartphone Application to Support Recovery From Alcoholism
David H. Gustafson, Fiona McTavish, Ming‐Yuan Chih et al. · JAMA Psychiatry · 2014 · 743 citations · Full text