A One-Armed Bandit Problem with a Concomitant Variable

Abstract

Abstract We consider a Bayesian sequential allocation problem that incorporates a concomitant variable in the model. Although the model is highly simplified, our results do have some mildly surprising aspects: (a) the approximate solution to the allocation problem is simpler in the presence of concomitant information than in its absence, (b) the exact value of the discount factor is less important than in previous work, and (c) the myopic procedure is asymptotically optimal.