Recursive estimation of prior probabilities using a mixture

Abstract

The problem of estimating the prior probabilities <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">q = (q_{1} \cdots q_{m-1})</tex> of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</tex> statistical classes with known probability density functions <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F_{1}(X) \cdots F_{m}(x)</tex> on the basis of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</tex> statistically independent observations <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(X_{l} \cdots x_{n})</tex> is considered. The mixture density <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">g(x|q) = \sum^{m-1}_{j-1}q_{j}F_{j}(x) + (1 - \sum^{m-1}_{\tau = 1}q_{\tau})F_{m(x)</tex> is used to show that the maximum likelihood estimate of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">q</tex> is asymptotically efficient and weakly consistent under very mild constraints on the set of density functions. A recursive estimate is proposed for <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">q</tex> . By using stochastic approximation theory and optimizing the gain sequence, it is shown that the recursive estimate is asymptotically efficient for the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m = 2</tex> class case. For <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m &gt; 2</tex> classes, the rate of convergence is computed and shown to be very close to asymptotic efficiency.