arXiv (Cornell University) · 2013 · 39 citations · 8 references
ComputingComputer ScienceMachine LearningStatistical LearningStatistical ModelingBayesian InferenceBayesian LearningBayesian ModelingBayesian StatisticsMaximum Likelihood EstimationParameter EstimationBayesian EstimationBayesian NetworksConvergenceEm AlgorithmMaximum Likelihood EstimatorOnline Machine LearningParameter InferenceUpdate Mechanisms
This paper re-examines the problem of parameter estimation in Bayesian networks with missing values and hidden variables from the perspective of recent work in on-line learning [Kivinen & Warmuth, 1994]. We provide a unified framework for parameter estimation that encompasses both on-line learning, where the model is continuously adapted to new data cases as they arrive, and the more traditional batch learning, where a pre-accumulated set of samples is used in a one-time model selection process. In the batch case, our framework encompasses both the gradient projection algorithm and the EM algorithm for Bayesian networks. The framework also leads to new on-line and batch parameter update schemes, including a parameterized version of EM. We provide both empirical and theoretical results indicating that parameterized EM allows faster convergence to the maximum likelihood parameters than does standard EM.
8
Donald B. Rubin · Biometrika · 1976 · 9.7K citations
Information theory and statistics
Journal of the Franklin Institute · 1959 · 7.2K citations
Local learning in probabilistic networks with hidden variables
Stuart Russell, John Binder, Daphne Koller et al. · 1995 · 146 citations
Nicolò Cesa‐Bianchi, Yoav Freund, David P. Helmbold et al. · 1993 · 132 citations
Management, Knowledge Management, Decision Support Systems +8