Journal of Machine Learning Research · 2017 · 361 citations · 0 references
Natural SciencesMathematicsMachine LearningProbability TheoryStochastic ProcessesBayesian InferenceBayesian LearningBayesian StatisticsMarkov Chain Monte CarloProbabilistic Machine LearningStatistical InferenceApproximate Bayesian InferenceEm AlgorithmGradient DescentKullback-leibler DivergenceLangevin DynamicsPosterior InferenceStochastic Gradient DescentVariational Em Algorithm
Stochastic Gradient Descent with a constant learning rate (constant SGD) simulates a Markov chain with a stationary distribution. With this perspective, we derive several new results. (1) We show t...