Probabilistic inference for computing optimal policies in MDPs
We investigate how the problem of planning in a stochastic environment can be translated into a problem of inference. Previous work on planning by probabilistic inference was limited in that a total time T has to be fixed and that the computed policy is not optimal w.r.t. expected rewards. The generative model we propose considers the total time T as a random variable and we show equivalence to maximizing the expected future return for arbitrary reward functions. Optimal policies are computed via Expectation-Maximization.