Explorer Variational Message Passing
Bayesian inference is now widely established as one of the pr inci al foundations for machine learning. In practice, exact inference is rarely possible, and so a variety of approximation techniques have been developed, one of the most widely used being a deter ministic framework called variational inference. In this paper we introduce Variational Me ssage Passing (VMP), a general purpose algorithm for applying variational inference to Bayesian N etworks. Like belief propagation, VMP proceeds by sending messages between nodes in the network an d updating posterior beliefs using local operations at each node. Each such update increase s a lower bound on the log evidence (unless already at a local maximum). In contrast to belief pr opagation, VMP can be applied to a very general class of conjugate-exponential models becaus it es a factorised variational approximation. Furthermore, by introducing additional variatio nal parameters, VMP can be applied to models containing non-conjugate distributions. The VMP fr amework also allows the lower bound to be evaluated, and this can be used both for model compariso n and for detection of convergence. Variational message passing has been implemented in the for m of a general purpose inference engine called VIBES (‘Variational Inference for BayEsian net workS’) which allows models to be specified graphically and then solved variationally withou t recourse to coding.