Variational Cumulant Expansions for Intractable Distributions Pi Erre Van De Laar
Intractable distributions are a common diculty in the probabilistic representation of knowledge and variational methods have recently been popular in providing an approximate solution. In this article, we describe a perturbational approach in the form of a cumulant expansion which, to lowest order, recovers the standard Kullback-Leibler variational bound. Higher order terms describe corrections on the variational approach without incurring much further computational cost. The relationship to other perturbational approaches such as TAP is also elucidated. We demonstrate the method on a particular class of undirected graphical models, Boltzmann machines, for which our simulation results connrm improved accuracy and enhanced stability during learning.