On Gaussian Expectation Propagation
In this short note we will re-derive the Gaussian expectation propagation (Gaussian EP) algorithm as presented in Minka (2001) and demonstrate an application of Gaussian EP to approximate multi-dimensional truncated Gaussians. 1 On Gaussian Distributions Here we will summarise some important equalities about the Gaussian density. A Gaussian density in Rn is defined by N (x;μ,6) := (2π)− 2 |6|− 1 2 exp ( − 2 (x− μ)T6−1 (x− μ) ) . (1.1) We will write x ∼ N (x;μ,6) to both denote x has a distribution P(x) and that the density of this distribution is given by (1.1). We will write N (x) as a shorthand for N (x; 0, I). For t ∈ R, we will denote the cumulative Gaussian distribution function by 8(t;μ, σ 2) which is defined by 8 ( t;μ, σ 2 ) = Px∼N (x;μ,σ 2) (x ≤ t) = ∫ t −∞ N ( x;μ, σ 2 ) dx . (1.2) Again, we write 8(t) as a shorthand for 8(t; 0, 1). We will write 〈 f (x)〉x∼P to denote the expectation of f over the random draw of x , that is 〈 f (x)〉x∼P := ∫ f (x)d P(x). The following results are given without proof; for a detailed derivation the reader is referred to Herbrich (2002). Linear transformation x ∼ N (x;μ,6) and y = Ax+ b⇒ y ∼ N ( y;Aμ+ b,A6AT ) . Convolutions Assume x ∼ N (x;μ,6) and y|x ∼ N (y;Ax,0) . Then x|y ∼ N ( x;9 ( AT0−1y+6−1μ ) ,9 ) , (1.3) y ∼ N ( x;Aμ,0 + A6AT ) , (1.4) where 9 := (AT0−1A+6−1)−1. Marginals Let us assume that a random vector x is composed such that x ∼ N ([ x1 x2 ] ; [ μ1 μ2 ] , [ 611 612 612 622 ]) .