Optimal model inference for Bayesian mixture of experts
We present an algorithm for inferring the parameter and model structure of a mixture of experts model (MoE) based on the variational Bayesian (VB) framework. First in the VB framework, we show that the model parameter and structure of a MoE can be simultaneously optimized by maximizing an objective function derived in this paper. Next, we present a deterministic algorithm to find the optimal number of experts of an MoE while avoiding local maxima. Our experimental results demonstrate the practical usefulness of the method.