General bounds on the mutual information between a parameter and n conditionally independent observations
Each parameter in an abstract parameter space is associated with a di erent probability distribution on a set Y A parameter is chosen at random from ac cording to some a priori distribution on and n condi tionally independent random variables Y n Y Yn are observed with common distribution determined by We obtain bounds on the mutual information be tween the random variable giving the choice of pa rameter and the random variable Y n giving the se quence of observations We also bound the supremum of the mutual information over choices of the prior dis tribution on These quantities have applications in density estimation computational learning theory uni versal coding hypothesis testing and portfolio selection theory The bounds are given in terms of the metric and information dimensions of the parameter space with respect to the Hellinger distance