Howhy is a theoretical seminar I organize at Queen Mary University of London every once in a while. The blog posts summarize the material I discuss at the seminar.

The EM algorithm

General presentation and practical example

The expectation maximization (EM) algorithm is a general technique for finding maximum likelihood (or MAP) estimates for models with latent variables. Let $x$ be the observed data, $z$ the hidden variables, and $\theta$ the parameters; the goal is to maximize the log likelihood function: [Read More]

Variational inference with exponential families

Introduction and applicability to conditionally conjugate models

A central task when working with probabilistic models is the evaluation of the posterior distribution. It is often the case the posterior is intractable (integrals with no closed form analytical solutions; exponentially many discrete states), so approximation methods need to be employed. Broadly, these methods fall into two categories, stochastic... [Read More]