The expectation maximization (EM) algorithm is a general technique for finding maximum likelihood (or MAP) estimates for models with latent variables. Let $x$ be the observed data, $z$ the hidden variables, and $\theta$ the parameters; the goal is to maximize the log likelihood function:
[Read More]
Howhy is a theoretical seminar I organize at Queen Mary University of London every once in a while. The blog posts summarize the material I discuss at the seminar.
A nonparametric mixture based on a stick-breaking process
Overview and inference using variational methods
Mixtures can be found in a large number of models, e.g., clustering and classification models, models of annotation, models with hierarchical structures, topic models, and many others. In many of these cases, the number of mixture components is unknown; choosing a small number of clusters means potentially distintive groups will...
[Read More]
Variational inference with exponential families
Introduction and applicability to conditionally conjugate models
A central task when working with probabilistic models is the evaluation of the posterior distribution. It is often the case the posterior is intractable (integrals with no closed form analytical solutions; exponentially many discrete states), so approximation methods need to be employed. Broadly, these methods fall into two categories, stochastic...
[Read More]