Classic machine learning
Gaussian mixture model
Imagine sorting apples from two orchards whose fruit sizes overlap. Instead of assigning each apple to one orchard, you give each origin a probability based on size. A Gaussian mixture model does this with bell-shaped components, and expectation-maximization (EM) learns their centers, spreads and proportions.
Component AComponent BMixture
How it works
- Choose the number of Gaussian components and initialize their means, covariances and mixing weights.
- In the E-step, compute each component’s probability of having generated each point, called its responsibility.
- In the M-step, use those responsibilities to update weighted means, covariances and mixing weights.
- Repeat until the log likelihood stops improving enough, and compare several initializations.
$$p(x)=\sum_{k=1}^{K}\pi_k\,\mathcal{N}(x;\mu_k,\Sigma_k)$$
The knobs that matter
- n_components
- more components fit finer groups but add parameters
- covariance_type
- full covariance allows tilted groups; diagonal uses fewer parameters
- reg_covar
- adds a small diagonal term to prevent singular covariances
- n_init
- more restarts reduce dependence on initialization
What interviewers ask
- How does a Gaussian mixture model cluster data differently from k-means?
- Can you walk me through the E-step and M-step?
- How would you choose the number of components?
- How would you use a mixture model to flag unusual transactions?
Common mistakes
- Treating responsibilities as hard labels throws away uncertainty.
- Without covariance regularization, a component can collapse onto a point and make the likelihood unbounded.
- EM can reach a local optimum, so one initialization is not enough to judge the fit.
When to use it
Use it for soft clustering or density estimation when a few Gaussian components reasonably describe the data. Avoid interpreting low density as proof of fraud, or assuming every component represents a real category.
Implement it (3 problems)
- Gaussian Mixture Model E-StepProMediumSolve