Gaussian Mixture Clustering
The problem, from first principles
01-kmeans-assignment and 02-kmeans-centroid-update cluster by making a hard call: every point belongs to exactly one centroid, full stop. That's fine when clusters are compact and well-separated, but it throws away real information when a point sits genuinely between two clusters — K-Means still forces a 100%/0% split on it, when 60%/40% would describe reality better.
A Gaussian Mixture Model keeps the same "clusters have centers" intuition but replaces each hard centroid with a full Gaussian distribution — its own mean, its own variance, and a mixing weight saying how much of the overall data it accounts for. A point no longer gets assigned to one cluster; it gets a probability of having come from each one. That's the same probabilistic machinery 03-gaussian-naive-bayes used to classify a point against known class distributions — the difference here is there are no known classes to check against, the "responsibilities" a point gets are the answer.
From theory to code
Implement gmm_e_step(input, weights, means, variances), which computes each point's responsibility (posterior probability) under each component, and gmm_m_step(input, responsibilities), which re-estimates each component's weight, mean, and variance from those responsibilities. Theory below shows the E-step is Bayes' rule reused from 03-gaussian-naive-bayes, and the M-step is 02-kmeans-centroid-update's averaging generalized to fractional (soft) group membership instead of all-or-nothing membership.
Constraints
input: shape(n_samples, n_features).weights: shape(k,), mixing proportions, sum to 1.means: shape(k, n_features).variances: shape(k, n_features)— diagonal covariance only, one variance per feature per component, no cross-feature covariance terms.gmm_e_stepreturns shape(n_samples, k); every row must sum to exactly 1 (a genuine probability distribution over components).- Reuse
03-gaussian-naive-bayes'sgaussian_log_likelihood, don't recompute the Gaussian log-density by hand. gmm_m_stepreturns(weights, means, variances), the same shapes asgmm_e_step's inputs.- Work in log-space and normalize with the row-max-subtraction trick before exponentiating — avoid underflow/overflow from raw probabilities.
Hints
Open one at a time. Each gives away a little more than the last.
Hint 1
weight[j] * P(x_i | component j) is a product of tiny numbers. Working in log-space turns the product into log(weight[j]) + gaussian_log_likelihood(...), a sum — compute that for every (sample, component) pair before doing anything else.
Hint 2
You can't exponentiate raw log-probabilities safely — they can be very negative, and exp of a very negative number underflows, but the real danger is the opposite: unnormalized values could differ enough to overflow before you divide. Subtract each row's max log-value before calling np.exp, then normalize by each row's sum. This doesn't change the final ratios, only the numerical stability of computing them.
Hint 3
For the M-step, responsibilities[:, j].sum() tells you how many samples' worth of "weight" component j currently claims — call it effective_counts[j]. Every other update for component j (weight, mean, variance) divides by this same number. The mean is a responsibilities[:, j]-weighted sum of input, divided by effective_counts[j]; when every responsibility is exactly 0 or 1 this collapses to 02-kmeans-centroid-update's plain group average.
Keywords
No results yet
Run your code to check it against the examples, or submit to run every test.