Unsupervised Learning
k-means and DBSCAN clustering, PCA and LDA, t-SNE/UMAP, and autoencoder-based anomaly detection.
17 questions
JuniorTheoryVery commonHow does k-means work step by step, and which quantity does each step decrease?
How does k-means work step by step, and which quantity does each step decrease?
k-means alternates two steps — assign every point to the nearest centroid, then move each centroid to the mean of its members. Both steps lower the same objective, the within-cluster sum of squared distances, so the run converges to a local optimum.
Common mistakes
- ✗Thinking k-means converges to the global optimum
- ✗Confusing the mean-based centroid update with a median-based one
- ✗Believing the two steps optimise different objectives
Follow-up questions
- →What stops the loop, and can the objective ever increase between iterations?
- →Why does the final partition depend on the starting centroids?
JuniorTheoryVery commonHow do you choose k for k-means, and why is the elbow so often ambiguous?
How do you choose k for k-means, and why is the elbow so often ambiguous?
Plot inertia against k and look for the bend, or score every k by silhouette and take the best. Inertia falls monotonically with k, so on smooth data no sharp bend exists and the elbow is read subjectively; silhouette gives a comparable number instead.
Common mistakes
- ✗Expecting inertia to bottom out at the right k: it keeps falling as k grows and reaches
0when every point is its own cluster, so you look for the bend, not the minimum. - ✗Expecting silhouette to rise with k the way inertia falls: cutting a real group in half puts its points near the neighboring cluster, so the score drops.
Follow-up questions
- →What does a silhouette value near zero say about a point?
- →How would a business constraint on the segment count change your choice of k?
JuniorTheoryCommonWhat happens when this program runs?
What happens when this program runs?
predict raises AttributeError: this class defines no predict method
Common mistakes
- ✗Expecting the new point to join the nearest cluster: the merge tree covers only the fitted points, so the class has no
predictat all; usefit_predictorlabels_. - ✗Expecting
NotFittedError: the model is fitted andlabels_is set; the call fails before any check, because the class defines nopredictmethod.
Follow-up questions
- →How would you assign new customers to segments found by agglomerative clustering?
- →Which clustering class in scikit-learn does have a
predictmethod, and what does it compare a new point with?
JuniorTheoryCommonIn the density-based algorithm DBSCAN, what do eps and min_samples control, and what is noise?
In the density-based algorithm DBSCAN, what do eps and min_samples control, and what is noise?
eps sets the radius of the neighbourhood and min_samples how many points must fall inside it for a point to count as core. Core points and their neighbours form clusters; a point that is neither core nor within eps of a core point is labelled noise.
Common mistakes
- ✗Swapping the meanings of eps and min_samples
- ✗Assuming every point is assigned to some cluster
- ✗Thinking noise is decided by distance to a centroid
Follow-up questions
- →How does raising eps change the number of clusters and the amount of noise?
- →What is a border point, and why can its assignment depend on processing order?
JuniorTheoryCommonWhat does this program print?
What does this program print?
16.0
Open full question →Common mistakes
- ✗Expecting
8.0, the sum of plain distances: inertia squares each distance to the centroid, so the four distances of2give4 * 2 ** 2 = 16.0. - ✗Expecting
4.0, an average per point:inertia_is a sum over all points, not a mean, so it grows with the number of points.
Follow-up questions
- →What would
inertia_be withn_clusters=4on the same four points? - →Why can inertia not be compared between two datasets of different sizes?
JuniorTheoryCommonWhat does this program print?
What does this program print?
0.5 6.0
Open full question →Common mistakes
- ✗Expecting
0.5 4.0: the centroid moves to the mean of its points3,4and11, which is6.0;4.0is their median. - ✗Expecting the centroids to move only part of the way: one k-means step puts each centroid exactly at the mean of its points, there is no step size.
Follow-up questions
- →Which centroid would the point
3join if the second centroid started at7.0instead of5.0? - →What should the update step do if no point is assigned to a centroid?
JuniorTheoryCommonWhat does this program print?
What does this program print?
(150, 2) (2, 4)
Open full question →Common mistakes
- ✗Expecting
components_of shape(4, 2): it holds one row per component and one column per original feature, so its shape is(n_components, n_features). - ✗Expecting
X_newto keep 4 columns:fit_transformreturns the projections, one column per kept component, so it has 2 columns.
Follow-up questions
- →How is one row of
X_newcomputed from a row ofXandcomponents_? - →What shape would
components_have withoutn_components?
MiddleTheoryCommonRound blobs, interleaved crescents, a nested taxonomy — pick a clustering method for each.
Round blobs, interleaved crescents, a nested taxonomy — pick a clustering method for each.
Round, similarly sized blobs suit k-means, matching its spherical assumption. Interleaved crescents need DBSCAN, which follows density and marks outliers as noise. A nested taxonomy calls for hierarchical clustering, whose dendrogram exposes the levels themselves.
Common mistakes
- ✗Forcing k-means onto non-convex shapes by raising k
- ✗Treating DBSCAN as universally applicable with a single eps
- ✗Choosing an algorithm by data size rather than cluster geometry
Follow-up questions
- →Which linkage would you use for the taxonomy, and why does the choice matter?
- →What breaks in DBSCAN when the crescents have very different densities?
MiddleTheoryCommonHow many principal components do you keep, and what does explained variance really tell you?
How many principal components do you keep, and what does explained variance really tell you?
Keep enough components for a target explained variance, or cut at the scree bend. But explained variance measures reconstruction of the inputs, not usefulness for the label — a low-variance direction may carry the signal, so tune the count by model metric.
Common mistakes
- ✗Treating 95 percent explained variance as a guarantee of downstream quality
- ✗Assuming variance and predictive power are monotonically related
- ✗Fitting PCA on the full dataset before the train and test split
Follow-up questions
- →Give a case where the discriminative direction has almost no variance.
- →Where in a cross-validation pipeline must the PCA fit actually live?
MiddleTheoryCommonHow does the supervised projection LDA differ from PCA, and when is its subspace better?
How does the supervised projection LDA differ from PCA, and when is its subspace better?
PCA is unsupervised and maximises total variance; LDA uses labels and maximises between-class scatter relative to within-class scatter. When the discriminative direction has low variance LDA wins, but it is capped at C-1 components and assumes similar class covariances.
Common mistakes
- ✗Believing LDA is unsupervised or that PCA uses labels
- ✗Forgetting the C-1 cap on the number of LDA components
- ✗Assuming supervision makes LDA unconditionally better
Follow-up questions
- →How many components can LDA return on a three-class problem?
- →What happens to LDA when one class has a far wider covariance than the others?
JuniorTheoryOccasionalWhich failure of random initialization does k-means++ fix, and what does it cost?
Which failure of random initialization does k-means++ fix, and what does it cost?
Random seeds can drop several centroids into one dense region and leave a real cluster unclaimed, so the run settles in a poor optimum. k-means++ samples each seed with probability proportional to squared distance from the nearest seed, costing extra passes.
Common mistakes
- ✗Expecting k-means++ to take the mutually farthest points: each seed is drawn at random, weighted by squared distance, so a far point is likely but not certain to be picked.
- ✗Expecting k-means++ to guarantee the global optimum: it only gives a better start, so the run can still stop in a local optimum, and
n_initrestarts still help.
Follow-up questions
- →Why does sampling proportional to squared distance beat picking the single farthest point?
- →Do you still need several restarts once k-means++ is enabled?
JuniorTheoryOccasionalWhat does the dimensionality-reduction method PCA do geometrically, and why scale first?
What does the dimensionality-reduction method PCA do geometrically, and why scale first?
PCA finds perpendicular directions of maximum variance and projects the data onto the leading ones. Because it maximizes raw variance, a feature measured in large units dominates the components, so features on different scales must be standardized before the fit.
Common mistakes
- ✗Thinking PCA keeps the original features with the highest variance: each component is a weighted sum of all the features, and its variance depends on their units.
- ✗Believing PCA uses the class labels: its
fitsees onlyXand looks for the directions of greatest variance, not of best class separation.
Follow-up questions
- →What happens to the components if one feature is in metres and another in millimetres?
- →Are the resulting components correlated with each other, and why?
MiddleTheoryOccasionalIn autoencoder anomaly detection, what is the score and how do you threshold it without labels?
In autoencoder anomaly detection, what is the score and how do you threshold it without labels?
The score is reconstruction error — the model trains on mostly normal data, and points it cannot rebuild score high. Without labels, put the threshold at a percentile of training error matching the alert budget, then tune it by how many alerts analysts can review.
Common mistakes
- ✗Training the autoencoder on data heavily contaminated with anomalies
- ✗Fixing the threshold by a rule of thumb instead of an alert budget
- ✗Believing labelled anomalies are required to produce a score
Follow-up questions
- →What happens to the score when the training data already contains many anomalies?
- →How would you re-check the threshold as normal traffic drifts over time?
MiddleTheoryOccasionalWhy does k-means fail on elongated or unevenly dense clusters — name the assumption.
Why does k-means fail on elongated or unevenly dense clusters — name the assumption.
Minimising squared Euclidean distance to a centroid assumes roughly spherical clusters of similar size and spread. Elongated or unevenly dense groups break that, so k-means cuts across them; a density-based method such as DBSCAN follows the true shape instead.
Common mistakes
- ✗Blaming scaling or the iteration count instead of the shape assumption
- ✗Assuming the correct k rescues k-means on non-spherical clusters
- ✗Not reaching for a density-based alternative when shapes are irregular
Follow-up questions
- →How does a Gaussian mixture model relax part of this assumption?
- →What does DBSCAN give up in exchange for handling arbitrary shapes?
MiddleTheoryOccasionalWhy can cluster sizes and gaps not be read off the nonlinear embeddings t-SNE or UMAP?
Why can cluster sizes and gaps not be read off the nonlinear embeddings t-SNE or UMAP?
Both preserve local neighbourhoods, not global geometry, so between-cluster distances, apparent sizes and empty gaps are artefacts of perplexity and layout. Treat the picture as qualitative only, and never feed the coordinates into a classifier as features.
Common mistakes
- ✗Interpreting cluster sizes or gap widths in the embedding
- ✗Using embedding coordinates as model features
- ✗Assuming tuned perplexity or averaged runs restore global geometry
Follow-up questions
- →Why can a t-SNE fit not be applied to new test points the way a PCA fit can?
- →What would you use instead when a compact numeric representation is genuinely needed?
SeniorDesignRareA retail app with 8M monthly buyers wants five to eight marketing segments the CRM team can act on and that stay recognisable quarter over quarter. You have two years of transactions, sessions and catalogue categories. Design the pipeline end to end — which features you build and how you handle the heavy right tail of spend, what you standardise and whether you reduce dimensionality first, which clustering algorithm you pick and how you settle on the number of segments, and how you convince a sceptical CRM lead that the segments are real rather than an artefact of the algorithm and that they hold up when you refit next quarter.
A retail app with 8M monthly buyers wants five to eight marketing segments the CRM team can act on and that stay recognisable quarter over quarter. You have two years of transactions, sessions and catalogue categories. Design the pipeline end to end — which features you build and how you handle the heavy right tail of spend, what you standardise and whether you reduce dimensionality first, which clustering algorithm you pick and how you settle on the number of segments, and how you convince a sceptical CRM lead that the segments are real rather than an artefact of the algorithm and that they hold up when you refit next quarter.
Build RFM-style features, log-transform skewed spend, standardise, then cluster with k-means and pick k by silhouette plus business usability. Show the segments are real by profiling them on held-out behaviour, and stable by refit agreement across time slices.
Common mistakes
- ✗Clustering unscaled features with a heavy-tailed spend column
- ✗Choosing k by the inertia minimum instead of a scored criterion plus usability
- ✗Confirming stability on the same data rather than across time slices
Follow-up questions
- →How would you keep segment identities aligned between two quarterly refits?
- →What would make you drop a mathematically clean segment as unusable for CRM?
