Ready when you are

The deck opens at chapter 1 and unlocks later chapters as you show you can hold the earlier ones. Older cards keep coming back on their own schedule, so nothing quietly rots.

Your way through the course

x teal bold = vector (always a column) S violet bold = matrix n plain italic = scalar

Keyboard: ↑/↓ or space flips a card · ← not known · → known · h hint

↑↓ flip · ← didn't know · → knew it · h hint

Focus mode

Tick any chapters and just drill them — pure random shuffle, forever, nothing scored. This never touches your Learn-mode progress or unlocks.

The whole deck

Every card, in teaching order, with its status. Use this to check coverage against your notes.

One dataset, the whole course

Every panel below uses the same five observations, so you can watch a data matrix turn into deviation vectors, a covariance matrix, an ellipse, principal components, a test statistic and a factor model. Numbers are chosen so the arithmetic stays exact — check any step by hand.

1The data matrix, and what a row and a column mean

Five students, two variables: X_1 = weekly study hours, X_2 = mock-exam grade. So n = 5, p = 2, and the data matrix \m{X} is 5 \times 2.

	X₁ hours	X₂ grade
A	3	3
B	5	5
C	6	5
D	7	7
E	9	5

A highlighted row is one observation \v{x}_1 = (3,3)\T — a single point in 2-dimensional variable space.

	X₁ hours	X₂ grade
A	3	3
B	5	5
C	6	5
D	7	7
E	9	5

A highlighted column is one variable as a vector in \mathbb{R}^5: \v{x}_{\cdot 1} = (3,5,6,7,9)\T. This is the picture used for deviation vectors and angles.

Two legitimate pictures of the same table. Rows live in \mathbb{R}^p (scatterplots, Mahalanobis distance, ellipses). Columns live in \mathbb{R}^n (projections, angles, correlation as a cosine).

2The mean is a projection; deviations are what is left

Split each column into a part along the ones-vector and a remainder orthogonal to it: \v{z} = a\one + \v{e}. Orthogonality forces a = \bar{z}, so the projection of a variable onto \span(\one) is \bar{z}\one.

\one\T(\v{x}_{\cdot 1} - a\one) = 0 \Rightarrow a = \frac{\one\T \v{x}_{\cdot 1}}{\one\T\one} = \frac{30}{5} = 6

column	raw	mean part	deviation \v{d}_j
X₁	(3,5,6,7,9)	6·(1,1,1,1,1)	(−3,−1,0,1,3)
X₂	(3,5,5,7,5)	5·(1,1,1,1,1)	(−2,0,0,2,0)

Each deviation column sums to zero — that single fact is why \rank(\m{D}) \le n-1.

mean vector\bar{\v{x}} = \mtx{6;5} lengths\|\v{d}_1\| = \sqrt{20}, \|\v{d}_2\| = \sqrt{8} inner product\v{d}_1\T\v{d}_2 = 6+0+0+2+0 = 8 angle\cos\theta = \frac{8}{\sqrt{20}\sqrt{8}} = 0.632 \Rightarrow \theta = 50.8° sor_{12} = 0.632 — the correlation is that cosine

Observation C sits exactly at the mean, so its deviation is the zero vector: it contributes nothing to \m{S}. An angle below 90° in \mathbb{R}^n is a positive correlation; exactly 90° would be r = 0.

3Deviation matrix → covariance → correlation → generalised variance

Stack the deviation columns into \m{D} = \m{X} - \one\bar{\v{x}}\T (5×2). Then one matrix product produces every variance and covariance at once.

\m{S} = \frac{1}{n-1}\m{D}\T\m{D} = \frac{1}{4}\mtx{20,8;8,8} = \mtx{5,2;2,2}

variancess_{11} = 5, s_{22} = 2 covariances_{12} = 2 scaling matrix\m{D}_\sigma = \diag(\sqrt{5},\sqrt{2}) correlation\m{R} = \m{D}_\sigma^{-1}\m{S}\m{D}_\sigma^{-1} = \mtx{1,0.632;0.632,1} generalised variance\det(\m{S}) = 5\cdot 2 - 2^2 = 6 inverse\m{S}^{-1} = \frac{1}{6}\mtx{2,-2;-2,5}

The two deviation vectors span a parallelogram of area A = \|\v{d}_1\|\|\v{d}_2\|\sin\theta = 9.80, and A^2/(n-1)^2 = 96/16 = 6 = \det(\m{S}).

Generalised variance is literally squared area (volume in 3D, hypervolume in pD) per (n-1)^p. Collapse the angle to 0° and the parallelogram — and \det(\m{S}) — vanishes: perfect collinearity.

4Why Euclidean distance lies, and Mahalanobis doesn't

Two candidate points, both measured from \bar{\v{x}} = (6,5)\T. Point E = (9,5) sits 3 units away along the high-variance, strongly correlated direction. Point Q = (6,7) sits only 2 units away, but straight up — a direction where the data barely spreads.

Dashed grey: Euclidean circle of radius 3. Dashed teal: the Mahalanobis contour through Q. E is on the circle yet inside the ellipse; Q is inside the circle yet on the ellipse.

d(\v{x},\bar{\v{x}})^2 = (\v{x}-\bar{\v{x}})\T\m{S}^{-1}(\v{x}-\bar{\v{x}})

	E = (9,5)	Q = (6,7)
deviation	(3, 0)	(0, 2)
Euclidean	3.00	2.00
quadratic form	⅙·2·3² = 3.00	⅙·5·2² = 3.33
Mahalanobis	1.73	1.83

The ranking flips. Moving 3 hours along a direction the data naturally varies in is less surprising than moving 2 grade points along a direction it doesn't.

\m{S}^{-1} does two jobs at once: it divides by the spread in each direction and it undoes the correlation tilt. If \m{S} were diagonal, Mahalanobis would collapse to plain standardised Euclidean distance — the tilt is exactly what the off-diagonal adds.

5Linear combinations: where the shapes go

Every transformation rule in the course is one of two lines. Read them as shape arithmetic and they stop being memorisation.

\E[\m{A}\v{X}] = \m{A}\v{\mu} \qquad \Var(\m{A}\v{X}) = \m{A}\m{\Sigma}\m{A}\T

shapes(q \times p)(p \times p)(p \times q) = (q \times q) scalar case\Var(\v{a}\T\v{X}) = \v{a}\T\m{\Sigma}\v{a}, a 1×1 number

With \m{\Sigma} = \mtx{5,2;2,2} and \m{A} = \mtx{1,1;1,-1}, i.e. total and difference:

\m{A}\m{\Sigma}\m{A}\T = \mtx{11,3;3,3}

Check by hand: \Var(X_1+X_2) = 5+2(2)+2 = 11; \Var(X_1-X_2) = 5-2(2)+2 = 3; \Cov(X_1+X_2, X_1-X_2) = 5 - 2 = 3. The matrix formula is doing all three simultaneously.

A quadratic form \v{a}\T\m{\Sigma}\v{a} is a variance, so it can never be negative: that is what "positive semi-definite" means in plain language, and it is why every eigenvalue of \m{\Sigma} is \ge 0.

6Eigenvectors turn the cloud; PCA reads off the new coordinates

Solve \m{S}\v{e} = \lambda\v{e} for \m{S} = \mtx{5,2;2,2}. The characteristic equation is \lambda^2 - 7\lambda + 6 = 0, so \lambda_1 = 6, \lambda_2 = 1 — and note \lambda_1 + \lambda_2 = 7 = \tr(\m{S}), \lambda_1\lambda_2 = 6 = \det(\m{S}).

The dotted feet are the projections onto the first principal axis. PC₁ keeps the long spread; PC₂ keeps what is left after that.

\v{e}_1 = \frac{1}{\sqrt{5}}\mtx{2;1},\quad \v{e}_2 = \frac{1}{\sqrt{5}}\mtx{-1;2}

Y_1 = \frac{2}{\sqrt 5}X_1 + \frac{1}{\sqrt 5}X_2,\quad Y_2 = -\frac{1}{\sqrt 5}X_1 + \frac{2}{\sqrt 5}X_2

	deviation	y₁ score	y₂ score
A	(−3,−2)	−3.578	−0.447
B	(−1, 0)	−0.894	+0.447
C	( 0, 0)	0.000	0.000
D	( 1, 2)	+1.789	+1.342
E	( 3, 0)	+2.683	−1.342

sample variance of y₁24.0 / 4 = 6 = \lambda_1 sample variance of y₂4.0 / 4 = 1 = \lambda_2 sample covariance0 — exactly, by construction explained by PC₁6/7 = 85.7% loading check\Corr(Y_1,X_1) = \sqrt{6/5}\,(0.894) = 0.98

PCA does not delete variance, it re-parcels it: \tr(\m{S}) = 5+2 = 7 = 6+1 = \lambda_1+\lambda_2. Keeping only PC₁ converts a 5×2 matrix into a 5×1 one and throws away 14.3% of the total spread. Rescale grade to a 0–100 scale instead of 0–10 and these eigenvectors change — that is what "PCA is not scale invariant" means.

7Conditioning: slicing the normal cloud

Treat the same numbers as a population: \v{X} \sim N_2(\v{\mu},\m{\Sigma}) with \v{\mu} = (6,5)\T and \m{\Sigma} = \mtx{5,2;2,2}. Someone tells you a student scored X_2 = 7. What do you now believe about their study hours?

\mu_{1|2} = \mu_1 + \Sigma_{12}\Sigma_{22}^{-1}(x_2-\mu_2) = 6 + 2 \cdot \tfrac{1}{2} \cdot (7-5) = 8

\Sigma_{1|2} = \Sigma_{11} - \Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21} = 5 - 2\cdot\tfrac12\cdot 2 = 3

beforeX_1 \sim N(6, 5) afterX_1 | X_2 = 7 \sim N(8, 3) meanmoved up, because the correlation is positive variancefell from 5 to 3 and does not depend on the value 7 if\Sigma_{12} = 0, both formulas collapse to the marginal — independence

Conditioning on a jointly normal partner is exactly linear regression: the slope is \Sigma_{12}\Sigma_{22}^{-1} = 1, and the leftover variance 3 is the residual variance. Knowing x_2 removed 2 units of the original 5.

8Testing a mean vector, and four flavours of interval

Test H_0: \v{\mu} = (5,4)\T with our sample (n=5, p=2, \bar{\v{x}} = (6,5)\T). The deviation is (1,1)\T.

T^2 = n(\bar{\v{x}}-\v{\mu}_0)\T\m{S}^{-1}(\bar{\v{x}}-\v{\mu}_0) = 5 \cdot \frac{1}{6}(2-2-2+5) = 2.5

critical value\frac{p(n-1)}{n-p}F_{2,3}(0.05) = \frac{8}{3}(9.55) = 25.5 decision2.5 < 25.5 → do not reject. With n=5 the test has almost no power; that is information, not a bug. same thing as a ratio\Lambda = (1 + T^2/(n-1))^{-n/2} = (1.625)^{-2.5} = 0.297

Now intervals for \mu_1 alone. Because \sqrt{s_{11}/n} = \sqrt{5/5} = 1, each half-width is its critical constant — the four methods can be compared directly.

Widest first. The T² intervals are valid for every linear combination \v{a}\T\v{\mu} at once; Bonferroni only for the two components you named; the t-interval only if you had asked one question; the asymptotic one only for large n — here it is honestly too narrow.

Coverage is a budget. Guard more statements simultaneously and you pay in width. The confidence ellipse itself is (\bar{\v{x}}-\v{\mu})\T\m{S}^{-1}(\bar{\v{x}}-\v{\mu}) \le 5.09, with semi-axes \sqrt{6 \cdot 5.09} = 5.53 along \v{e}_1 and \sqrt{1 \cdot 5.09} = 2.26 along \v{e}_2.

9A factor model, assembled from parts

Four exam-anxiety items, one latent factor (p = 4, m = 1), standardised so \sigma_{ii} = 1. Loadings \m{L} = (0.9, 0.8, 0.7, 0.6)\T.

\m{\Sigma} = \m{L}\m{L}\T + \m{\Psi}

LLᵀ	1	2	3	4
1	.81	.72	.63	.54
2	.72	.64	.56	.48
3	.63	.56	.49	.42
4	.54	.48	.42	.36

Ψ	1	2	3	4
1	.19	0	0	0
2	0	.36	0	0
3	0	0	.51	0
4	0	0	0	.64

Σ	1	2	3	4
1	1	.72	.63	.54
2	.72	1	.56	.48
3	.63	.56	1	.42
4	.54	.48	.42	1

communality of item 1h_1^2 = 0.9^2 = 0.81 — 81% of its variance is common specific variance\psi_{11} = 1 - 0.81 = 0.19 implied covariance\Cov(X_1,X_2) = 0.9 \times 0.8 = 0.72 loadings are covariances\Cov(\v{X},\v{F}) = \m{L}, so 0.9 is the correlation of item 1 with the factor rotationmultiply \m{L} by any orthogonal \m{T}: \m{L}^*\m{L}^{*\text{T}} = \m{L}\m{L}\T, so \m{\Sigma} cannot tell them apart

Because \m{\Psi} is diagonal, every off-diagonal number in \m{\Sigma} comes from the loadings alone. That is the whole idea: one unobserved factor explains all 6 correlations among 4 items using 4 numbers. Compare with PCA, which is a rotation of the observed variables, not a model with an error term.

Same five observations throughout: A(3,3) B(5,5) C(6,5) D(7,7) E(9,5).