Foundation 8 min 1 of 3

A covariance is a shape

Eigenvectors are the directions; square-rooted eigenvalues are the extents. Correlation is a tilt.

What this repairs

Reading a covariance as "the error in x and the error in y", which discards the off-diagonal information that is usually the interesting part.

For a Gaussian $\mathcal{N}(\boldsymbol{\mu}, \Sigma)$ , the set of points at a fixed probability density is

$(\mathbf{x} - \boldsymbol{\mu})^\top \Sigma^{-1} (\mathbf{x} - \boldsymbol{\mu}) = c,$

an ellipse in 2D and an ellipsoid in 3D. Diagonalise $\Sigma$ and the picture becomes concrete: the eigenvectors are the axis directions, and the semi-axis lengths are $\sqrt{\lambda_i}$ scaled by however many sigmas you chose to draw.

So $\sigma_x$ and $\sigma_y$ — the diagonal entries' square roots — are the bounding box of the ellipse, not the ellipse. A strongly correlated estimate has a long thin ellipse tilted away from the axes, and reporting only the diagonal describes a large box containing a thin sliver. That is a very different claim.

Insight The inverse is more informative than the covariance

$\Lambda = \Sigma^{-1}$ is the information matrix, and it is the more natural object in estimation. Information from independent sources adds: $\Lambda_{\text{post}} = \Lambda_{\text{prior}} + \Lambda_{\text{measurement}}$ . Zero information is $\Lambda = 0$ , which is a perfectly representable state of total ignorance; the corresponding covariance is infinite and is not.

There is a second, sharper distinction. A zero in $\Sigma$ means two variables are uncorrelated; a zero in $\Lambda$ means they are conditionally independent given everything else. The second is the structural property, it is what makes a SLAM problem sparse, and it is invisible in the covariance — where the same problem is dense.

Two operations on Gaussians look similar and behave oppositely, and confusing them is common:

Marginalisation — integrating a variable out. In covariance form: delete the rows and columns. Cheap. In information form: a Schur complement, which creates fill-in.
Conditioning — fixing a variable to an observed value. In information form: delete rows and columns. Cheap. In covariance form: an expensive update.

Each form makes one operation trivial and the other awkward. Which representation a system uses is therefore a statement about which operation it performs most.

Check yourself

A 2D position estimate has $\sigma_x = \sigma_y = 1$ m and correlation $0.99$ . Sketch the ellipse.

Show answer

A very long, very thin ellipse along the $45°$ diagonal. The eigenvalues are $1 + 0.99 = 1.99$ and $1 - 0.99 = 0.01$ , so the semi-axes are $\approx 1.41$ and $\approx 0.1$ — a fourteen-to-one ratio. The estimate knows the difference $x - y$ to $10$ cm and the sum to $1.4$ m. Reporting "1 m in each axis" describes a square that is almost entirely empty.