Core 8 min 4 of 4

Lens distortion

The pinhole model is a lie about real lenses, and the correction is not invertible in closed form.

What this repairs

Assuming distortion is a small cosmetic effect, and that "undistort" and "distort" are the same operation run backwards.

Assumes

Intrinsics: from metres to pixels

A pinhole has no lens, and therefore no distortion, and also almost no light. Real cameras use lenses, and real lenses bend rays by an amount that depends on where they hit the glass. The standard model is a polynomial correction applied to the normalised coordinates, before $K$ :

$\begin{pmatrix} x_d \\ y_d \end{pmatrix} = (1 + k_1 r^2 + k_2 r^4 + k_3 r^6) \begin{pmatrix} x_n \\ y_n \end{pmatrix} + \begin{pmatrix} 2p_1 x_n y_n + p_2(r^2 + 2x_n^2) \\ p_1(r^2 + 2y_n^2) + 2p_2 x_n y_n \end{pmatrix},$

with $r^2 = x_n^2 + y_n^2$ . The first term is radial — the dominant effect, symmetric about the principal point. The second is tangential, and models a lens whose axis is not quite perpendicular to the sensor.

Pitfall Undistortion has no closed form

The model maps ideal $\to$ distorted. Rectifying an image needs the inverse, and a polynomial of this shape is not invertible analytically. Libraries solve it iteratively (a fixed-point or Newton iteration per pixel), and the iteration is only guaranteed to converge for moderate coefficients. Fisheye lenses break it outright, which is why they get their own model — $\theta_d = \theta(1 + k_1\theta^2 + \dots)$ , in terms of the incidence angle rather than the radius, because at $180°$ field of view the radius is not a function of the angle any more.

The practical consequence: never assume that "distort then undistort" returns your original point to machine precision, and never feed distorted pixels to a function expecting the pinhole model. A $2$ -pixel distortion residual at the image edge is enough to bias a calibration by more than the reprojection error will admit.

Check yourself

Your reprojection error is 0.3 px in the image centre and 2.5 px at the corners after calibration. What is the likely cause?

Show answer

An under-parameterised distortion model. A radially growing residual is the signature of unmodelled radial distortion — you probably fitted $k_1$ only, on a lens that needs $k_2$ as well, or you fitted a pinhole model to a wide lens. Adding parameters blindly is not the answer either: with few observations near the edge, a high-order model will fit noise and produce wild extrapolation outside the calibrated region.