Core 11 min 1 of 3

The epipolar constraint

A point in one image restricts its match in the other to a line — before anything is known about depth.

What this repairs

Memorising $x'^\top F x = 0$ without a picture of where the line comes from or why the epipole is special.

Assumes

The pinhole model · Projective Geometry & the Camera Model
Frames, and a notation that catches mistakes · Rigid Transforms & Camera Poses

Take a pixel in image $A$ . It back-projects to a ray in space. You do not know where along that ray the world point is — that is exactly what one view cannot tell you — but you know the point is somewhere on it.

Now project the whole ray into image $B$ . A line in space projects to a line in an image. So the match, wherever it is, lies on that line. That line is the epipolar line, and this is the entire content of the epipolar constraint. It is a statement about the ray, made before the depth is known, which is why it can be used to search for the match rather than to verify one.

From coplanarity to the essential matrix

$\mathbf{x}_B \;\propto\; R\,\mathbf{x}_A + \tfrac{1}{Z}\mathbf{t}$

The normalised direction in B, for a point at depth Z along the ray x_A in A. R and t are the relative pose ᴮT_A.
$\mathbf{x}_B,\; R\,\mathbf{x}_A,\; \mathbf{t} \;\text{ are coplanar}$

All three lie in the plane through the two camera centres and the world point — the epipolar plane. This is the geometric fact; everything below is algebra.
$\mathbf{x}_B^{\top}\,(\mathbf{t} \times R\,\mathbf{x}_A) = 0$

Coplanarity of three vectors is exactly the vanishing of their scalar triple product.
$\mathbf{x}_B^{\top}\,\underbrace{[\mathbf{t}]_\times R}_{E}\,\mathbf{x}_A = 0$

Write the cross product as a matrix. E is the essential matrix — it depends only on the relative pose, never on the scene.
$F = K_B^{-\top} E\, K_A^{-1}$

In pixels rather than normalised coordinates: the fundamental matrix. F needs no calibration, which is its whole reason to exist.

Insight The epipole is the other camera

All epipolar lines in image $B$ pass through one point: the image of camera $A$ 's centre. That is forced — every ray from $A$ starts at $A$ 's centre, so every projected ray passes through the projection of that centre. It gives you a free diagnostic: if your computed epipolar lines do not meet at a single point, your $F$ does not have rank $2$ , and an $F$ that does not have rank $2$ is not a fundamental matrix.

When the second camera moves straight along its optical axis, the epipole sits in the middle of the image and the epipolar lines radiate outward — the "flying through a tunnel" pattern. When it moves sideways, the epipole is at infinity and the lines are parallel. Rectification is nothing more than warping both images so that this second case holds and the lines become image rows.

Check yourself

Why does the epipolar line have a finite start and end in practice, even though it is mathematically infinite?

Show answer

Because only part of the ray corresponds to physically possible depths. Points behind camera $A$ are excluded, and points beyond the scene's extent are not plausible. The visible segment is bounded by the projections of the minimum and maximum depth along the ray — which is exactly what a stereo matcher's disparity range is, expressed in the other language.