Core 9 min 3 of 3

From depth map to 3D

A depth map plus the wrong intrinsics produces a confident, wrong reconstruction.

What this repairs

Blaming a bent or skewed point cloud on the depth model when the depth values are fine and the camera parameters are not.

Assumes

Intrinsics: from metres to pixels · Projective Geometry & the Camera Model
Scale, shift, and the alignment protocol

Back-projection is the inverse of the pinhole equations. For pixel $(u, v)$ with depth $Z$ :

$X = \frac{(u - c_x)}{f_x}Z, \qquad Y = \frac{(v - c_y)}{f_y}Z, \qquad Z = Z.$

Notice the asymmetry. $Z$ passes through untouched; $X$ and $Y$ are computed from $Z$ and the intrinsics. So an error in $f_x$ does not scale the cloud — it scales two axes out of three, and the result is a scene stretched or squashed in depth relative to its width. Flat walls tilt. Right angles open or close. Circles become ellipses.

Check yourself

You back-project a depth map of a corridor and the two walls converge instead of staying parallel. Depth error or intrinsics error?

Show answer

Almost certainly intrinsics. Walls converging or diverging with distance is the signature of a wrong $f_x$ : the lateral extent $X = (u - c_x)Z/f_x$ is scaled wrongly, and the error grows with $Z$ , so the effect is a systematic taper. A depth error would have to be a very specific depth-dependent bias to imitate it. Test it directly: if scaling $f_x$ by a single constant makes the walls parallel, the depth map was right all along.