Core 11 min 4 of 5

Everything that is not linear any more

Tone, gamma, local contrast and sharpening run before anything you train sees the image.

What this repairs

Treating the ISP as a colour-space conversion, so tone mapping, local contrast and sharpening read as cosmetic rather than as an irreversible nonlinearity applied before the network sees anything.

Assumes

Scene-linear and sRGB

The ISP is where a camera stops measuring and starts deciding what the picture should look like. The order in euler-preprocess is fixed and worth memorising, because each stage's effect depends on the space the previous one left behind:

denoise, on the raw-ish signal;
colour matrix, sensor primaries to the working primaries;
tone map, compressing scene range into display range;
gamma, the sRGB encode;
local contrast, a large-radius unsharp on the encoded image;
sharpen, a small-radius unsharp;
saturation, a lerp toward luma.

Stages 1–3 act on light. Stages 5–7 act on codes, because gamma has already happened. That split is the single most useful thing to know about the chain: a sharpening halo is a fixed fraction of a code value, not of a radiance, so its apparent strength depends on where in the tone curve it landed.

Two nonlinearities, stacked

$y = \frac{x}{1 + \sigma x}, \qquad y' = \frac{1}{(1 + \sigma x)^{2}}$

The Reinhard tone map and its slope. Monotone, fixes zero, and asymptotes below 1 — which is what "compressing the highlights" means quantitatively. The shipped profile interpolates a 16-entry LUT instead; Reinhard is the one you can differentiate by hand, and it has the same shape.
$\frac{y'(1)}{y'(0)} = \frac{1}{(1 + \sigma)^{2}} = \tfrac{1}{4} \quad (\sigma = 1)$

A difference between two bright surfaces survives at a quarter of its original size, while the same difference in the shadows passes through untouched. Contrast has been moved, not removed.
$z = 1.055\,y^{1/2.4} - 0.055, \qquad \frac{dz}{dy} \propto y^{-0.583}$

Then the gamma encode, whose slope diverges as y goes to zero — so it expands exactly the shadow region the tone map left alone.
$\frac{dz}{dx} = \frac{dz}{dy}\cdot\frac{dy}{dx}$

The composed response is the product of the two slopes, and the two curves pull in opposite directions. That product is what a histogram of the delivered image is showing you — never the scene.

Pitfall The point stages are invertible; the neighbourhood stages are not

It is worth being precise about which part of "the ISP is irreversible" is true.

Reinhard, gamma, the colour matrix and the saturation lerp are point operations: the output at a pixel depends only on the input at that pixel, and each is monotone, so each has an inverse — up to clipping and up to the 8-bit quantisation that follows. Given the parameters, you can linearise an image and be close.

Denoise, local contrast and unsharp masking are neighbourhood operations. The output at a pixel depends on its surroundings, and no per-pixel inverse exists at all: you cannot undo a smoothing without knowing what was smoothed away. So "linearise before you measure" recovers the curve and never the structure, and any claim that a network can invert the ISP is a claim about the first group only.

Insight Unsharp masking creates edges that were not there

Sharpening subtracts a blurred copy and adds back the difference. Around any strong edge this produces an overshoot on the bright side and an undershoot on the dark side — a halo, whose width is the blur $\sigma$ used to make it. With a sharpen $\sigma$ of about a pixel, the halo is exactly the scale of the small structure a detector cares about, and it is a reversal of gradient sign right next to every real edge.

To an edge-sensitive model this is not a cosmetic difference. It is extra evidence, spatially locked to the true edges, which is present in training data that was sharpened and absent in data that was not. Local contrast is the same operator at a radius of tens of pixels, where the halo becomes a slow shading gradient around large objects instead.

Saturation closes the chain as a luma-preserving interpolation, $\mathbf{c}' = Y + \alpha(\mathbf{c} - Y)$ with $Y$ the luminance of the pixel. $\alpha < 1$ pulls colours toward grey and $\alpha > 1$ pushes them away, in both cases leaving $Y$ untouched — which is why saturation adjustments do not change a luminance histogram at all.

Two things follow. Pushing $\alpha$ above one can send a channel outside $[0, 1]$ on a colour that was in range, and the clip that catches it changes the hue, not just the saturation. And when the scene is already desaturated — fog being the obvious case — an $\alpha$ below one compounds with the physical desaturation of the veil, so the delivered image is less colourful than the fog model actually predicted.

Check yourself

A faint texture is clearly present in the render and absent from the delivered image. How do you find the stage that removed it?

Show answer

By asking at what level it vanished. If the texture sits in a bright region, suspect the tone map: its slope there may be a quarter of what it is in the shadows, so a difference that was several code values becomes less than one and the quantiser finishes the job. If it disappears at every level, suspect denoise — a smoothing removes low-amplitude high-frequency content regardless of where it sits. And if it survives in a PNG of the same render but not in the delivered JPEG, it was compression, which throws away high-frequency content per 8×8 block and does so most aggressively in the chroma channels.