Core 12 min 3 of 5

Photons, electrons, and why σ is not a constant

The noise is a line through variance and mean, and ISO changes its slope by shrinking the well.

What this repairs

Modelling image noise as additive Gaussian with one σ, so both the denoiser and the augmentation are calibrated to a distribution the sensor does not produce.

Assumes

Exposure is a decision, not a measurement

A pixel does not measure brightness. It counts electrons, released by arriving photons, until it is read out — and counting is Poisson, so the variance of the count equals its mean. That single fact makes the noise signal-dependent, and everything else in this concept follows from it.

To connect it to a normalised image, you need the well capacity $N_e$ : the number of electrons that a normalised signal of $1.0$ corresponds to. A signal $s$ then means $s N_e$ electrons, and the noise on it is the noise of that count.

From Poisson counting to the photon transfer curve

$k \sim \operatorname{Poisson}(\lambda), \qquad \lambda = s\,N_e, \qquad \operatorname{Var}(k) = \lambda$

The defining property of a counting process: mean and variance are the same number. Nothing about the sensor is in this line except the capacity.
$\hat{s} = \frac{k}{N_e} \;\Longrightarrow\; \operatorname{Var}(\hat{s}) = \frac{\lambda}{N_e^{2}} = \frac{s}{N_e}$

Dividing a random variable by a constant divides its variance by the square, so back in normalised units the shot-noise variance is linear in the signal, with slope 1/Nₑ.
$\operatorname{Var}(\hat{s}) = \frac{s}{N_e} + \left(\frac{r_e}{N_e}\right)^{2}$

Read noise is a fixed number of electrons added at readout, independent of the signal, so it contributes a constant variance — the intercept of the line.
$N_e = \frac{\text{full well}}{\text{ISO}/\text{ISO}_0}$

And the capacity itself depends on ISO. Raising ISO adds no electrons; it declares that fewer of them already count as white, so the well the signal is measured against shrinks.
$\text{SNR}(s) = \frac{s}{\sqrt{s/N_e}} = \sqrt{s\,N_e}$

The consequence worth carrying: signal-to-noise goes as the square root of the collected electrons, so four times the light buys two times the SNR — and half the well capacity costs a factor of √2.

Insight ISO is a divisor, not a gain

Take a 12 000 e⁻ well at base ISO 100 and shoot the same scene at ISO 800. The capacity becomes $12\,000 / 8 = 1500$ electrons. In normalised units:

shot-noise variance $s/N_e$ is eight times larger, so its $\sigma$ is $\sqrt{8} = 2.8$ times larger;
read noise $r_e / N_e$ is eight times larger as a $\sigma$ , because it is a fixed electron count against a smaller white point.

The two terms therefore move at different rates, which is why the photon transfer curve is a line with two independently meaningful parameters rather than one number. Fit variance against mean over a step wedge and the slope hands you the well capacity in electrons and the intercept hands you the read noise — neither of which appears anywhere in the image metadata.

Because the variance is proportional to the signal, absolute noise is largest in the highlights and relative noise is largest in the shadows: at signal $s$ the relative error is $1/\sqrt{s N_e}$ , which grows without bound as $s \to 0$ . This is the correct way to read a noisy image. The shadows are not noisy because the sensor is bad there; they are noisy because there are few photons there, and that is a property of the light, not of the electronics.

It is also why a single global $\sigma$ is such a poor model. Fit one to a real frame and you have averaged a shadow $\sigma$ and a highlight $\sigma$ that differ by a factor of several — then trained a denoiser to over-smooth the highlights and under-smooth the shadows, and an augmentation to add noise where the sensor would not have.

Pitfall Fixed-pattern noise does not average out

Shot and read noise are independent between frames, so averaging $N$ frames reduces them by $\sqrt{N}$ . Three of the sensor's noise sources do not behave that way at all:

PRNU — per-pixel gain variation, multiplicative and identical in every frame;
DSNU — per-pixel offset, additive and identical in every frame;
row and column banding — a bias drawn per row or column and smoothed over a correlation length of tens of pixels, so it is a structured, low-frequency pattern rather than a speckle.

Average a thousand frames and all three are exactly where they started. They are also the terms that break the i.i.d. assumption a denoiser is usually trained under: banding is correlated along a whole row, so it looks like scene structure to a model whose noise prior is per-pixel independence.

In practice Two thirds of every pixel is interpolated

Before any of this reaches an RGB array it passes through a colour filter array — one colour per photosite, in a Bayer pattern with twice as many green sites as red or blue. Demosaicing interpolates the two missing channels at every pixel, so two thirds of the values in an RGB image were never measured.

Two consequences. Noise stops being per-pixel independent, because neighbouring pixels share the samples they were interpolated from — which is a second, quite separate reason the i.i.d. Gaussian model is wrong. And fine coloured structure near the sampling limit produces zipper and fringe artefacts, which look like edges to an edge-sensitive model. The ADC's 12-bit quantisation sits in the same region of the chain and adds a floor of its own, below which no amount of averaging finds anything.

Check yourself

You lose a stop of light and let auto-exposure double the gain, which resolves a higher ISO. The image is as bright as before. What happened to the noise?

Show answer

It got worse, and by a computable amount. The brightness is restored, so the normalised signal $s$ is where it was; but the resolved ISO shrank $N_e$ , and every noise term is measured against that. If the ISO doubled, shot-noise $\sigma$ rose by $\sqrt{2}$ and read-noise $\sigma$ by $2$ , at identical brightness. This is the entire mechanism behind gloom in the fog topic: the picture is not much darker, and it is much noisier.