Advanced 11 min 4 of 4

Why 25 steps and not 1000

Training defines a loss over a thousand timesteps. Inference visits twenty-five of them, and which twenty-five is a config key with consequences.

What this repairs

Confusing the training schedule with the inference schedule, and assuming more steps is strictly better.

Assumes

The thousand timesteps are a property of the loss, not of inference. Nothing in the derivation of the objective says a sampler has to visit all of them, and the shipped model visits twenty-five.

What licenses that is a change of sampler. DDPM's ancestral step draws fresh noise at every timestep, so the chain is a genuine stochastic process and skipping links in it changes the distribution being sampled. DDIM removes the noise draw, and what is left is a deterministic map from $x_t$ to $x_{t'}$ that happens to agree with the DDPM marginals — a map you can compose in any strides you like.

The DDIM step, and why a stride is exact

$\hat{x}_0 = c\,x_t - s\,\hat{v}, \qquad \hat{\varepsilon} = s\,x_t + c\,\hat{v}$

Start by converting whatever the network emitted into both endpoints, using the rotation from the previous concept. Every sampler in this family begins here, whatever its prediction type.
$\sigma_t = \eta\,\sqrt{\frac{1-\bar{\alpha}_{t'}}{1-\bar{\alpha}_t}}\,\sqrt{1 - \frac{\bar{\alpha}_t}{\bar{\alpha}_{t'}}}$

The noise budget for this jump, as a fraction η of the DDPM posterior standard deviation. Setting η = 1 reproduces the ancestral sampler exactly; η = 0 is DDIM proper.
$x_{t'} = \sqrt{\bar{\alpha}_{t'}}\,\hat{x}_0 \;+\; \sqrt{1 - \bar{\alpha}_{t'} - \sigma_t^{2}}\,\hat{\varepsilon} \;+\; \sigma_t z$

The step. Read the three terms as a budget: the next timestep needs total variance 1 − ᾱ_{t’}, σ² of it is spent on fresh randomness, and whatever is left is filled by pointing along the estimated noise direction rather than by drawing.
$\eta = 0,\ \hat{\varepsilon} = \varepsilon \;\Longrightarrow\; x_{t'} = \sqrt{\bar{\alpha}_{t'}}\,x_0 + \sqrt{1-\bar{\alpha}_{t'}}\,\varepsilon$

And that is the closed form of the forward process at t’, exactly. With a perfect network the step lands on the right point for a stride of any length — nothing about skipping is approximate, which is what makes 25 steps a defensible number rather than a shortcut.

Insight The stride is not what costs you

Since a DDIM step with a perfect $\hat{\varepsilon}$ is exact at any stride, the error in a real sampler comes entirely from $\hat{\varepsilon}$ being wrong — and from that error compounding along a trajectory that curves.

The mechanism: $\hat{x}_0$ is re-estimated at every step and it changes between steps, most in the middle of the chain where the model is still deciding what it is drawing. A long stride pins $\hat{x}_0$ across an interval over which it would have moved, which is a first-order integration error on a curved path. So the returns from more steps are large while the trajectory is still bending and then stop, which is why sampler step-count curves flatten rather than improving forever.

The corollary is the one people resist: more steps is not strictly better. Past the flat part you are paying linearly for nothing, and with a stochastic sampler ( $\eta > 0$ ) you are also injecting more noise draws into the same trajectory, which is a different result and not automatically a better one.

Pitfall The last training step still has the image in it

Here is a real defect in the textbook schedule, with a real fix that ships in the model this arc is built on.

At $t = 999$ the schedule leaves $\bar{\alpha} = 0.00466$ , so $\sqrt{\bar{\alpha}} = 0.0683$ : the noisiest training input a model ever sees still carries $6.8\%$ of the image by amplitude. Inference starts from pure noise, which carries none. The model is therefore first asked to work on a kind of input it was never trained on — and its first step is the one that decides what is being generated at all.

The symptom is well documented: models trained this way struggle to produce very dark or very bright images, because the residual signal at the last training step always carried the average brightness of the training image, and at inference there is nothing to carry it.

Both schedulers in sd2_joint.py are constructed with rescale_betas_zero_snr=True, which fixes it in closed form.

The rescale, and the spacing that has to go with it

$\sqrt{\bar{\alpha}_{T-1}} = 0.0683 \quad\Longrightarrow\quad \mathrm{SNR}(T-1) = 0.00468 \neq 0$

The defect, as a number. The terminal signal-to-noise ratio is small but not zero, and inference assumes it is zero.
$\sqrt{\bar{\alpha}'_t} = \left(\sqrt{\bar{\alpha}_t} - \sqrt{\bar{\alpha}_{T-1}}\right)\cdot\frac{\sqrt{\bar{\alpha}_0}}{\sqrt{\bar{\alpha}_0} - \sqrt{\bar{\alpha}_{T-1}}}$

The whole fix: a two-point affine map on √ᾱ. Shift the tail to zero, then stretch what is left so that √ᾱ₀ comes back to where it was. Nothing is retrained and nothing is approximated — it is a rescaling of a lookup table.
$\bar{\alpha}'_{T-1} = 0 \quad\Longrightarrow\quad x_{T-1} = \varepsilon \quad \text{exactly}$

Now the last training input really is pure noise, so training and inference agree about what the first sampler step is looking at. This is also the timestep at which ε-prediction went degenerate in the previous concept — the rescale is why that topic mattered.
$\sqrt{\bar{\alpha}'_{961}} = 0.0183 \neq 0$

And this is why the spacing is not a free choice. The diffusers default, "leading", starts a 25-step sampler at t = 961 for this model — 960 from the stride, plus the steps_offset: 1 the shipped scheduler_config.json carries — where the rescaled schedule still expects 1.8 % signal. A sampler fed pure noise there reintroduces exactly the mismatch the rescale was added to remove. Without the offset the timestep is 960 and √ᾱ′ is 0.0188; the argument is the same number to two figures either way, but 960 is not what the library prints for this configuration.

Convention Three spacings, each dropping a different end

Twenty-five timesteps have to be chosen out of a thousand, and the three available rules are not interchangeable:

spacing	first	last	drops
leading (the library default)	961	1	the noisiest end
trailing (what this model selects)	999	39	the clean end
linspace	999	0	neither

trailing is the one that pairs with the terminal-SNR rescale, because the rescale's whole purpose is to make $t = 999$ correct and leading never visits it. What trailing gives up instead is the clean end: it stops at $t = 39$ and takes the final $\hat{x}_0$ directly, which is a smaller sin because the sampler's estimate of $x_0$ is at its best there.

The leading row is $961$ and not $960$ for a reason worth having once: this model's shipped scheduler_config.json sets steps_offset: 1, and DDIMScheduler adds it to every leading timestep. The stride gives $960$ and the offset makes it $961$ . Nothing in the argument turns on the extra step — leading still never reaches $t = 999$ , which is the entire objection to it — but a figure or a notebook that prints $960$ beside a real diffusers run for this model is printing a number the library does not produce.

One implementation trap, since it is invisible by construction: the strided indices are rounded with numpy's half-to-even rule, not JavaScript's round-half-up. At 16 trailing steps the raw sequence contains $812.5$ , which numpy sends to $812$ and a naive port sends to $813$ — a one-timestep drift that no figure and no eyeball would ever reveal.

The shipped numbers are 25 steps for the dehazing branch and 20 for depth, from the training config's validation block.

It is worth being honest about what those numbers are. Nothing in the configuration says why, and neither is derived from anything on this page: they are the result of somebody sweeping the step count and finding the knee. The useful thing is knowing what to measure to defend or move them — run the same input at 5, 10, 25 and 50 steps and look at where the outputs stop differing from each other, not at where a metric stops improving, because a per-pixel metric against a single reference will keep improving past the point where the samples have converged.

That the depth branch runs fewer steps than the RGB branch is the kind of asymmetry worth noticing and not worth explaining away. The plausible reading is that a depth field is a smoother target with less trajectory curvature, so it tolerates a coarser stride — but that is a hypothesis about the model, and the config records a decision rather than a reason.

Check yourself

Someone reports that raising a sampler from 25 to 100 steps made their outputs worse. Is that possible, or did they make a mistake?

Show answer

Entirely possible, and there are three separate mechanisms to check before calling it a mistake.

The trajectory changed. More steps is a different discretisation of the same ODE, so with $\eta = 0$ you land somewhere else — near the old answer, not on it. "Worse" may mean "different, and they preferred the old one".

More noise went in. If $\eta > 0$ then step count and injected randomness are the same knob, and 100 steps means 100 draws instead of 25.

The spacing moved. Most libraries recompute the subsequence from the step count, so a schedule that started at $t = 961$ with 25 leading steps starts at $t = 991$ with 100 — both including this model's steps_offset: 1 — which, on a zero-terminal-SNR schedule, changes how much mismatch the first step is carrying. The step count and the spacing are not independent, even though they are two arguments.