Core 11 min 3 of 4

ε, x₀ or v

Three exact coordinate systems for one quantity. Choosing one chooses a loss weighting across the schedule, and nothing else.

What this repairs

Treating the prediction target as an implementation detail, when it changes the loss weighting across timesteps and is why SD2.1 is not SD1.5.

Assumes

Three quantities are on offer as the thing a network should output: the noise $\varepsilon$ , the clean image $x_0$ , or the velocity $v$ . They are usually introduced as three ideas that happen to be related, which makes the choice between them look empirical.

It is not. Given $x_t$ and $t$ , any one of the three determines the other two exactly, with no approximation and no information lost. They are one quantity written in three coordinate systems, and the coordinate change is a rotation.

The rotation

$c = \sqrt{\bar{\alpha}_t}, \qquad s = \sqrt{1-\bar{\alpha}_t}, \qquad c^2 + s^2 = 1$

The variance-preserving identity from the first concept, given names. Two non-negative numbers whose squares sum to one are a cosine and a sine, so there is an angle here whether or not anyone mentions it.
$x_t = c\,x_0 + s\,\varepsilon, \qquad v \;\overset{\text{def}}{=}\; c\,\varepsilon - s\,x_0$

The forward process, and the definition of velocity. The second line is not derived from anything — it is a definition, chosen so that the pair below comes out orthogonal.
$\begin{pmatrix} x_t \\ v \end{pmatrix} = \begin{pmatrix} c & s \\ -s & c \end{pmatrix} \begin{pmatrix} x_0 \\ \varepsilon \end{pmatrix}$

Stack the two lines. Because c² + s² = 1 this matrix is a rotation — orthogonal, determinant one — which means it preserves lengths and is invertible everywhere on the chain.
$\begin{pmatrix} x_0 \\ \varepsilon \end{pmatrix} = \begin{pmatrix} c & -s \\ s & c \end{pmatrix} \begin{pmatrix} x_t \\ v \end{pmatrix}$

A rotation inverts by transposing. So x₀ = c·x_t − s·v and ε = s·x_t + c·v: two multiply-adds, no division, and no timestep at which either is ill-conditioned.
$\hat{x}_0 = \frac{x_t - s\,\hat{\varepsilon}}{c}, \qquad c = 0.0683 \text{ at } t = 999 \text{ (textbook)}$

The same recovery from an ε estimate instead, and the contrast is the whole practical argument. Going through ε divides by √ᾱ, which on the textbook curve is 0.068 at the end of the chain, so whatever error the network made comes out fifteen times larger. On the rescaled curve the model actually ships, √ᾱ at t = 999 is exactly zero and the division does not exist at all — which the pitfall below is about.

Insight The v-loss is the ε-loss plus the x₀-loss, exactly

Errors transform through the same rotation as the quantities do. Hold $x_t$ fixed and perturb the prediction, giving $\delta_{x_0} = -s\,\delta_v$ and $\delta_\varepsilon = c\,\delta_v$ . Measure all three losses in the same units — squared $\varepsilon$ -error — and the weights are:

$w_\varepsilon = 1, \qquad w_{x_0} = \frac{s^2}{c^2} = \frac{1}{\mathrm{SNR}}, \qquad w_v = \frac{1}{c^2} = \frac{1}{\bar{\alpha}_t}.$

And since $1/\bar{\alpha} = 1 + (1-\bar{\alpha})/\bar{\alpha}$ , the third is the sum of the first two — not approximately, identically, and here evaluated on the textbook curve:

$t$	SNR	$w_\varepsilon$	$w_{x_0}$	$w_v$
0	1175	1	0.0009	1.0009
354	0.999	1	1.0007	2.0007
999	0.0047	1	213.59	214.59

The identity is a property of the algebra, not of the curve, so it holds on the shipped one too — where the equal-weighting row is $t = 337$ instead, and the last row diverges rather than reaching $214.59$ , because $\bar{\alpha}$ there is exactly zero and $1/\bar{\alpha}$ is not a number. That divergence is the pitfall two sections down, seen from the loss side.

Read the table as the answer to "which target should I predict?". Choosing $\varepsilon$ weights every timestep equally in $\varepsilon$ -space and therefore almost ignores the low-noise end in image terms. Choosing $x_0$ does the opposite, by a factor of a quarter of a million across the chain. Choosing $v$ takes whichever of the two is larger at each $t$ , which is why its loss is conditioned across the schedule and the other two are not.

Why v is never approximately anything

$v = c\,\varepsilon - s\,x_0$

The definition again, to be read as a sum of two terms rather than as an approximation of either. Both terms are always present; only their coefficients move.
$\bar{\alpha} \to 1 \;\Longrightarrow\; c \to 1,\ s \to 0 \;\Longrightarrow\; v \to \varepsilon$

At the clean end the sine vanishes and v is the noise exactly. This is the regime where predicting the image is nearly hopeless, because there is almost none of it missing to predict.
$\bar{\alpha} \to 0 \;\Longrightarrow\; c \to 0,\ s \to 1 \;\Longrightarrow\; v \to -x_0$

At the noisy end the verdict flips exactly: v is the negated image. This is the regime where predicting the noise is hopeless, because the input has become the noise, and copying the input scores perfectly.
$\bar{\alpha} = \tfrac{1}{2} \;\Longrightarrow\; v = \tfrac{1}{\sqrt{2}}\left(\varepsilon - x_0\right)$

And the crossover, where the two are weighted equally, is exactly ᾱ = ½ — which is SNR = 1, which is t = 354 on the textbook curve and t = 337 on the rescaled one the model trains on. The one number in this topic a reader will not believe until they compute it turns out to be the hinge of the target choice as well, on whichever curve they compute it.

Pitfall A zero-terminal-SNR schedule makes ε-prediction degenerate

The model in this codebase builds both of its schedulers with rescale_betas_zero_snr=True, which pins $\bar{\alpha}$ at the last timestep to exactly zero rather than the textbook $0.00466$ . The next concept is about why. The consequence for this concept is immediate and easy to miss.

At $\bar{\alpha} = 0$ the forward process reads $x_t = 0 \cdot x_0 + 1 \cdot \varepsilon$ , so $x_t$ is $\varepsilon$ and nothing of the image survives. A network asked to predict $\varepsilon$ there is being asked to copy its own input: a task with a perfect solution, no gradient and no information in it. Every training step spent at that timestep teaches nothing.

Predicting $v$ at the same timestep asks for $-x_0$ — the whole image, from pure noise — which is exactly the question the first sampler step needs answered. So the terminal-SNR rescale and the $v$ target are not two independent configuration choices that happen to co-occur in sd2_joint.py; the first one requires the second. A pipeline with the rescale and an $\varepsilon$ target has a dead timestep at the exact place it starts sampling.

This is the concrete difference between SD1.5 and SD2.1. Both are latent diffusion models with nearly the same UNet; SD1.5 was trained against $\varepsilon$ and SD2.1 against $v$ , and the joint model here inherits the latter — prediction_type: "v_prediction" is the class default and what the shipped config asks for.

The practical consequence is unpleasant, and it is why the field treats this as a convention rather than a detail: a checkpoint's prediction type is not recoverable from its weights. Load $v$ -trained weights into a pipeline configured for $\varepsilon$ and nothing raises. The sampler computes $\hat{x}_0 = (x_t - s\,\hat{\varepsilon})/c$ on an output that is really $v$ , and what it gets is

$\hat{x}_0 = 15.58\,x_0 + 13.62\,\varepsilon \quad \text{at } t = 999 \text{ on the textbook curve,}$

falling to $2.37\,x_0 + 0.76\,\varepsilon$ at $t = 499$ and to $x_0$ itself by $t \approx 0$ . The error is largest at the start of sampling and vanishes at the end, so the output is not noise — it is a structured, over-saturated, clipped image that looks like a model with a bad prompt rather than a model with a misconfigured target. That is the failure mode worth recognising by sight.

On the shipped curve that first line is not finite at all: $c$ is exactly zero at $t = 999$ , so the division does not exist, and trailing spacing makes that the first timestep the sampler visits. The rescale turns a silent misconfiguration into a loud one, which is the rare case of a numerical degeneracy being the friendlier behaviour.

Check yourself

If ε, x₀ and v carry exactly the same information, why does swapping the target change what the trained model is good at?

Show answer

Because the loss is not invariant under the coordinate change even though the information is. Squared error in $v$ -space is squared error in $\varepsilon$ -space multiplied by $1/\bar{\alpha}_t$ , which on the textbook curve runs from $1.0009$ to $214.6$ across the chain, and on the shipped one runs from $1.0009$ to unbounded. The optimiser sees a different trade between timesteps and allocates capacity accordingly.

Two things follow. First, the target is a weighting scheme in disguise, so arguing about it in terms of accuracy is arguing about the wrong axis. Second, if you wanted an $\varepsilon$ model to behave like a $v$ model, you would not have to change the architecture — you would have to change the per-timestep loss weight to $1/\bar{\alpha}_t$ , which is a one-line change that most codebases do not expose.