DDPM Forward Chains, Cumulative Noise and Training Targets

Task: Construct Exact Noisy Samples Without Recursive Simulation

The flow time of the previous chapter goes from noise to data. $j=0,1,\ldots,J$ ： $x_0$ It's clean data. $x_J$ DDPM is the acronym for a stand-off probabilities model; first, a non-learning frontal Markov chain, then learning to reverse. This chapter only completes forward chain, random moment distribution and noise monitoring, and does not refer to noise creation as being generated.

Markov here means that the next transfer will no longer be dependent on an earlier state after the current state has been determined. It is a stand-alone assumption of the conditions for our noise process, not the pixels or two-dimensional coordinates of the original data. $z_j$ is a separate standard normal vector and independent of clean samples; per-step deviation parameters $\beta_j$ Strictly between zero and one, and remember. $\alpha_j=1-\beta_j$ 。

1. Why Signal Retention Factors Multiply: A Two-Step Expansion

The first step of the data coefficient is $\sqrt{\alpha_1}$. The first step is the second step and the data coefficient becomes $\sqrt{\alpha_2\alpha_1}$, and the two noise band coefficients $\sqrt{\alpha_2\beta_1}$ and $\sqrt{\beta_2}$. The linear combination of independent standard normals is still normal, with the variance being the sum of the coefficient squares; so the noise variance is $\alpha_2\beta_1+\beta_2$ Simplify $1-\alpha_1\alpha_2$. Each of these places depends on sound independence, and if the same noise is repeated without cross-drawn, the formula will fail.

Define cumulative retention rate $\bar\alpha_j=\prod_{i=1}^j\alpha_i$ And I'll make a deal. $\bar\alpha_0=1$. Assumption number one $j-1$ Noise Difference to $1-\bar\alpha_{j-1}$ Do the transfer again. Get it. $\alpha_j(1-\bar\alpha_{j-1})+\beta_j=1-\bar\alpha_j$;data coefficients synchronised $\sqrt{\alpha_j}$ And that's the proof that there's no need to write all the steps into a huge noise. And so we get the margin distribution of the conditions that can be sampled directly.

The last sentence is very practical. If training takes only one random step, you just have to build a sample of the conditions for that moment, without having to go all the way before. To study the combined difference of the same chain at two moments, two direct sampling noises cannot be set aside and claim that they are the same real forward trajectory. The same edge does not fix coupling; the warning of the previous chapter ' s punctuation remains valid in the discrete chain.

Example 1: Accumulated Variance and Noise Recovery

Think of a two-dimensional clean, take both steps. $\beta=0.2$ So twice. $\alpha$ The second-step condition is zero eight times six; the difference is zero thirty-six, and the standard difference is zero six. If you take negative one, the sample is minus one six, and gets one. On the other hand, it is known that one of the samples is negative and can be reduced first by sound reduction and the data coefficient is returned to clean two.

The two-step exercise can also independently examine the difference: the noise coefficient before step 2 is zero eight times zero two, equal to zero one-six; the second-step noise contributes zero two-two, both of which are zero-six. Wrongly adding the two differences directly to zero-four, and missing the first noise is shrunk. The cumulative retention rate at zero-eight plus zero-eight will exceed one, and the square root noise items will not even have real meaning. These are the failures of the agenda that should be clearly reported, rather than the mysterious phenomenon of training instability.

2. Gaussian Conditionals Do Not Imply a Gaussian Data Marginal

Fixed $x_0$ After that, the condition is Goss; Jean. $x_0$ When you come from a complex data source, you have to get the data points. $q_j(x)$. Some of the net data are in chapter I, four columns, and each cluster centre shrinks. $\sqrt{\bar\alpha_j}\mu_k$ The inner gap has changed to $\bar\alpha_j\sigma^2+1-\bar\alpha_j$ So the whole is still a mixture of four Goss, not necessarily a single Goss. Using the form of the distribution of conditions as an unconditional edge, it is a common logical leap to induce reverse noise.

When the accumulated retention rate is very hours, centres approach zero, the inner margin is close to one, and the entire edge can be close to standard normal. But “Close.” is not “Completely equal.” under limited steps, especially when the scale or end of the data is large. Reverse sampling begins with standard normality, an agreement that requires an examination of terminal errors. The end is not made precisely because many steps are used; cumulative retention rates, original data scales, and the rectangulations and shapes of the actual terminal distribution should be examined.

Example 2: Signal-to-Noise Ratios Compare Variance Scales

Standardization of clean variables to single coordinate difference one for the time being, and definition of the letter noise ratio as a signal variance other than a noise differential, i.e. $\mathrm{SNR}_j=\bar\alpha_j/(1-\bar\alpha_j)$ The cumulative retention rate is one-nine, one- and nine-point. Note that the coefficient is square root and the ratio is different and can no longer be referred to as the same indicator after taking square root from this value. For unstandardized targets with a four-point-nine difference, the signal difference is multiplied by four-nine and the values vary according to the data scale.

Let's say both schedules are written at one step. $\beta_j=0.01$, which does not indicate the same amount of noise: the first multiplier may be one zero nine and the second zero one. The direct condition sample scale is the aggregate amount. The training code should either keep a complete schedule or recreate its parameters and total number of steps, not just a vague label such as “One percent of the noise was used.”. Step infusion should also correspond to this agenda, and the old step sign should be used to allow the network to read incorrect terms after changing the schedule.

3. Where Noise-Prediction Targets Come From

When we build the training with a direct style above, we smoke it ourselves. $\epsilon$ So it's a monitoring label. Noise network. $\epsilon_\theta(x_j,j)$ Receive noise samples and step numbers, output the same vector, parameters $\theta$ The number of things to train. The usual square error requires that it be close to the noise actually smoked; but given the same noise point, there may be many clean samples and noise combinations, so the network usually does not get the hidden noise back perfect on a case-by-case basis. The optimal regression is average, and chapter IV leads to a connection with score.

The target formula does not indicate whether sampling is by coordinates and average, time steps are taken, and whether weights are weighted for different periods of time. This must be done with fixed protocols. Taking and averaged differences in constants will change the interpretation of gradient scales and learning rates without changing the ideal best; changing time weights will change the distribution of limited network capacity. Do not call “Same. MSE” for all different realizations, and make an unconditional and fair comparison with exactly the same learning rate.

The running projects that follow us train flow speeds, not this noise network. So the independent code in this chapter only checks forward agenda and condition algebras, and does not claim to have completed DDPM training. Dispersing diffusion deductions and continuous flow experiments are combined to compare monitoring objectives with sampling mechanisms; they share generation tasks, but not the same model weight or the same time direction.

from math import sqrt
beta=[0.2,0.2]
bar=1.0
for value in beta:
    assert 0<value<1
    bar*=1-value
mean=sqrt(bar)*2
variance=1-bar
observed=mean+sqrt(variance)*(-1)
reconstructed=(observed-sqrt(variance)*(-1))/sqrt(bar)
assert abs(mean-1.6)<1e-12
assert abs(variance-0.36)<1e-12
assert abs(reconstructed-2)<1e-12
print(bar,mean,variance,observed,reconstructed)

Training steps are usually taken from one to the total step, and do not contain zero. No noise is added at zero hour. If you still require a random noise to be pulled back with a noise sample, the label does not have the same message as input; there are zero denominators involved in formulae other than the standard deviation of noise. The endpoint should be explicitly rejected as a single defined limit issue, rather than adding a small constant after the denominator, claiming that it has not changed. Value patches can be used to stabilize the program, but they must indicate what their calculations have changed.

Noise targets also rely on data pre-processing. If the two-dimensional coordinates as a whole are multiplied by ten without changing the calendar, the real signal difference corresponding to the same cumulative retention rate is magnified by 100 times, and the network faces different effective albino ratios. The preservation of standardized centres, scales, and inverse variations is part of the model definition; it must be returned to the same coordinates unit when evaluating output. Noise regression curves under different pre-treatment cannot simply be seen as matching conditions, especially if smaller coordinates are automatically interpreted as better distribution modelling.

Exercises and Complete Solutions