Euler, Heun and Separable Sampling Errors

Task: Validate the Learned Velocity Field and Numerical Error Separately

The previous chapter describes what marginal speeds should be studied and how the score should be counted. $x(t)$ Satisfy $dx/dt=v(x,t)$, starting from standard normality, with zero to one target zone. The ODE trajectory is determined for fixed primary and stationary values; the diversity of samples is derived from different initial noises and does not require extra random injections at each step. Euler and Heun are ODE methods in the project, not SDE discreters in reverse. Adding noise to them will change density evolution and cannot be called minor fixes for the same solver.

Set up partitions $N$ A long walk. $h=1/N$, the current grid $t_n=nh$, numeric status as $x_n$. The course is limited to a fixed step length, which clearly records the number of times the speed field is called. The fixed step, the fixed call and the fixed real time are three different budgets; network reasoning, data removal and solver extra operations can affect time. This chapter uses a simple equation check program with precision, then uses the same code in the learning field, avoiding the reversal of a sample that looks like a target.

1. Deriving Euler's Method from an Integral Identity

Exact state meets $x(t+h)=x(t)+\int_t^{t+h}v(x(s),s)ds$ Euler uses the left end speed to be similar to that of the entire area, and takes the current state plus the current speed. If the field is smooth enough to get a second order with time deflection and space accompaniment speed along the true solver; Euler drops this item, so that the local error of one step from the exact state is a long square step. This level does not mean that the overall error after many steps is still square, because each step produces errors and spreads.

Assuming that the velocity meets the Lipschitz boundary in space, i.e. how different is the state, the velocity difference is at most a constant $L$ Multiply the difference, and there is a boundary between the local cut-off constants. The error forward can be multiplied by the upper limit as the previous step. $(1+hL)$ Add a second tier of local entry. Push this forward, adding up about one step to the count down, and eventually getting a single step of global error. This conclusion does not apply to random operations that are incoherent, have exploded or do not fall into the area where the fault line was formed.

2. Heun's Method: Correcting Velocity with a Prediction

Heun first makes an Euler prediction point. $x_{\rm pred}=x_n+h v(x_n,t_n)$, recalculate the speed at the projection point and the next moment, updating the average of the starting speed and the predicted end speed. The second call must use the next time, not the original time; it must also use the forecast state, rather than simply replace it with the original state. A function called Heun, but these two faulty functions, do not have the second order nature that we will be deduced.

Start the second speed at the beginning. Get it. $k_2=v+h(\partial_t v+D_xv\,v)+O(h^2)$. Once updated on average over time, the exact second order is restored, with a single step local error becoming a third step; under similar stable and smooth conditions, the global error is a second step. This is the scale of the method, not that the arbitrary error of Heun must be a fixed proportion of the Euler error, let alone that the same number of steps represents the same cost.

There is another easy to ignore cache error: Heun ' s second speed is calculated at the prediction point, whereas the final state is the correction point, which is usually different. The second speed of this step cannot be considered as the first speed of the next step to reduce calls without any extrapolation. Some special methods support the re-use of endpoint guides, and this Heun set of projects has not been agreed. The number of calls will be accumulated over time, requiring Euler to be N and Heun to two N.

Example 1: Using an Exact Solution to Check Convergence Order

Marked equation $dx/dt=x$ The endpoint is the natural constant e. Two steps in half, Euler multiplied one step by one five, and Heun multiplied one sixty-two, and Heun multiplied one hundred sixty-five, and two hundred-six forty-two. The relative accuracy was about two hundred and seventy-eight, and the absolute error was about four hundred and eighty-three and zero seventy-seven, respectively. Here Heun called four times, Euler only twice, and could not call the difference directly as a budget advantage.

If Euler is also called four times, one-fourth step, the result being four parties of 1:25, about two-four-four-four-six, with a margin of error of about zero-two-seven-69. In this smooth example and this budget, Heun is still more accurate; the scope of the conclusions is this. The project module test compares ten steps to twenty steps, and Euler should approach four steps in order to check the scores rather than just checking the final fractions of an accident.

3. Stability and Learning Error Are Different Questions

Equation for contraction $dx/dt=-\lambda x$ 、 $\lambda>0$, the precise solution is reduced over time. Euler zooms in to $1-\lambda h$, which is required to be less than one in order to maintain near-sustainability, so that the length of the step must be less than two divided by the lambda. Heun's amplification factor is $1-\lambda h+(\lambda h)^2/2$ The same inter-segment limits apply to negative material axes. The second-order approach does not automatically solve all rigid problems; if speed changes in position are particularly dramatic, both visible methods may need to be very detailed.

Even with a very precise score of a network field, the final distribution may be incorrect if the network itself is not different from the correct margin. Conversely, having a precise solution for teachers does not compensate for the fragmentation error caused by a giant Euler update. The project provides both a learning ground and a resolution field and uses multiple steps for the same initial noise: the solution field helps to locate solver errors, the network validation error in the resolution field helps to locate learning errors, and the final statistical indicator checks output distributions. The three types of evidence cannot be combined into a vague “Quality enhancement”.

Example 2: An Exact Teacher Can Still Fail with One Euler Step

The course's separate linear bridge meets the edge speed at zero. $v_*(x,0)=\mathbb E[X_1]-x=-x$, because the overall average is zero, the initial point is independent of the target. If Euler takes only one step and takes one step, the end point is $x+(-x)=0$ Each of the different noises is presented to its original point. The target has four columns, with a 4:09 margin. This is a completely predictable failure of values, not a teacher ' s formula error, or a network failure; it reveals a direct error in using instantaneous speed as a constant rate across the region.

As heun takes a step, the prediction points fall to zero and the next time the teachers are at zero, the result of which is half the original noise, and not the four targets. “Look at the speed.” is therefore not any longer a panacea. As the grid is refined, the two may approach the right flow under normal conditions. We will keep these two thick grids, not being deleted because they are hard to see, as they examine whether the evaluation process can detect a positive comparison of a manifest error.

4. Aligned Comparisons and Saved Inputs

When comparing the two solvers, the same model is frozen and the same initial noise is fixed. If the network is changed at the same time, the number of training steps or random seeds are changed, the difference is no longer merely a fractional method. The fine grid decoupling can be used as a reference for approximate values, but not for mathematical truths; it should continue to encrypt grids to check whether the reference is stable. A point-by-point error check requires matching the initial sample, while the distribution indicator does not require either approach to a real target point.

The project evaluation orders support the presentation of steps one, ten, twenty and forty, which allows for a comparison of the 40 Euler calls with the twenty-step Heun calls. The time-consuming records are the observations that run, not the general performance law. Three groups of sampling seeds share a training model that reflects only sampling randomity; they are not independent training three times and are not sufficient to claim that the stability of training has been duplicated.

The two types of error relationships can be further clarified. Assuming that the difference between the learning and the real fields does not exceed a single boundary in all the relevant locations, and the real field has Lipschitz constants, the distance of the two precise ODE tracks meets the diversity of "no more than a constant number of guides than a constant multiplier error". After the add-up, the field deviation is magnified over time, and then the value is solved separately. However, the average square field error of the validation set is not this uniformity; It cannot rule out a very small number of large errors, so it cannot directly replace a small average MSE with strict trajectories.

The fine grid pairs in the project also detect another type of failure: if the initial point is repeated every time a calculator is called, the output difference contains both input changes and solver errors. The program should first take the initial load and then pass the same amount to different methods, and only cloned within the blocker without rewrite the caller's input. The unit test will check that the original count is not modified in situ. This object identity agreement seems small, but it determines whether multiple steps are actually comparing the same initial value.

from math import e
def solve(n,method):
    x=1.0;h=1/n;calls=0
    for i in range(n):
        k1=x;calls+=1
        if method=="euler":x+=h*k1
        else:
            k2=x+h*k1;calls+=1;x+=h*(k1+k2)/2
    return x,calls
assert solve(2,"euler")== (2.25,2)
assert solve(2,"heun")== (2.640625,4)
for method in ["euler","heun"]:
    a,_=solve(10,method);b,_=solve(20,method)
    print(method,a,b,abs(a-e)/abs(b-e))

Exercises and Complete Solutions

# integrate
def integrate(field,initial,grid,method='heun'):
    raise NotImplementedError('ode')

# integrate
def integrate(field,initial,grid,method='heun'):
 if method not in ['euler','heun']or len(grid)<2:raise ValueError('method/grid')
 times=[float(t)for t in grid]
 if not all(torch.isfinite(torch.tensor(t))for t in times)or any(b<=a for a,b in zip(times,times[1:])):raise ValueError('strict finite increasing grid')
 x=initial.clone();path=[x.clone()];calls=0
 for t,end in zip(times,times[1:]):
  h=end-t;k1=field(x,t);calls+=1
  if method=='euler':x=x+h*k1
  else:k2=field(x+h*k1,end);calls+=1;x=x+h*(k1+k2)/2
  path.append(x.clone())
 return torch.stack(path),calls