A rolling horizon outruns the history that was meant to quiet it

Jittered frames rebuild a camera’s horizon, saw-tooth and all found that a renderer which moves its sample a fraction of a pixel every frame and averages the frames turns each pixel’s single sample into an average over its area, which is what a camera’s pixel records. Along a walker’s hazy horizon, rolled three degrees, eight Halton frames put the last ground pixels within a hundredth of a camera’s, and the saw-tooth that remained was the camera’s own. Its price was time: jitter shown frame by frame flickered by 0.29, and only a history — a running average of recent frames — turned the flicker into the camera’s picture.

That essay measured a still camera, and it named the weakness it had not tested. A history averages frames that show the same pixel. When the camera rolls, the pixel the history holds is no longer the pixel being drawn, and the history must either follow the motion — reprojection, each stored sample moved to where its ray now falls — or forget faster, which lets the flicker back. The question was how far a moving horizon then sits from a camera’s moving picture, and at what rate of roll a renderer would do better to give up the history and paint four rows of band instead.

A history, two ways of keeping it, and a camera to judge it by #

The scene is the earlier essays’: an eye 1.62 metres up, 740 pixels of focal length, a uniform distance fog of five kilometres on a flat ground, a sky of uniform haze. The camera now rolls about its own principal point at a stated rate, starting from three degrees, and is drawn at sixty frames a second. Each frame samples every pixel once, at its centre moved by the next offset of a Halton sequence in bases two and three, and the picture shown is a running blend that keeps nine-tenths of the history and takes a tenth of each new frame, the proportion such renderers commonly use.

The history is kept one of two ways. Left in place, each pixel’s history is the same pixel’s last shown value, as if nothing had moved. Reprojected, it is last frame’s picture read at the point where this pixel’s ray fell a frame earlier. For a roll that point is known exactly — a camera turning about its own centre carries every ray round the principal point, with no depth to guess — so this is reprojection at its best, the motion a renderer can follow without error. What it cannot do is read between pixels without interpolating, and it reads last frame’s picture by blending the four stored pixels round the point.

The judge is a camera’s moving picture: each frame’s scene averaged over every pixel’s area at that frame’s roll. The scene is computed directly from each page point’s ray rather than through the rasteriser’s triangles, and along a horizon rolled three degrees the two agree to 0.0007 of the fog’s range on average — the rasteriser’s horizon is snapped to 1/256 of a pixel, and that is all that separates them.

Rolling at four degrees a second, the reprojected history keeps the pixels within two rows of the horizon 0.048 of the fog’s range from a camera’s picture on average; the history left in place, 0.101; one centred sample a frame with no history at all, 0.025. Still, the same history was 0.0059 from the camera and the centred sample 0.0247. Four degrees a second has taken the history from four times better than no anti-aliasing to twice as bad.

The crossing is at a degree and a half a second #

The rate at which the history stops paying is the number the question asked for.

One centred sample a frame stays 0.025 from the camera at every rate from still to thirty-two degrees a second: it is never more than a pixel’s worth of point sampling wrong, and that error does not depend on what the camera did a frame ago. The history left in place is 0.015 at half a degree a second and 0.028 at one — already worse than no history — then 0.055 at two, 0.10 at four, 0.23 at sixteen. The reprojected history is 0.012 at half a degree a second, 0.019 at one, and 0.0251 at one and a half, the centred sample’s own figure; past that it is worse, 0.031 at two, 0.048 at four, 0.093 at sixteen.

A degree and a half a second is slow. A camera held in a hand drifts that much; a car’s camera on an ordinary road rolls by more; a head tilting to look at something rolls at tens of degrees a second. On a horizon, which is the steepest edge in most outdoor pictures, a jittered history earns its keep only on a camera that is very nearly still — or for the part of the picture where nothing steep is moving.

Reprojection smears the step it reads #

Reprojection was exact, so the error is not where the history looks. It is how it reads.

The point where a pixel’s ray fell a frame earlier is almost never a pixel’s centre, so the history there is interpolated from the four pixels round it. On the horizon those are two of sky and two of ground, and the interpolated value is a blend — a step drawn a little softer than it was. Next frame the softened step is read again, between pixels again, and softened again. The history keeps nine-tenths of what it held, so the softening of a dozen frames is all present at once, and the horizon has moved a long way under it. Rolling at sixteen degrees a second, the row the horizon crosses in this column is shown at 0.72 where a camera records 0.86, the first full row of ground at 0.41 against 0.25, and the last full row of sky at 0.88 against 1.00. The step has been spread both ways — the ground’s fog carried up into the sky and the sky’s haze down into the ground — and it stays spread while the roll continues.

Left in place, the history smears much worse, because it does not follow the horizon at all: it averages each pixel with what that pixel showed before the horizon moved across it. Reprojected, it follows the horizon and smears only by its own interpolation. Both errors grow with how far the horizon moves while the history still remembers, which is the rate of roll times the history’s length.

The pops are the jitter’s below, the edge’s above #

The earlier essay measured jitter’s flicker as a mean, and on a moving edge the mean hides the part that shows. A point-sampled edge does not shimmer evenly; it pops — one pixel jumping by most of the fog’s range as the horizon crosses its centre — and the pops are rare and large.

The jump exceeded once in a hundred pixel-frames is the measure here, taken of each picture’s departure from a camera’s, so that the horizon’s own motion — which a camera shows too — is set aside. A centred sample does not pop at all on a still camera, and pops 0.014 at half a degree a second, 0.055 at two, 0.10 at four and 0.25 at sixteen: the faster the roll, the more pixel centres the horizon crosses each frame. The reprojected history pops 0.045 on a still camera — that is its jitter, which a blend of a tenth leaves at that size — and barely more until two degrees a second, 0.048; then 0.063 at four and 0.21 at sixteen, as its smear starts to move. Below about two degrees a second the history pops more than no history; above it, less. Its error crosses the other way at a degree and a half. Between one and a half and two degrees a second, then, the history is worse on one count and better on the other, and outside that narrow window it is worse on one count without being better on the other by much.

What a viewer sees at each rate #

Put as pictures rather than numbers, the three ways of drawing a rolling horizon look like this. With one centred sample, the horizon is crisp and correct in each frame and crawls as it moves: pixels along it switch between sky and ground one at a time, a little run of pops travelling along the line, faster as the roll quickens. With a history left in place, the horizon leaves a trail: wherever it has just been, the sky’s haze lingers in pixels that are now ground, fading over a dozen frames, a ghost horizon dragging behind the real one. With a reprojected history the ghost follows the horizon instead of trailing it, but the line itself goes soft, a row or so of haze laid along its ground side, and on a still camera the same line is crisp, quiet and right.

None of the three is what a camera shows. A camera’s rolling horizon is the still one’s, carried round: neither crawling nor ghosting nor softened. The earlier essay found that a history reaches that picture on a still camera to within a hundredth. On a rolling one, the nearest of the three to it below a degree and a half a second is the history, and above, the plain centred sample — whose fault is the crawl, and whose crawl is the part of the picture a viewer notices first.

No blend wins both #

A renderer’s one free choice is the blend — how much of each new frame the history takes — and the obvious hope is that some blend wins on a moving horizon.

At four degrees a second, taking a twentieth of each frame gives an error of 0.071 and pops of 0.054; a tenth, 0.048 and 0.063; a fifth, 0.030 and 0.10; a third, 0.025 and 0.19; a half, 0.028 and 0.29. Taking the whole frame is jitter with no history, 0.060 and 0.70. One centred sample is 0.025 and 0.10. A longer history quiets the jitter and smears the moving step; a shorter one keeps the step and lets the jitter through; and the curve the blends trace passes above and to the right of the centred sample everywhere. There is no blend at which a horizon rolling four degrees a second is drawn both nearer a camera and more quietly than by not jittering at all. On a still camera the same curve passes far below and to the left of it, at a tenth: 0.0059 and 0.045. The history is a tool for pictures that hold still.

What the band buys on a moving horizon #

The band’s case was the opposite of the history’s from the start: it changes the scene rather than the sampling, and it does so in every frame alone.

Four rows of blended haze, sampled at pixel centres with no history, pop 0.0027 at four degrees a second against a camera of the banded scene — a fortieth of the centred sample’s 0.10 and a twentieth of the history’s 0.063 — and 0.0093 at sixteen degrees a second. Eight rows pop 0.0007. The band is quiet in motion for the reason it was quiet still: it has taken the steep rise out of the last pixels, so a moving horizon has little left to change as it crosses their centres. A history on top of it adds its own resampling to a scene that no longer needed it, and pops 0.018.

What the band costs is the same as before and does not move with the roll: four rows of band are 0.25 of the fog’s range from a camera of the world, eight rows 0.29, because that is the ground they paint hazier than its distance. A fog band softens the horizon only by repainting the ground priced that repainting in metres of falsified ground, and nothing about motion changes the price.

So the question’s last part has an answer that is not a rate. If the renderer’s aim is a horizon that moves quietly, the band wins at every rate, including still, and the history never does better than the band on that count. If the aim is a camera’s horizon, the history wins only below a degree and a half a second and loses above it to a single centred sample, which costs nothing. The band is never the camera’s horizon at any rate. The choice is between looking like a camera on a still picture, looking like a camera roughly on a moving one, and looking quiet on both, and the three tools take one each.

What the comparison assumes #

The motion is a roll. A roll about the optical axis turns every ray round the principal point and is reprojected exactly. A turn of the head about the vertical moves the horizon sideways instead, and a step forward moves near ground and far ground differently; the second needs depth to reproject and adds the depth buffer’s own error, which the precision a depth buffer has left found least in exactly these rows. The roll is the kindest motion a history can face, so the crossing at a degree and a half is an upper bound for the others.

The history is a plain blend. Renderers usually clamp the history read for each pixel to the range of the current frame’s neighbouring samples, precisely to stop a smeared step from persisting. That is not modelled here, and it is the obvious next question.

The scene is the earlier essays’. A uniform fog on a flat ground meeting a uniform sky, the same five kilometres of visibility; ground and sky meet without a crack is why the page function and the rasterised scene agree to within the vertex snap, and a sky is carried as a direction is why the sky’s value above the horizon is the sky’s own. A horizon of hills, or an atmosphere that thins with height, changes where the rise falls and how steep it is, and with it every number above, but not which tool removes which part.

The camera’s picture is instantaneous. A real camera exposing for a sixtieth of a second smears a horizon rolling at sixteen degrees a second by about half a pixel a hundred pixels from the centre — a motion blur of its own, which would move the judge a little towards the history. At a degree and a half a second the smear is a twentieth of a pixel, and the crossing would barely move.

Where this leaves the horizon #

A pixel is not a point is the general argument for averaging over area, and a texture reaches the horizon as a rate found the same three-way division for a texture as the earlier essay found for a fog. Motion adds a fourth consideration that a still picture hides: averaging in time is averaging over what the pixel showed, and a pixel on a moving horizon showed something else. The divide is postponed carried the ground’s attributes so that nothing divided by depth until the pixel; a history is the same trick played over frames, carrying the pixel’s past so that nothing is decided until the average, and like every carried quantity it is only as good as the carrying. On a still horizon it carries exactly. On a rolling one it carries by interpolation, and interpolation is the one thing a steep step cannot survive repeatedly.

Still open: whether clamping the history to its neighbourhood keeps the step without bringing back the pops #

The smear persists because nothing stops the history from holding a value the current frame could not have produced: sky, in a pixel whose samples this frame all land on ground. Renderers guard against that by clamping each pixel’s reprojected history to the smallest and largest of the current frame’s samples in the three-by-three pixels around it, so that a history value outside what the neighbourhood now shows is pulled back inside before it is blended.

The measurement that settles what that is worth adds the clamp to the reprojected history here and asks three things: how far the rolling horizon then sits from a camera’s picture at one, four and sixteen degrees a second, and whether the crossing with a centred sample moves from a degree and a half a second towards the rates a hand or a head actually rolls at; whether the clamp, which on the last ground pixel lets through anything between the sky’s one and the deep ground’s fog, is too loose there to stop the smear at all; and what it does to the pops, since a clamp that snaps a history back to the current frame’s range brings back part of the current frame’s jitter.