A clamp keeps the sky clean and leaves the rolling horizon smeared

A rolling horizon outruns the history that was meant to quiet it put a renderer’s jittered history on a camera that rolls. Each frame samples every pixel once at a point moved by a Halton offset, the picture shown keeps nine-tenths of last frame’s and takes a tenth of the new one, and last frame’s picture is reprojected — read where each pixel’s ray fell a frame earlier, which for a roll about the camera’s own centre is known exactly. Still, that history keeps the horizon’s last ground pixels within 0.006 of a camera’s picture. Rolling, it loses: from a degree and a half a second, a single centred sample a frame with no history at all is nearer a camera, and at sixteen degrees a second the history is nearly four times further away.

The loss was a smear. Reprojection reads between pixels, so the step it reads is a little softer each frame, and a history that keeps nine-tenths of what it held keeps a dozen frames of softening at once. That essay ended on the guard every renderer that keeps a history uses against exactly this: clamp each pixel’s reprojected history to the smallest and largest of the current frame’s samples in the three-by-three pixels round it, so that a value the neighbourhood could not now produce is pulled back inside before it is blended. It asked three things — how far the clamp moves the crossing, whether it is too loose where the smear is, and what it does to the pops.

One correction first, because it changes what the smear is. The earlier essay’s row-by-row figure read the column at the very edge of the window the history was kept for, where reprojection had nothing to read from and the picture there was the current frame’s raw sample. Read one column further in, the smear is not the sky’s haze carried a row down into the ground, as that essay said: it runs both ways. On a horizon rolling sixteen degrees a second the last full row of sky is shown at 0.88 where a camera records 1.00, the row the horizon crosses at 0.72 against 0.86, and the first full row of ground at 0.41 against 0.25. That essay’s figure and paragraph now say so.

The clamp on the horizon’s edge #

The scene is the earlier essays’: an eye 1.62 m up, 740 px of focal length, a uniform haze of five kilometres’ visibility over flat ground, a sky of uniform haze, the camera rolling about its principal point from three degrees, sixty frames a second. The clamp takes, for each pixel, the smallest and largest of the current frame’s nine jittered samples round it and limits the reprojected history to that range before blending a tenth of the new sample in.

Rolling at four degrees a second, the clamped history keeps the pixels within two rows of the horizon 0.039 of the fog’s range from a camera’s picture on average, against 0.048 unclamped and 0.025 for one centred sample. The clamp has taken about a fifth off the history’s error and left it half as far again from the camera as no history at all. Still, it changes nothing: on a camera that does not move every reprojected history value already lies inside its neighbours’ range, and the clamped history is the unclamped one to the last digit, 0.0059 from the camera.

The crossing moves by three-tenths of a degree a second #

At half a degree a second the clamped history is 0.011 from a camera where unclamped it was 0.012; at a degree, 0.017 against 0.019; at a degree and a half, 0.022 against 0.025; at two, 0.026 against 0.031. The centred sample stays at 0.025 throughout, so the clamped history’s error crosses it at about 1.8° a second instead of 1.5. At four degrees a second the clamped history is 0.039, at eight 0.054, at sixteen 0.070 — a fifth to a quarter better than unclamped at every rate from four up, and at sixteen still nearly three times the centred sample. A clip to the neighbours’ mean plus or minus one standard deviation, within the same range, does a little better again — 0.038 at four degrees a second, 0.064 at sixteen — and crosses the centred sample at much the same rate.

The question asked whether the clamp moves the crossing towards the rates a hand or a head actually rolls at. A hand-held camera drifts by a degree or two a second; a head tilting to look at something rolls at tens. The clamp moves the crossing from the bottom of that range to a little further inside its bottom.

Loose exactly where the smear is #

Down one column, eighty-five pixels left of the principal point, the clamp’s window says why. Three rows above the horizon the nine neighbours are all sky; the window is a sliver at 1.00, and the history, which had been darkened there to 0.98, is pulled back to 1.00. One row above, the window is 0.98 to 1.00 and the history goes from 0.88 to 0.99. The sky side of the smear is gone.

On the row the horizon crosses, the nine neighbours are some sky and some ground, and the window runs from 0.22 to 1.00. The history there, 0.72, is inside it and is left alone; it becomes 0.75 only by the tenth of a new sample blended in. On the first full row of ground, the window runs from 0.12 to 1.00 — its neighbours on the row above still reach the sky — and the history, 0.41 where a camera records 0.25, is inside it and stays 0.42. Two rows down, the window has closed to 0.08–0.40 and the history, 0.23 against 0.13, still sits inside it; three rows down, the window is 0.06–0.16 and the history, 0.13 against 0.08, is still inside. The window narrows as fast as the smear does, and never quite past it.

So the clamp is loose exactly where the smear lives. The rows a horizon has just crossed are the rows whose neighbourhood straddles it, and a straddling neighbourhood can produce anything between sky and ground. The guard works by refusing a history value the neighbours could not have produced, and next to an edge the neighbours can produce almost anything. It cleans the sky because the sky’s neighbourhood is all sky; it leaves the ground because the ground’s nearest rows are not yet all ground.

A tighter clip brings the jitter back #

The obvious repair is to tighten the window — clip to the neighbours’ mean plus or minus less than one standard deviation, so that a straddling neighbourhood’s range is narrowed towards what the pixel’s own sample says.

On the rolling horizon, a clip of three-quarters of a standard deviation is 0.037 from the camera and pops 0.068 — the best the clip does, and not much better than one standard deviation, 0.038 and 0.065. Half a standard deviation is 0.047 and 0.097; a quarter, 0.069 and 0.19. Tightening past three-quarters makes the rolling horizon worse on both counts, not better on one. The reason shows on the still camera. There the unclipped history was 0.0059 from a camera and popped 0.045; clipped to three-quarters of a standard deviation it is 0.011 and 0.080, to a half 0.041 and 0.11, to a quarter 0.068 and 0.18 — four times the pops, and eleven times the error, of the history it was meant to guard.

A tight clip snaps each pixel’s history towards what this frame’s jittered samples say, and what they say is the jitter: a pixel on the horizon’s edge sampled a quarter of a pixel up this frame and a quarter down the next sees sky one frame and ground the next. That variation is exactly what the history was there to average away. Clipped hard, the history follows the samples and becomes them. No setting of the clip reaches the corner of the plot below and to the left of the centred sample on the rolling horizon — nearer a camera and quieter at once — and every setting tighter than one standard deviation damages the still picture the history was built for.

The pops are the history’s own #

The third question was whether the clamp brings back the current frame’s jitter as it pulls the history inside the current frame’s range.

At the looseness renderers use, it does not. The clamped history pops 0.045 on a still camera, exactly as unclamped; 0.047 at a degree and a half a second; 0.065 at four, against 0.063; and 0.196 at sixteen, against 0.209 — a little less than unclamped, since a smaller smear moves less from frame to frame. A clamp this loose pulls on almost nothing near the horizon, so it has nothing of the jitter to bring back. The centred sample’s pops are 0.10 at four degrees a second and 0.25 at sixteen, so between about two and twenty degrees a second the clamped history is quieter than no history and further from a camera, as the unclamped one was; the clamp has shifted both curves a little and crossed neither.

What the clamp is for #

None of this says the clamp is a poor tool. It is a guard against a history holding a value the neighbourhood rules out, and most of a moving picture is neighbourhoods that rule out a great deal: a dark object sliding over a bright wall, a wall’s edge passing a window. Away from the edge, a stale history is outside a tight window and is pulled back within a frame, which is the sky rows here. What a clamp cannot guard is the band of pixels right at the edge, whose neighbourhood holds both sides, and on a picture whose only feature is one steep edge — a horizon — that band is where all the error is.

A pixel is not a point is the reason the band exists: a pixel the horizon crosses is a mixture in a camera, and any reading of it from a few points is a guess at the mixture. Jittered frames rebuild a camera’s horizon made the guess good by averaging many frames of one still mixture. A moving horizon changes the mixture every frame, and a clamp can only say whether a remembered value is a possible mixture, never whether it is this frame’s.

A fog band softens the horizon by repainting the ground found the other way out: change the scene, not the sampling, so that the steepest edge in the picture is no longer a step. A band drawn into the haze is the same in every frame, needs no history, and pops a tenth as much as a centred sample on a moving horizon. The clamp is the history’s attempt to keep the step and lose the smear, and on this edge it keeps the step about as well as it did without the clamp.

Why the window narrows with the smear #

The pattern in that column has a simple cause, and it is worth stating because it is not particular to fog. The clamp’s window for a pixel is the spread of what nine nearby samples say, and near an edge that spread is set by the edge: a pixel one row from the horizon has neighbours on the horizon’s row, whose samples land on either side of it as the jitter moves them. The smear is set by the edge too: it is the edge’s softened copy, carried in the history, and it is widest in the rows nearest the edge and fades with distance at the rate the reprojection softens it. Both fall off over the same one or two rows, for the same reason — the edge is there — and there is no row near the horizon where the window has closed and the smear has not.

A texture reaches the horizon as a rate found the same structure in a texture’s footprint: everything the renderer does near the horizon is decided within a row or two of it, by the rate at which the ground’s distance changes there. Ground and sky meet at the horizon without a crack found the meeting itself exact — no gap, no overlap — so the step a history smears is a true step, as sharp as the pixel grid allows. And one depth per sample is not enough is the general warning that a pixel near an edge needs more than one sample’s say before anything can be decided about it; the clamp decides with one sample from each of nine pixels, and at the edge nine are not enough.

So the obvious improvements to the clamp all buy the same thing at the same price. A window drawn from more samples a pixel narrows near the edge, because each neighbour’s say is less noisy; that costs samples, which is what the jittered history was meant to save. A window drawn from a blurred frame narrows too, and blurs the edge it then lets through. A window drawn from the neighbours’ colours and their depths would separate sky from ground cleanly — the sky’s depth is infinite — and that is the one improvement this scene invites, since a horizon is the rare edge where depth says outright which side a sample is on.

What the measurement leans on #

The clamp sees the current frame’s jittered samples, one per pixel. A renderer that clamped against a frame rendered at several samples a pixel, or against a blurred copy of the frame, would have a narrower window at the edge — and would pay for it in the samples it draws.

The camera only rolls. For a roll, reprojection is exact — a sky is carried as a direction is why a turn about the eye moves every ray by a rule that needs no depth; for a camera that also moves, reprojection guesses each pixel’s depth, and on a horizon the guess is easy — the sky is at infinity — but a nearer edge would add a reprojection error the clamp would be asked to catch as well.

The judge is an instantaneous camera. A camera exposing for a sixtieth of a second blurs a horizon rolling at sixteen degrees a second by about half a pixel this far from the centre, which would move the judge a little towards the history; at two degrees a second the blur is a twentieth of a pixel and the crossing would barely move.

The window the history is kept for has edges. Reprojection near the window’s own edge reads outside the stored picture and falls back to the raw sample; one column at each side of two hundred is affected, and the earlier essay’s row-by-row figure happened to sit on one of them.

Still open: whether a band and a history together keep a rolling horizon nearer a camera than either #

The fog band and the history have been measured apart and against each other, and on a rolling horizon each wins one count: the band is quiet and further from the world it replaced, the history is near the world and pops. The earlier essay measured them together once, at four degrees a second and against the band’s own scene. A band widens the step a clamp could not reach into several rows of gradual change, and gradual change is what a clamp’s window holds tightly — a neighbourhood of three nearly equal haze values rules out almost everything.

The measurement that settles what that is worth paints the haze in a band of two, four and eight rows, keeps the reprojected history clamped to its neighbours, and asks how far the result then sits from a camera of the unbanded world and from a camera of the banded one, at rolls from one to sixteen degrees a second; whether the clamp, on a band, at last catches the smear it missed on the step; and how narrow the band can be before its rows straddle the window again and the clamp goes loose.