A rolling frame on a bend is a coarse curvature gauge

A rolling frame on a bend is right at its centre followed a camera car round a fifty-metre bend with a sensor that reads its frame one row at a time, and found the rolling readout displacing every still point by its streak times its row’s time over the exposure. Two things were left exactly where a global shutter would put them: the bend’s centre, which turns with the camera, and every point at eye height, which lands on the middle row and is read at the global shutter’s moment. A vertical straight toward the centre therefore leans in the rolling frame by an amount proportional to one over its range less one over the radius, and a vertical at the centre does not lean at all.

That essay ended by turning the law round. If the lean vanishes at the bend’s radius, then verticals at several known ranges ahead, read for their lean, should say where the lean vanishes — and a camera with a rolling shutter would carry a curvature gauge in every frame it takes on a bend, the distortion that stabilisers are fitted to remove becoming a measurement of the road.

The gauge exists and is exact. Read with the errors a real frame has, it is coarse, and what makes it better is the opposite of what sensor makers work towards.

Posts that lean both ways #

The scene is the earlier essay’s: a camera car at ten metres a second on a fifty-metre bend, aimed at the bend’s centre, its frame read from top to bottom over 33 milliseconds with the middle row read at the moment a global shutter would take. Posts three metres tall stand straight toward the centre at several ranges, their feet at eye height.

Each post’s foot, at eye height, sits on the middle row and is undisplaced; its top is read earlier, while the camera was a little further back round the bend, and is displaced sideways by the image’s speed at that column times the time between the two rows. So each post leans by a fixed number of pixels per row. The post twelve metres out leans by 0.039 pixels per row, the one at twenty-five metres by 0.012, the one at fifty metres — the bend’s centre — by nothing, and the posts beyond it lean the other way: −0.0062 at a hundred metres, −0.0082 at a hundred and fifty. The camera turns and travels at once, and at the centre the two carry a still point’s image exactly with the frame; nearer than that the travel wins and beyond it the turn does. Turning and travelling blur different worlds is where the two motions were first set against each other; the rolling frame records their difference as a lean. Drag the bend’s radius and the upright post moves with it: on a 25-metre bend the post at twenty-five metres stands straight and the one at fifty leans back by 0.012 pixels per row, and on a 100-metre bend the straight post is the one at a hundred. Wherever the row stands upright, that is how far away the centre is.

A straight line through the radius #

The earlier essay’s law says what these leans should do against range.

Plotted against one over the range — the quantity a stereo pair’s disparity is linear in, as depth is a reciprocal found — the leans lie on a straight line: lean $= a\,(1/Z - 1/R)$, with $a$ the camera’s focal length times its speed times one row’s time. Every bend at one speed shares the slope; each bend’s line crosses zero at one over its own radius. Fitted exactly, the three lines cross zero at 30.000, 50.000 and 100.001 metres. A straight road is the line through the origin, since a straight road’s radius is infinite and every post leans by travel alone.

So two posts at known ranges give the radius, and more posts give it by least squares. The ranges have to be known — from a map, a second camera, the posts’ own known height in the picture — but the speed and the readout time do not, because they are all in the common slope, which the fit finds.

Off the axis, the sharp circle #

Posts are not usually lined up toward the centre of a bend. A post at some bearing off the camera’s axis leans too, and its zero is somewhere else.

Along a bearing ten degrees off the axis the leans cross zero at 49.23 metres; twenty degrees off, at 46.96; thirty, at 43.26. A camera on a bend is sharp on a circle found the places where a still point draws no streak at all over an exposure, and they lie at 49.27, 47.04 and 43.38 metres on the same bearings. Both are $R\cos b$: the circle whose diameter runs from the eye to the bend’s centre. Every still point on that circle turns with the camera, and a rolling readout, which moves only what moves in the image, leaves it standing.

The gauge therefore works on any bearing. Posts along any line from the camera give the range at which their leans vanish, and that range divided by the cosine of the line’s bearing is the radius. A roadside’s lamp posts, seen at different bearings as the road curves away, are as good as posts lined up toward the centre, provided their ranges and bearings are known.

One frame, read to a pixel #

Exact leans are one thing. A real frame’s posts are read with an error, and the lean of a post is the displacement of its top against its foot divided by the rows between them — a small number divided by a number that shrinks with range.

Eight posts from twelve to a hundred and fifty metres, their tops placed to a tenth of a pixel, give the fifty-metre radius as 50.3 metres with quartiles 46.9 and 54.4 — a gauge good to about eight per cent. Placed to a quarter of a pixel, 50.7 metres with quartiles of 43 and 64. To half a pixel, 44.6 metres, 35 to 74, and to a pixel the estimate is 31 metres with its lower quartile below zero: the gauge reads nothing at all.

The reason is where the answer lives. The radius is where the line crosses zero, one over fifty metres, and that is found from the line’s intercept — a small number, $-a/R$, that the fit has to extrapolate from posts whose leans are largest near the camera and smallest far away. The far posts, whose leans pin the crossing, are exactly the posts that are short in the picture: three metres of post at a hundred and fifty metres spans fifteen rows, and a pixel of error in its top is a lean error of a tenth of a pixel per row, ten times the lean itself. The ratio that turns slope and intercept into a radius is also biased once the intercept is noisy, and biased towards small radii, which is why the median slides down as the reading worsens.

More frames #

A camera on a bend takes many frames, and each one draws the same posts leaning by the same amounts if the car holds its speed and line. Averaging their leans averages their reading errors.

With tops placed to half a pixel, four frames give 49.5 metres with quartiles of 43 and 61; sixteen frames — about half a second of video — give 50.9, 47 to 55; sixty-four, 49.9, 48 to 52. With tops placed to a pixel, sixteen frames already do better than one frame read to half a pixel, 51.6 metres with quartiles of 44 and 60; at half a pixel, sixty-four frames make the gauge good to about four per cent. The spread falls as the square root of the frames once the averaged leans are good enough for the ratio’s bias to fade.

That is an optimistic figure, and it says so. On a real drive the posts do not hold their ranges from frame to frame: the car approaches them, and a post’s lean changes as its range falls. The frames’ leans cannot simply be averaged; they have to be fitted together, each frame’s posts at their own ranges, with one radius for all. That uses the same information and more, and it should reach at least this precision; the figure is the averaging’s ceiling, with posts that wait.

A slow sensor measures the road #

The last variable is the one sensor makers control, and it runs the wrong way for this purpose.

Every lean is proportional to the readout time: a frame read over sixty-seven milliseconds leans its posts eight times as far as one read over eight. The reading error does not change, so the gauge improves in proportion. With sixteen frames and tops placed to a quarter of a pixel, a 67-millisecond readout reads the fifty-metre bend as 50.2 metres with quartiles of 49.2 and 51.1 — two per cent — while a 4-millisecond readout gives 47.0 with quartiles of 35 and 67, no gauge at all.

Sensor makers work to shorten the readout, because the leaning verticals and the sheared cars are what photographers complain of. A sensor built to make the distortion small is the sensor whose distortion measures the road least well. Every row is a different camera put the underlying fact plainly: a rolling frame is a set of cameras at different moments, and the longer those moments are spread, the more the frame knows about how the camera moved between them.

Where the ranges come from #

Every reading above supplies each post’s range, and the gauge is only as good as those ranges. The law makes the dependence exact: the fit finds where the leans cross zero on an axis of one over the range, so a common error of a few per cent in every range scales the radius by the same few per cent, and an error that grows with range — the usual kind — bends the line and moves its crossing.

The natural source of range in a single frame is the posts themselves. A lamp post or a fence post of a standard height spans a number of rows inversely proportional to its range, and the rolling frame already requires that span to turn a displacement into a lean. But a post’s height in pixels is read to the same fraction of a pixel as its top’s displacement, and for the far posts that pin the crossing it is a small number read coarsely: a three-metre post at a hundred and fifty metres spans fifteen rows, so a pixel of error in its height is a range error of about seven per cent — on exactly the posts where the line is decided.

A second camera, a map or a depth sensor gives ranges independently of the frame and does better at range than a post’s own height. A frame’s shear knows travel only over depth is the underlying reason the ranges cannot be avoided: a rolling frame’s distortion couples the camera’s travel to each point’s depth, and anything read from it about the travel — here, where the travel curves — has to be given the depths from somewhere.

What a bend of fifty metres is #

The fifty-metre bend used throughout is a tight one: a roundabout’s circle, a sharp corner in a town, an exit ramp. The gauge’s difficulty grows with the radius for a reason that follows from the law. The crossing sits at one over the radius, and a gentle bend puts it close to zero on the one-over-range axis, where the posts that fix it have to be far away — beyond the radius, where the lean changes sign. A five-hundred-metre bend on a main road needs posts past five hundred metres to bracket its crossing, which are a few rows tall and leaned by a hundredth of a pixel over a frame.

So the rolling-shutter gauge is a tool for tight bends and slow sensors, and a poor one for the gentle curves of fast roads. For those, the scroll camera that a scroll can be asked its own radius measured is the better instrument: a scroll accumulates a bend over its whole length rather than over one frame’s readout, and it gave the radius from a single point and its neighbour to a part in a billion with exact readings.

Between the two lies a difference of time scale. A rolling frame’s rows span tens of milliseconds, in which a car at ten metres a second moves a few tenths of a metre and turns by a few tenths of a degree; everything the gauge reads is that small motion, magnified by the posts’ depths. A scroll spans the whole drive. The rolling frame is a curvature gauge that fits in one exposure, and it is coarse in proportion to how little the camera moves in one exposure.

What the gauge is, and is not #

The measurement answers the earlier essay’s question with a qualified yes. A rolling frame on a bend carries the bend’s radius exactly in the leans of its verticals, straight toward the centre or off it, and with ranges known it can be read from any two posts. As a practical gauge from one frame it is coarse — eight per cent with tops placed to a tenth of a pixel, nothing at a pixel — because the answer is an intercept extrapolated from short, far posts. Averaged over frames, or read from a slow sensor, it becomes a useful instrument: four to eight per cent from half a second of video at the readout times phone sensors actually have, sixteen to thirty-three milliseconds.

It measures the bend as the camera drives it, which is not quite the road’s centreline radius, and it measures it relative to the ranges supplied: an error of a few per cent in every range scales the radius by the same few per cent. A turning frame can be straightened found the turn alone correctable row by row and the travel not; this is the travel’s revenge — the part a stabiliser cannot remove is the part that says where the road goes.

What was assumed #

The car holds its speed and its line. A car that brakes on the bend changes its speed between rows, and the leans’ common slope with it; one that drifts across its lane changes its radius during the frame. Both add errors the fit reads as noise in the lean.

The posts are vertical and their ranges known. A leaning post leans in every frame by its own lean plus the rolling one; a post whose range is read from its height in the picture carries that reading’s error into the one-over-range axis, where it matters most for the far posts.

The readout is linear from top to bottom. Sensors that read in interleaved fields, or whose row time varies across the frame, lean their verticals by a pattern the straight-line law does not describe.

Still open: whether a moving car’s own frames fit one radius better than they average #

The frames figure averaged leans as if the posts stood still in range. On a real drive each frame sees each post at a smaller range than the frame before, so the post’s lean moves along the line in one-over-range as the car approaches, sweeping the line itself rather than sitting on one point of it. A single post tracked through many frames samples the line over a range of distances, including the far part where the intercept is decided.

The measurement that settles it drives the car round the bend, tracks a handful of posts through half a second of frames, fits one radius and one slope to every post’s lean in every frame at its own range, and asks how the precision compares with the averaged figure — whether a single post seen at many ranges is worth as much as many posts seen at one, and so whether a camera needs a roadside of posts or only one lamp post it can follow round the bend.