A rolling frame on a bend is right at its centre

A camera on a bend is sharp on a circle put a camera car on a fifty-metre bend at ten metres a second, aimed into the bend, and found that one exposure of a thirtieth of a second blurs the still world everywhere except at the bend’s centre. Off the axis the sharp place comes nearer as the cosine of the bearing, on the circle through the camera and the centre; above the ground only the vertical line through the centre stays sharp. The two motions — the turn, which moves every point alike, and the travel, which moves near points more — cancel on that circle.

Every frame in that essay was taken with a global shutter, all rows at once. Most cameras on vehicles read their sensor a row at a time, so each row is a picture taken at a slightly different point on the bend, turned by a slightly different amount. A turning frame can be straightened found that a pure turn read row by row can be undone exactly, because a turn moves every point by an amount that does not depend on depth, and that a travel cannot, because it does. The earlier essay closed by asking whether the circle that is free of streaks is also free of shear — whether a frame could be straightened about the bend’s centre without knowing any depth.

It is free of shear wherever it is sharp, and the frame needs no straightening there at all. It is the straightening that does the damage.

Every mark moves but two sets #

The frame below is read from its top row to its bottom row over a thirtieth of a second, with the middle row read at the moment a global shutter would take the picture. Posts stand at several ranges and bearings, each marked at four heights — a little below the eyes, at eye height, and two and five metres above — and each mark carries an arrow six times the displacement the rolling readout gives it against the global frame.

Most marks move. The largest arrow, on a post twelve metres out and two metres above the eyes, is 5.07 pixels. Two sets of marks do not. The marks on the bend’s centre, at every height, move by $10^{-14}$ pixels: the centre turns with the camera, so wherever the camera is on the bend the centre is in the same place in its picture, and it does not matter which row reads it when. And every mark at eye height, whatever its range, moves by nothing: with the camera level, eye height is the middle row, and the middle row is read at the global shutter’s own moment.

The second set is a fact about when a row is read and the first is a fact about the bend. Together they are the whole of what a rolling frame on a bend gets right for free.

The readout’s length is a scale, not a shape #

The slider on the frame changes how long the readout takes, from a hundred-and-twentieth of a second to a fifteenth, and the arrows change length and nothing else. The largest displacement is 1.27 pixels at the fastest readout, 2.53 at a sixtieth, 5.07 at a thirtieth and 10.16 at a fifteenth: in proportion to the readout time, because each mark’s row time is a fixed share of it. The two undistorted sets do not move at all. A faster sensor shrinks every arrow and changes none of their directions, and the places where there are no arrows are the same at every speed.

Every row is a different camera is the general statement this is a case of: a rolling frame is a stack of rows, each taken by the camera at its own moment. On a bend the cameras in the stack differ by a turn about the bend’s centre and nothing else, so the stack is a fan of pictures about one axis. A frame’s shear knows travel only over depth found, for a camera travelling straight past posts, that the lean a rolling frame gives a vertical contains the travel only as speed over depth; here the same lean contains it as speed times the difference between one over the depth and one over the bend’s radius, and so it vanishes at the radius instead of at infinity. The bend moves the place where travel stops showing from the horizon in to the centre of the curve.

One law for the streak and the shear #

The earlier essay’s streak and this essay’s displacement are two samplings of one motion, and the figure below shows that they are.

A still mark’s image moves across the picture at some velocity, set by where it is and how the camera moves. An exposure open for $T$ seconds smears it along that velocity into a streak $T$ times as long. A rolling readout reads the mark’s row $t$ seconds from the middle row’s moment and records it displaced by $t$ times the same velocity. So each mark’s displacement is its streak scaled by its row time over the exposure, and across 130 marks spread through the frame the two agree to three per cent — the remainder being the image motion’s own change over the few milliseconds involved.

That is the answer to the earlier essay’s question, and it is a stronger answer than the question hoped for. The rolling frame is undistorted exactly where the global frame is sharp, because both are measuring the same image velocity and the sharp places are where that velocity is zero. The earlier essay found those places: the vertical line through the bend’s centre at every height, and the sharp circle at eye height. The rolling readout adds one set of its own, the horizon row, where every row time is zero whatever the velocity. A frame is an interval described an exposure as an integral of projections over time; a rolling frame is the same integral taken row by row, each row over its own short interval, and the law above is what that integral reduces to when the interval is short.

Up a vertical line, only the centre is kept #

The sharp circle deserves a closer look, because at eye height it is undistorted for two reasons at once, and above eye height for neither.

At the bend’s centre the displacement is nothing at every height, below the eyes and eight metres above. Twenty metres straight ahead it grows in proportion to the height above or below the eyes — 0.68 pixels a metre above them, 5.47 at eight metres — because the mark’s image velocity is fixed and its row time grows with its height in the picture. And at a point of the sharp circle twenty degrees off the axis, it is nothing at eye height and grows slowly above it, to 0.11 pixels at eight metres.

The circle’s point grows because the circle is sharp only at eye height. A camera on a bend is sharp on a circle found that above the ground the rest of the circle blurs, and only the centre’s vertical stays sharp; a mark with image velocity has a displacement proportional to its row time, and above eye height the circle’s marks have both. So the circle is undistorted only at eye height, where every mark is undistorted anyway. The one place on the bend that is sharp and undistorted at every height is the vertical line through its centre.

A gyroscope makes the near world worse #

The standard repair for a rolling frame is a gyroscope. It records how the camera turned during the readout, and each row is turned back by its own share — which a turning frame can be straightened found is exact for a camera that only turns. On a bend the camera also travels, and the gyroscope knows nothing of that.

Along the axis, four metres above the eyes, the frame as read is displaced by 14.6 pixels at ten metres, 2.74 at twenty, nothing at the bend’s centre fifty metres out, and a fraction of a pixel beyond. The gyroscope’s correction turns every row back by the camera’s turn over its row time and leaves the travel: 18.3 pixels at ten metres, 4.56 at twenty, 0.73 at the centre, and less than the uncorrected frame only past a hundred metres.

The reason is the cancellation the earlier essay found. On a bend the turn and the travel move a mark’s image in opposite directions for everything nearer than the far side of the circle, and exactly cancel at the centre. The uncorrected frame’s error goes as $|1/Z - 1/R|$ — the streak’s own law — and the gyroscope, by removing the turn, leaves an error that goes as $1/Z$. The two are equal where $1/Z = 1/R - 1/Z$, which is twice the radius. Nearer than that, the correction undoes more cancellation than it removes error. The centre, which the frame had right, it gets wrong by 0.73 pixels.

That turns the practical advice round for a vehicle on a curve. A frame taken on a bend by a camera aimed into it is, as read, already the frame straightened for the bend’s centre: the readout has done the correct thing for a point at that depth. A gyroscope-only stabiliser is the right tool for a camera that turns in place and the wrong one for a camera that turns because it is travelling round a curve, unless everything it looks at is more than twice the curve’s radius away.

The sharp circle, and the streak it came from #

The two figures below are the earlier essay’s, for comparison: the sharp place in plan, and the streak along the axis.

The sharp circle is the set of eye-height points whose image velocity is zero. At eye height a rolling frame is undistorted along the circle and everywhere else too, because every eye-height point is read on the middle row, so for a rolling shutter the circle is not special at eye height and not undistorted above it. What survives from the earlier picture is its central fact: the bend’s centre is a fixed point of the camera’s motion, and a fixed point is right in every picture, however it is read.

The streak’s curve is the rolling displacement’s curve, scaled. That is the practical form of the law: anything a photographer knows about where a camera on a bend is sharp is something they now know about where its rolling frame is undistorted, and by how much elsewhere — the streak, times the row time, over the exposure. Turning and travelling blur different worlds separated the two motions for a camera that follows a subject; on a bend the two are one motion, a rotation about the bend’s centre, and every reading of the picture — blurred, sheared, or corrected — is a reading of how far each point is from that centre’s depth.

What a rigid turn about a far point is #

The whole of this follows from one description. A camera car rounding a bend at steady speed, aimed at a fixed angle to the road, is a rigid body turning about the bend’s centre. Every picture it takes is the same picture of a world turning the other way about that point, and the only points that do not move in its pictures are the points on the axis of the turn: the vertical line through the centre. The rows count hands, not cameras is a reminder from a different field that rows record the order in which a picture was made; here they record it in time, and the axis of the turn is the one place where time does not show.

A rolling frame is therefore a picture in which each row was taken by the same camera at a different angle of the turn. A row-by-row correction that knew the turn and its axis would carry every row to the middle row’s moment exactly for every point on any surface — but it would need the depth of each point, since a turn about a far axis moves near points by their distance from it. A correction that knows only the turn treats the axis as through the camera, and so treats every point as infinitely far away; on a bend, that is right beyond twice the radius and wrong nearer.

What a camera on a curve should do with it #

Three practical rules follow, for anyone mounting a rolling-shutter camera on a vehicle that corners.

Aim into the bend, and the frame is right where it matters most. A camera aimed at the curve’s centre has its one undistorted vertical in the middle of the frame, and everything near that depth is displaced by little: at forty metres, four metres above the eyes, 0.23 pixels. A camera aimed along the road has no such place in front of it. A dolly zoom is a step and a zoom found the same kind of fixed depth for a camera that steps and zooms at once — two motions that cancel at one distance — and the bend’s centre is that depth for a camera that steps and turns.

Correct with the travel, or not at all. A gyroscope’s row-by-row turn is the right correction for a camera turning in place and half of the right correction for a camera on a curve. Applied alone it adds error inside twice the radius. A correction that also knows the speed and a depth for each point can remove both motions; one that knows only the turn should be withheld for anything nearer than twice the radius of the curve the vehicle is on — which, on a road, is most of what a camera sees.

Keep the horizon near the middle row. Every mark at eye height is undisplaced because it is read at the middle row’s moment. A camera pitched down to see the road puts the horizon high in the frame, reads it early, and gives up the one set of rows that was free for every depth. Levelling the camera, or timing the readout from the horizon’s row, keeps it.

What was assumed #

The camera is level and aimed into the bend. A camera aimed along the road, as a camera on a bend is sharp on a circle found, has no sharp range in front of it, and so no undistorted range either; the centre is behind its field of view. A pitched camera puts the horizon off the middle row, and the undistorted row moves with it.

The bend is steady. A car entering or leaving a curve changes its rate of turn during the readout, and the centre of the turn moves; the fixed point is then only instantaneous, and the rolling frame is right about it only to first order.

The readout runs top to bottom with the middle row at the global moment. A readout timed from the first row shifts every row time by half the readout, and adds to every displacement the image velocity times that shift: the horizon row is then displaced too, and the only undistorted set left is the centre’s vertical.

Still open: whether a rolling frame on a bend gives back the bend’s radius #

The law above says the rolling displacement along a column is the image velocity times the row time, and the image velocity is $f\,v\,|1/Z - 1/R|$ along the axis. A column crossing the bend’s centre — a lamp post standing at the centre, or any vertical at a known range — therefore leans in the rolling frame by an amount proportional to $1/Z - 1/R$, and a vertical at the centre does not lean at all.

The measurement that settles what that is worth takes a rolling frame of a road with verticals at several ranges ahead, reads each vertical’s lean to a pixel, and asks whether the range at which the lean vanishes — found by fitting the leans against one over the range — returns the bend’s radius, and how precisely, against the readout time and the number of verticals. If a single rolling frame gives a curve’s radius to a few per cent, a camera with a rolling shutter carries a curvature gauge in every picture it takes on a bend, and the distortion that gyroscopes are fitted to remove is a measurement of the road.