A drift across the lane closes the blind point, slowly

Steering swings a camera too little to see ahead looked for a sideways movement that would give a forward-driving camera’s blind point — the direction it is driving towards, which a camera moving straight along its own line of sight cannot range, because there the image of the other eye lies on the point itself — a depth. A car yaws about its rear axle, so a camera mounted ahead of the axle is swung sideways by every steering correction: half a degree swings a windscreen camera by 1.3 centimetres and a bumper camera by 3.1. Over three frames, that left a pixel of reading error costing 96 per cent and 40 per cent of the depth eight metres ahead. A usable depth needed the arm times the yaw to reach about seven metre-degrees a frame — two degrees of steering a frame on a bumper, which nobody drives.

The essay ended on the sideways movement a car actually makes. Drivers do not hold the centre of their lane to the centimetre. A car wanders across its lane by tens of centimetres over a few seconds, slowly, and that wander moves every camera on it sideways by the same amount — far more than any lever arm’s swing. The question was what the wander is worth, and whether the blind point is closed not by where the camera is mounted but by how long the car is allowed to look.

Looking back from now #

The car drives at fifteen metres a second and films at thirty frames a second. Its lateral offset in the lane follows a sine of twenty centimetres either side and a four-second period, and its heading follows its path, as a car’s must. The question is put the way a car has to answer it: at this moment, with the frames of the last so many seconds, how well is the road a stated distance straight ahead of the car known? Each past frame saw that piece of road from further back and from a little to one side. The depth error a pixel of reading in every frame costs is computed from all of them at once, each frame’s mark a ray and a point is a line over there in every other frame, and since it depends on where in its wander the car happens to be, the median over the wander’s phase is reported.

In one second the camera moves 27.8 centimetres sideways. That is nine times what half a degree of steering swings a bumper camera, from a car that is not steering at all in any sense its driver would notice. The road twenty metres ahead, seen from every frame of that second, is known to 22.8 per cent a pixel: still not a usable depth, but already far better than any lever arm gave the earlier essay.

The slider lengthens the look. Half a second back, the camera has moved less and the depth twenty metres ahead costs 45 per cent a pixel; two seconds back, 13.5; three seconds, 11.3. A longer look buys more sideways movement, but every frame it adds is older and further behind, and the piece of road ahead that matters is smaller and further from it.

A slow approach to a usable depth #

The curves bend over. Twenty metres ahead, the error falls from 82 per cent with a quarter-second’s look to 23 per cent with one second’s, and from there only to 13.5 with two and 10.5 with four. Eight metres ahead and forty metres ahead behave alike and end in the same place, near eleven per cent after four seconds. No look of this length brings the point straight ahead under ten per cent a pixel.

The reason the gain slows is the forward motion. In four seconds a car at fifteen metres a second has driven sixty metres, so the frame four seconds back stood eighty metres from a point twenty metres ahead of the car now. Its sideways displacement is a smaller angle at that distance than the same displacement a second ago, and every older frame adds less: far enough away, a pair is one eye, and the old frames are far away. What a car can learn about the road just ahead comes mostly from its last second or two, whatever it remembers before that. A camera that keeps four seconds of frames is spending most of its memory on viewpoints that add almost nothing: the second second of the look cuts the error by two-fifths, and the third and fourth seconds together take it only from 13.5 per cent to 10.5.

The answer to the earlier essay’s question is therefore yes and not enough. A car’s ordinary drift across its lane gives the direction of travel a depth that steering never could, and gives it within a second. It does not, by itself, give it a usable one, from a car that keeps its lane as well as this one does.

How much wander it takes #

The sideways movement is the baseline, and a depth’s error falls as one over its baseline, so the error here falls in proportion to the wander. A car that holds its lane to five centimetres either side gives the road twenty metres ahead 91 per cent a pixel from a second’s look: blind, in effect. One that wanders ten centimetres gives it 45 per cent; twenty, 23; thirty, 15; fifty, 9. With two seconds of looking, a wander of thirty centimetres either side reaches nine per cent, inside the ten that the earlier essays took as a usable depth.

So the point the car is heading for is ranged best by the car that keeps its lane worst. That is not a recommendation to weave. It is a measure of how little a well-driven car’s own motion offers the one direction it most needs to see, and of why that direction is normally ranged by something other than the forward camera’s motion: a second camera beside it, a radar, the size of a known object.

The period, and the speed #

The wander’s amplitude sets how far the camera can move; its period and the car’s speed set how much of that movement falls within a look, and how useful it is when it does.

The same twenty centimetres either side, swept every two seconds, gives the road twenty metres ahead 12 per cent a pixel from a second’s look; swept every sixteen seconds, 91. Within one second a quick wander carries the camera through most of its range, and a slow one through a small part of it, so what counts is not the wander’s amplitude but how far the car moves sideways within the look. A slow drift across the lane, over many seconds, is the least useful kind, and it is the kind a car that is driven smoothly makes.

The car’s speed works the other way. The same wander gives the road twenty metres ahead 13.4 per cent a pixel at five metres a second and 40.2 per cent at thirty. A second ago a fast car was thirty metres further back, the road ahead of it now was fifty metres from it then, and its sideways displacement subtended a smaller angle; a slow car’s past is near where it is. The wander is a fixed sideways distance in a fixed time, and a car that covers more road in that time spends it on a smaller angle. The blind point is least blind for the slow car in traffic, which needs it least, and most blind on the open road at speed.

The wander’s own turn #

A car that wanders also turns: its heading follows its path, and a twenty-centimetre wander every four seconds at fifteen metres a second turns the car by up to 1.2 degrees either way. The frames here carry that turn, and a turn moves the picture, not the blind point: it shifts where the direction of travel falls in each picture without ranging it. What ranges it is the sideways movement alone.

The turn does have one consequence the frames here leave out, because their camera stands over the point the car turns about. A camera mounted ahead of the rear axle is swung by that 1.2 degrees as well, by the earlier essay’s lever: a bumper camera 3.6 metres ahead by up to 7.5 centimetres, a windscreen camera by 3.1. Those swings are a quarter of a cycle ahead of the wander — the heading is steepest as the car crosses the middle of its lane, where its offset is nothing — so they add to the wander’s twenty centimetres at right angles rather than in line: a bumper camera’s sideways movement swings through about 21.4 centimetres either side instead of 20, seven per cent more, and a windscreen camera’s through 20.2. The mounting matters a little after all, through the turn the wander brings with it rather than through any steering the driver does on purpose, and a little is all it matters: the car’s own movement across the road is the baseline, and no place on the car changes it by more than a few per cent.

What the lever arm and the wander are each worth #

Put beside the earlier essay, the wander and the lever arm are the same kind of thing at very different sizes. Both are sideways movements of the camera that give the direction of travel a baseline. The lever arm’s is the arm times the yaw, centimetres for any steering a driver makes; the wander’s is the car’s own movement across its lane, tens of centimetres within a second or two, and it moves every camera on the car alike, wherever it is mounted.

So the earlier essay’s conclusion — that where the camera is mounted hardly matters, because steering swings it too little — holds with a sharper reason. The swing that matters is not the camera’s about the car but the car’s across the road, and the mounting cannot change it. What changes it is how the car is driven and how long the camera’s past is kept. A sway gives the blind centre a depth, not a good one found the same shape for a deliberate sway: any sideways movement gives the epipole a depth, and the size of the movement decides whether the depth is any good. The wander is the sway a car makes for free, and at ordinary sizes it is not quite enough.

What remains blind #

The measurements above are straight ahead, at the epipole itself. An epipole in the picture leaves a blind disc found that the blindness is not a point but a disc around it, whose size is set by the baseline; the wander shrinks the disc as it grows the baseline. With the road twenty metres ahead and ten per cent a pixel as the line, the disc within which the depth is worse than that has a radius of 34 pixels with half a second’s look, 17 with one second, 7.6 with two and 2.5 with four. At this camera’s focal length, 17 pixels is about a degree and a half — about half a metre either side of the point twenty metres ahead.

So a second’s wander leaves a blind patch about a metre across, centred exactly where the car is going, and everything outside it ranged to better than ten per cent; four seconds shrink the patch to a few tens of centimetres. That is a more useful statement than the error at the centre alone. The road surface, the lane markings, the kerbs, the vehicles to either side all lie outside the patch within a second or two; what stays inside it is whatever is directly in the car’s path at the distance being asked about — which is, unfortunately, the thing it most needs to range.

What the wander cannot give at all is a depth that is current. What it does give has a size — which two views give shape and no size would deny it — only because the car’s odometry supplies the scale. Every frame that contributes is in the past, and anything that has moved since — another car, a pedestrian — has moved in the frames being combined. A depth from the car’s own motion is a depth of the still parts of the scene; for the moving parts, the earlier essays’ two-camera rigs are the only answer that does not wait.

More frames, and faster ones #

The figures here use thirty frames a second. A camera that films faster has more frames in the same look, each a fresh reading of nearly the same geometry, and the error falls roughly as the square root of their number: a second’s look twenty metres ahead costs 31 per cent a pixel at fifteen frames a second, 22.8 at thirty and 16.4 at sixty. Doubling the frame rate buys what about forty per cent more wander would — the wander’s gain is in proportion, the frames’ only as the square root — but a car’s camera can choose its frame rate where it cannot choose how its driver steers.

The gain has the same limit as the wander’s. The frames added are not new viewpoints but more readings of the same few: a camera filming at sixty frames a second through a twenty-centimetre wander still has only twenty-eight centimetres of baseline in a second, read more often. It narrows the reading error, which is what a pixel of error per coordinate per frame assumes is the whole of the uncertainty. A feature tracker’s errors from one frame to the next are not independent, and correlated errors do not average away, so the square-root gain is an upper bound on what a faster camera buys.

What was assumed #

The wander is a single sine. A real car’s lateral motion is irregular, a mixture of slow drifts and quicker corrections. What matters is the sideways distance covered within the look, which an irregular wander delivers in bursts; the median over the sine’s phase here stands for that mixture, and a burst of correction would range the road ahead better for the second it lasts.

The car’s past positions are known. Every frame’s camera is placed exactly. A real car’s past positions come from its own odometry and from the pictures themselves, and a sideways movement of twenty centimetres must be known to a centimetre or so for its parallax to be read; an error in the camera’s track is an error in the baseline, and the depth inherits it.

The road ahead is still and the camera’s picture distortion is removed. The road is ranged as a fixed point, and the pictures are exact pinhole projections read to a pixel in each coordinate.

Thirty frames a second. More frames in the same look add readings of nearly the same geometry, and narrow the error roughly as the square root of their number; fewer widen it.

Still open: whether a second camera’s short baseline and the wander’s long one combine #

The two-camera rigs of the earlier essays range the scene in an instant over a short fixed baseline across the car; the wander ranges it over a second, over a baseline that comes and goes. A pair of cameras thirty centimetres apart ranges the road twenty metres ahead to about eleven per cent a pixel — the square of the distance over the focal length times the baseline, as depth is a reciprocal found — which is no better than a second of a twenty-centimetre wander gives, and the wander’s own baseline is at the mercy of the driver.

The measurement that settles what the two are worth together takes a stereo pair on the car — two cameras a stated distance apart across it — wandering in its lane as above, and asks for the depth twenty metres ahead from both cameras’ frames over the last second: whether the pair’s instantaneous baseline and the wander’s accumulated one add, so that a modest pair on a car that keeps its lane well still ranges the road ahead to a few per cent, or whether the pair’s baseline is spent across a direction the wander already covered, and the combination is no better than the larger of the two.