An epipole in the picture leaves a blind disc

The image of the other eye found that in each of two photographs there is a point that is the other camera, and that for two cameras aimed at a common subject from a few metres apart it sits thousands of pixels off the picture. It computed how far: the epipole lands inside a 690 px frame only if the cameras stand more than two and a half times further apart than the subject is away. For a pair verged on a subject the epipole is off the paper essentially always.

A camera that walks forward is not a verged pair. Its second picture is taken from a point straight ahead of the first, so the image of the first eye in the second picture — and of the second eye in the first — is in the middle of the frame. Every car-mounted camera, every walking phone and every drone flying toward something is this case, far more often than it is the textbook side-by-side pair.

The epipole being in the picture changes what the picture can say about depth, and the change has a shape.

Why the middle is blind #

Depth from two pictures is read from how far a mark moves between them. A point is a line over there set out the constraint: a mark in one picture must lie on a line in the other, every such line runs through the epipole, and where along its line the mark falls is its depth. Two cameras side by side make those lines horizontal and the movement a disparity, and depth is a reciprocal turned that disparity into a distance.

For a camera stepping forward, the lines are not horizontal. They run radially out from the centre of the picture, and every mark slides straight away from the epipole between the first picture and the second, faster the nearer it is and the further from the centre it sits. A mark a long way out slides a long way. A mark near the epipole slides hardly at all — and a mark exactly at the epipole does not move, whatever its depth, because it lies on the line through both eyes and both eyes see it in the same direction.

So near the epipole the whole signal a depth is made from shrinks toward nothing, and a reading error of fixed size does not. The consequence is a region around the epipole where the reading error is a large share of the slide, and therefore of the depth. That region is the disc.

One pixel’s cost, at one distance #

The measurement is direct. For each mark in the first picture, place a surface at a stated distance, find where the point lands in the second picture, move it one pixel along its epipolar line away from the epipole, and intersect the two rays again. The change in the recovered depth, as a share of the true depth, is what one pixel of reading costs that mark.

For a surface 8 m away, with the camera stepping 0.5 m and a 60° field, the curve inside which one pixel costs half the depth has a radius of 13.1 px. The quarter curve is 41.1 px out and the tenth curve 122.7 px — a disc about a third of the picture’s height across, in the middle of the frame, which is where a forward-looking camera usually wants to see what it is heading toward.

The curves are circles because the step is straight along the camera’s axis and the arrangement is symmetric about it. A step with a sideways component moves the epipole off the centre, and the disc moves with it: it is centred on the image of the other eye, not on the picture.

What one pixel costs, closer and further from the epipole #

Reading the cost along one radius shows how steeply the disc’s edge is set.

At 8 m, one pixel costs 6.5 per cent of the depth at 190 px from the epipole, 9.6 per cent at 128, 17.7 at 64, 29.9 at 32, 45.4 at 16, 61.1 at 8, 74.0 at 4 and 82.7 at 2.

Far from the epipole the cost falls almost in proportion to the distance from it — a slope of −0.92 on logarithmic axes between 64 and 190 px — which is the simple account: the slide is proportional to the distance from the epipole, so the cost of a fixed error is inversely proportional to it. Close in, the curve flattens to a slope of −0.22 between 2 and 8 px. There the slide itself is a fraction of a pixel, so a one-pixel error is more than the entire signal, and the recovered depth has already lost most of what it could lose. The cost does not grow without bound; it saturates near all of the depth.

That is a more useful shape than a single power law would be, because it says where the transition between usable and useless sits: at the distance from the epipole where the slide equals a few pixels, which on this arrangement is somewhere between 30 and 100 px from the centre.

The disc grows with distance #

Holding the edge at a tenth of the depth and moving the surface further away shows the disc’s size depends on what is being photographed.

The tenth-of-the-depth disc is 4.0 px in radius for a surface a metre away, 19.5 px at 2 m, 54.0 at 4, 122.7 at 8, 247.4 at 16 and 443.7 at 32. The half-the-depth disc goes from a fifth of a pixel at a metre to 60.7 px at 32 m.

Doubling the distance a little more than doubles the radius — from 2 to 4 m the tenth disc grows 2.8 times, from 8 to 16 m 2.0 times — because a surface twice as far away slides roughly a quarter as far between the pictures, and the fixed reading error is correspondingly a larger share. Past 16 m the tenth disc is wider than the 200 px from the picture’s centre to its top edge, so for anything that far away a half-metre forward step recovers no depth to within a tenth anywhere up the middle of the frame, and only the picture’s corners carry usable depth.

This is the same reciprocal law depth is a reciprocal found for a side-by-side pair and the range a pair cannot see past turned into a hard bound, arriving in a different geometry. There, the whole picture loses depth together as the scene recedes. Here, the loss starts at the epipole and spreads outward.

The arithmetic of the slide #

The shape of all three figures follows from one line of arithmetic, and working it through says which of their numbers are general and which belong to this lens.

A point a little off the camera’s axis, at depth $Z$, appears at a distance $r$ from the epipole in the first picture. After the camera steps forward by $b$ it is at depth $Z - b$, and since its distance from the axis has not changed, its image distance grows in proportion: $r' = rZ/(Z - b)$. The slide is the difference, $r' - r = rb/(Z - b)$. Reading depth back from the slide and asking how far one pixel of error moves it gives, to first order, a relative depth error of

$$\frac{\delta Z}{Z} \approx \frac{(Z - b)^2}{Z\,b\,r}$$

per pixel, with $r$ in pixels.

The focal length has dropped out, because $r$ is already measured in the picture’s own pixels. And the formula accounts for each figure’s shape. The cost falls as one over the distance from the epipole, which is the −0.92 slope far out. It grows roughly in proportion to the depth for depths much larger than the step, which is why the disc widens as the surface recedes. And it depends on $Z$ and $b$ only through their ratio, which is the subject of the next figure.

As numbers, the first-order formula is somewhat pessimistic. At 8 m and 190 px it predicts 7.4 per cent against the measured 6.5; at 64 px, 22 per cent against 17.7; and for the tenth-of-the-depth disc it predicts radii of 22.5, 61.3, 141 and 300 px at 2, 4, 8 and 16 m against the measured 19.5, 54.0, 122.7 and 247.4. The difference is the difference between a derivative and a one-pixel step: moving a mark outward by a whole pixel makes it look nearer, and the depth it gives changes by less than the tangent line says. Close to the epipole, where a pixel is more than the whole slide, the formula stops describing anything and the measured cost flattens instead.

Doubling the step is halving the distance #

The formula’s dependence on $Z/b$ alone is not an approximation. It is exact, and it can be checked directly.

With a step of a metre instead of half a metre, the tenth-of-the-depth disc is 4.0 px at 2 m, 19.5 at 4 m, 54.0 at 8 m and 122.7 at 16 m. Those are the half-metre step’s radii at 1, 2, 4 and 8 m, exactly. At 8 m the three nested discs shrink from 13.1, 41.1 and 122.7 px to 5.2, 17.5 and 54.0.

The reason is the one two views give shape and no size established: scale the whole arrangement — the step and the scene together — by any factor, and every picture is unchanged, mark for mark. A one-metre step past a surface at 16 m takes the same two photographs as a half-metre step past a surface at 8 m. The reading error is fixed in pixels, so it is the same in both, and every cost built from the pictures and the reading error is the same too.

That makes the practical rule unusually clean. The disc a forward-moving camera cannot see depth in depends on the ratio of how far away things are to how far the camera moves between pictures, and on nothing else about the geometry. A camera that wants to halve the disc for a given scene can take pictures twice as far apart — every other frame of a video instead of every frame — and it will see exactly what a camera twice as close would see.

The same step, taken sideways #

The disc is a property of where the epipole is, and the cleanest demonstration is to take the same half-metre step in a different direction.

At 35 marks spread across the picture, for a surface 8 m away, stepping 0.5 m forward costs from 3.6 per cent of the depth at the corners to 14.7 per cent beside the centre — and the grid was placed so that no mark sits on the epipole itself, where the cost would be the whole depth. Stepping the same 0.5 m sideways costs between 2.41 and 2.75 per cent at every one of the 35 marks.

The sideways step is better everywhere, including at the corners, and nearly uniform. It sends the epipole to infinity, so no mark in the picture is near it, and every mark slides by roughly the same amount between the two pictures: the baseline divided by the depth, times the focal length. The forward step spends part of its baseline moving marks away from a centre, and the part it spends near the centre is wasted.

The comparison is not an argument against moving forward — a vehicle moves the way it moves. It is a statement about what the pictures a forward-moving camera takes can and cannot supply. For a camera driving toward a wall, the wall straight ahead is the one thing whose distance the two pictures are worst at telling.

What else lives at the epipole #

The blind disc is one face of a point that has several, and the others are worth placing beside it.

The disc is where a flat scene’s repair works best. Two marks off a known plane find the other eye found that the parallax of raised marks, measured against a plane’s map, points at the epipole — and that it finds the epipole to under three pixels when the epipole is inside the picture and misses by hundreds when it is not. The arrangement that puts the epipole in the frame is the good one for finding it, and the bad one for depth near it.

A turn of the head has no epipole to be near. A turn of the head is not a step sideways measured the case with no baseline at all: every mark moves, none of the movement carries depth, and the failure is uniform across the picture. The forward step is between that case and the sideways one: it has a baseline, and a region of the picture where the baseline buys almost nothing.

Far enough away, the whole picture is the disc. Far enough away, a pair is one eye measured parallax falling as one over the distance. For a forward step the same fall is what grows the disc until, past a few tens of metres, it has covered everything.

The triangulation’s convention matters more near the epipole. The midpoint is a choice of ruler found that the midpoint of two nearly parallel rays is not a measurement but a convention. Near the epipole the two rays to a mark are as close to parallel as they get, so the difference between intersecting them one way and another is largest exactly where the disc is.

What this does not settle #

One reading error, applied one way. The cost here is the change in depth when a mark in the second picture moves one pixel along its epipolar line, away from the epipole. A real matcher’s error has a component across the line as well, which the epipolar geometry partly rejects, and a component along it of either sign; moving toward the epipole rather than away gives a different, larger number near the centre. The disc’s radius is a statement about a one-pixel slide outward and is quoted as that.

One step, one field of view, one principal point. A half-metre step with a 60° lens. A longer step shrinks every disc in proportion, and a wider lens makes each pixel a larger angle and grows them. The shape — a disc about the epipole that grows with the scene’s distance — does not depend on those choices; the radii do.

No scene structure. Every mark here is placed on a surface at a stated distance, independently. A real forward-moving camera sees a road that gets further away toward the horizon, which sits near the epipole, so the most distant part of the scene and the blindest part of the picture tend to coincide.

Still open: whether a third picture closes the disc #

The disc exists because two pictures taken along one line share an epipole, and near it the slide is small. A third picture taken from off that line has an epipole somewhere else with respect to each of the first two, and its own lines through a mark near the first pair’s epipole run in a different direction.

The question that leaves is how much a small sideways offset buys. A camera driving forward sways slightly from side to side; three pictures of which one is displaced sideways by a tenth of the forward step would give each mark near the centre a second slide, across rather than along. Measuring the depth cost at the centre of the picture as that sideways displacement grows — from nothing to the size of the forward step — would say whether a vehicle’s natural wobble is worth anything to a depth estimate or whether the disc survives any offset a steering wheel produces.