A longer lens buys parallax and not the direction

Far enough away, a pair is one eye held a baseline of thirty centimetres and walked the scene away, growing the scene as it went so that the picture would not change. The parallax that a homography cannot absorb fell as one over the distance, and the recovered direction of travel fell apart along with it, from 3.3° at four metres to 74.5° at two hundred and fifty-six. The whole of the failure was in the ratio of the baseline to the depth. That essay’s closing rule was stated in terms of parallax in pixels against reading error in pixels.

That suggests a remedy nobody needs a theory to think of. If the trouble is too few pixels of parallax, use more pixels. Anyone facing a distant subject reaches for a longer lens, or a closer crop on a larger sensor, and a pair of cameras gets exactly the benefit it seems to: the parallax on the page goes up in proportion. The parallax is what the earlier essay’s rule of thumb asks for. So a longer lens should help, in proportion. The lens’s effect on parallax is simple and exact. What it does for the recovery is a separate question, and the answer to that is that it does almost nothing.

Three ways of walking away #

The earlier essay’s scene grew with the distance on purpose, to keep the picture from changing. There are two other natural arrangements, and comparing all three is the first move. Take a scene about 10 m across at six metres. Then there are three things to do as the distance grows:

• The scene grows and the lens stays. This is the earlier essay’s arrangement: same picture, same relative relief, a bigger world.
• The scene stays and the lens stays. A real scene, seen from further off. It shrinks in the frame.
• The scene stays and the lens lengthens, in proportion to the distance, so that the scene keeps filling the frame.

The fitted slopes are tidy. The growing scene has $-1.00$, as the earlier essay found. The fixed scene under a fixed lens has $-2.02$: it loses a power for the depth and another because its relief has shrunk relative to its distance. The fixed scene under a lengthening lens has $-1.02$. So a longer lens does exactly what it seems to. It gives back one power of the distance. At 24 m, against a fixed lens’s 1.7 px or so of parallax, the longer lens offers about four times as many, the same as if the scene had been made larger to fill the frame.

The closed form explains the slopes. With focal length $f$, baseline $b$, relief $A$, at distance $Z$, the parallax beyond the plane is about $fbA/Z^2$. If the scene is fixed and the lens fixed, that falls as $Z^{-2}$. If $f$ grows as $Z$, it falls as $Z^{-1}$. If the scene grows and so $A$ grows as $Z$, it also falls as $Z^{-1}$. Those two arrangements put the same pixels on the page and look the same in a table. They are not the same.

The same parallax, a different result #

The parallax is only a means. What a pair wants from its parallax is the direction between the cameras, from which everything else follows. So plot that direction’s error, with marks read to 0.3 px, against the parallax on offer.

If parallax in pixels were the whole story, those curves would coincide, and they do not. At 24 m the two arrangements with about the same 1.8 px of parallax have very different directions: 18.5° for the scene that grew and 58.0° for the longer lens. At 48 m it is 37.1° against 78.5°. A longer lens, delivering just as many pixels of parallax, is three times further from the answer.

So the earlier essay’s rule of thumb, for all its usefulness, was incomplete. It held the arrangement fixed, and in that case parallax does measure the conditioning. Change the arrangement and the number of pixels stops being the thing that says how well a direction is fixed.

What the lens cannot change #

The reason lies in what fixes a direction. When a camera steps sideways, a mark near the middle of its picture moves one way and a mark far off to the side moves in a slightly different direction, by an amount set by how far off to the side it is. That pattern, across the picture, is what tells a step sideways from a turn of the head, or from a step forward, and it is a function of the angle at the camera between the marks.

A scene 10 m across occupies a cone of 82° from 6 m, 47° from 12 m, 24.5° from 24 m and 6° from 96 m. Nothing in front of the camera changes that. A longer lens spreads the same narrow bundle of rays across more pixels, so reading errors in those rays are smaller in angle, and that is real. But the rays have become nearly parallel, and in the limit of parallel rays a camera is an orthographic one, which is the arrangement with a famous and much-studied ambiguity between a turn and a slope in depth, called the bas-relief ambiguity. A step forward does nothing to an orthographic picture. So a very long lens does not fail because of too few pixels. It fails because there is less and less of what needs to be read, however many pixels are spent on it.

That explains why the error curves saturate near 90° rather than falling: with no information about the direction left, a direction read from noise is uniform, which a turn of the head also noted when the baseline fell to nothing. It explains too why the scene that grows does better at the same parallax. It keeps its cone. It is also a scene nobody can have, in the sense that a mountain cannot be arranged to grow as the observer walks away. It is the arrangement in which a small baseline faces a large wide scene.

Zooming in on one scene #

A second experiment makes it plainer. Stand still at 24 m in front of a very large scene, with marks all over it. Lengthen the lens, and use the marks that remain in the frame, always the same number. The parallax on offer, measured over those marks, rises as the lens does.

From the usual focal length to six times that, the parallax on offer increases by a factor of 4.81, and the error in the direction stays between 21° and 37°, with no trend. The pixels have been spent on a smaller and smaller cone, and the two effects cancel. The wiggles in the curve are just the changing sample of marks, and they are about the sampling variation of this measure with 30 marks.

So here is one case where the arithmetic gives a flat answer to a common intuition. A person with a fixed position and fixed marks, considering a longer lens for this one purpose to improve the recovery of the camera’s motion, gets nothing from it. It gives a much better reading of the parts of the scene in the frame, and it gives better details on what is there, so it may well be a good idea for other reasons. It does nothing for the pose.

Why a table of pixels misleads #

The figure for the three arrangements shows parallax in pixels, and it is easy to see why a person would read that as the score. In whole pixels cut space into shells, for a calibrated rig, a pixel of disparity really is a fixed quantity of depth, and a longer lens does cut finer shells. For a pair of known cameras, pixels are what count. The range a pair cannot see past is stated in those terms, correctly, for that reason. The trouble starts when the same intuition is carried to a pair that has to find out where its cameras are.

The two questions differ in what they are asking of a single mark. A depth comes from one point’s disparity, and all its information is in that number. A direction has to come from how marks in different parts of the picture disagree with one another. So a depth is a function of a quantity read at one place, and a direction is a function of a pattern read across the frame. Magnifying the picture scales a single reading up, and it also scales the pattern. But the pattern’s departure from a pure shift has its own size, proportional to how far the marks are from the centre in angle, and that has not changed. Relative to the shift, the pattern has shrunk as the cone has.

The arithmetic gives one small form of this, and it can be written down. For a small baseline, the direction’s recoverable part goes as the marks’ angular spread times the parallax, in pixels. A longer lens raises the second and lowers the first, when a fixed scene is the subject, and the product is what the figure in place shows flat. A larger real scene at the same distance keeps the spread and keeps the lens, and it gets the benefit of the first with no loss of the second, which is why the wide scene’s curve lies so far below. It is also why another picture of the same sweep does better by turning than by waiting: anything that widens the set of directions a camera has seen is directly useful for the direction, whereas a higher magnification is not.

What does work #

What would help is any change in the geometry of the rays themselves, since those are what the direction is being read from. The obvious one is the baseline.

At 24 m with the longer lens, widening the baseline from 0.15 m to 4.8 m takes the error from 70.6° to 5.1°. With the wider scene it goes from 35.7° to 1.3°. Doubling the baseline roughly halves the error throughout, at each arrangement, as long as the direction is reasonably well defined at all. So a pair’s condition is really in the ratio of baseline to depth. The earlier essay said so. A longer lens does not touch that ratio, though it does raise the number of pixels that the same ratio produces.

That makes sense of practice. Aerial surveys do not put a long lens on a pair and expect it to help the pose. They fly the cameras far apart, with a base-to-height ratio in the region of a half. A long lens is used when the job is resolution on the ground, and then the base-to-height ratio has to be held by moving further between exposures, since it has nothing to do with the lens. Nothing in that was unknown. What the figures here add is a number for the size of the mistake made by anybody who reads a table of parallax values without looking at the cone they came from.

There is one other way to see the same limit, by asking how good the reading must be.

At a reading error of 0.1 px, the wide scene is at 7° and the long lens is at 28°. At 0.05 px, roughly half of both. The error falls with the reading error, in the same way for both, and the ratio holds. So a long-lens pair could always read its way out, with a reading better by the ratio of the two curves. That is a large ratio to find.

An honest role for the longer lens #

None of this makes a longer lens useless to a pair. Three things it does improve are worth keeping separate from what it does not.

The point’s depth, once the direction is known. Triangulating a point with a fixed baseline and known cameras gives a depth error that falls in proportion to the focal length, by the formula depth is a reciprocal measured. If the pose has come from somewhere else, a calibrated rig or a good survey, then all those pixels go straight into a better depth. A calibrated stereo head with a long lens is a perfectly good instrument. It is the recovery of its own geometry from its pictures that is ill-conditioned.

The homography itself. A turn of the head treats that map as what a pair reduces to at the limit. A plane’s map reads well in pixels, and if the job is to warp one picture onto another at a stated depth, then the longer lens is a real gain. How flat is flat enough lives in that setting, where whether the parallax exceeds the reading is the right question.

Whether a scene is degenerate at all. A pair’s own check for whether a homography explains everything is exactly about pixels. A long lens makes that check more sensitive. It will say sooner that parallax exists, and the earlier essay’s reading on whether a scene is too distant holds in a more forgiving form. So a person asking “is there any parallax here to use?” gains from the lens. A person asking “which way did the camera step?” does not, and the same lens can leave the first person satisfied and the second badly misled.

Where the comparison is fair to the lens, and where it is not #

The marks are scattered on a slanted plane with relief as in the earlier essay, and all arrangements use the same relative relief at the reference distance. The growing scene’s relief grows with it. The fixed scene’s stays the same in metres, so its relative relief shrinks with distance. That is the honest comparison for a fixed scene, and it contributes to the lens’s disadvantage. A scene with more relief would help every arrangement, though by different amounts, and the difference between the curves would not disappear.

The recovery is the plain five-parameter estimate from 30 marks, with no refinement. A bundle adjustment would reduce the errors a good deal and might change the size of the separation, though it could not recover what the rays do not contain. Reading errors are independent, of 0.3 px, which is optimistic for a low-texture distant scene and pessimistic for a good feature detector. Holding that error fixed in pixels is itself a concession to the long lens, because the same 0.3 px subtends a smaller angle through it. The curves already grant it that advantage.

The median of 60 trials is reported at each point. Near 90° that median is simply noise, and a reader should not read shape into the top of those curves.

Still open: what a known rotation does for the long lens #

The longer lens fails to separate a turn from a step. If the turn were known from somewhere else, from an inertial sensor or from a distant reference such as the horizon, the remaining unknown would be a direction alone, and the parallax in pixels might be just what it needs. In that case, the long lens’s pixels would count, since nothing else would be competing with them.

The measurement that settles it repeats these sweeps with the rotation given, to stated accuracies from exact to a few tenths of a degree, and asks three things: whether the three arrangements’ curves then collapse onto one curve in parallax, as the rule of thumb would have them; how accurately the rotation has to be known before a long lens beats a short one; and whether that accuracy lies within what an inexpensive inertial sensor actually delivers.

Parallax in pixels measures a pair’s condition only when the arrangement has not changed. A lens spends more pixels on the same rays, and what fixes a direction is how far apart the rays are.