What two parallel views leave free

The twoviews field ends on a clean sentence: two perspective pictures give shape and no size. Every pairwise distance ratio in a reconstructed courtyard matches the world’s to fourteen digits, and the courtyard’s actual size is absent rather than uncertain.

Ask the same question of two parallel views — two axonometric drawings of the same object from two directions, or two photographs from far enough away that the divide has gone — and the answer is worse in a way that is worth being precise about. The shape is not determined either.

What the recovery is shown #

The separation this site insists on applies here: the recovery is given the pictures and nothing else.

Two parallel views of $n$ points are $2 \times 2 \times n$ numbers. Subtract each view’s own centroid — which removes the translations, since a parallel view of a translated object is the same picture moved — and stack them into a $4 \times n$ measurement matrix.

That matrix has rank three, and the reason is the whole of the method. Each row is a linear functional of the world point: view $v$’s first row is $\mathbf{p}\cdot\mathbf{r}_v$, its second is $\mathbf{p}\cdot\mathbf{d}_v$. Four linear functionals of a three-dimensional quantity cannot be independent, so the matrix factors as a $4\times3$ times a $3\times n$ — a set of camera axes and a set of world points.

The fourth singular value is therefore the check on whether the pictures were parallel views of a rigid set at all, and it is the only quantity here computable from the pictures alone. Everything else in this essay is a comparison with something the recovery never saw.

The first ambiguity is the factorisation’s own #

A factorisation is not unique. Slip any invertible $3\times3$ matrix $A$ between the two halves — replace the camera axes by (axes $\times A$) and the points by ($A^{-1} \times$ points) — and the product is unchanged, so the new pair reproduces both pictures exactly.

So the raw recovery determines the scene up to an arbitrary affinity: nine free numbers. That much is standard and is not the interesting part, because most of it can be removed by insisting that the recovered camera axes are what a parallel view’s axes have to be — two perpendicular unit vectors.

Each view supplies three equations for that: the two rows are unit, and they are orthogonal. The unknowns live in a symmetric $3\times3$, which has six entries.

Two views: six equations, six unknowns. Which reads as exactly determined, and is not.

The rank is not the count #

This essay’s own draft said “exactly determined” on the strength of the arithmetic above, and the measurement disagreed.

The system’s six singular values, for a two-view pair, come out at 1.4, 1.0, 0.80, 0.71, 0.66 — and 9.4e-9. The last one is zero to the precision the decomposition can offer, with the next one up at 0.66, so the gap is not a judgement call. The rank is five. Six equations, six unknowns, one free direction.

The free direction is not a defect of the solver and it is not noise. It is a one-parameter family of metric upgrades, each of which produces a different solid, and every one of them redraws both original pictures to 7e-16 m.

The deformation the parameter applies is a stretch along one axis — the solid gets deeper or shallower between the two viewing directions while both outlines stay put. That is the ambiguity Koenderink and van Doorn identified for two orthographic views, and the figure is it, walked from end to end.

What makes a family a family, and not a solver’s excuse #

A one-parameter family of answers is exactly what a badly conditioned solve looks like from the outside, so the claim needs three separate things to be true, and each is checked on its own.

Every member redraws both pictures. If they did not, the family would be the solver failing rather than the problem being underdetermined. Measured across eleven members: 7e-16 m, which is arithmetic noise on a scene about a metre across.

The members are different solids. If they differed by less than the noise, the family would be a rounding artefact. The measure used is the departure from a similarity — the ratio of the largest stretch to the smallest in the best affine map from the scene to the member — which is 1 for a solid that is the scene in another frame and runs to 8.59 at the end of the family.

And the scene is one of them. If it were not, the family would be the ambiguity of some other problem. It is: searching along the parameter finds the scene at a specific position, fitting to 1e-8 of the set’s own size.

That third check has a property worth naming rather than hiding. It is done by asking the scene, which is precisely the information the two pictures do not contain. It is not a recovery and is not offered as one; it is the measurement that turns “the solver returned a family” into “the answer is in the family and the pictures cannot say where”.

The finding this essay had to reverse #

The count in the section above was written here first as exactly determined — six equations, six unknowns, done — and it stood for as long as it took to run the singular values.

That is the second time in three phases this site has written down an over-determination that was not there. The anamorph recovery was believed over-determined, and fed a wrong aspect ratio it returned a residual of 5.9e-16 and a different eye. Both mistakes have the same shape: a count of equations was taken for a rank, and a rank is a measurement.

The general form is worth carrying: equations can be dependent for reasons that are not visible in how they were written down. Here the dependence comes from the two view directions spanning only a plane, so the constraints they impose on the upgrade cannot reach the direction perpendicular to both. Nothing in “three equations per view, two views, six unknowns” shows that, and nothing except computing the rank would have caught it.

Where the free direction lives, and how it appears in the limit #

The rank deficiency has a location, and saying where it is makes the family less mysterious than “one free direction” suggests.

The two view directions $\mathbf d_1$ and $\mathbf d_2$ span a plane. Their common perpendicular $\mathbf n$ lies in both image planes, so both pictures measure the $\mathbf n$ component of every point directly and exactly — nothing about that axis is free. What is free is a deformation acting inside the plane the two directions span: the solid stretches or compresses between the two viewing directions while both outlines stay where they are. That is the relief transformation, and it is why the figure’s members differ in depth and not in width.

Three views whose directions do not all lie in one plane close it, which is the classical count for orthographic structure from motion and is what the rank measurement reports. Two views leave a one-parameter family; three leave nothing but the mirror image, and the mirror is the reversal this field measures separately rather than a further continuous freedom.

The perspective case is the same problem with a term that vanishes #

The contrast with two perspective views is usually stated as a difference in kind — one determines the shape and the other does not — and it is better read as a limit.

What a perspective view adds is the divide: the picture carries $1/z$, and it is that term which breaks the affine freedom the factorisation leaves. Its size relative to the rest of the picture is of order $L/D$ for an object of extent $L$ at distance $D$. So the information that removes the relief family is proportional to $L/D$, and the conditioning of the recovered depth degrades as $D/L$ — which means two perspective views of a small object across a large room determine its shape in principle and scarcely at all in practice.

Taking $D \to \infty$ with the focal length raised to match is the limit this field is built on, and along it the constraint thins continuously to nothing and the one-parameter family opens up. So the two results are not two regimes: the relief ambiguity is the $L/D \to 0$ end of a conditioning that is always present. A photographic pair at fifty metres is nearer the parallel case than to the textbook one, and a reader would not know from the pictures.

That also places the finding among the collection’s other freedoms. Two views give shape and no size reports one free number that survives any number of perspective views, because scale is a symmetry of the whole arrangement; seven numbers no picture can name counts the gauge a reconstruction floats in. The relief parameter is a different animal from both: it is not a symmetry of the world and it is not a gauge, it is a genuine shape ambiguity that a third view removes — the only one of the three that more pictures fix.

Why a second view removes so much less than expected #

The comparison with the perspective case is sharp, and it says exactly which piece of information the divide was carrying.

A perspective view of a point encodes its direction from the eye. Two of them intersect two directions, and the intersection is a point — up to the global scale, which the pair cannot fix because scaling the scene and the baseline together changes nothing.

A parallel view of a point encodes two of its three coordinates and discards the third outright. Two parallel views discard two different thirds, and two projections of a three-dimensional quantity along two directions do determine the point — but only given the two directions, and the directions are what the factorisation is trying to recover at the same time. The chicken-and-egg is the whole difficulty, and the residue of it is the one free parameter.

That is why the perspective case leaves one number and the parallel case leaves one number plus a shape change. In the perspective case the free number is a scale, which acts on everything equally and leaves shape alone; here it is a relief, which acts on one direction and does not.

The third view, and the thing it does not buy #

A third parallel view, taken along a direction not in the plane of the first two, makes the system rank six. The family collapses and the reconstruction is the scene.

Except in one respect, and it survives every count.

The upgrade determines a symmetric matrix $Q$; recovering the transformation from it means taking a square root, and a square root is determined only up to an orthogonal factor. An orthogonal factor of determinant $+1$ is a rotation, which is a choice of frame and not a fact about the scene. An orthogonal factor of determinant $-1$ is a reflection, and it is.

Both square roots reproduce every picture exactly. At three views the mirror image redraws every one of them to 8e-16 px, and sits 100% of the object’s own size away from the scene under any rotation whatever. Adding a fourth, fifth or sixth view changes neither number.

So the Necker reversal of a wireframe cube is not a defect of having only one drawing. It is a property of parallel projection, and the correct statement is that no number of parallel views ever removes it. A perspective view removes it immediately, at a rate inverse in the eye’s distance.

What the numbers are, in one place #

Written as an accounting, because the differences between the three cases are easy to blur:

One parallel view. Two of three coordinates per point. Nothing about the third, and the reflection is free.

Two parallel views. Rank five of six. A one-parameter relief family, running here from a stretch of 1.00 to 8.59, all of it redrawing both pictures to 7e-16 m; and the reflection.

Three or more parallel views. Rank six. The scene, up to a rigid motion, and the reflection.

Two perspective views. The scene, up to a similarity — shape exact, size free.

The odd feature of this ladder is that the parallel case needs one more picture than the perspective case to reach a weaker result. That is not an artefact of the method: it is the divide, which is a nuisance for every measurement in the metrology field and is the thing carrying depth here.

The gauge, and how it differs from the ambiguity #

The manyviews field has a result that looks like this one and is a different kind of statement, and running them together would be a mistake.

Seven numbers no picture can name counts the flat directions of a bundle adjustment’s Jacobian: three of position, three of orientation and one of scale. Those are gauge freedoms — the reconstruction and the cameras move together, so nothing observable changes, and there is nothing to be uncertain about. Fixing them is a choice of coordinates.

The relief family is not a gauge. Its members are genuinely different solids — one is 8.59 times further from being a similarity of the scene than another — and a reader shown two of them would say they are different objects. The reflection is not a gauge either: a left hand and a right hand are not the same hand in a different frame.

The test that separates them is the one the figures apply. Compare the reconstructions under the transformations that are choices of coordinates — rotation, translation, and for a scale-free problem, scale — and see whether anything is left. If nothing is left, it was a gauge. If something is, it is an ambiguity, and the picture is short of information rather than short of a convention.

Where the family lives, geometrically #

Naming what the free parameter does makes the result easier to hold than the rank does.

The two viewing directions span a plane. Perpendicular to that plane is one direction in the world, and the family stretches the reconstruction along it. Both pictures are unaffected, because that direction is being seen edge on in a particular sense by both views: whatever the stretch does to it, each view’s own pair of image axes measures the same two combinations it measured before.

Two consequences follow that a reader can check without any of the algebra.

Widening the angle between the two views does not close the family. It changes which direction is free, and it never removes it — the two directions always span a plane. This is the same lesson another picture of the same sweep records for a perspective rig, arrived at from the other side: what a reconstruction is short of is not pictures.

And the third view has to be out of the plane. A third parallel view taken along a direction lying in the plane of the first two adds three equations that are already implied, and the rank stays five. Which is why “three views” in a drawing office means three mutually perpendicular directions rather than three drawings.

What this says about a drawing #

The practical residue is worth stating, because two axonometric drawings of a part is a common thing to be handed.

Two isometric views of an object do not determine it. Not even up to size. They leave a family of solids that draw both, differing by a stretch along the axis between the two viewing directions, and no reasoning from the drawings can choose among them.

A third view from a direction out of the plane of the first two does determine it, up to the reflection — which is to say, up to whether the object or its mirror image is being described. In a drawing that is settled by convention: first-angle or third-angle projection, marked with the truncated-cone symbol, and it is marked because the drawings themselves do not say.

That symbol is usually explained as a convention about where to place the views on the sheet. It is that, and it is also the disambiguation of the one thing every parallel projection leaves free — and the essay’s own measurement says how much would be at stake without it: 100% of the object’s own size.