Seven numbers no picture can name

The reconstruction in the previous essay has a hundred and sixty-eight parameters and three hundred and eighty-four measurements. That looks like a comfortably overdetermined system, and in most directions it is.

In seven directions it is not determined at all.

Those seven are the subject of this essay, and the reason they get one rather than a footnote is that they are not a defect. They are the exact statement of what a set of photographs contains, and every comparison anyone makes between a reconstruction and anything else has to remove them first or report them as error.

What the seven are #

Three for position, three for orientation, one for size.

Take a reconstruction — every camera pose and every point — and move the whole thing a metre to the left, cameras included. Every camera now stands a metre to the left of a scene that is a metre to the left, so the relationship between each camera and each point is unchanged, so every projection is unchanged. The photographs are identical.

The same argument works for a rotation of everything about any axis, and for a uniform scaling of everything. Together those are the similarity transformations of space: three translations, three rotations, one scale. Seven.

That is the whole derivation and it is not subtle. What is worth doing is measuring it, because “the pictures are unchanged” is a claim that could be wrong — through an error in the code, a rotation applied to points and not to cameras, or a scale that was not applied to the camera translations.

Counting them #

The Jacobian is the derivative of every residual with respect to every parameter. A direction in parameter space that changes no residual is a direction in which the Jacobian is zero, which is to say a null direction, which is to say a singular value of zero.

So the count is a count of small singular values, and the figure draws the smallest sixteen as fractions of the largest.

Seven of them sit below 10⁻⁸. The eighth is 1.3 × 10⁻², which is a factor of 4.6 × 10⁶ above the seventh.

The measurement is the gap rather than the count, and that choice matters. Counting how many singular values are “small” needs a threshold, and a threshold is an opinion — a different one would give a different count, and the reader has no way to know whether the answer was chosen or found. A gap of six and a half orders of magnitude is not an opinion. There is nowhere sensible to put a threshold that gives any answer other than seven.

That is a habit worth carrying beyond this field. Wherever a result takes the form of a count, the honest presentation is usually the quantity whose ordering produced the count, with the discontinuity visible.

Walking along them #

A null singular vector says a derivative vanishes. That is a statement about an infinitesimal neighbourhood, and it is consistent with a direction that is flat to first order and curves away immediately.

The claim being made here is stronger. These directions are not nearly flat; they are exactly flat, all the way. So the figure walks along each of them a finite distance and checks.

The reconstruction is shifted by up to 1.3 m, turned by up to 0.55 radians — a bit over thirty degrees — and scaled by a factor of 2.7. These are not perturbations. A scene scaled by 2.7 is a different scene by any ordinary standard.

Every picture stays where it was to 3.8 × 10⁻¹¹ px.

The comparison that makes that number mean something is drawn beside it. Move one camera by fifty millimetres and the pictures move by 2.1 px. Turn one camera by a hundredth of a radian and they move by 0.77 px. Move one point by fifty millimetres and they move by 0.70 px. The smallest of those is 2 × 10¹⁰ times the largest of the seven.

A figure that showed only the flat directions would be equally consistent with a bug — with a residual function that never moves, or a moveWorld that does nothing. The three stiff directions are the rejection test this site puts at the end of every gate: machinery that has never refused anything has demonstrated nothing.

What moves the world #

The operation being walked along is worth describing, because it is the content of the claim rather than its implementation.

Applying a similarity to a reconstruction means applying it to the points and the cameras together. A point moves; the camera that saw it moves the same way; the picture is unchanged. Applying it to the points alone would move every picture enormously, and applying it to the cameras alone likewise.

The scale is the subtle one. Scaling the points by 2.7 and scaling the camera positions by 2.7 leaves every ray at the same angle, so the pictures are unchanged. Scaling the points and leaving the cameras where they were changes every angle and every picture. Written in the solver’s parameters — where a camera is stored as a rotation and a translation vector rather than as a position — that means the translation has to be scaled and the rotation must not be, which is exactly the kind of thing that gets written the other way round and produces a “gauge direction” that is not flat.

So the walk is a check on the transformation as much as on the geometry, and the 3.8 × 10⁻¹¹ is evidence about both.

Why more pictures do not help #

The most common misreading of this result is that the seven are underdetermined for want of data.

They are not, and the next essay but one measures the flat count at every view count from three to seven and gets seven every time, with a cliff of at least a million at each. Adding a picture adds observations and adds six parameters — the new camera’s pose — and the seven survive unchanged.

The reason is structural rather than statistical. The similarity acts on the whole configuration, including any camera one might add. There is no photograph that could be taken which would be changed by shifting everything, because a photograph is taken by a camera which is part of everything.

That is a different situation from a poorly conditioned direction, and the difference is the point of walking a finite distance. A poorly conditioned direction produces a small but non-zero change, which more or better data would resolve. These produce zero, which no data resolves.

What the solver does about it #

A singular system is not usually something an optimiser survives, and it is worth saying how this one does, because the answer is more interesting than “add a small number to the diagonal”.

The normal equations of a least-squares problem are built from the Jacobian, and a Jacobian with seven null directions gives a normal matrix with seven zero eigenvalues. Asking such a system for a solution is asking for a vector that is only determined up to a seven-dimensional space, and a direct solve will either fail or return whichever member of that space the arithmetic happens to produce.

Three responses are available and they are genuinely different.

Fix seven parameters by decree. Nail the first camera’s pose — six numbers — and one distance in the scene. The system becomes non-singular and the answer is a reconstruction in a stated frame. This is what most implementations do and it is perfectly correct. What it costs is that the reported uncertainties are then uncertainties relative to the fixed camera, which makes the point nearest the anchor look best-determined and the furthest look worst, entirely as an artefact of where the anchor was put.

Add a constraint that treats everything alike. Require, for instance, that the reconstruction’s centroid stay put and its mean radius stay fixed. This is a gauge choice too, but a symmetric one, and the resulting uncertainties do not privilege any camera. It is more work and is what a careful survey adjustment does.

Or damp. Adding a multiple of the identity to the normal matrix, as Levenberg–Marquardt does, moves the seven zeros to something small and positive. The system becomes solvable, and — this is the useful part — the resulting step has almost no component along the flat directions, because the damping penalises movement in exactly the directions that buy nothing. The solver simply declines to travel along the gauge.

That third route is what the figures here use, and it is the one that requires no decision. It does not fix a gauge; it stays wherever the starting point put it, which is why the recovered track has to be aligned before it is compared. The alignment is doing the work that a fixed camera would have done, after the fact and symmetrically.

What removes them #

One thing removes each of the seven, and it is not a photograph.

The three translations are removed by knowing where something is. The three rotations, by knowing how something is oriented — a vertical, a compass bearing, a horizon. The scale, by knowing how long something is.

That is the arithmetic of a survey control point, and it is why photogrammetric surveys have them: a single control point with a known position and orientation, plus one known length, converts a reconstruction into a map. In this site’s terms it is the same supplied length that fixed the two-view scale, with six more supplied quantities for the rest of the similarity.

It is worth noticing which of the seven the previous fields already met. Two views gave shape and no size, which is one of the seven — the scale. The other six were present there too and went unremarked, because a two-view reconstruction is conventionally written in the first camera’s frame, which fixes them by decree rather than by measurement. That convention is a gauge choice, and this essay is what it looks like when the choice is not made.

Seven, and not eight, and not six #

The count is exact, and it is worth checking each of the seven off against something the pictures could in principle have contained, because a reader is entitled to wonder why the list stops where it does.

Why not more? Any transformation that leaves every picture unchanged has to leave every angle at every camera unchanged, which means it has to preserve the shape of the whole configuration — cameras and points together — exactly. The transformations of space that preserve all shapes are precisely the similarities, and there are seven of them. A reflection preserves shapes too, and it is excluded not by the pictures but by cheirality: reflecting a reconstruction turns it inside out and puts the scene behind the cameras. So a reflection is a discrete ambiguity resolved by a sign condition, not an eighth continuous freedom, and it does not appear in the Jacobian at all.

Why not fewer? Each of the seven can be exhibited separately, and the figure does exhibit them separately — three shifts along different axes, three turns about different axes, and a scale, each walked independently. If any of them were secretly determined by the pictures, its walk would have moved something.

There is a case where the count genuinely changes, and it is worth naming to show that seven is a result rather than a definition. If the internal calibration is not known — if the focal length is being solved for as well — the ambiguity is larger, because the pictures then determine the scene only up to a projective transformation rather than a similarity. That is the bottom rung of the ladder, and its gauge group has fifteen dimensions rather than seven. Every figure in this field has a stated focal length, which is what puts them at the metric rung and makes the answer seven.

The comparison this makes impossible #

The practical consequence is a warning, and it is the reason this essay sits second in its field rather than last.

A recovered quantity that is not invariant under a similarity is not a measurement. Not approximately, not with error bars — it is a statement about the solver’s starting point.

That rules out a great deal. The position of a reconstructed point, in coordinates. The distance from a camera to a point, in metres. The height of anything. The orientation of a wall relative to any axis of the reconstruction’s own frame. Each of those changes when the gauge changes, and the gauge is free.

What survives is ratios of distances, angles between directions, and the shape of the configuration. Those are what every figure in these three fields reports, and the reason the track comparison is quoted as a fraction of the track’s mean radius rather than in metres.

The trap is easy to fall into because the numbers look fine. A solver produces positions in metres, and metres are a unit, and a table of positions in metres looks like a survey. It is a survey of a scene whose size, position and orientation were decided by whatever the solver started from.

The habit, one more time #

There is a family resemblance across everything this site measures, and this is its clearest member.

A single picture determines a camera up to nothing: the round trip closes exactly. A single picture determines a scene up to ratios along a line. Two pictures determine a scene up to a scale. Many pictures determine a scene and a camera track up to a similarity.

The qualifiers grow as the problem grows, and the pattern is not that the answers get worse. It is that each recovery is exact, and exactness is a statement about a quotient — the reconstruction is determined exactly, as an element of the set of reconstructions modulo the transformations that no picture can see.

Saying it that way makes the seven sound like bookkeeping. Walking 1.3 m, half a radian and a factor of 2.7 along them, and finding every picture unmoved to a hundredth of a billionth of a pixel, is what turns the bookkeeping into a measurement.

And there is a last reading of the result that is worth leaving with, because it connects this field back to the one the site is named for.

A perspective picture states the point it is correct from. That has been the site’s premise since the first essay, and it is a statement about a relationship — between a picture, the size it is shown at, and a place in the reader’s room. It is never a statement about absolute position, because there is no absolute position for it to be about.

The seven are the same fact, arrived at from the other end. A photograph records the directions from one eye at one instant, and directions are relationships. Nothing in a photograph refers to where its subject is in the world, how it is turned with respect to anything outside the frame, or how large it is, because none of those is a direction from the eye. A hundred photographs contain a hundred times as much of the same kind of information and none of a different kind.

Which is why the seven do not go away, and why that is the right answer rather than a shortfall.