The ladder of assumptions is a ladder of conditioning

What one picture determines sets out the ladder. One picture of a plane fixes it projectively; add the vanishing line and it is fixed affinely; add the circular points and it is fixed metrically. Each rung is a structure recovered and a set of assumptions paid for.

The ladder is always presented that way — as a hierarchy of what is known. This essay measures it as a hierarchy of what a pixel costs, and the two orderings turn out to agree while the sizes do not.

One map, three quantities #

The confound to avoid is obvious once named. Comparing a projective recovery against an affine one against a metric one compares three different procedures, and any difference could be a difference between the procedures.

So there is one recovery. A board of four known plane points is photographed, the four image corners are pushed by a pixel, and a homography is fitted to the perturbed correspondences. All three quantities are then computed through that one map.

• Projective — the cross-ratio of four collinear points on the plane, read straight from the image.
• Affine — the ratio of two lengths along one plane direction, which survives an affine map and not a projective one.
• Metric — the angle between two plane directions, which needs the map’s shape and not only its parallelism.

Any difference between them is now a difference between the quantities.

The cross-ratio does not move #

At every obliquity from twenty degrees to seventy-eight, the cross-ratio’s error is exactly zero — not small, zero, because the perturbation of the map does not enter it at all.

That is worth stating precisely rather than as a slogan. A cross-ratio of four collinear image points is computed from those image points. It is invariant under the projection, so its value in the picture is its value on the plane, and no recovered map is needed to read it. Perturbing the map’s four corners perturbs the map; it does not perturb four other points elsewhere in the picture.

So the projective quantity sits below the ladder rather than on its bottom rung. It is the thing that survives without any recovery at all, which is what the collection’s first result says and which is easy to lose sight of once a homography is in the room.

There is a practical consequence and it is the reverse of how these things are usually ranked. A quantity that survives the projection is not merely cheaper to obtain than one that does not; it is immune to the recovery’s errors, because it never touches the recovery. So in any pipeline that ends in a comparison, expressing the comparison projectively where possible removes an entire error source rather than reducing it.

That is available less often than one would like — most questions a reader has are metric — but it is available more often than it is used. A test of whether four points are in a given projective relation, for instance, needs no rectification at all.

The two that do move #

The affine and metric quantities both go through the map, and both get worse with obliquity, and they get worse at different rates.

At twenty degrees the length ratio is out by 0.123% and the angle by 0.016 degrees. At seventy-eight the length ratio is out by 1.606% — thirteen times worse — and the angle by 0.914 degrees, fifty-eight times worse.

So the metric quantity degrades about four times faster than the affine one across the same sweep. That ordering is what the ladder predicts qualitatively; the factor is what measuring it adds.

The reason is not mysterious. A near-grazing view compresses the far half of the plane toward the vanishing line, so the map’s own coefficients become large and its behaviour near that line becomes violent. An angle depends on the map’s local linear part — its shape — which is where that violence lands; a length ratio along one direction depends on less of it.

Why the far half of the plane is where it goes wrong #

The mechanism deserves a paragraph, because it says which part of a photograph to distrust.

A homography maps the plane to the picture and it sends the plane’s line at infinity to the vanishing line — the horizon of that plane. Points near the plane’s infinity crowd toward that line, so the map’s inverse is expanding violently there: a pixel near the horizon corresponds to metres of plane.

Obliquity moves the vanishing line into the picture and moves the useful part of the plane toward it. At twenty degrees the board sits well away from its own horizon; at seventy-eight it is close to it, and every pixel is worth more world.

That is the whole of the conditioning story and it is the same statement the horizon makes in the metrology field — how far a point is from the horizon in the picture is how much of the plane a pixel there is worth. The stratification’s rungs differ in how much of that expansion each one inherits.

The linearity, and what it lets a reader do #

Multiply the corner perturbation by four and every error multiplies by four. The comparison is linear in the noise across the whole sweep.

That is a useful property and not an obvious one. The map is a nonlinear function of its correspondences — a homography’s coefficients come out of a matrix solve — so there is no reason in advance for the downstream errors to be linear in the input.

They are, over this range, and the reason is that a pixel is a small perturbation of a well-conditioned solve. It stops being linear when the four corners approach a degenerate configuration, which is the regime the fitting routine’s own guard is about: four points that nearly line up produce a map that satisfies its own four correspondences and misses everything else.

So the numbers above scale, and a reader with a different noise level can multiply. What they cannot do is extrapolate past about ten pixels on a small board, where the linearity goes.

What the ladder is really ordering #

The usual statement of the stratification is about information: each rung needs more of it than the one below.

The measurement above says the same ordering holds for stability, and the two are connected rather than coincidental. A quantity that needs more structure to compute depends on more of the recovered map, so more of the map’s error reaches it.

Stated that way the result sounds like a tautology and it is not, because “depends on more of the map” is not a quantity until somebody measures it. The angle could have depended on a well-conditioned combination of the map’s coefficients and the length ratio on a badly conditioned one; nothing in the hierarchy of assumptions forbids it. It happens not to, and the factor of four is the measurement.

Obliquity is the parameter nobody varies #

The strongest practical point is about the sweep rather than about any of the numbers in it.

A conditioning is almost always reported at one configuration — “a pixel costs about a per cent” — and the configuration is whichever one the author had. Across this sweep the same quantity costs 0.12% and 1.6%, a factor of thirteen, and both are correct statements about the same method.

So a quoted conditioning without its configuration is not much of a number. That is the same complaint the metrology field makes of a height recovery, where the error turns out to track distance rather than the height being measured, and the essay that measured it reports the curve rather than a figure.

The instruction is short. A conditioning is a curve, not a number, and the parameter it is a curve in is usually the one the experimenter did not vary.

What the sweep does not cover #

Two limits are worth naming, because a curve fitted over a range invites extrapolation past it.

Below twenty degrees the plane is seen nearly face-on, and the interesting failure there is a different one: a face-on view of a plane has its vanishing line far outside the picture, so recovering it from parallel lines in the image is badly conditioned even though the map is well conditioned. That is the vanishing point that runs off the paper and it is the affine rung’s own trouble rather than the metric one’s.

Above seventy-eight degrees the board’s far edge approaches its own vanishing line, the map’s condition number runs away, and the linearity in the noise goes with it. What happens there is not a steeper version of the curve; it is the regime where a pixel of corner error moves the recovered plane by an unbounded amount, and no scaling law describes it.

So the sweep covers the range where a photograph of a floor is actually taken, and both ends have their own failure that this measurement does not describe.

Two terms, and this law has only one #

Read with the instrument of this row, the conditioning curves here have a particular shape: they are all falling term and no floor. Reduce the noise and every error reduces proportionally, with nothing left over.

That is worth stating because it is not automatic and it distinguishes this failure from most of the others in the row. A floor would mean something in the recovery is wrong in a way better measurement does not fix — a lens the pinhole model cannot see, a shape family that does not contain the object. Here the model is exact: the scene really is a plane, the camera really is a pinhole, and the only error is the pixel.

So this is the clean case, and it is worth having one. The obliquity multiplies what a pixel costs and does not add anything a pixel cannot buy away.

Reading the curves backwards #

A conditioning curve is usually read forwards — here is the noise, here is what it costs. Read backwards it is a rule for taking the photograph, and that is the more valuable direction because one of its two parameters cannot be revisited afterwards.

The reading is available because of the linearity established above. Every curve scales with the corner noise, so a required accuracy in a recovered quantity divides through: fix what a recovered angle must be good to, divide by what one pixel costs it at a given obliquity, and the quotient is the marking precision the photograph demands. Alternatively fix the marking precision — a pixel is what a careful reader achieves on a sharp corner, a quarter of one with subpixel fitting on a good target — and read off the obliquity beyond which the requirement cannot be met at all.

The asymmetry between the two parameters is what makes this worth doing before the shutter rather than after. Marking precision is recoverable and obliquity is not. A photograph marked carelessly can be marked again; one shot at seventy-eight degrees to the plane is oblique forever, and no amount of later care buys back a conditioning that was decided at capture. Every quantity on the ladder inherits that decision, and the higher rungs inherit more of it.

So the practical instruction is short. Move, if moving is possible, before improving the marks: ten degrees less obliquity is worth more than a factor of two in marking precision across most of the sweep above. When moving is not possible — a façade across a street, a floor photographed from a doorway — the curves say what the picture can support, and the honest response is to report the recovered angle with the tolerance the obliquity implies rather than with the digits the arithmetic prints.

Where the floor would come from #

It is worth naming what would put one there, because a real photograph has all three.

Lens distortion. The board’s corners are not where a pinhole would put them, so the fitted map is wrong in a way that does not average down — which is exactly the floor a camera recovery has, measured there as proportional to the distortion coefficient.

A board that is not flat. The correspondences are then inconsistent with any homography, and the fit returns the best available compromise with a residual that reports the departure without naming it.

A misidentified corner. A gross error rather than a small one, which no amount of subpixel care touches.

Each of those is a term with no pixel count in it, and each would appear in this measurement as a curve that flattens instead of falling. None of them is present here, which is what makes the numbers above a measurement of conditioning rather than of anything else.

A note on which four points #

Everything above fits the map on the board’s four extreme corners, and that choice is load-bearing rather than incidental.

Four points in general position determine a homography. Four points that nearly line up determine one that satisfies its own four correspondences and misses everything else — the matrix solve is nearly singular, and its answer is nearly arbitrary in the direction the four points do not span.

So the corners are chosen to be as far from collinear as the board allows, which is the standard advice and which is worth restating with the reason attached: it is not that corners are convenient, it is that a bad choice makes the conditioning curve above a curve about the choice rather than about the obliquity.

The same trap has a worked instance in this collection. An anamorph’s design is usually a grid, and a grid’s first four points are a column — so a routine that took “the first four” would fit on four collinear points and produce exactly this failure, silently.

The same reading, one field over #

The result has a twin in the metrology field that arrived by a different route and is worth putting beside it.

How wrong a measurement can be asks what a pixel costs a single-view height, expecting the error to grow with the object’s height — the estimator divides by a quantity that tends to zero for a tall object, so the argument is sound and the conclusion is wrong. Measuring it says the error tracks distance, and tracks it linearly, because the whole object shrinks toward the horizon and a pixel buys more world.

That is the identical mechanism as this essay’s obliquity. In both cases the conditioning is governed by how close the thing being measured is to its own vanishing structure — the horizon for a height, the plane’s vanishing line for a rectification — and in both cases the parameter that governs it is not the one the algebra suggests.

Two fields, two recoveries, one rule: conditioning is distance from the vanishing structure, and every quantity that goes through a recovered map inherits it in proportion to how much of the map it uses.

What to take from the ladder #

The stratification is taught as a sequence of assumptions and it is one. What the measurement adds is that the sequence is also ordered by fragility, by a factor that is not small, and in a parameter — the obliquity — that most treatments hold fixed.

A reader rectifying a floor from a photograph therefore has a decision the textbook version does not present. A shallower view costs coverage and buys precision, and the exchange rate is steep at the oblique end. Thirteen times, for a length ratio, between a comfortable view and a grazing one; fifty-eight for an angle.

Which is worth knowing before the photograph is taken rather than after, and is the sort of thing that only appears once somebody sweeps the parameter that seemed like a nuisance.