What one picture of a plane determines

Photograph a tiled floor at an angle. What can be got back out of the photograph, and what has to be supplied from elsewhere?

The question sounds like it should have a vague answer — most of it, roughly — and it has an exact one in three parts. The photograph determines the floor up to a projective transformation. Adding one line determines it up to an affine one. Adding one circle determines it up to a similarity, which is to say completely, up to overall scale.

Stage one: what the photograph gives #

A homography has eight degrees of freedom, and a photograph of a plane leaves all eight of them free. That is not a small residual uncertainty; it is total. Any two quadrilaterals are related by a homography, so the photograph is consistent with the floor being any quadrilateral whatsoever.

What survives is the cross-ratio, and the only thing that survives is the cross-ratio and things built from it. Four points along a receding line of tiles carry a number, and that number is the same in the photograph as on the floor, to twelve decimal places. Four points on the floor and their four images are two quite different-looking configurations that agree about exactly one quantity.

The midpoint is not among the survivors. A tile’s centre line, photographed, lands 0.397 of the way along rather than half — for that camera, and at some other value for any other camera.

Stage two: buying the midpoint with a line #

Supply one thing: the plane’s vanishing line — the horizon, for a ground plane. That is two numbers, and it is the line the plane’s points at infinity image onto.

Applying the homography that sends that line back to infinity turns the projectivity into an affinity, and what an affinity preserves comes back exactly. Parallel lines are parallel again. Ratios of lengths along a line are right, so the midpoint of the receding side lands at 0.500000. Ratios of areas are right.

What does not come back is anything comparing directions. Two sides of a square still measure differently and the corner is still not a right angle. The affine stage buys the midpoint and buys nothing else, and it is worth being precise about how nothing: the angle at the corner comes out at 177.58° in this particular rectification, and that number is not a measurement of anything.

A quantity an affinity is free to change has no determined value. Any further affinity can be applied to the rectified picture and it is still a valid affine rectification, and the angle will be something else. So printing 177.58° in that cell would invite it to be read as the angle, as recovered at this stage, and there is no such thing. The figure prints a dash.

That distinction — between a number that is wrong and a number that does not exist — is the whole reason to state the chain in stages rather than as a single procedure. It is the same distinction the gauge freedom in a bundle adjustment turns on, and the same one behind the seven numbers no picture can name.

Stage three: buying the angle with a circle #

Supply one more thing: the image of a circle lying in the plane. A drain cover, a round table, a manhole, a wheel.

After the affine step, that circle’s image is an ellipse — the affinity has not made it a circle, because an affinity can shear a circle into any ellipse of the same area ratio. The transformation that turns that ellipse back into a circle is the remaining upgrade, and it is determined by the ellipse up to a rotation and a scale, which is exactly a similarity.

Apply it and the plane is a scale drawing of itself. The right angle comes back at 90.000000°. Two equal lengths at right angles measure 1.000000. Every angle in the plane is now correct, not just the ones used.

Why the cross-ratio column is the control #

Four columns, and the fourth is a control rather than a result.

The cross-ratio is 1.333333333 at the projective stage, at the affine stage and at the metric stage. It has to be: every stage applies a projective transformation, and the cross-ratio is invariant under all of them. A stage that changed it would mean the arithmetic was wrong somewhere.

So the table reads: one quantity that was never lost, one bought with a line, two bought with a circle. And the ordering is not a convenience — the angle cannot be bought without first buying the midpoint, because the vanishing line is contained in what the circle supplies and the upgrades compose in that order.

What each purchase actually costs #

It is worth being concrete about where the supplied information comes from, because “supply the vanishing line” sounds like cheating and is not.

The vanishing line is measurable in the picture. Two sets of lines that are parallel on the floor — the two directions of a tile grid, the two edges of a road, a pair of parallel wall bases — give two vanishing points, and the line through them is the vanishing line. So stage two costs a picture containing two known-parallel directions, which most pictures of built environments contain by default.

The circle is also measurable, when there is one. Any round object lying flat does it. If there is none, two known right angles will do instead, or a known length ratio in two directions, or a known angle — the circle is one of several equivalent ways to spend the last three degrees of freedom.

And scale is never recoverable. The chain ends at a similarity, not at a congruence, because nothing in a single picture carries a length. That is the one-view scale ambiguity, and the whole metrology field is what to do about it, which is: supply one known length, anywhere.

Reading it as a budget #

The count is tidy enough to be worth stating as arithmetic. A homography has 8 parameters. An affinity has 6. A similarity has 4. So:

Stage one to stage two costs 2, and the vanishing line is 2 numbers. Stage two to stage three costs 2, and the circle’s image supplies them — an ellipse has 5 parameters, of which 3 go on position and size, leaving 2 for shape, which is exactly the deficit.

The budget balancing is not a coincidence and it is a useful check when adapting the chain: any fact worth 2 parameters can play the vanishing line’s role, and any fact worth 2 more can play the circle’s. A pair of known right angles supplies 2; a known length ratio in two directions supplies 1 and leaves the chain one short.

The chain as it is actually used #

Almost every practical rectification collapses the three stages into one, and it is worth saying how, because the collapse is what makes the stages invisible.

Four known corners of a rectangle determine the homography from the picture to that rectangle’s own plane directly — eight numbers from eight equations, and the answer is metric in one step. No vanishing line is computed and no circle is needed, because the four corners contain both.

That is the operation the metrology field uses and it is the right operation when four corners are available. What it hides is which ingredient is doing which job, and the hiding matters exactly when the four corners are not available: a reader with three corners, or with a rectangle whose proportions are unknown, needs to know which stage they can still reach.

The chain answers that. Three corners and the horizon reach the affine stage. A rectangle of unknown aspect reaches the affine stage and stops, because its shape is the thing that would have supplied the metric constraint. Two parallel directions and a manhole cover reach the metric stage without any rectangle at all.

What the chain says about a claim #

The most useful thing the three stages provide is not a procedure but a way of grading a sentence, and the grading is quick.

These two lines are parallel on the floor — affine. It needs the vanishing line, and a reader who asserts it from a photograph without one has asserted nothing.

This paving is evenly spaced — projective, and available from the photograph alone, because even spacing is a cross-ratio.

This corner is a right angle — metric. It needs the vanishing line and one more constraint, and the constraint has to be identified.

This tile is 30 cm across — beyond metric. It needs everything above plus a known length from outside the picture.

Applying that grading to a claim before checking it saves the checking, because most wrong claims about photographs are wrong in the same way: a metric assertion supported by projective evidence. A photograph shows a quadrilateral; the quadrilateral is asserted to be a square; nothing in the picture distinguishes it from any other quadrilateral, and the assertion has come from the writer’s knowledge of what the object was rather than from the picture.

That failure is not always an error — the writer may well know the object was square — but it is a different kind of statement, and the chain is what separates the picture says from the writer already knew. On a site whose whole habit is that every claim gets a test it could fail, the separation is the point.

What each stage is worth in practice #

Projective sounds useless and is not. A cross-ratio along a line of tiles identifies the tiling — it says whether the divisions are equal, which is a yes-or-no answer available from a photograph with nothing supplied. That is enough to catch a constructed image whose paving was spaced by eye.

Affine is where most architectural photography lands and it is worth more than it sounds. Parallelism, midpoints, ratios along a line and ratios of areas are all back. A floor plan traced at the affine stage has the right topology and the right proportions along each direction, and is sheared.

Metric is a scale drawing, which is what everybody wanted. Its one remaining freedom is the overall size, and one known length anywhere fixes it.

The ordering is strict and the strictness is useful. A claim that skips a stage — a right angle asserted from a picture with no circle and no known perpendicular — has not been derived from the picture at all, and the chain says which piece of information is missing rather than merely that something is.

Why the stages are groups #

There is a reason the three stages are the ones they are rather than an arbitrary sequence of refinements, and it is the reason the word “stratification” is used.

Each stage is a group of transformations: projective (eight parameters), affine (six), similarity (four). Each contains the next as a subgroup. And the quantities that are meaningful at each stage are exactly the quantities invariant under that group — the cross-ratio under all projectivities, the midpoint and area ratio under all affinities, the angle and length ratio under all similarities.

That is what makes the dashes in the table right rather than fastidious. A quantity that is not invariant under the stage’s group does not have a value at that stage, because the stage has not distinguished the picture from every other picture the group can reach. Printing the value the particular rectification happened to produce would be reporting an artefact of an arbitrary choice.

The same logic runs through this site wherever a recovery is up to something. A reconstruction up to a similarity has no size; a bundle adjustment’s solution is up to a gauge; a parallel drawing’s cube is up to a reflection. In each case the honest report names the group and lists what survives it.

What the chain does not price #

Everything above is a statement about determination: which quantities have values at which stage, and what has to be supplied to move up. Not one line of it says how well a determined quantity comes out, and on a real photograph that is the question that decides whether the answer is usable.

The rectification makes the omission concrete. Sending the vanishing line to infinity is the map $\mathbf{x} \mapsto \mathbf{x}/(\boldsymbol{\ell}^{\mathsf T}\mathbf{x})$, and putting the vanishing line at $v = 0$ turns picture coordinates $(u, v)$ into plane coordinates $(u/v,\; 1/v)$. Differentiate. The Jacobian’s determinant is $1/v^{3}$ and its two singular values are $1/v$ and $1/v^{2}$, so a patch of photograph at distance $d$ from the vanishing line rectifies with

• its area multiplied by $1/d^{3}$,
• its shape stretched by $1/d$ along the receding direction.

Both blow up as the patch approaches the horizon, which is the arithmetic behind something every reader has seen: the far end of a rectified pavement is a smear. A mark placed to within $\varepsilon$ pixels lands in the metric plane with an error of about $\varepsilon/d^{2}$ across the depth direction, so halving the distance to the horizon quadruples the error in a recovered depth while only doubling it across. The tiles nearest the horizon are the ones a reader instinctively measures — they are the ones the grid runs out on — and they are worth least.

None of that changes a single entry in the stages. The midpoint still lands at $0.500000$ at the affine stage, because the stage determines it; what the conditioning says is how many digits of that come from the photograph rather than from the arithmetic. The two readings are independent and both are needed, and the ladder of assumptions read as a ladder of conditioning is the same chain measured the second way.

There is a practical rule in it, and it is the rule flattening a façade follows without stating: spend the picture’s information where $d$ is large. Find the vanishing line from the longest, nearest, best-separated parallels available; take the circle from a round object well down in the frame rather than a distant one; place the known length across the near half — and if the round object is a circle, take its centre as the pole of the horizon rather than as the centre of its own drawn ellipse, which is the near-horizon error in its most compact form. Every one of those is free — the same photograph, a different choice of which parts to believe — and together they are worth more than an extra stage of supplied information.

It also explains why the two circular points are the demanding purchase. They sit on the vanishing line, which is exactly where $d = 0$ and every magnification above is infinite, so they are read off the worst-conditioned line in the picture. That the metric stage works at all is because an angle is a cross-ratio with those points, and a cross-ratio is computed from four positions rather than from a difference of two nearly-equal ones.

What it is for #

Two things, and the second is why it is a rung on this site rather than a note in a textbook.

It says what to look for in a picture that is going to be measured. Not is this photograph good enough but does it contain two parallel directions and one circle, which is a checkable question with a yes-or-no answer.

And it says what a claim about a picture is worth. A statement about a photographed plane belongs to one of the three stages, and knowing which says what supporting information it needed. These two lines are parallel is affine and needs the horizon. This corner is square is metric and needs the circle. These four points have this cross-ratio is projective and needs nothing.

The last one is the reason to keep the stages apart even though almost every practical rectification supplies everything at once. A procedure that always works is one nobody can reason about when it fails. The chain says exactly which piece of information is missing, and there are only three places to be stuck.