The answer is an ellipse

Every recovery in this field so far has reported its answer as a number with an error beside it. That is the wrong shape for the answer, and this essay is about what the right shape is.

A mark on a photograph is read with an error that is round. There is nothing about a pencil point, a cursor or a corner detector that prefers a direction, so the error in the picture is the same in $x$ as in $y$. What comes back on the ground is not round at all.

Why the answer stretches #

Unprojecting a mark onto the ground plane is a division: the ray from the eye through the mark, intersected with $y = 0$. Move the mark across the picture and the ground point slides sideways by $d/f$ metres per pixel. Move it up the picture, toward the horizon, and the ground point slides away along the ray by $d^2/(fh)$ metres per pixel — the same grazing factor an area, out of one photograph finds in its two error laws.

Two different scalings applied to two perpendicular directions turn a circle into an ellipse. Its long axis runs along the line of sight and its short axis across it, and the ratio between them is $d/h$.

Five point zero seven to one at eight metres from a camera 1.62 metres up, and $8.2/1.62 = 5.06$. The identification is exact to the precision of the linearisation.

The propagation, and the check on it #

The ellipse above is computed by differentiating the unprojection: perturb the mark by a small amount in each image direction, collect the two resulting ground displacements as the columns of a Jacobian $\mathbf{J}$, and form $\boldsymbol{\Sigma} = \sigma^2 \mathbf{J}\mathbf{J}^{\mathsf{T}}$. Its eigenvalues are the squared semi-axes and its eigenvectors are their directions.

That is a statement about a derivative, and derivatives on this site are checked.

Six thousand draws, and the sampled long axis is 23.3 millimetres against a propagated 23.5, with the short axes at 4.6 against 4.6. Half a per cent, which is about what six thousand samples support.

The check matters more here than in most places, because the unprojection is a division and its linearisation is only as good as the region it is linearised over. A mark near the horizon has a Jacobian that changes rapidly across the very region the error covers, and there the propagated ellipse understates the true spread — which the sampling would show, and does, if the mark is put close enough to the horizon.

The control: a ray that is not grazing #

An elongation of five to one might be a property of the arrangement or a property of the method. The way to tell is to remove the grazing.

Put the camera six metres up looking almost straight down at a mark directly beneath it. The ray meets the ground at nearly normal incidence, $d/h$ is near one, and the same computation returns an elongation of 1.00.

So the ellipse is the geometry rather than the arithmetic, and the quantity that produces it is named: the angle at which the ray meets the plane. A camera on a mast, a drone or a satellite reads its ground nearly round; a camera at head height reads it in long slivers.

Where this sits in the field’s own accounting #

The metrology field has measured error twice before and both times reported a scalar, so it is worth saying what this adds rather than replaces.

How wrong a measurement can be prices a height recovered from one photograph and gives a sensitivity in metres per pixel, which is the right answer to the question it asks: a height runs vertically, its uncertainty is one-dimensional, and one number describes it completely. The ladder of assumptions is a ladder of conditioning shows that each rung of the projective-to-metric ladder is bought at a conditioning cost, again reported as a number per rung.

Neither is wrong and neither is enough for a point on a plane, because a point on a plane has two degrees of freedom and its error therefore has three — two magnitudes and an orientation. A scalar is a projection of that onto one number, and every projection of a three-parameter object onto one loses something specific.

What it loses here is the orientation, and the orientation is the actionable part. Knowing that a mark is uncertain by fifteen millimetres says nothing about how to photograph the next one; knowing that it is uncertain by twenty-three along the sight line and five across it says to turn the camera.

How far the cheap direction stays cheap #

The advice to measure across the picture is worth qualifying, because the ellipses do not all point the same way and the advice quietly assumes they do.

Every ellipse points at the camera. Two marks separated laterally by $w$ at distance $d$ therefore have long axes at an angle of about $w/d$ to each other, and neither of them is quite perpendicular to the separation being measured. The variance of the separation along its own direction is $\mathbf{u}^{\mathsf T}(\boldsymbol{\Sigma}_1 + \boldsymbol{\Sigma}_2)\mathbf{u}$, and to second order in the angle that is

$$\sigma^{2} \;\approx\; 2\left[\sigma_s^{2} + \sigma_\ell^{2}\left(\frac{w}{2d}\right)^{2}\right].$$

The long axis leaks in, quadratically in the span. On the arrangement measured here — $\sigma_\ell$ = 23.5 mm, $\sigma_s$ = 4.6, $d$ = 8 m — two marks two metres apart give 7.7 mm rather than the 6.5 the short axis alone would suggest, an eighteen per cent surcharge.

The two terms are equal when $w/2d = \sigma_s/\sigma_\ell = h/d$, that is at

$$w \;=\; 2h.$$

The cheap direction stays cheap out to a span of about twice the camera’s height, and no further — 3.2 metres for a camera at 1.62, independent of how far away the marks are, because both the ellipse’s aspect and the angle between the two long axes carry the same $d$ and it cancels.

That is a clean boundary for a piece of advice that otherwise sounds unlimited. A doorway measured across the picture is inside it. The width of a room is not, and a room measured corner to corner is paying most of the long axis whichever way it is framed — which is a real limit rather than a framing mistake, and is the point at which a second view stops being a luxury.

Why one number for the error is worse than no number #

The practical content of the essay is what quoting a single figure throws away.

A recovered ground point 23.5 millimetres uncertain along one axis and 4.6 along the other has an error that depends entirely on what is being asked. A measurement across the picture — the width of a doorway, the separation of two marks at the same depth — uses the short axis and is good to five millimetres. A measurement into the picture — how far one object stands in front of another — uses the long axis and is good to twenty-three. Quoting the average, or the worst, or the square root of the determinant, gives a number that is wrong for both questions.

Worse, it hides the one piece of advice that would help. A reader who knows the ellipse knows to arrange the measurement across the picture wherever possible — to photograph a room from a corner so that its diagonal runs across the frame rather than into it — and that is worth a factor of five for nothing.

Two marks, and the ellipse of their difference #

Most measurements are differences of two recovered points rather than single points, and the shape of a difference’s uncertainty is not obvious from the shapes of its parts.

If the two marks are read independently, the difference’s covariance is the sum of the two — so a separation between two marks at similar depth has a short-axis-plus-short-axis uncertainty in the across direction and a long-plus-long in the depth direction, and it inherits the same anisotropy.

If the two marks are at very different depths, the far one dominates entirely. The near mark’s ellipse is small, the far mark’s is large as $d^2$, and the sum is very nearly the far mark’s alone — which is why a measurement anchored at the horizon is a measurement with no anchor.

And if the two marks share a systematic error — both read through the same slightly wrong rectifier — the errors are correlated and partly cancel, which is the effect that makes a ratio of areas so much better behaved than either area. Correlated errors are the difference between a difference and a sum, and treating them as independent when they are not overstates the uncertainty in the useful direction.

The ellipse is the right object for a plan #

Reporting a recovered plan as a set of points with ellipses attached rather than as a set of points with numbers attached changes what a reader can do with it, and it is worth spelling out what changes.

A fit to those points — a wall’s line through five recovered marks, say — should weight each by the inverse of its covariance rather than equally, because the marks near the camera are worth much more than the ones far from it and worth much more in one direction than the other. An unweighted fit gives the far marks equal say and is dominated by their long axes.

A comparison between a recovered plan and a survey wants a distance that accounts for the ellipse — the same quantity a statistician calls a Mahalanobis distance — because a two-centimetre discrepancy along a short axis is a real disagreement and the same discrepancy along a long axis is nothing.

And an error bar drawn on a figure should be an ellipse rather than a cross, which is what the figures in this essay do. A cross drawn with the long axis vertical and the short horizontal is not the same object and is not even a projection of it.

Where the linearisation gives out #

Every propagated covariance is a first-order statement, and this one has a definite place where the first order stops being enough.

The Jacobian’s depth term grows as $d^2$, so its own derivative grows as $d^3$ — which means the map changes appreciably across the region the error covers once the ellipse’s long axis is comparable to the depth. Near the horizon that happens quickly: a mark whose ray is within a pixel or two of the horizon has an unprojection that runs to infinity, and no ellipse describes it.

The honest report there is not a larger ellipse but a region that is unbounded — the mark is consistent with everything past a certain distance. That is the same shape of answer the range a pair cannot see past gives for a stereo pair, and it is a refusal rather than a large number.

Where the linearisation does hold, the sampled check above says how well: half a per cent at eight metres. Pushing the same check outward until it breaks is how the boundary was found rather than assumed.

The same shape, one field over #

The anisotropy here is not special to a ground plane, and naming where else it appears is what makes it a fact about grazing rather than a fact about floors.

A stereo pair’s triangulated point has exactly this structure: well determined across the baseline and badly determined along the line of sight, with a ratio set by the parallax angle. Two rays that do not meet is where this collection measures the miss, and depth is a reciprocal is where the depth direction’s law is derived. A stereo point’s ellipse is the same object as a ground point’s, with the crossing angle playing the part the grazing angle plays here.

A shadow’s recovered lamp has it too: a lamp far away is poorly determined along its own direction and well determined across it, which is why a light far enough away finds that the direction comes back and the distance does not.

And a mirror pair’s triangulated point has it, with the epipole’s position in the frame setting the ratio — which is the whole argument of square to the camera is the worst mirror.

Four fields, one shape. What differs is which angle sets the ratio: grazing incidence on a plane, parallax in a pair, the angular size of a lamp’s baseline, the epipole’s offset in a mirror shot. In every case the useful summary is the same: recovered geometry is anisotropic, the anisotropy is computable before the photograph is taken, and it is the thing to arrange rather than the thing to apologise for.

The boundary, stated #

A plane, a known camera, and marks read with independent round errors.

The independence is the strongest assumption and the least often true. Marks read by one person on one print share a systematic bias; marks found by one detector share its failure modes; and marks on one edge of an object are correlated along that edge. All of those correlate the errors, and correlated errors give a different covariance than the sum of the individual ones.

The round error is nearly always fine and is worth checking once: an edge detector localises across an edge much better than along it, so marks taken from edges are anisotropic in the picture before any geometry is applied, and the two anisotropies compound.

And the camera is assumed known here. A camera recovered from the same picture has its own uncertainty, which propagates into every ground point and correlates all of them — the effect that makes a whole recovered plan wrong together rather than each point wrong separately.

Reading a plan that has ellipses on it #

A worked reading is worth more than a principle, so here is what the five ellipses in the first figure say about the five marks that produced them.

The two near marks have short axes of a few millimetres and long axes of a centimetre or so; a separation measured between them is good to about a centimetre either way, and the direction hardly matters because both ellipses are nearly round at that depth.

The two middle marks have long axes several times their short ones. A separation between them measured across the picture is good to under a centimetre; the same separation measured into the picture is good to four or five, and a reader who quotes the first figure for a measurement of the second kind is out by that factor.

The far mark’s ellipse is longer than the distance between the two middle marks. Any measurement anchored on it is a measurement whose uncertainty is that mark’s alone, and the sensible thing to do with it is not to use it as an anchor.

That last observation is the general rule, and it is a rule about choosing rather than about computing: given several recovered points and a measurement to make, anchor on the ones whose ellipses are small in the direction of the measurement. Which is a rule the next essay turns into a number.

What is measured here #

Four numbers.

A mark read to four tenths of a pixel, eight and a fifth metres from a camera 1.62 metres up, unprojects to a ground region whose semi-axes are 23.5 and 4.6 millimetres — an elongation of 5.07, against a predicted depth-over-height of 5.06. Sampling the same arrangement six thousand times gives 23.3 and 4.6, agreeing with the propagation to 0.5 and 0.4 per cent. And a camera six metres up looking almost straight down returns an elongation of 1.00, which is what says the ellipse is the grazing rather than the method.

The short version #

A round reading error on a photograph becomes an elongated region on the ground, because the ray meets the plane at a grazing angle and the two directions are scaled differently. The elongation is the depth divided by the camera’s height, exactly, and it points at the camera.

Quoting one number for that region is worse than useless: it is wrong by a factor of five in one direction and wrong by a factor of five the other way in the other, and it conceals the advice the shape contains — measure across the picture rather than into it, and raise the camera if the depth direction matters.