A survey is trusted at its own accuracy, unless its error has a shape

The eighth held number bends the scene put four surveyed points into a six-picture courtyard, each 10 mm out in a different direction, and compared two ways of using them. Holding seven of their coordinates during the adjustment chooses a frame and changes nothing; holding all twelve makes a claim about the scene’s shape that the pictures can disagree with, and the reconstruction bends to meet it. Fitting a similarity to the four points afterwards bends nothing, and leaves whatever error the pictures made where the pictures made it.

Those two procedures are the two ends of one dial. In between, every surveyed coordinate enters the adjustment as a measurement with a stated standard deviation, pulling the reconstruction toward its surveyed value in proportion to how accurate it is said to be. Holding control is a stated accuracy of zero. Fitting afterwards is a stated accuracy of infinity. And the textbook says where to set the dial: weight each surveyed coordinate by the inverse of its variance, and each mark by the inverse of its own, and the result is the best estimate there is.

That answer has a condition attached which is easy to forget, because it is built into the word variance: it assumes the errors are random. A survey is an instrument, and an instrument’s errors are frequently not. The question this essay measures is whether the textbook answer lands on the smallest shape error when it is tried — and what happens to the best setting of the dial when the survey’s error is the same every time.

The dial, and how it is read #

The courtyard is the earlier essay’s: six pictures across a sixty-degree arc of a scene of blocks and posts, and four control points spread across it. The adjustment is linearised at the true scene — the Jacobian of every mark is computed once, and each setting of the dial is one solve — which is exact enough for shifts of millimetres in a scene metres across. That is checked rather than assumed: with the earlier essay’s four fixed survey directions at 10 mm and the survey held, the linear solve bends the courtyard by 0.601 mm, against the 0.59 mm the full adjustment in that essay reports.

The dial is each control coordinate’s stated standard deviation, entered as a row that weighs the coordinate’s disagreement with its surveyed value against a pixel’s disagreement with a mark. The survey’s true error is drawn separately. Nothing about the one is told to the other.

What is read off is the shape error: the root-mean-square distance of every non-control point from where it truly is, after the best similarity has been fitted, so that only shape is measured and the seven numbers no picture can name are excluded. Each setting is averaged over two hundred trials, each with its own reading error on every mark and, for a random survey, its own survey error on every coordinate.

A random survey is best trusted at its own accuracy #

The first measurement gives the textbook its ideal case: survey errors random and independent, with standard deviations of 1, 5 and 20 mm, and marks read to a whole pixel.

The textbook is right. The 5 mm survey is lowest at a stated 5 mm, the 20 mm survey at a stated 20 mm, and the 1 mm survey at 0.5 mm, the step of the sweep beside its truth on a curve almost flat there. No survey is best trusted more than it deserves or less.

The minimum is also very shallow. With marks read to a pixel the courtyard’s shape error is about 4.5 mm whatever the survey does, because that is the error the pictures themselves leave; the best survey buys it from 4.61 to 4.46. Where the adjustment stops found the floor a pixel’s reading puts under the adjustment’s fit, and a survey is fighting that floor for its gains. What stands out in the figure is not the minima but the asymmetry either side of them. Stated looser than it is, every survey settles at 4.61 mm — the fitted similarity’s value. Stated tighter, the 20 mm survey climbs to 8.16 mm, nearly double.

Finer marks make the dial matter #

The shallow minimum is a consequence of coarse marks, and reading the marks ten times more finely shows what the survey is actually doing.

At a tenth of a pixel the pictures leave 0.46 mm of shape error, and each survey is still best stated at, or one step from, its true accuracy. But now the cost of getting it wrong on the tight side is enormous. The 20 mm survey, stated to a tenth of a millimetre, drives the shape error to 6.26 mm — thirteen times what the pictures alone give. Stated anywhere above its truth, it costs nothing measurable.

This is the essential shape of the dial, and it has a plain cause. A survey stated looser than it is has its information discounted: the pictures carry the shape and the survey contributes less than it could, but what it contributes is still right. A survey stated tighter than it is has its error amplified: the adjustment is told to believe coordinates that are wrong, and bends the courtyard to fit them. The earlier essay’s eighth held number is the limit of that second mistake.

Swept across how finely the marks are read, the asymmetry has a clear boundary. A 5 mm survey stated twenty times too tight costs 2.6 times the best shape error when the marks are read to a twentieth of a pixel, and nothing detectable when they are read to two pixels. Stated twenty times too loose it costs nothing detectable at either end. So the practical instruction is simple and one-sided: when unsure, state a survey loosely. The loss from under-trusting a good survey is a few per cent at most; the loss from over-trusting a bad one grows as the pictures improve, which is exactly when a surveyor would be most tempted to hold the control.

What the textbook weighting is a claim about #

The textbook’s weighting is not a rule of thumb; it is a theorem, and knowing what the theorem says makes both of the results so far predictable. Among all estimates that are linear in the measurements and right on average, the one weighted by inverse variances has the smallest expected squared error — provided every measurement’s error is random, centred on zero and uncorrelated with the others. The adjustment the track and the scene together solves is exactly such an estimate once the survey is added as rows, so for a random survey the theorem applies and the dial’s minimum has to sit at the truth. The figures confirm it rather than discover it.

What the theorem does not promise is a deep minimum. It says the true weight is best; it says nothing about how much better than a nearby weight. How much depends on how much information the survey adds that the pictures do not already hold, and with coarse marks the answer is very little — which is why the first figure’s curves are nearly flat above their minima. The picture of the loop in closing a loop mends its ends is the same shape in another setting: a new measurement buys most where the existing ones are weakest, and a survey laid over a well-photographed courtyard is laid where the pictures are already strong.

And the theorem is silent the moment its proviso fails. A survey whose error is the same in every trial is not centred on zero from the adjustment’s point of view; it is a bias, and a weighted estimate carries a bias through in proportion to the weight. A scale chain leans rather than wanders found the same distinction along a street, where the random part of a chain’s error cancelled and the systematic part accumulated. Here the systematic part does not accumulate; it bends, by an amount the weight controls and the variance does not describe.

A systematic survey is not described by its size #

A variance is one number per coordinate. It says how large an error is expected to be, and nothing about which way. For a random survey that is all there is to say, because every direction is equally likely. A survey whose errors come from a mis-set instrument, a wrong benchmark or a reflector height misread in the same way at every station has errors with a pattern, and the pattern is the same every time the survey is used.

The earlier essay’s four fixed directions are one such pattern. The measurement here draws twelve more: each puts every control point exactly 10 mm out, in directions chosen once and then kept, so that only the marks vary from trial to trial. All twelve surveys are the same size. A random survey of that size — 10 mm per point — is best stated at 5 mm per coordinate, as the textbook predicts.

The best stated accuracy for the twelve runs from 0.3 mm — hold the survey — to 10 mm. Five are best held harder than the random survey of their size; three are best trusted less. And the scatter is not arbitrary. Each survey’s error bends the courtyard by a different amount when it is held, from 0.77 mm to 3.02 mm for the same 10 mm of error per point, and the best weight loosens as that bend grows: the four surveys that bend it least are all best held at 3 mm or tighter, and the surveys best trusted loosest are among those that bend it most. A random survey of the same size bends it by 2.06 mm on average.

The relation is loose, and the figure is drawn so that its looseness shows: two surveys that bend the courtyard by almost exactly the same amount are best trusted at 5 mm and 7 mm. But the direction is not in doubt, and it says what a stated variance cannot. How much an error should be trusted depends on how much damage it can do to this scene, and that depends on which way it points against the geometry of the points and the pictures — which is the question the spread a point gets left open about a single point’s error, and a relative of what an uncertainty is quoted from something found for the pictures’ own: a number that does not say what it is measured against says less than it appears to.

Why a wrong survey can be worth holding #

The low end of that scatter is the surprising part. A survey put 10 mm out and best held outright is a survey whose errors are doing no harm — and the measurement the earlier essay drew shows the harmlessness directly.

With that survey held and the marks read to a pixel, the shape error is 4.42 mm with a perfect survey and 4.30 mm with a survey 20 mm out. The wrong survey does slightly better than the right one, up to 20 mm, and only past 50 mm does holding it cost more than fitting it afterwards. The reason is that this particular error pattern bends the courtyard by 0.6 mm at 10 mm of error, which is small against the pictures’ own 4.5, while holding twelve coordinates hard stiffens the adjustment against the pictures’ reading error in every direction. The stiffening is worth more than the bend costs, and it keeps being worth more until the bend grows past it.

A random survey of the same size bends the courtyard more than three times as much. That is why the textbook, which averages over all directions, is right for it and wrong for the fixed survey: the average direction does more damage than this direction does.

None of which makes a known-wrong survey a good thing to hold. It makes the stated accuracy the wrong question to ask of a survey whose errors are systematic, because the right weight for such a survey depends on what the error is, and a survey whose error were known would simply be corrected.

What a surveyor can do with this #

Three statements, in decreasing order of confidence.

For random survey error the textbook weighting is right, and the mistake to avoid is over-trust. Stating a survey at its true accuracy gives the smallest shape error at every accuracy and every reading precision measured. Stating it too tight is costly and gets costlier as the pictures get better. Stating it too loose is nearly free.

For systematic survey error the stated accuracy is not enough to set the weight. Surveys with identical error sizes want weights a factor of thirty apart. The only information that would set the weight is the pattern of the error against the scene, which is precisely what an unknown systematic error withholds.

The asymmetry makes a loose statement the safe default in both cases. Every one of the twelve systematic surveys was, at a stated 1 m, within 3.5 per cent of its best shape error. Holding the worst of them outright cost 15 per cent. A surveyor who does not know whether their errors are random has one setting that is never far from right, and it is the loose one.

What this measurement leaves out #

One scene, four control points. The courtyard, the six-picture arc and the choice of which four points are surveyed are the earlier essay’s. More control points spread differently would give a survey’s error more directions to bend the scene in, and the spread of best weights across patterns would change.

Linearised, and shape only. The linear solve is checked against a full adjustment at one setting and agrees to within a few per cent. The error measured is the scene’s shape after a similarity is removed; a survey is often wanted for exactly the similarity — to place the reconstruction in the world — and how the dial trades shape against placement is not measured.

Systematic error as a fixed pattern. Twelve drawn patterns are twelve samples of what a systematic error can look like. A real instrument’s error has a structure — a constant offset, a scale, a tilt — some of which a similarity absorbs entirely and some of which it does not, and those particular patterns are not singled out.

The marks are random. Every trial’s reading error is independent. A systematic error in the marks — a lens model slightly wrong, which a loop’s far side is a length and the scale-chain measurement both leave aside — would interact with a systematic survey error, and whether the two cancel or compound is open.

The dial and the pattern #

Weighted into an adjustment, a survey whose errors are random is best trusted at exactly its own accuracy — 1, 5 and 20 mm surveys all give their smallest shape error when stated at the truth, with marks read to a pixel and to a tenth of one. The minimum is lopsided: stated twenty times too loose a survey costs almost nothing, and stated too tight it can cost thirteen times the pictures’ own shape error when the marks are read finely.

A survey whose error has a fixed pattern is not described by its size. Twelve surveys all 10 mm out are best stated anywhere from 0.3 mm to 10 mm, the weight loosening as the particular error bends the scene more, and the earlier essay’s own survey does better held than fitted until it is 20 mm out. The textbook weighting is an average over error directions, and a systematic survey has only one.

Still open: whether a survey’s own residuals can tell its pattern #

The weight a systematic survey deserves depends on how much its error bends the scene, and the survey cannot report that. The adjustment might. When a survey is weighted in, its coordinates end up some distance from their surveyed values, and those residuals are the adjustment’s statement of where the pictures disagree with it.

The measurement that follows sweeps the dial for many systematic patterns and, at each setting, records the control residuals alongside the shape error; then asks whether some statistic of the residuals — their size, or their size against what the stated accuracy predicts — reaches a minimum or a turning point at the weight that gives the smallest shape error. If it does, a surveyor could set the dial from the adjustment’s own output, pattern unknown. If the residuals are as uninformative about the pattern as the variance is, then a systematic survey can only be weighted safely by being weighted loosely, and the loose default above is the whole of the practical answer.