The residual has a shape

The floor that is not a plane measures the defect. A shadow on a flat floor is a homology, so four marks of an occluder’s outline determine the whole map and every other mark comes back exactly. Dish the floor and the same four marks mispredict the rest by millimetres; ridge it and by more; put a step in it and by a great deal more.

The curvature a shadow reports turns that round into a measurement and finds the trap in it. Fit a curvature to the mispredict and a dished floor’s own curvature comes back to a per cent — and a floor made of two planes, whose curvature is zero at every point of it, comes back as 0.482, with a residual of ten microns.

Both essays quote one number: the worst mispredict. This one asks what is being thrown away by quoting one number, and the answer is most of the measurement.

Matching the scalar on purpose #

The comparison only means something if the three floors genuinely agree on the number under test, so they are made to.

Each receiver’s amplitude is searched by bisection until its worst mispredict hits 25.0 mm. A dish needs a curvature of 0.289, a ridge 0.183, a step 0.040. Those are three quite different floors — the dish drops nearly thirty centimetres over a metre and a half, the step is twenty-four centimetres tall — and after the search they all mispredict the four-point map by the same 25.0 mm, matched to under a twentieth of a millimetre.

The search is a scan then a bisection rather than a formula, and the reason is worth a sentence because it is a real property of the step. The worst mispredict is not monotone in the step’s height. Below about twelve centimetres the whole shadow lands on one side of the seam, so the floor is a plane at a constant height, so the map is exactly a homology and the mispredict is zero. The residual jumps from nothing to two and a half centimetres as the shadow first crosses the crease. A solver that assumed a smooth increasing function would return the wrong floor and say nothing about it.

Why matching is the right experiment #

It would have been easier to show three floors with three different mispredicts and observe that their residuals look different. That would prove nothing, because the residuals would differ in size and size is the thing already known to differ.

Matching the scalar is what makes the comparison a controlled one. After the search, the three floors are indistinguishable on the quantity the field has been quoting for four rungs — not approximately, to four figures — so any statistic that separates them is reading something the quoted number never contained.

That is the same design as the matched comparison of a swing-lens camera against a stitched one, and the same reason: a comparison between two things that already differ in the obvious way cannot say whether the non-obvious way is real.

The residual as a field #

The four-point predictor already computes everything needed; it simply discards it.

Fit the map from four marks of the shadow to four marks of the occluder, ask the remaining sixty-eight where they landed, and instead of taking the worst, keep them in order and keep their signs. The sign is taken against the outward direction from the outline’s own centroid, so a floor that pushes the whole shadow outward and one that ripples it are different fields rather than two numbers that happen to be equal.

What comes out is a signal — sixty-eight numbers around a closed ring — and a signal has structure a maximum does not.

There is a reason this was not done four rungs ago and it is not oversight. The four-point predictor was written to answer a question about whether the homology holds, and for that question a worst case is the right summary — it is the number that says how far the description can be trusted. The mistake is in carrying that summary forward into a question it was not built for, which is what the floor is.

Every measurement in this collection that has gone wrong in this particular way has gone wrong at the same join: a statistic computed for one purpose, quoted in a second, and used as though it carried what the second purpose needed.

Three statistics, and what each is for #

Three readings of that signal, chosen because a different floor is the one each answers.

Sign changes — how many times the signed mispredict crosses zero around the ring. The intuition is that a dish pushes one way everywhere it can, while a ridge alternates.

The second harmonic — the fraction of the signal’s power in the ring’s second Fourier mode. A ridge is $k x^2$, quadratic in one direction and flat in the other, and the second harmonic is what “one direction only” looks like on a closed ring.

Spikiness — the largest second difference against the root mean square of them. A step is a discontinuity, and a discontinuity is a spike in the second difference where nothing else here is.

All three are scale free. Multiply the residual by seven and every one of them is unchanged to fifteen figures, which is the property that makes a matched comparison mean anything at all: none of them is reading the size that was deliberately equalised.

What the numbers actually say #

Two of the three predictions were right and the interesting one was wrong, and both outcomes can be derived rather than observed.

Take a ring of radius $r$ about the outline’s centroid and write the floor’s height on it. For a dish, $h = k(x^{2}+z^{2}) = kr^{2}$ — a constant, with no angular dependence whatever. For a ridge, $h = kx^{2} = \tfrac12 kr^{2}\bigl(1 + \cos 2\theta\bigr)$ — a constant plus a pure second harmonic, and nothing else at all.

So the ordering of the first two statistics is a trigonometric identity. A dish has exactly no second harmonic and a ridge has nothing but one, which is the 0.09 per cent against 2.79 per cent, and no tuning of the amplitudes can change it because the amplitude multiplies both terms alike. The statistic was not chosen to separate them by luck; $\cos 2\theta$ is literally what “quadratic in one direction and flat in the other” is when written on a circle.

And the same two lines explain why the sign-change prediction failed. The guess was that a ridge alternates because it curves one way; what the identity says is that a ridge’s residual is $\tfrac12 kr^{2}$ plus $\tfrac12 kr^{2}\cos 2\theta$, and those two have equal amplitude — so the signed field just touches zero and never crosses it cleanly. Whether it changes sign at all is decided by how much of the constant the four-point fit happened to absorb, which is a property of where the four marks were rather than of the floor. A statistic that depends on the fit’s incidental choices is not reading the floor, and that is the honest account of why the prediction was wrong.

The step is the one the identity does not cover, and its 19.37 per cent is the reason. A discontinuity has no Fourier expansion that terminates: its energy spreads across every mode, so it registers substantially on the second harmonic as well as on everything else, and the statistic that isolates it is the one reading the smoothness rather than any particular mode. That is what the spikiness measure is for, and the division of labour between the two is now visible — the second harmonic separates the two smooth floors from each other, and the spike separates the non-smooth floor from both.

Which suggests the natural completion of the set, and it is one more line of the same algebra. A floor tilted in one direction is $h = ax$, a first harmonic, entirely absorbed by the fit and therefore invisible; a saddle is $h = k(x^{2}-z^{2}) = kr^{2}\cos 2\theta$ with no constant at all, a pure second harmonic that does change sign four times. So the harmonic index of the floor’s height field is the classification, and the three statistics are three ways of reading it.

The second harmonic orders all three cleanly: dish 0.09%, ridge 2.79%, step 19.37%. That is a factor of two hundred across three floors whose headline number is identical to four figures. It is the single statistic that separates the whole set rather than picking one out.

The spikiness picks out the step on its own — 3.72 against 2.48 and 1.98 — which is what a discontinuity should do.

And sign changes, the obvious statistic and the one this essay was expected to be about, reads 8, 10 and 8. It does not order them at all.

Why the ridge was not the second-harmonic one #

The prediction that failed is worth working out rather than quietly correcting, because the reason is general.

The expectation was that a ridge — quadratic in one direction, flat in the other — would be the second-harmonic case, since a function of $x$ alone evaluated around a ring has a strong mode-2 component. That reasoning is about the floor. The measurement is of the residual, which is what a four-point homography failed to absorb of the floor.

A homography has eight parameters and they are not neutral. Among other things it can apply a projective shear, which is a great deal of the mode-2 content of a smooth quadratic field — so the ridge’s second harmonic is largely fitted away before the residual is formed, and what is left is the higher-order part.

A step’s discontinuity has no representation inside a homography at all. Nothing is absorbed. Its energy stays where it was, spread across every mode including the second, and it dominates.

So the ordering is not by how much second harmonic each floor has; it is by how much of it survives being fitted. That is the same reading a wrong shape family on a mirror needs — a residual measures what the model failed to disguise, not what was there.

The scale-freedom check, and why it is not decoration #

Multiply the dish’s signed residual by seven and recompute all three statistics. They agree with the originals to fifteen figures.

That check is doing real work rather than confirming an obvious property. A statistic that drifted under scaling would be reading the amplitude through a back door, and in a comparison whose whole design is that the amplitudes are equal, a statistic that read the amplitude would appear to work while measuring nothing.

It is also the property that makes the statistics usable outside the designed comparison. A real floor’s residual is whatever size it is; a reader wants a number that says what kind of floor it is without first having to normalise anything, and scale-freedom is exactly that requirement.

The sharpest form of the check is that the equality is not approximate. Every one of the three statistics is a ratio of two quantities that are homogeneous of the same degree in the signal — a count, a power fraction, a ratio of second differences — so the scale cancels algebraically rather than numerically, and the fifteen figures are the arithmetic confirming an identity rather than a coincidence.

The control #

A flat floor’s mispredict is $10^{-16}$ metres, so there is no residual to have a shape and every statistic reads whatever arithmetic noise reads.

That is the control that makes the rest a measurement of the floors rather than of the machinery. Without it, three sets of statistics from three receivers could be three sets of statistics from one estimator, and there would be no way to tell.

It also sets the sensitivity question honestly. These statistics are ratios, so they are undefined when the residual is zero and unstable when it is small; a floor whose mispredict is comparable to the mark accuracy has shape statistics that are reading the marks. The detection floor a shadow has applies here unchanged and is the number to compare against before any of these ratios is quoted.

What this repairs, and what it does not #

It repairs the specific complaint the earlier rung left standing. A fitted curvature returned 0.482 for a floor with no curvature anywhere, and nothing in the output said so. The shape statistics do say so: the step’s spikiness is nearly twice the dish’s and its second harmonic is two hundred times, both from the same residual the fit had already computed and thrown away.

So the answer to “the estimator cannot report a surface outside its model” is not a better estimator. It is to stop compressing the evidence before looking at it.

What it does not repair is the deeper problem, and the rest of this row is about that. A shape statistic still assumes the residual is worth reading as a description of the floor, and there is a prior question — whether the floor’s shape belongs in this reading at all. The floor is a choice of coordinates argues that it does not, and that every one of these mispredicts is an artefact of asking the question in the plan rather than in the rays.

Read in that light, this essay is a way of extracting the floor’s shape from an error that need never have been made. That is worth having — sometimes the floor is what a reader wants — but it is not a defence of the reading that produced it.

Two further statistics that were tried and dropped #

Two more readings were computed and are not in the table, and saying why is more useful than the ones that stayed.

The residual’s own maximum position — where around the ring the worst mispredict happens — separates the floors well and is not scale free in the way that matters: it moves with which four points the map was fitted on. Change the four and it changes, on all three floors, so it is a statistic about the fit rather than about the surface.

The total power is simply the size again, in a different unit. It is what the matching search equalised, so it is guaranteed to say nothing here — which is a useful reminder that a statistic can be perfectly informative in general and identically uninformative in a designed comparison.

The three that stayed are the ones that are both scale free and independent of which four points were chosen. That pair of requirements is a good filter and it removes most of what comes to mind first.

What a reader should take from a shadow #

A photograph of a shadow on a floor of unknown shape carries more than the earlier rungs used. Four marks give the map, the rest give a residual, and the residual has a shape as well as a size.

The size says how far from flat the floor is. The shape says what kind of departure it is, and the three kinds this collection distinguishes — a smooth bowl, a developable ridge, a crease — are the three that matter for everything downstream, because a crease costs a reading something a curve does not and a smooth floor cannot do things a creased one can.

The instruction is short. Do not take the worst. Keep the residual in order, keep its sign, and look at it before reducing it to anything.

One more thing the shape settles #

A practical question the size cannot answer, and it is the question a reader with a photograph usually has.

A shadow measurement on an unknown floor is trustworthy if the floor is smooth and is not if it has a crease, because a crease takes one direction out of the plan entirely rather than distorting it. So before any recovery is run, a reader wants to know which case they are in.

The spikiness answers it from data they already have. It does not require a second lamp, a second photograph, or any knowledge of the floor — the residual of the fit that was going to be run anyway carries the answer, and it carries it before the fit’s result is used for anything.

That is the most useful form of any of this. A diagnostic that runs on the by-product of the computation it is diagnosing costs nothing, and the reason it was not there was that the by-product was being discarded one line before anybody looked at it.