The wedge recovered with the camera

A wedge of glass turns the camera behind it recovers a pinhole camera through a wedge and finds it turned three times too far — 3.37 degrees against the glass’s own 1.14 — with nothing in the fit to say so. The residual is small because a rotation absorbs most of what a wedge does, and the part it cannot absorb is 0.94 pixels.

The obvious repair is to put the glass in the model. What is not obvious is whether that helps, because near the axis a wedge and a rotation are very nearly the same map, and a fit cannot recover two things it cannot tell apart.

What the joint fit is #

Five unknowns and two per reading. The camera’s orientation is three numbers; the wedge’s angle and the azimuth of its thick edge are two more; and each world direction read through the glass supplies two equations. Four directions give eight equations for five unknowns, so the fit has three left over — which is what “the fit finally has something left over” means, and it is the difference from the earlier recovery, where three vanishing points gave exactly as many equations as unknowns and the residual could only ever be zero.

The observations here are six directions spread evenly round the axis at a stated angle from it, read to half a pixel, with the camera’s focal length and principal point known.

Near the axis there is nothing to recover #

With the read directions confined inside eighteen degrees of the axis, the fit returns a wedge angle of essentially zero. The median error over sixty trials is two degrees, which is the whole of the wedge’s own angle: nothing is recovered at all.

That is not a failure of the solver. Near the axis a wedge deviates every ray by the same angle in the same direction, and a deviation that is the same for every ray is a rotation of the camera. The fit has two models that agree on the data and it settles on the one with fewer parameters, which is correct behaviour.

Put in the model’s own terms: the wedge’s angle and the camera’s orientation are not identifiable from directions near the axis, so there is nothing for a residual to reject. The earlier recovery’s 3.37-degree error is this fact, seen from a fit that was not allowed to notice.

Off the axis they separate #

Past about twenty-five degrees the two maps part company, because a wedge’s deviation varies with the angle at which the ray strikes it and a rotation’s does not.

The median error falls from two degrees to 0.81 at twenty-five, 0.27 at forty, 0.11 at fifty, and 0.068 at fifty-five, where 95 per cent of the trials settle near the truth. So the answer to “how far off the axis must the reading reach” is about fifty-five degrees for a tenth of a degree, on six directions read to half a pixel.

That is a demanding reading. Fifty-five degrees off the axis is the corner of a frame taken with a wide lens, and the directions have to be known directions — the vanishing points of straight world features, which is what supplies them in practice. A photograph of a building through a display case, with three sets of parallel edges reaching the corners, is about what it takes.

What the exact case says, and what the rotation-only fit leaves #

Two controls make the numbers above a measurement rather than a demonstration.

With exact readings and the wedge in the model, the fit closes: the worst residual is under 10⁻⁶ pixels and the recovered angle is the glass’s own to the same precision. So the model is right and the search finds it.

With exact readings and only a rotation in the model — the fit a reader makes who does not know there is glass in front of the lens — the worst residual is 0.579 pixels at forty degrees of spread. That is the wedge’s whole signature, and it is the same order as the 0.94 pixels the earlier recovery reports by a different route.

Half a pixel is a real marking error. So the joint fit’s advantage is not that the wedge announces itself: it is that a reading spread far enough off the axis, and precise enough, can tell the two apart, and a reading that is not cannot.

The index is weakly determined, not undetermined #

A pane gives a product before it gives two numbers finds that a flat pane’s displacement is the thickness times one minus the reciprocal of the index, so the two numbers arrive multiplied: four panes from 6.5 to 13.2 millimetres thick, with indices from 1.35 to 2.1, agree to under two microns over an eight-degree fan.

A wedge’s small-angle deviation has the same shape — the index less one, times the angle — so the same question arises. Holding the index at a series of assumed values and refitting everything else gives the answer.

At the glass’s own index of 1.52 the residual is nothing. Assuming 1.50 gives a wedge angle of 2.021 degrees instead of 2.000 and a residual of 0.068 pixels — well under any plausible marking error. Assuming 1.45 gives 2.067 degrees and 0.24 pixels. Assuming 1.95 gives 1.289 degrees and 1.23 pixels, which a careful reading would notice.

So across the ordinary optical glasses, from 1.45 to 1.62, the index and the angle are not separable and assuming a catalogue value costs about one per cent in the angle. Across a wide range of index they are separable, which is a weaker statement than the pane’s and a different one.

And the product is not what is determined #

The sharper form of the comparison is what happens to the product itself.

For a flat pane the product is exactly what a picture fixes: any pair of numbers with the same product draws the same displacement, over a narrow fan, to microns. For the wedge the recovered product does not hold constant along the profile. It runs from 0.930 at an assumed index of 1.45 to 1.225 at 1.95, against a true 1.040.

So the picture is not merely determining a product and leaving its factors free; it is determining something closer to the angle itself, with the index entering weakly through how the deviation varies with incidence. The small-angle expression (n − 1)α is the leading term, and the joint fit is reading the next one.

That difference between the pane and the wedge is worth stating in one sentence. A pane’s displacement is a position effect and its dependence on incidence is second order over a small fan; a wedge’s deviation is a direction effect and its dependence on incidence is what the whole recovery lives on. The two look like the same algebra and they are read from opposite ends of it.

Why over-determination is the whole of the repair #

The earlier recovery used three vanishing points, which is exactly enough to fix a camera, and the essay’s own complaint is that it had nothing left over. It is worth saying what that phrase costs a reader who takes it for granted.

A fit with as many equations as unknowns always succeeds. Its residual is zero whatever the data, so the residual carries no information about whether the model is right — it only says the arithmetic worked. Recovering the camera is the standing account of the three-vanishing-point recovery, and everything it establishes about what a picture gives is a statement about a model assumed correct.

Adding a fourth direction changes the character of the fit rather than its precision. With four directions and a pinhole model the residual is no longer forced to be zero, so a picture taken through glass finally has a number that is not small. With four directions and a wedge in the model the residual is forced back toward zero, and the difference between those two residuals is the evidence that there is glass.

That is the general shape and it is worth carrying past this case. A model that cannot be contradicted has not been tested, and the cheapest way to make a geometric model contradictable is one more measurement than it needs.

Two ways a wedge is not a distortion #

There is a temptation to treat a wedge as a kind of lens distortion and fit it with the radial model already in hand, and it is worth saying exactly why that does not work.

A radial distortion is symmetric about a centre: it moves every point along its own radius by an amount depending only on how far out it is. A wedge is not symmetric about anything in the picture. It moves every point in one direction — the azimuth of its thick edge — by an amount depending on how far the ray leaned toward or away from that edge.

The tangential coefficients of the usual model exist for exactly the asymmetric case, and a tilted sensor is not a distortion measures what they do when asked to absorb a departure they do not contain: they take part of it, leave 1.87 pixels, and bend straight rows by four pixels when used as a correction. A wedge is a second such departure, and a second reason that the tangential terms are where a calibration puts whatever it does not understand.

What the measurement here adds is the positive half. A wedge does have a model — two numbers, an angle and an azimuth, with the index taken from a catalogue — and with enough field the two come back. It is not a distortion to be absorbed; it is an object to be recovered.

What this changes about a photograph through glass #

Three practical statements.

A photograph through a wedge, read near the middle of the frame, gives a camera that is wrong and a residual that is fine. That is the earlier recovery’s finding and nothing here softens it. The repair is not a better fit; it is more field.

With the frame’s corners read, the wedge can be recovered and removed. That is the same demand the lines that calibrate a lens makes of a distortion fit — the information is off the axis and a reading confined to the middle of the frame determines nothing — arriving here for a quite different quantity. At fifty-five degrees of spread the angle comes back to under a tenth of a degree, and with the angle and its azimuth known the whole map is known and the camera behind it follows.

And an index taken from a catalogue is good enough. Ordinary display glass, window glass, acrylic and crown glass span 1.45 to 1.52, and across that range the assumed value costs a per cent in the angle and a residual far below a marking error. The index is the one number in this problem that does not need measuring, which is the opposite of the pane’s case and worth knowing before anybody tries.

Where else a small deviation hides in a recovery #

The shape of this failure — a systematic deviation that a free parameter of the camera happens to be able to absorb — is one the recovery field keeps meeting, and the three instances together say what to look for.

A wedge is absorbed by the camera’s orientation, which is what this measures. A tilted sensor is absorbed by the principal point, exactly, which is a tilted sensor is not a distortion’s finding: a calibration that frees its principal point takes the whole of it and leaves nothing. And a flat pane is absorbed by nothing at all — what survives a pane of glass finds that it moves every point and no direction, so the camera recovered through a display case is exactly the camera that took the picture, and there is nothing to absorb.

Setting the three side by side says what decides which happens. A departure that is a rotation of directions is absorbed by the orientation; one that is a translation of the picture is absorbed by the principal point; and one that leaves directions alone is not a departure in a recovery’s terms at all, however visibly it moves the picture.

So the question to ask of any element in front of a lens is not how much it moves the picture but what it does to directions, and that is the reading under which a flat pane is harmless and a two-degree wedge is not.

What this does not settle #

The wedge has flat faces and one angle. Real glass in a display case is a sheet with a small wedge in it and a small curvature as well, and a curvature is a lens — what survives a pane of glass establishes what the flat case preserves, and a curved sheet preserves less.

The camera’s intrinsics are known. Freeing the focal length and the principal point adds three unknowns that a wedge’s displacement can partly imitate, particularly the principal point, whose shift a tilted sensor is not a distortion measures for a different cause. Whether a joint fit of all eight survives is not measured here.

Nothing here is about a thick wedge. The model is a wedge of glass treated as a deviation of direction alone, which is exact for a thin one and not for a thick one: a thick wedge also displaces every ray sideways, and what survives a pane of glass is the measurement of what that displacement does to a recovered camera when the faces are parallel. The two effects combine in a real prism and only one of them is fitted here.

And the readings are directions rather than lines. The measurement uses vanishing points, which a photograph supplies only where it contains long parallel features. The frame’s straight lines carry more information than their vanishing points alone — a line bent by a wedge is bent in a particular way — and using the whole line rather than its meeting point is a stronger fit that has not been made.

Still open: whether the wedge or the lens gets the blame #

Everything here assumes the only thing between the scene and the sensor is the wedge. A real camera has a lens with its own distortion, and the two are fitted from the same picture.

The concern is specific rather than general. A wedge’s leading effect is a rotation, which a distortion model cannot imitate; but its next effect is a displacement that grows with the angle off the axis, which is exactly the shape a radial distortion coefficient describes. So a joint fit of a wedge and a lens has two parameters competing for the same part of the signal, and the question is how strongly they are correlated and which one wins when the reading is short of the field it needs.

The measurement builds a picture with both a known wedge and a known distortion in it, fits them together, and reports the correlation between the wedge angle and the first radial coefficient against how far off the axis the reading reaches — and, more usefully, what a fit that leaves the wedge out reports as the distortion. If a modest wedge reads as a plausible distortion coefficient, then a calibration done through a display case is returning a lens that does not exist, and there is nothing in the residual to say so.

The short version #

Putting the glass in the model turns an exactly-determined recovery into an over-determined one, and with it the residual becomes able to say something. The fit has five unknowns — three for the camera’s orientation, two for the wedge’s angle and the azimuth of its thick edge — and two equations for every direction read.

Inside eighteen degrees of the axis nothing is recovered: a wedge deviates every ray alike there, which is a rotation, and the fit correctly declines to invent a second explanation. Past twenty-five degrees the two separate, and by fifty-five the angle comes back to 0.068 of a degree with 95 per cent of the fits settling on it. With exact readings and no wedge in the model, the residual left over is 0.579 pixels — the wedge’s whole signature, and the same size as a marking error.

The index is not separable from the angle across the ordinary glasses: assuming 1.50 for a true 1.52 costs 0.021 of a degree and 0.068 pixels. Across a wide range it is, and the recovered product runs from 0.930 to 1.225 rather than holding — so unlike a flat pane, a wedge does not hand back a product before it hands back its factors.