A calibration through glass reports a prism

A wedge moves the centre, not the lens put a two-degree wedge of glass in front of a camera whose lens has no distortion, and calibrated the camera with a model that knows nothing of the glass: rotation, focal length, principal point and one radial coefficient. The calibration did not invent a lens. It reported a coefficient of about 0.002, moved its principal point by 4.5 to 14 pixels, and left a residual that swirled with the wedge’s thick edge — a pattern no radial model takes, visible from twenty-two degrees off the axis.

Standard calibrations carry two more parameters than that model had, for exactly the kind of displacement a radial coefficient cannot describe: the decentring, or tangential, terms of the Brown–Conrady model, which describe a lens whose elements are not quite coaxial. A wedge in front of a lens is, loosely, a very badly decentred element. The earlier essay closed by asking three things of those terms: how much of the swirl they absorb, whether the principal point then moves less, and whether the terms a glass-blind calibration reports are large enough to look like a defective lens.

They absorb most of it. The principal point moves more, not less. And the terms they report describe a lens so decentred that a calibration engineer would send it back — while describing the glass correctly would have needed a different pair of terms, which describe a prism.

Three calibrations of one camera behind glass #

The arrangement is the earlier one exactly: a 620-pixel camera with no lens distortion at all behind a two-degree wedge of crown glass, read at twelve directions on two rings, forty and twenty degrees off the axis, with exact readings. Four calibrations are fitted, each blind to the glass: the radial model the earlier essay used; the same with the two Brown–Conrady decentring terms added; the same with two thin-prism terms instead; and with both.

The field of arrows is the difference between where the glass puts each direction and where the calibration puts it, drawn over the whole disc the readings span and not only at the readings, magnified forty times. With the decentring terms free, the swirl the radial calibration left — 1.88 pixels root-mean-square across the disc, 3.44 at worst — falls to 0.54 and 1.15. The slider swaps the model; the thin-prism terms take it to 0.41.

The decentring terms describe a specific pattern. In normalised coordinates they add $2t_1xy + t_2(r^2 + 2x^2)$ to the horizontal coordinate and $t_1(r^2 + 2y^2) + 2t_2xy$ to the vertical: a displacement that grows as the square of the distance from the centre, in a direction that turns with position. A wedge’s leading effect after the rotation the calibration already absorbs is also a displacement growing roughly as the square of the field angle, with a direction fixed by the wedge’s thick edge. The two are not the same pattern, but they share enough that fitting one absorbs most of the other.

How much each set of terms takes #

The comparison worth making is across reach, since the earlier essay found that a wedge hides near the axis and shows further out.

The decentring terms take between half and three-quarters of the swirl at every reach: at twenty-five degrees off the axis the residual across the disc falls from 0.57 pixels to 0.16, and at forty from 1.88 to 0.54. The thin-prism terms take more: 0.048 at twenty-five degrees, and near the axis almost everything, 0.009 against 0.084 at ten degrees. Both sets together are the thin-prism result and a little more near the axis, and nothing more further out.

The thin-prism terms are the ones written for this. Weng, Cohen and Herniou added them to the calibration model for a lens with a slightly tilted element — a displacement $s_1r^2$ horizontally and $s_2r^2$ vertically, the same direction everywhere and growing as the square of the radius. A thin wedge is a thin prism, and near the axis a thin prism’s departure from a rotation is exactly that. It is a different thing from a tilted sensor, which a tilted sensor is not a distortion found is a pinhole camera with a moved principal point and no distortion at all; a tilted plate of glass bends rays, and a tilted sensor only catches them at a slant. What the thin-prism terms cannot take is the wedge’s behaviour further out, where the deflection depends on the incidence angle in a way no square law describes; past about forty degrees the residual under every model is growing as fast as under the radial model alone, and at fifty-five degrees all four leave three pixels.

So the answer to the first question is: the decentring terms take most of the swirl, and a different pair of terms, written for tilted elements, takes more. Neither takes it all, and nothing short of modelling the glass takes it at wide field. The wedge recovered with the camera did model the glass, and the same readings with the wedge in the model leave $6\times10^{-14}$ of a pixel.

The swirl before any extra terms #

It helps to see the pattern the extra terms are competing for, drawn as the earlier essay drew it.

The arrows turn with position: they point one way near the wedge’s thick edge and the other near its thin edge, and round the sides in between. A radial coefficient can only push every point straight out or straight in, and so it takes almost nothing. The decentring terms push points in directions that turn with position, which is why they take so much. The thin-prism terms push every point the same way, harder with distance from the centre — which is the part of the swirl that remains once the rotation has taken the wedge’s uniform deviation, and why near the axis they take nearly all of it.

The difference between a plumb-line calibration and this one is worth noting. The lines that calibrate a lens fits distortion from straight edges alone, and a straight edge is bent by a wedge in a way no radial coefficient describes either; a plumb-line fit through glass would face the same choice of terms, and the same temptation to accept whichever makes the lines straight. A barrel model folds at a radius it sets itself is the reminder that every distortion model is a polynomial with a range, and a polynomial asked to describe something it was not written for describes it well only near where it was fitted.

The residual that announced the glass goes quiet #

The earlier essay’s comfort was that the glass announced itself: the residual a calibration leaves at its own readings is the only evidence a calibration has, and a radial calibration through a wedge leaves a swirl there that no reading error explains. More terms take that evidence away.

A calibration can only see its residual at its own readings. With marks read to 0.4 pixels, a residual smaller than that is indistinguishable from reading error. The radial calibration’s residual passes 0.4 pixels at about twenty-two degrees off the axis: a reading that reaches further sees the glass announced, and one confined inside that reach does not. With the decentring terms free, the residual stays under 0.4 pixels until about forty-one degrees. With thin-prism terms, until about forty-six.

So the extra terms do not remove the glass. They hide it. The glass is still bending every ray exactly as much as it did; what has changed is that the calibration now has a place to put the bending that is not the residual, and the residual was the only place a reader of the calibration could have seen it. A report of a clean fit is a report about the model’s freedom as much as about the camera. A calibration whose readings reach thirty degrees off the axis — most of the frame of a normal lens — reports a clean fit when it carries decentring terms, and a swirling one when it does not. The standard model, with the standard terms, is the one that makes the glass invisible over the widest range. That is the worst outcome of the three the earlier essay anticipated, and it is the one that happens.

What the calibration says about the camera #

The third question was whether the terms a glass-blind calibration reports look like a defective lens. They look like a very defective one.

The decentring terms come out at a combined magnitude of 0.011 to 0.014, nearly the same at every reach, because they are describing one fixed thing — the glass — as a property of the lens. Calibrations of assembled lenses commonly report decentring terms of a thousandth or less; a value ten times that is the signature of an element badly off its axis, and the camera here has a lens with no distortion at all. The calibration has not found a defect; it has invented one, and attributed the glass to it.

The principal point, which the earlier essay found was the radial calibration’s scapegoat, does not move less. With decentring terms it moves further — 11.6 to 13.9 pixels out to thirty-two degrees, against 4.5 to 6.1 for the radial calibration — because the decentring pattern and a shift of the centre are partly the same motion, and the fit trades between them. Only at the widest reach, where the decentring terms have more of the field to fit, does the principal point settle nearer its true place. The thin-prism terms do better near the axis, where they leave the principal point within half a pixel of the truth at ten degrees, and worse far out, where they move it seventeen pixels at fifty-five.

The thin-prism terms report a combined magnitude of about 0.015, constant with reach. That is at least a true description. A thin prism is what a two-degree wedge in front of a lens is, to first order, and a calibration that reports a thin prism has reported the glass — as part of the lens, where it is not, but as the right kind of thing.

The prism the glass is #

The thin-prism calibration’s residual field shows what is left when the right kind of term is used.

At forty degrees off the axis the thin-prism calibration still leaves 0.41 pixels across the disc, and the field is no longer the earlier swirl. Near the axis every arrow points the same way — a uniform offset of about 0.3 pixels, which the fit has traded between the principal point and the rotation — and between the two rings of readings the arrows turn with position again, reaching 0.72 pixels at thirty degrees off the axis, where no reading constrains them. The readings themselves sit where the residual is smallest, which is what a fit does and why the residual at the readings understates the residual everywhere else. Its reported thin-prism terms, about 0.015, are the wedge’s deflection written in the lens’s coordinates.

That is a strange kind of success. The calibration’s report is true of the image and false of the camera. Any use of the calibration that keeps the glass where it was — a camera in a fixed housing, looking through the same port for its whole life — is served perfectly well by a lens model that contains the glass. Any use that moves the glass, or reasons about the lens alone, is served by a calibration that is wrong in a way its residual will never show.

Why the blame moves as it does #

The earlier essay found a principle worth restating: a calibration assigns a disturbance to whichever of its parameters describes the disturbance’s pattern best, and it has no way to know that the disturbance is outside the camera. A rotation takes the wedge’s uniform deviation; a principal-point shift takes some of its field-dependent part; a radial coefficient takes almost none of it, because the wedge’s pattern is not radial.

Adding parameters adds candidates for the blame. The decentring terms share part of their pattern with a principal-point shift, so adding them does not relieve the principal point — it gives the fit two partly interchangeable ways to describe the same displacement, and the solution moves along the direction they share. The thin-prism terms share less with the principal point near the axis, where their pattern is nearly the wedge’s own, and so they take the wedge’s part directly and leave the centre alone.

This is the same finding a pane gives a product before it gives two numbers made for a flat pane of glass, in a different quantity: what a picture determines is a combination of parameters, and a model with more parameters than the picture determines distributes the combination among them by the accident of their shapes. What survives a pane of glass is the reassuring limit — a pane with parallel faces moves no direction at all, so a calibration through it is the camera’s own — and a wedge is where that reassurance ends.

What a calibration through glass should do #

The practical conclusions are three, and they pull against each other.

A calibration through glass should not carry decentring terms unless it has reason to. They take the glass’s swirl, invent a defective lens, and move the principal point further; the clean residual they leave is the least trustworthy outcome of all, because it removes the only warning. A calibration that is going to be used behind a display case, a housing port or a windscreen is better done with the radial model alone and a reach wide enough for the residual to speak.

If it carries extra terms, thin-prism terms are the honest ones. They describe what a tilted plate of glass does, they take nearly all of a thin wedge’s effect near the axis, and the magnitude they report is a direct statement that something prism-like is in the path.

And if the glass can be modelled, it should be. A wedge of glass turns the camera behind it found that a camera recovered through a wedge is turned three times too far with nothing left over to say so; the wedge admitted into the model recovers both the camera and the glass. Any calibration model that absorbs the wedge into the lens is a model that will be wrong the moment the glass is moved and the lens is not.

What was assumed #

The lens has no distortion. Every term a calibration reports here is the glass’s. A real lens with its own radial and decentring distortion adds its terms to the glass’s, and a calibration through glass then reports the sum; the glass’s share is identifiable only by moving the glass.

The readings are exact. With reading error, every model’s residual has a floor, and the reaches at which the glass is announced move out further; the figure on quiet residuals uses the reading error only as a threshold, not as noise in the fit.

Two rings of six readings. A calibration from a dense chessboard fills the frame and constrains every model much more, which leaves each set of terms less freedom to trade with the principal point — and more readings far off the axis, where no model takes the wedge.

Still open: whether moving the glass separates it from the lens #

The first assumption names the one test that can tell the glass’s terms from a real lens’s: move the glass. A calibration done twice, with the wedge turned between the two by a known angle about the camera’s axis, sees the lens’s terms stay where they are and the glass’s terms turn with it.

The measurement that settles what that is worth calibrates a camera whose lens has a stated radial and decentring distortion, behind a two-degree wedge, twice — the second time with the wedge turned by a stated angle — with the same glass-blind model and 0.4-pixel reading error, and asks whether the difference between the two calibrations recovers the glass’s terms and the lens’s separately, and how small a turn is enough. If a turn of a few tens of degrees separates them to the precision a calibration needs, a camera behind a window can be calibrated without modelling the window at all, by rotating it; if it needs a half turn, the window has to be modelled or removed.