A barrel model folds at a radius it sets itself

Every distortion essay before this one has used one model. Straight lines that are not introduced it: a lens moves each point along its own radius from the principal point, by a factor $1 + k_{1}r^{2} + k_{2}r^{4}$ that depends only on how far out the point is. A lens destroys the invariant measured what it costs the cross-ratio, and the render is distorted on purpose ran it backwards to pre-warp a headset’s picture. Fitting a lens from straightness alone recovered its coefficients from nothing but bent lines.

All of that treats the polynomial as a lens. It is a formula, and a formula has a domain. This essay finds the edge of that domain, measures what a routine that uses the formula does when a direction crosses it, and asks how close to the edge a real calibrated lens sits.

The map, drawn against the field angle #

The radial model sends a direction whose undistorted radius is $r$ — in focal lengths, the tangent of its angle off the axis — to a picture radius

$$r_{d} = r\,(1 + k_{1}r^{2} + k_{2}r^{4}).$$

For a pinhole, $r_{d}$ is simply $r$: the further off the axis a direction is, the further out it lands, without limit. For a barrel coefficient, $k_{1}$ negative, the correction subtracts a term growing as the cube of $r$, and a term growing as a cube eventually overtakes a term growing linearly.

At $k_{1} = -0.28$ the picture radius climbs to 0.727 focal lengths and then turns back down. The turn happens where the slope $dr_{d}/dr = 1 + 3k_{1}r^{2}$ reaches zero, at

$$r = \frac{1}{\sqrt{-3k_{1}}} = 1.0911,$$

which is a field angle of 47.49°. The figure does not take the closed form on trust: it searches the map for its maximum by a dense scan and a golden-section refinement, and lands on the formula’s radius to within five parts in a billion.

Past that angle the model is still defined, still smooth, and still returns a number — but it is no longer a lens in the one sense a lens has to be. Directions further out land nearer the centre. A picture radius just under the peak, 0.713 focal lengths, is reached from 43.91° and again from 50.54°: two directions, one place in the picture. Nothing in the picture says which of them put a mark there.

The fold and the zero are different places #

The model has a second landmark, and it is the one a careful undistortion routine watches for. The radial factor $1 + k_{1}r^{2}$ itself falls to zero at $r = 1/\sqrt{-k_{1}}$, which at $k_{1} = -0.28$ is 62.11° of field. Past that, the factor is negative and directions are thrown through the centre to the other side of the picture — an obvious absurdity, and easy to test for.

The two radii are in a fixed ratio when $k_{2} = 0$: the zero is exactly $\sqrt{3}$ times further out than the fold. So between 47.49° and 62.11° lies a band where the radial factor is still comfortably positive, nothing looks absurd, and the model has already stopped being invertible. A check for a positive radial factor passes throughout it.

What an undistortion routine does across the field #

This matters because the direction that gets used is almost never the forward one. A calibration’s product is a routine that takes a mark in a picture and returns the direction it came from, and that routine has to invert the polynomial, usually by fixed-point iteration: guess the undistorted radius, divide the mark’s radius by the radial factor there, and repeat until the guess stops moving. The routine measured here does exactly that, and refuses if the radial factor at any guess is not positive.

The figure runs that routine, as it stands, across the field, a quarter of a degree at a time. Up to about 46° every direction comes back to the arithmetic floor. Beyond that the failures come in three different kinds, and they are worth separating because only one of them is visible to anything the routine could check.

Near the fold, the iteration stops short. Where the map is nearly flat, each correction moves the guess very little, and the iteration runs out of steps before it arrives. At 46.75° the answer is 0.012° out, at the fold itself 0.30°, at 48° a little over a degree. These answers do not quite reproduce the mark they came from: at 48° the round trip misses the mark’s radius by $5 \times 10^{-5}$ of a focal length, which is 0.02 px on this frame.

Past the fold, the iteration converges — to the other direction. Every mark from a direction beyond 47.49° lies at a radius that a direction inside the fold also reaches, and the iteration, starting from the mark’s own radius, finds the inner one. 50° comes back as 44.63°, 55° as 35.56°, 60° as 15.83°. These answers are not approximations. They are exact inverses of the mark, to the last digits of the arithmetic.

Past the zero, the mark is on the other side. From 62.25° the radial factor is negative, so the mark lands through the centre on the far side of the frame, and the routine returns the direction on that side that produces it: 65° comes back as −35.85°. Thirteen of the directions swept fall here, and they include the worst of all, 65.25° returned as −41.30°, which is 106.5° from where it arrived.

Only from 65.5° does the routine refuse, and not because the radial factor went negative. It refuses because the mark now lies 0.764 focal lengths from the centre, beyond the 0.727 the model can reach at all, so no direction maps there and the iteration wanders until a guess makes the factor non-positive. Across the whole sweep from 0.5° to 89°, 21.1% of the directions are returned silently wrong.

A round trip cannot catch it #

The ordinary defence against a bad inverse is to check it: take the direction returned, push it forward through the model again, and confirm it lands on the mark. That check is what a careful pipeline runs, and it is worth measuring what it sees here.

It sees the first kind of failure, faintly — the 0.02 px left by an iteration that stopped short near the fold. It sees nothing of the other two. At 55° the returned direction is 19.4° wrong, and pushed forward it lands on the mark to $2 \times 10^{-15}$ of a focal length. Past the zero the closure is just as good. A returned direction that genuinely produces the mark cannot be distinguished, by any test applied to the mark and the answer, from the direction that actually produced it, because both are correct answers to the question the routine was asked.

That changes what the problem is. The routine is not badly written, and a better stopping rule or a stricter refusal would not fix it. The information that a mark came from beyond the fold is not in the mark. The guard has to be on the field, not on the answer: a model can be trusted to invert only a picture whose directions all lie inside its fold, and whether they do is a fact about the camera and the frame, not about any one mark.

The same sign flip, behind the eye #

The third kind of failure has a close relative in an earlier essay, and the resemblance explains why it passes unnoticed.

What happens behind the eye found that a point behind a camera has a perfectly plausible image: dividing by a negative coordinate flips both signs, and the point lands through the principal point on the far side of the frame, inside it, looking like any other point. A radial factor that has gone negative does the same thing by a different route. The direction at 65° is on one side of the axis; its mark is on the other; nothing about the mark’s coordinates is unusual; and the inverse, faithfully, reports a direction on the mark’s side.

Both are a sign hidden inside an ordinary-looking number, and both are caught only by asking a question about where the point could have come from — in front of the camera, or inside the model’s domain — rather than about the number itself.

Whether the fold is inside a real frame #

A fold at 47.49° would be a curiosity if no picture reached that angle. The wide frame the lens figures are drawn in has a 78° horizontal field on a 690 × 400 canvas, and its corners sit 43.1° off the axis. So at $k_{1} = -0.28$ the fold lies about four degrees outside the corners. The picture itself is entirely on the invertible side, and the silent band begins just beyond it.

That margin is not a property of wide lenses. It moves with the coefficient.

At $k_{1} = -0.5$ the fold is at 39.23°, inside the same frame’s corners. The corners of such a picture carry marks whose inverse is ambiguous, and an undistortion that runs across the whole frame hands back wrong directions for the outermost of them — directions that, by the argument above, no check on the answers will flag. Since the fold radius goes as $1/\sqrt{-3k_{1}}$, doubling the strength of the barrel moves it inward by a factor of $\sqrt{2}$, and the band where a correction is silently wrong walks into the frame from the corners.

The second coefficient, and what it is quietly doing #

A single barrel coefficient always folds. A second coefficient of the opposite sign need not.

With $k_{2} = +0.09$ the slope becomes $1 + 3k_{1}r^{2} + 5k_{2}r^{4}$, a quadratic in $r^{2}$ with a negative discriminant, and it never reaches zero. The map rises across the whole field, every picture radius is reached from exactly one direction, and the inverse is well defined everywhere.

That is worth reading carefully, because it gives the second coefficient a job nobody asked it to do. A calibrated wide lens with a negative $k_{1}$ and a positive $k_{2}$ is, among other things, a model that has been kept from folding. And the plumb-line fit found that $k_{1}$ and $k_{2}$ sit in a long valley, correlated at −0.997, where trading one against the other barely changes the fit.

Walk along that valley and the fold comes back. On that essay’s own five-line fit, a model 0.075 of the way along the soft direction leaves the lines 0.444 px from straight — well inside what ordinary marking noise would produce — and folds at 57.8° of field. It does not fold inside the frame, whose corners are at 43.1°, and no model along the walk does. But a coefficient pair that the data cannot tell from the true one has a fold, and the fold is fourteen degrees outside the frame rather than nowhere.

So the practical reading is specific. A model fitted to a picture is invertible over that picture, because the fit would have seen the fold otherwise. It is not safely invertible beyond the frame it was fitted on — a wider crop from the same lens, a mark extrapolated past the corner, a direction computed for a point just outside the field — and the second coefficient that keeps it from folding is the least determined number in the calibration.

A fisheye is not a strong barrel #

The last place the fold matters is the one where it is most tempting to ignore it: lenses whose field reaches toward, or past, ninety degrees.

A fisheye does not follow a perspective mapping at all. Its designers choose a different law — equidistant, where the picture radius is proportional to the angle itself, $r_{d} = f\theta$; or stereographic, equisolid, orthographic — and every fisheye is a different rule set out why there are several: one is a protractor, one a counting instrument, one preserves shape. Every one of them, written as a function of $r = \tan\theta$, has a series beginning $r(1 + c\,r^{2} + \dots)$ with a negative $c$. For the equidistant law $c = -1/3$, and the figure recovers that coefficient numerically, from the law alone, to six decimal places. So near the axis every fisheye law is a barrel polynomial, and a two-coefficient model fits it well.

Fitted out to 60°, the best two-coefficient polynomial is within 2.9% of the equidistant law’s picture radius everywhere in its field, and it does not fold anywhere in the field drawn. A calibration done this way, on the central part of a fisheye, would report a small residual and be right to.

Fitted out to 80°, the best two-coefficient polynomial is 58.7% out, and it folds at 67.4° — inside the field it was fitted to. The other three laws do the same, folding between 65° and 69° when fitted to 80°. The least-squares fit has done the best a polynomial in $\tan\theta$ can, and the best is a model that cannot be inverted across its own data, because $\tan\theta$ runs to infinity at 90° and a law that stays finite there cannot be followed by any finite polynomial in it.

That is the precise sense in which a fisheye needs a different model rather than more coefficients. Adding terms pushes the fold outward and shrinks the residual; it does not change the fact that the model is built on a variable the lens does not have. Past 90° the variable does not merely grow without limit — $\tan\theta$ changes sign, so a direction behind the lens would be written as one in front of it — and a mirror ball is an equal-area fisheye photographs every direction there is, which no polynomial in $\tan\theta$ can describe at all.

The constructive half of the result has already been measured. Which rule a fisheye obeys from straightness alone fits the laws themselves to bent lines, in their own variable, the angle, and names the one that took a photograph reliably from about 45° of half-field. That is the same plumb-line idea as the barrel fit, pointed at a family whose members do not fold.

What this does not say #

It does not say calibrated lenses are routinely wrong. A careful calibration of an ordinary wide lens produces coefficients that are invertible across the frame they were fitted on, and the numbers here show why: the fit would have noticed a fold in its own data. The hazard is extrapolation past that frame, and it is a hazard of using a model outside its data rather than of the model.

It does not say anything about a particular library. The undistortion routine measured here is a plain fixed-point iteration, run with the setting a calibration’s search uses. How widely used software handles the fold is a question about that software, and nothing here tests it. What the measurement does establish holds for any routine: past the fold, the answer returned is a true inverse of the mark, so no test applied to the mark and the answer can separate it from the right one.

And the fold is a property of the model, not of glass. A real lens does not fold its image; a polynomial describing it does, outside the range where it describes it. Every statement above is about the formula, which is the thing that gets inverted.

Still open: whether a tilted sensor is a distortion at all #

This essay found where the radial model stops describing a lens. There is also a departure from a pinhole that the same calibration models describe without its being a departure at all. Brown–Conrady’s model carries tangential coefficients alongside the radial ones, and they are the terms a calibration reaches for when a picture’s distortion is not symmetric about the principal point. A sensor mounted a few degrees out of square with the lens produces exactly that asymmetry. A tilted sensor is not a distortion shows that such a picture is an exact pinhole picture — its lines straight, its cross-ratio intact, its principal point moved — and measures what the tangential terms do instead: absorb part of a homography with a polynomial, leave a residual, and, when used to correct the picture, bend lines that were straight.