A stereographic fisheye is a division model

A model that inverts has a horizon instead of a fold set two families of radial model side by side. The polynomial every calibration fits, $r_d = r_u(1 + k_1 r_u^2 + k_2 r_u^4)$, rises to a maximum and turns back, so past a radius its own coefficients fix it is no longer a map from direction to picture. The division model, $r_u = r_d/(1 + \lambda_1 r_d^2 + \lambda_2 r_d^4)$, never turns back: for a barrel lens its picture radius rises for ever toward a horizon, and the whole hemisphere of directions lands inside a finite disc.

The essay then fitted both to the four fisheye laws and reported that the polynomial was the better model below sixty degrees and the division model better past seventy, by factors of three to five. It closed by asking whether the division model’s horizon — which a fit places wherever the residual likes — should be put somewhere deliberately.

Answering that question meant fitting the division model again with a constraint on it, and the first thing the new fit found was that the old one had not been fitting the division model.

The model that was fitted instead #

The fitting routine solved $r_d(1 + \lambda_1 r_d^2 + \lambda_2 r_d^4) = r_u$ for the picture radius. That is a polynomial, run from picture to pinhole: it multiplies the picture radius by a polynomial where the division model divides by one. The two agree to first order — dividing by $1 + \lambda r^2$ and multiplying by $1 - \lambda r^2$ differ only in the square of a small quantity — which is presumably why the substitution went unnoticed, and they part completely at a wide field.

The polynomial run backwards has none of the properties the essay was about. With the positive coefficients the fits returned, its pinhole radius grows without bound as the picture radius does, so it has no fold and no horizon either: it maps the whole picture plane onto the whole pinhole plane. The essay’s figures of the fold and the horizon were drawn from the true models and are unaffected. Only the residuals — the comparison on the four laws — came from the substitute.

Fitted to the equidistant law, the three models are far apart. The polynomial in the direction a renderer uses reaches $3.1\times10^{-3}$ of the picture’s radius at forty-five degrees and $2.6\times10^{-1}$ at seventy-five. The polynomial run backwards — the numbers the earlier essay reported — reaches $4.8\times10^{-3}$ and $5.5\times10^{-2}$. The division model reaches $2.6\times10^{-5}$ and $5.7\times10^{-4}$: a hundred times closer than the polynomial at forty-five degrees and nearly a hundred times closer than its own impostor at seventy-five.

Fitted as itself, it wins at every field #

With the right model in the fit, the comparison the earlier essay drew changes shape entirely.

At every field from forty degrees to eighty and for every law, the division model is the more accurate. For the equidistant law the margin is a factor of 105 at forty degrees and 452 at seventy-five. For the orthographic law, the one that departs most from a pinhole, it is $1.5\times10^{-2}$ against $4.9\times10^{-1}$ at seventy-five. There is no field in the range where the polynomial is the better model; the earlier conclusion that it was below sixty degrees came from a comparison with the wrong curve.

The margins are largest where the polynomial was supposed to be at home. At forty-five degrees the division model follows the equisolid law to $8.0\times10^{-5}$ of the picture’s radius, where the polynomial manages $3.9\times10^{-3}$, a factor of nearly fifty; the orthographic law, which is the hardest for both, it follows to $7.5\times10^{-4}$ against $6.5\times10^{-3}$. These are the fields most calibrations are run over, so the correction is not a matter of wide-angle curiosities. A two-coefficient division model is a better description of every fisheye law at every field a camera is likely to be calibrated over, and a markedly better one at the edge.

The practical reading shifts with it. At a narrow field the polynomial remains safe — its fold is far outside the data — and it is what every toolkit expects. But it is not the more accurate model there. The reason to keep it is convention, and the earlier essay’s rule of thumb has been rewritten in place to say so.

The stereographic law is the model #

The flattest line in that figure is not a small residual. It is none.

The stereographic fisheye puts a direction at angle $\theta$ at picture radius $2f\tan(\theta/2)$. A pinhole puts it at $f\tan\theta$. Write $t = \tan(\theta/2)$; the double-angle identity gives

$$\tan\theta = \frac{2t}{1 - t^2} = \frac{r_d}{1 - r_d^2/4}, \qquad r_d = 2t,$$

which is the division model with $\lambda_1 = -1/4$ and $\lambda_2 = 0$, exactly.

The figure draws the law and the model at that coefficient on top of one another; they agree to $2\times10^{-16}$ of a focal length from the axis to within a tenth of a degree of ninety, and a least-squares fit over seventy-five degrees finds the coefficient as $-0.250000000$. The horizon, $1/\sqrt{1/4} = 2$ focal lengths, is where the law puts ninety degrees.

So one of the four standard fisheye designs is not approximated by the division model but described by it, which says something about why the model works as well as it does on the other three. Every fisheye is a different rule found the four laws agreeing near the axis and parting toward the edge; the stereographic is the one that bends least, and the division model with one coefficient is that bend. The other laws are near it, and a second coefficient accounts for most of the difference.

Where a free fit puts ninety degrees #

A division model fitted over part of the field extrapolates to all of it, because its horizon is ninety degrees by construction. The question the earlier essay asked is where on the picture that horizon falls.

Every free fit puts it too far out. Fitted to the equidistant law over seventy-five degrees, the horizon is at 1.5804 focal lengths where the law has ninety degrees at 1.5708, six tenths of a per cent beyond. Over forty-five degrees it is 1.4 per cent beyond. The equisolid law behaves alike at twice the size — 1.4 per cent beyond over seventy-five degrees and 3.1 over forty-five — and the orthographic is the outlier, its horizon 8.4 per cent beyond at seventy-five degrees and 15.9 per cent at forty-five. A renderer extrapolating any of these fits draws the edge of the hemisphere outside the lens’s own edge.

All three laws bend further from the pinhole than the stereographic does, and a fit over a narrower field sees less of that bending and misplaces the horizon further — every row of the figure moves outward as the field shrinks. Why the error falls outward for all three was not derived here. The orthographic’s size has a plain reason: its radius $f\sin\theta$ stops growing at ninety degrees, reaching $f$ with zero slope, while a division model’s radius arrives at its horizon with a slope that is finite and not zero. No setting of the coefficients can level off the way that law does, so the fit can only overshoot and meet it late.

Pinning the horizon trades one error for another #

Pinning the horizon is one line of algebra: the denominator $1 + \lambda_1 r^2 + \lambda_2 r^4$ must vanish at the law’s own ninety-degree radius, which fixes $\lambda_2$ once $\lambda_1$ is chosen, and the fit then has one coefficient left to spend.

The pin is not free. Over seventy-five degrees it raises the worst error inside the fitted field by 2.7 to 2.9 times: for the equidistant law from $5.7\times10^{-4}$ to $1.7\times10^{-3}$, for the orthographic from $1.5\times10^{-2}$ to $4.1\times10^{-2}$. And it is not useless. Beyond the fitted field, from seventy-five degrees to 89.5, it cuts the worst error 2.1 to 3.8 times: for the equidistant law from $5.8\times10^{-3}$ to $1.5\times10^{-3}$. At a forty-five degree fit the trade is steeper in both directions — the free fit is very good inside and very wrong outside — and the slider shows it.

The earlier essay’s question asked whether a pinned horizon was “nearly free and should simply be imposed”. It is neither. It is a choice about where a model’s error goes, and the choice depends on what the model is for. A calibration used only inside the field it was fitted over should be left free; one handed to a renderer that will draw the whole hemisphere should be pinned, because the renderer will use exactly the part of the model the free fit got wrong.

What the pin does to the edge #

The trade is easier to see as a curve than as four numbers.

Inside the fitted field, the free model’s error is the smaller and the pinned model’s is larger and smoothly spread — it has had to bend its interior to reach the law’s value at ninety degrees. In the band the fits were never shown, the free model’s error climbs steeply toward its misplaced horizon and reaches 0.58 per cent of the radius at 89.5 degrees. The pinned model’s error in the same band turns round and comes back: it ends at −0.02 per cent at 89.5 degrees and never exceeds 0.15 per cent anywhere beyond the field, because both of its ends — the axis and the horizon — are the law’s.

On the orthographic law, which the slider reaches, the same shape appears larger: the free fit’s horizon is 8.4 per cent out, and its error at the edge is correspondingly large. The pin halves it, at the cost of a worse fit inside. No setting of two coefficients follows the orthographic law’s flattening at ninety degrees, since the division model’s radius is still rising steeply there, and that is the one law of the four for which neither choice is good.

The two directions a model is run in #

The earlier essay recommended the division model partly for its algebra: with one coefficient its inverse is a quadratic with a closed-form root, so going from a direction to a picture radius needs no iteration. That remains true, and it matters most for a renderer, which runs exactly that direction for every pixel. The render is distorted on purpose measured a headset’s pre-warp closing its round trip to a thousandth of a millionth of a pixel; a warp built on a one-coefficient division model gets the same closure with a square root instead of a loop.

The second coefficient costs that convenience. With $\lambda_2$ in the denominator the inverse is a quartic in the picture radius rather than a quadratic, and the fits in this essay find it by bisection inside the horizon. That is still a well-posed problem — the map is monotone below the horizon for every barrel pair fitted here, so the root is unique — but it is an iteration, and a renderer that adopts the two-coefficient model for its accuracy gives up the closed form it was chosen for. The stereographic law is the case where nothing is given up: one coefficient is exact, and the inverse is the quadratic.

The polynomial runs the other way round. Its forward direction, from direction to picture, is arithmetic, and its inverse is the iteration; a barrel model folds at a radius it sets itself found that iteration returning wrong directions silently past the fold. The division model’s iteration cannot fail that way, because below its horizon there is only one root to find.

What changes for a calibration read from lines #

A calibration does not usually fit a law; it fits marks — often the bend of lines that ought to be straight. Fitting a lens from straightness alone and the lines that calibrate a lens measured how well a picture’s straight edges fix the polynomial’s coefficients, and a known target sharpens the fit and does not separate it found the two coefficients correlated by the shape of the model rather than by the instrument.

None of those measurements is changed by the correction here, since all of them fitted the polynomial, and said so. What changes is the choice they sit inside. If a lens is wide, the model a straightness fit should be asked to recover is the division model, since it follows every fisheye law more closely at every field; and whether the division model’s two coefficients are correlated as the polynomial’s are is a question the line-based calibrations have not been asked. Straight lines that are not holds for both families — a radial map leaves straight exactly the lines through the principal point — so the lines themselves carry the same kind of information to either model.

Why the substitution survived #

The polynomial run backwards is a reasonable model in its own right; some calibration tools use it. What went wrong was not the model but the name, and it is worth saying why nothing caught it.

The earlier essay’s claims about the two families were drawn from the families’ formulas, and those drawings were right. The residuals were computed by a routine whose name said “division”, and every check on the routine asked whether its fits were good fits — whether they converged, whether they beat the polynomial where the essay expected — and they were good fits, of a different model. The slip that leaves no trace described the same shape in a construction: an error that lands back on the set of correct-looking answers. A fitted residual is always a residual of something, and a check that asks only whether it is small cannot tell which something it belongs to.

The stereographic law is what exposes it. A model that is exact for one of the four laws should return a residual at the arithmetic’s floor on that law, and the substitute returned $3.5\times10^{-2}$ at seventy-five degrees. The exactness is a test the fitting routine can now fail, and it is the test that would have caught the substitution on the day it was made.

What the horizon cannot do #

Past ninety degrees. The division model’s horizon is ninety degrees by construction, since it is built on the pinhole radius $\tan\theta$, which is infinite there. A lens whose field exceeds a hemisphere — many fisheyes cover 180 degrees or more — cannot be described past ninety degrees by any setting of it, pinned or free. The polynomial in the picture-to-direction sense, or the laws themselves, can.

A real lens. The four laws are targets, as they were in the earlier essay. A real fisheye follows none of them exactly, and its own ninety-degree radius is a measured quantity with an uncertainty; pinning the horizon at a radius that is itself uncertain moves the trade in ways not measured here.

Other constraints. Pinning the horizon fixes the model’s value at ninety degrees and not its slope. A model constrained to meet the law’s slope there as well would spend both coefficients on the edge and none on the interior.

Still open: whether a fit can find the horizon from the picture #

Everything above pins the horizon at a radius read off the law, and a calibration does not have the law. What a calibration has is a picture that stops somewhere: the image circle of a fisheye, the dark ring beyond which the sensor sees nothing, is exactly the picture radius at the lens’s widest direction.

The measurement that settles whether that is enough fits the division model to a set of marks inside a field, adds the image circle’s radius as one more observation with its own uncertainty — the radius at which the model reaches the lens’s stated field — and asks two things: how precisely the image circle has to be measured before it improves the extrapolation more than it harms the interior fit, and whether a stated field of 180 degrees, which a division model cannot reach, can still be used by pinning the horizon at the image circle and accepting that the outermost directions are wrong. If the image circle is enough, every fisheye picture carries its own pin.