An image circle pins a fisheye's horizon, if its field is known

A stereographic fisheye is a division model fitted the division model to the four fisheye laws and found two things about its horizon, the picture radius at which the model sends a direction to ninety degrees. A free fit puts that horizon in the wrong place — beyond the law’s own ninety degrees by 0.6 to 16 per cent, depending on the law and on how much of the field the fit was shown — and pinning it at the right place trades error inside the fitted field for error beyond it, at roughly three to one in each direction. The pin was set at the law’s own ninety-degree radius, which a calibration does not know.

What a calibration has is a picture, and a fisheye picture stops. The ring beyond which the sensor sees nothing — the image circle — is the picture radius at the widest direction the lens accepts. For a lens covering exactly 180 degrees, that direction is ninety degrees off the axis, and the circle is exactly the radius the division model calls its horizon. The earlier essay ended by asking whether that is enough: whether the circle, measured from the picture with its own error, can be handed to the fit as one more observation, and how well it has to be measured before it helps.

One more observation #

The calibration measured here is a standard one reduced to its radial core. A 180-degree fisheye with a focal length of 600 pixels is shown 24 targets of known direction, evenly spaced in angle out to sixty degrees, and each target’s distance from the centre of the picture is read to half a pixel. The two-coefficient division model is fitted to those 24 readings by least squares, each weighted by its own error. That is the calibration without a circle, and the free fit of the earlier essay with noise added.

The calibration with a circle adds one reading: the circle’s radius, measured with an error of its own, and a residual asking the model’s horizon to sit at that radius, weighted by that error. It is a soft pin. Read to a hundredth of a pixel, it holds the horizon on the circle; read to a thousand pixels, it says nothing and the fit is the free one; between the two it pulls as hard as its measurement deserves.

The solid curve is the earlier essay’s free fit seen through a calibration’s eyes. Inside the targets’ sixty degrees it stays within a fifth of a pixel of the law. Beyond them it runs off, reaching 9.4 pixels at 89.5 degrees, because its horizon is at 952.3 pixels where the lens has its ninety degrees at 942.5: ten pixels too far out, one per cent of the radius, which is the equidistant row of the earlier essay’s horizon figure.

The dashed curve adds the circle, read to two pixels. The horizon comes in to 945.4, the error at the edge falls to 2.6 pixels, and inside the targets’ field the worst error rises from 0.17 to 0.57 pixels. The slider changes how well the circle is read: at half a pixel the error at 89.5 degrees falls to 0.22; at twenty pixels the circle barely moves the fit, and the dashed curve lies on the solid one.

How well the circle has to be read #

A single calibration is one draw of the targets’ errors. The sweep below repeats the calibration forty times at each setting and takes the median of each calibration’s worst error, inside the targets’ field and beyond it, as the circle’s error runs from a quarter of a pixel to a hundred.

For the equidistant law the answer has two steps. A circle read to two pixels or better brings the edge down to about two pixels and no further, however well it is read, because two pixels is what the division model itself cannot do: the earlier essay found the hard pin leaving an error of that order beyond a sixty-degree fit with no noise at all. A circle read worse than that gives back its gain gradually — 5.6 pixels at five, 7.8 at ten — and read to fifty pixels keeps less than a tenth of it.

Inside the targets’ field the price is small and moves the other way. With the circle held tightly the worst interior error is 0.72 pixels against 0.28 alone; as the circle loosens the interior returns to the free fit’s accuracy. The trade the earlier essay described at three to one is, with realistic targets, closer to a factor of five gained at the edge for a factor of two and a half lost inside — and the loss inside is under a pixel.

The slider shows the other laws, and they are not alike. The equisolid law behaves like the equidistant at twice the size: 18.4 pixels at the edge alone, 5.8 with a two-pixel circle, 3.4 with a one-pixel circle, for an interior cost of about a pixel. The stereographic law, which the division model describes exactly, takes the edge from 8.3 pixels to 1.0 and costs nothing inside — the circle there only removes the targets’ extrapolation noise, since there is no model error to trade. The orthographic law, whose radius levels off at ninety degrees where no division model can follow, gains least and pays most: 70.6 pixels at the edge alone, 53.6 with a two-pixel circle, and at a quarter of a pixel the edge falls to 22 while the interior rises to 17. For that lens the circle is a real trade, and it is the choice the earlier essay called one about what a model is for.

The floor is the model’s, not the circle’s #

The two pixels at which the equidistant sweep levels off are worth tracing to their source, because they say what a better circle cannot buy. Fitted to the equidistant law itself — no targets, no noise, every direction out to sixty degrees known exactly — with its horizon held exactly at the law’s ninety degrees, the division model still misses the law by 1.9 pixels somewhere between sixty degrees and the edge. That is the hard pin of the earlier essay on this lens, and it is the level the soft pin reaches: 2.0 pixels with the circle read to a quarter of a pixel, to half, to one or to two. The circle has done everything it can once the horizon is in the right place; the remaining error is the shape of a two-coefficient model failing to be an equidistant lens between the targets and the rim.

The same comparison holds at the other reaches, and it separates what the circle fixes from what it cannot. Pinned exactly, the model misses the law by 2.5 pixels beyond a forty-five-degree fit and 1.4 beyond a seventy-five-degree one; freed, it misses by 12.6 and 5.4. The circle read to two pixels brings the noisy calibrations to 2.9 and 4.0. At forty-five degrees that is the floor plus the circle’s own error. At seventy-five it is well above the floor, and the gap is the pull the next section measures — the targets arguing with the circle, and winning part of the argument.

In angle, which is what a stitcher or a sky camera cares about, the numbers are small but not negligible. An equidistant lens with a 600-pixel focal length turns a pixel at any radius into 0.095 degrees of direction. The free fit’s 9.4 pixels at the edge is nearly a degree — a seam between two fisheye pictures that visibly fails to meet — and the circle’s two pixels is under a fifth of a degree, which a stitcher’s blending hides. The circle turns an edge that shows into an edge that does not, and that is the whole of its practical value.

Where the circle buys most #

The circle stands in for targets the calibration did not have. It should matter most when the targets stop soonest.

It does. Targets out to forty-five degrees leave half the field to extrapolation, and the circle cuts the error there by a factor of six for the equidistant law, five for the equisolid and thirty-seven for the stereographic, whose free fit with noisy targets over so narrow a field puts its second coefficient almost anywhere. Targets out to seventy-five degrees already see most of the edge; the circle then takes the equidistant error only from 5.1 pixels to 4.0 and the equisolid from 10.6 to 9.0.

This is the useful practical reading. A calibration done with a flat target — a printed board held in front of the lens, or the straight lines which rule a fisheye obeys from straightness alone used instead — rarely reaches past sixty degrees on a fisheye, because a flat board cannot fill a hemisphere, and the targets near the edge are seen so obliquely that their centres are hard to read. That is precisely the calibration the circle helps most, and it costs nothing to collect: every picture the lens takes carries it.

The targets pull the horizon off the circle #

The seventy-five-degree row has a feature that looks like a failure. With the circle read to two pixels, the fitted horizon should sit within about two pixels of it; for targets out to seventy-five degrees it does not.

The fit settles 4.9 pixels outside the circle, two and a half times the circle’s stated error, and the further the targets reach the harder they pull. Targets to forty-five degrees let the horizon sit within a fifth of a pixel of a two-pixel circle; targets to sixty hold it 2.3 pixels off; to seventy-five, 4.9.

The reason is that the model is not the law. A two-coefficient division model can meet the equidistant law’s ninety-degree radius or follow its radius out to seventy-five degrees closely, but not both at once: the earlier essay found the hard pin raising the interior error of a seventy-five-degree fit threefold. Targets that reach far are numerous exactly where the model is strained, each read to half a pixel, and together they outweigh one circle read to two. The fit then puts the horizon where the disagreement between the two kinds of evidence is balanced by their weights, which is not where either of them says.

That is not an error in the calibration; it is the calibration reporting, in the one number that can show it, that its model cannot satisfy its observations. A calibration that holds its horizon several of the circle’s errors away from the circle has found the model’s limit, and a reader who needs the edge right should either trust the circle more than its error bar (and pay inside) or use a model with a third coefficient. The distance between the fitted horizon and the circle, in units of the circle’s error, is a free diagnostic of model inadequacy near the edge. No other observation in a radial calibration provides one.

The field is part of the observation #

Everything above assumed a lens covering exactly 180 degrees, so that its circle is its ninety-degree radius. Fisheyes are sold at 180 degrees, but many cover more — 185, 190, 200, 220 — and some, cropped by the sensor or the lens barrel, less. The circle of such a lens is the law’s radius at half its field, not at ninety degrees.

The pin is only as good as the field it assumes. At the true field of 180 degrees the circle brings the edge to 1.9 pixels. Five degrees either way — a 175-degree or a 185-degree lens — it makes the edge 23 or 24 pixels wrong, two and a half times worse than no circle at all. A 190-degree lens misread as 180 is 49 pixels wrong at the edge, five times worse; a 220-degree lens, 198 pixels.

A lens wider than 180 degrees is worse than it looks, because its circle is at a direction the division model cannot represent. The model is built on the pinhole radius, which is infinite at ninety degrees, so every one of its settings ends at ninety; a 190-degree lens’s circle marks ninety-five degrees, and pinning the horizon there puts ninety degrees five degrees too far out. The directions past ninety, which a 190-degree lens does see, are not wrongly placed by the model. They are not placed at all. A model that inverts has a horizon instead of a fold described this limit as the price of a model that never turns back; the image circle makes the price visible in the picture, where it is the ring of directions the lens sees and the model cannot.

So the circle is not an observation of the horizon. It is an observation of the radius at the lens’s field, and a statement of the field is part of it. A 180-degree lens whose field is known to a degree makes the circle the best single observation its pictures hold for the edge of the calibration; a lens of unknown field makes it a trap, because nothing in the fit says the assumption is wrong until the residual between the horizon and the circle — the diagnostic above — grows past what the circle’s error allows.

What the circle gives a calibration #

The measurement answers both halves of the earlier essay’s question. A circle read to a couple of pixels improves the extrapolation of an equidistant, equisolid or stereographic calibration far more than it harms the interior — an edge three to eight times better for under a pixel inside — and read to twenty pixels or worse it carries almost nothing the targets have not said. And a 180-degree field, which a division model reaches only at its horizon, is exactly the case in which the circle can be used; any other field makes it worse than useless unless the field is itself known and the model can reach it.

Every fisheye is a different rule found the four laws agreeing near the axis and parting at the edge, which is why the edge is where a calibration’s choice of model shows. The circle is the edge’s own witness. It does not say which law the lens follows; it says where the law ends, and that turns out to be the thing a calibration from targets knows least.

What was assumed #

The circle is sharp. Most fisheye pictures fade towards their circle through vignetting, and the ring a reader measures is where the brightness falls to some fraction of its centre value — which depends on the exposure, the lens’s stop and the scene. A circle read as a brightness threshold carries a bias that no amount of averaging round the ring removes. The sweep above measured random error only — a circle known to five pixels gives back about half its gain — and a bias of the same size was not measured, though it cannot be expected to cost less.

The circle is centred on the principal point and round. A real image circle can be offset from the distortion centre by several pixels, and a sensor that crops it leaves arcs rather than a ring. An offset circle read as centred reports one radius on one side and another on the other; fitting the centre and radius together from the ring’s edge points is a small extension that was not made here.

The targets are at known directions. A calibration from straight lines, which fitting a lens from straightness alone measured for the polynomial, has no known directions, only lines that should be straight; whether the circle constrains a line-based division calibration as usefully is a separate question, since the lines do not supply the focal length that turns the circle’s pixels into a radius in focal lengths.

The lens follows one of the four laws. Real fisheyes are designed near a law and depart from it. The diagnostic in the pull figure would report a departure as model inadequacy, which is correct, but would not say which way the lens departs.

Still open: whether the ring’s edge points can find the centre and the radius together #

The circle here is one number, its radius, read about a centre the calibration already knew. A picture offers far more: every point along the ring where the lens’s picture meets the dark is an observation, and there are thousands of them.

The measurement that settles what they are worth fits a circle to the ring’s edge points — each read across the edge with a stated error, some fraction of the ring hidden by the sensor’s corners — and hands the fitted centre and radius to the division calibration together, the centre as the distortion centre’s prior and the radius as the horizon’s. It asks two things: how precisely a partial ring fixes a centre that the targets alone leave uncertain by a pixel or two, and whether a ring cropped to arcs, as on a sensor narrower than the circle, still fixes the radius to the two pixels this essay found to be enough — or whether the arcs’ short reach lets the centre and the radius trade off along the axis the crop leaves open, in the same way two coefficients of one model trade along a valley.