Shot on one surface, shown on another

A picture taken on one surface and displayed on another is an ordinary thing to want. A fisheye photograph shown rectilinear; an equirectangular panorama shown as a flat view; a cylindrical stitch flattened for print. The operation is called a reprojection and it feels as though it ought to lose something.

It loses no geometry whatever, and the reason is structural rather than fortunate. Both surfaces are charts of the same object — the pencil of rays through one point — so a direction that has a mark on one has a mark on the other, and the map between them is a change of coordinates. There is nothing in a picture but directions, and a change of coordinates does not disturb them.

That control matters more than the result. A round trip returning its input is the easiest thing in this collection to achieve by accident — compose a map with something that is not quite its inverse and the errors can still cancel, and a pipeline that never actually consults the surfaces would also return its input perfectly. The 15.6° is the same machinery with one step replaced by a plausible wrong one, and it says the zero was produced by the geometry rather than by the plumbing.

So the price is not geometric, and there are two of them #

If nothing is lost in the mapping, a reader is entitled to ask what the difficulty is, since reprojected pictures plainly do look worse. The answer is two prices, and neither is about the geometry of the map.

The first price is reach, and it is worth naming as a property of the target rather than as a failure of the operation. Every surface in this collection has a bound on how far off its axis it can put a mark, and the bound is a fact about the surface’s own law rather than about its size — a plane made a metre wide still cannot hold 90°.

The first price is reach. The plane cannot hold a half-turn of directions at any size, so a fisheye’s outer sky has no image on it. Nothing is distorted; those directions simply have no target. Widening the output frame recovers some of it and never all of it, because the plane’s reach is bounded by 180° and it approaches that bound at infinite width.

What “the same pencil of rays” is claiming #

The exactness rests on one sentence and it is worth unpacking, because it is the premise that fails in every case where a reprojection is not exact.

A picture, on any of these surfaces, is a rule assigning a mark to a direction. It is not a rule assigning a mark to a point of the world — depth never appears, and two objects on one ray are drawn on top of each other whatever their distance. That is the whole content of a pinhole projection, and it is what a line is a closed curve reads from the algebraic side: what a camera records is a set of directions through a point, and the surface is only the paper it is written on.

So two surfaces sharing a centre are two notations for one thing. Converting between them is like rewriting a number from decimal to binary: the notations differ in which properties are easy to read off, and the number is untouched. The reason there is nothing to lose is that there was never anything in the picture except directions, and the reprojection is a bijection on directions wherever both surfaces reach.

That also says exactly when the claim fails, and it is not subtle. It fails when the two pictures do not share a centre — a stitched panorama assembled from exposures about the wrong pivot, or a photograph of a photograph taken from somewhere else. Then there are two pencils rather than one, the map between them depends on the depth of what is being looked at, and no coordinate change can express it. A projection of a projection measures that case and finds a homography through a plane and something measurably not a homography through a curve.

The distinction is therefore sharp rather than a matter of degree. One centre: exact, depth-free, computable from the two surface laws alone. Two centres: depth-dependent, and the geometry is a different subject.

The second price is resolution, and it is the one nobody budgets for #

The marks a picture has were laid down by the surface it was taken on. A reprojection can move them but cannot make more of them, and the target surface has its own opinion about where marks ought to be.

This is where a surface spends its pixels arriving as a practical consequence. The equidistant fisheye spends nearly evenly; the flat plane spends heavily at its edges. Reproject from the first to the second and the target’s outer regions are asking for a density the source never recorded, so those marks have to be invented by an interpolator. The geometry is exact and the picture is soft, and both statements are true at once.

The demand is worse the wider the output frame, which sets the two prices against each other.

So the choice of output field is a genuine trade with numbers on both sides. Narrow it and more of the source is discarded; widen it and more of the output is interpolated. There is no setting at which a fisheye becomes a rectilinear picture for free, and the essay’s whole practical content is that the cost is in coverage and sampling rather than in the geometry that everybody worries about.

Where the marks were, and why the shortfall has that shape #

The shape of the resolution curve is not arbitrary and it follows directly from the two surfaces’ own spending.

Read that way the resolution price is a subtraction rather than a phenomenon. The target’s demand is one curve, the source’s supply is another, and the factor a reprojection needs is the quotient. Both are known before any picture is taken, which means the cost of a reprojection is computable in advance — a photographer choosing a fisheye for later rectilinear output can work out exactly how soft the corners will be, from the two surfaces alone.

It also explains the sign change in the 120° case. At a wide output the plane’s axis is spending less than the fisheye did there, because the plane’s marks have been pulled outward to serve its edges; so the centre has marks to spare and the deficit begins only at 24° off axis.

Which direction to reproject, and why one way round is cheaper #

The two prices are not symmetric between a pair of surfaces, and the asymmetry decides which way a workflow should run.

Going from the evenly-spending fisheye to the edge-hungry plane, the target asks for marks the source did not record — ×5.49 at the frame’s edge. Run the same pair the other way and the shortfall largely disappears: the fisheye asks for a nearly flat density, and a plane’s picture has marks to spare almost everywhere, having concentrated them at its own edges. The coverage price reverses too, and more brutally — a plane at 100° carries only a third of what a 180° fisheye holds, so reprojecting outward invents nothing and simply has nothing to put in the outer sky.

That gives a rule with a reason behind it. Capture on the surface that spends most evenly and reproject toward whatever is wanted, because an even source can supply any target’s demand up to a bounded factor, while a source that has concentrated its marks somewhere cannot supply a target that wants them elsewhere. Every fisheye is a different rule sets out the four laws a lens may obey, and this is the practical criterion for choosing between them when the output surface is not yet decided.

It is also why the intermediate storage format for a panorama matters more than it looks. A picture reprojected twice pays the sampling price twice, and the second payment is made on marks that were already interpolated. The cheapest pipeline stores on the evenest surface available and reprojects once, at the end, to whatever a reader is looking at.

The reprojection is not a projection, and that is worth being exact about #

A reader who has followed a projection of a projection will want to know whether this composition is itself a projection, and the answer separates two things that are easy to run together.

A reprojection between two surfaces sharing a centre is not a new projection of the scene. It is the same projection, read off a different chart. Nothing about the scene enters — the operation takes directions to directions and knows nothing of depth, so a reprojected picture is correct for exactly the same eye position the original was, and the viewing distance changes only because the picture’s width and law have changed.

That is what distinguishes it from the compositions that essay measures, where a picture is photographed and the two centres are genuinely different. There the composite is a projection with its own centre, and through a curved surface it is measurably not a homography at all. Here there is one centre throughout, and the whole operation is a relabelling.

The distinction has a consequence. Because no scene enters, a reprojection can be performed on a picture of anything whatever without knowing the depths — which is why it is a routine operation and why stitching software does it without asking questions. The moment two centres are involved, as in a stitched panorama’s unavoidable parallax, depth enters and nothing is exact any more.

What a reader ends up looking at #

Setting the two surfaces side by side on one scene is the plainest summary of what has actually been traded.

Neither picture is more correct. They are two charts of one pencil of rays, and a reader standing at the right distance from either is receiving the same directions. What differs is which properties are legible — straight edges on one, even spending and a wide field on the other.

Which of those a picture should make legible is a question about the picture’s purpose rather than about geometry, and it is the one thing in this essay the measurements cannot settle. A drawing whose subject is a building wants its edges straight; a drawing whose subject is the whole sky wants its marks spread evenly; a texture sampled by a machine wants neither and cares only about the sampling ratio. The reprojection is the operation that moves a picture between those, exactly, and the choice of destination is editorial.

So the honest way to state the whole operation is that a reprojection is a change of what is easy to read, bought with coverage and sampling. That is a genuine thing to want, and it is a much smaller claim than the one implicitly made when a fisheye is “corrected” — a word that suggests the original was wrong.

The viewing distance moves, and the picture does not #

One consequence deserves its own paragraph because it is this collection’s signature and a reprojection is where it is easiest to get wrong.

Every figure here prints the distance it is correct from, and a reprojected picture prints a different one from its source — the flat 100° frame is correct from 7 centimetres at 160 millimetres wide, the 120° frame from 5. Those numbers moved. Nothing about the scene or the eye moved with them.

The reason is that the viewing distance is a property of the picture as an object on a page: it is the focal length scaled to the width the picture is displayed at, and a reprojection changes both the law and the width. A reader standing at the new distance in front of the new picture receives exactly the directions a reader standing at the old distance in front of the old one received — that is what the exactness means, restated in the room.

So a reprojection is a change of where a reader has to stand, and the picture’s correctness is preserved by moving them. Matching buys one seat is the companion statement for curved displays: a picture is right for a place, the place is computable, and changing the picture changes the place rather than breaking anything.

That is worth stating because “the fisheye has been corrected” invites the opposite reading — that the reprojected version is right and the original was wrong. Both are correct pictures, from two different points, and neither is a repair of the other.

Nothing here is a filter, and the word “resampling” hides that #

A last distinction, because the vocabulary works against the geometry.

The operation this essay measures is a map on directions. The operation a program performs is that map plus an interpolation, because the source’s marks sit on a grid and the target’s grid asks for values between them. Those two are habitually named together as “resampling”, and running them together is what makes a reprojection seem lossy in a way the geometry says it is not.

Separating them is worth doing because they fail differently. The geometry is exact everywhere both surfaces reach and undefined outside — a hard boundary, no gradation. The interpolation is approximate everywhere, by an amount set by how much finer the target’s grid is than the source’s, which is exactly the ×1.66-to-×5.49 curve above. One of those is a fact about surfaces and the other is a fact about a program, and only the first is computable from the picture surfaces alone.

The practical form: the resolution curve tells a reader where an interpolator is being asked to invent, and how much. It does not say how well any particular interpolator does it. A picture reprojected with a good filter and one reprojected with a bad one have the same geometry and the same demand curve, and differ entirely in the part this collection does not compute.

That boundary is the same one the third column is area draws between a surface’s area scale and what a sensor does with it. The geometry says what is being asked for; the instrument decides what arrives.

What this does not settle #

Three limits, and the first is the one that would change the numbers most.

Everything here assumes the two surfaces share a centre of projection. That is true of a reprojection performed on one photograph, and it is exactly what fails when a picture is stitched from several exposures taken about the wrong pivot — the case the pivot that is not the eye measures, where there is no single pencil of rays and the reprojection is not a change of coordinates at all.

The resolution costs are quoted as a demand, not as a result. A factor of ×5.49 says the target’s grid is that much finer than the marks arriving; what a reader sees depends entirely on the interpolator, and no amount of interpolation creates information. This collection computes the geometry and the sampling ratio; the filtering is somebody else’s subject.

And the exactness is bit-exact on this pair of surfaces, which is a stronger statement than it needs to be and is partly luck of the arithmetic. Both maps here are radial functions with clean closed-form inverses, so the round trip returns the identical double. A pair whose inverse needs iteration would return something a few ulps away and the claim would be “to the arithmetic floor” rather than “exact” — the same geometric statement, differently housed.

One pencil, two charts #

The object is a set of directions through a point, and the finding is that moving a picture from one surface to another does nothing to it at all.

What it does is discard the directions the new surface cannot reach — 70 per cent of a 180° fisheye’s picture, going to a 100° flat frame — and ask for marks in places the old surface did not put any, by a factor of 5.49 at the frame’s edge. Both prices are computable before the shutter opens, from the two surfaces alone, and neither of them is the loss of geometry that the word reprojection seems to promise.