When the picture surface is not flat

Everything so far has assumed the picture is flat. It usually is: a canvas, a print, a screen, a wall. The flatness is a fact about the medium rather than about the geometry, and relaxing it changes what a picture can do.

What flatness buys #

The flat picture plane has one enormous property: straight lines image to straight lines. That follows immediately from the projection being through a point — a world line and the centre define a plane, and a plane cuts a plane in a line.

Everything architectural depends on it. A building photographed rectilinearly has straight edges; a drawing of one can be made with a straightedge; a construction can be carried out with incidences alone. The whole apparatus of vanishing points exists because families of parallel straight lines image to families of concurrent straight lines, and none of it survives on a curved surface.

The price is the edge stretch. A shape at angle θ off the optical axis is elongated radially by 1/cos θ, which is 3% at 14° off and 41% at 45° off and unbounded as θ approaches 90°. A flat picture cannot contain a half-turn of view at all, because the rays at 90° are parallel to the plane and never meet it.

What a cylinder buys #

Wrap the picture surface into a cylinder about a vertical axis through the eye and the trade reverses.

Horizontally, the surface is now everywhere the same distance from the eye and everywhere perpendicular to the ray. So the horizontal scale is uniform: equal angles occupy equal widths, and there is no horizontal stretch at the edges at all. A cylinder can also hold a full 360°, which no flat surface can.

The cost is that straight lines no longer stay straight. A world line that does not pass through the cylinder’s axis images to a curve — the intersection of the plane through the eye and the line with the cylinder, which is an ellipse in general and appears as a sinusoid when the cylinder is unrolled.

That is why a panoramic photograph bends the horizon into an arc if the camera was not exactly level, and why straight architectural edges bow. It is not an artefact of the stitching; it is what a cylindrical projection does, and the same photograph reprojected onto a plane would have straight edges and unusable corners.

The full menu #

Once the surface is a free choice there is a family of projections, and the same trade-offs appear that cartographic projection has to make when flattening a sphere — for the same reason, since both are trying to represent a curved field of directions on a flat sheet.

Rectilinear (flat plane). Straight lines straight. Unbounded stretch at the edges. Cannot exceed 180°, and is unusable well before it. Used for essentially all ordinary photography and all architecture.

Cylindrical. Vertical lines straight, horizontals bowed. Uniform horizontal scale. Handles a full turn horizontally. Used for panoramas.

Equirectangular (sphere unrolled). Handles the full sphere. Straight lines bow except meridians and the horizon. The standard storage format for 360° imagery, and never viewed directly — a viewer reprojects a rectilinear window out of it.

Stereographic. Preserves angles locally, so small shapes stay the right shape everywhere. Straight lines become circular arcs. Used for the “little planet” images, and the only wide projection that keeps faces round at the frame edge.

Equisolid and equidistant fisheye. Preserve area and radial angle respectively. Both bend straight lines heavily and both hold close to 180°.

Every one of these is a correct projection. None preserves everything, and which properties are given up is the entire design of the projection.

Why no surface does both #

The impossibility is worth stating precisely rather than asserting.

Preserving straight lines requires that the surface meet every plane through the eye in a straight line. The only surfaces with that property are planes. So any non-flat picture surface bends some straight lines, necessarily.

Preserving angular scale uniformly requires the surface to be everywhere perpendicular to the ray and everywhere at the same distance from the eye, which means a sphere. So any non-spherical surface stretches somewhere.

There is a third step the argument needs, and it is the one that makes the menu above a menu rather than a single right answer. The sphere is not a picture anybody can hold. A picture has to end up on a flat sheet — a page, a screen, a canvas — and a sphere cannot be laid flat without stretching, by Gauss’s theorem rather than by any failure of technique. So the surface that would preserve angular scale exactly cannot be delivered, and every entry on the list above is really a flattening of the sphere, paying the flattening’s own irreducible price on top of whatever it gives up as a projection.

That price has a size. A spherical picture reaching $\Theta$ from its centre cannot be flattened with a strain below about $\Theta^{2}/6$: three per cent at 25° from the axis, ten per cent at 45°, forty at 90°. So a wide picture is distorted twice over — once in the choice of surface and once in getting that surface onto paper — and the second term is the one that cannot be traded, because it is the same for every choice made in the first.

A surface cannot be both a plane and a sphere, so no picture surface preserves both straightness and uniform scale. That is not a limitation of current technique; it is a theorem, and it is the same shape of theorem as the one that says a projection cannot preserve both convergence and measure.

Curvilinear perspective as a drawing system #

Artists have arrived at curved picture surfaces independently and repeatedly, usually by drawing what they see rather than by construction.

The observation that starts it: standing in a long corridor and looking straight ahead, the lines where the walls meet the ceiling appear to converge toward a point behind. Turn to look at one of them directly and it appears straight. Both observations are correct, and they are incompatible with a single flat picture plane — because the eye is not photographing a plane, it is scanning a field of directions.

Systems built on that go by the name of curvilinear or five-point perspective, and they amount to drawing on a spherical picture surface with the sphere then flattened by some convention. Hauck in the nineteenth century and Barre and Flocon in the twentieth worked out constructions for it.

Whether such a drawing is “more correct” than a rectilinear one is a question the geometry cannot settle, because the two are answering different questions. Rectilinear is correct as a projection onto a plane, exactly, and looks right from one point. Curvilinear is correct as a record of angular directions and looks right nowhere in particular — but matches what a viewer with a moving eye reports, which is a fact about scanning rather than about projection.

What the eye actually does #

Worth being careful here, because this is where geometry stops and something else begins.

The retina is approximately spherical, which invites the conclusion that human vision is a spherical projection and rectilinear pictures are therefore wrong. That conclusion does not follow.

The eye has about one degree of sharp vision at the centre and moves constantly — several fixations per second — building a scene representation from many samples. What is perceived is not a projection of any kind onto any surface; it is a reconstruction, informed by memory, expectation and the knowledge that walls are flat and heads are round.

That is why viewers accept rectilinear photographs of wide scenes with mild complaint, accept panoramas with bowed horizons with mild complaint, and accept neither as what it looked like. The projection question and the perception question are separate, and answering the first does not answer the second.

This site computes the first and says so wherever the second is in play.

Choosing a surface, in practice #

For anyone making a picture, the decision reduces to a few cases.

Under about 70° across: use a flat plane. The edge stretch is under 20%, straight lines are straight, and every other option costs more than it saves.

70° to 120°, with architecture in it: still flat, and compose so that nothing shape-critical goes near a corner. The alternative bends the building.

70° to 120°, with people in it: consider cylindrical or stereographic. Faces stay round, and the bowing of straight lines is less objectionable than stretched heads.

Beyond 120°: flat is not available in any useful sense. Choose by what has to survive — cylindrical for a landscape with a level horizon, stereographic for shapes, equirectangular if the image is going to be reprojected later anyway.

The one thing worth avoiding is treating the choice as a matter of which looks least distorted. Every option is undistorted in its own terms and distorted in every other’s, and the useful question is which property the picture cannot afford to lose.

Stitching, and what a panorama really is #

A panorama assembled from several frames is not a photograph of anything. It is a set of measurements of the field of directions, reprojected onto a chosen surface.

Each source frame is a rectilinear projection from the same centre in a different direction. Because they share a centre, any two overlapping frames are related by a homography — the same eight-parameter map that appears in anamorphosis and in façade rectification — and that is what stitching software solves for.

Once the frames are registered, the combined data is a function on the sphere of directions, and the output image is that function sampled onto whatever surface was chosen. Which is why panoramic software offers a projection menu: the data does not have a projection, and one has to be picked.

The requirement that the frames share a centre is the reason panoramas taken by pivoting a hand-held camera fail near objects. Pivoting about the wrist moves the entrance pupil, so the frames are not related by a homography — they are two different viewpoints, with genuine parallax — and no single warp registers them. Panoramic heads exist to rotate the camera about its entrance pupil for exactly this reason.

Why cartography is the same subject #

Flattening a field of directions onto a sheet is the same problem as flattening a sphere onto a sheet, and the two literatures reached the same conclusions independently.

A map projection cannot preserve both angle and area — that is Gauss’s Theorema Egregium in its practical form, and it is why Mercator is conformal and enormously wrong about area while equal-area projections distort shape. A picture projection cannot preserve both straightness and uniform scale, for the same underlying reason: a curved field cannot be laid flat without stretching something.

The correspondences are close enough to be useful. The rectilinear projection is the gnomonic map projection, which sends great circles to straight lines and blows up at 90° from its centre. The stereographic picture projection is the stereographic map projection, conformal in both. The equirectangular storage format for 360° imagery is the plate carrée.

So a photographer choosing between rectilinear, cylindrical and stereographic is making the same decision a cartographer makes between gnomonic, Mercator and stereographic, with the same trade-offs and the same absence of a right answer.

The one surface that is not a compromise #

There is a picture surface that preserves everything, and it is worth naming because it explains what the compromises are compromising about.

A spherical picture surface centred on the eye preserves angular relationships perfectly. Every direction maps to a point on the sphere, all scales are uniform, and nothing is stretched anywhere.

It is not a picture, because it is not flat. It cannot be printed, hung, or looked at as a surface — the viewer would have to be inside it, which is what a planetarium dome and a head-mounted display arrange.

That is the honest statement of the whole problem: the perfect picture surface exists and is unusable, and every practical projection is an attempt to get most of its virtues onto something flat. Flattening it is where the loss happens, and the choice of projection is the choice of which loss to take.

Which also explains why virtual-reality displays feel different from pictures rather than like very good pictures. They are not projections onto a flat surface at all; they present the field of directions directly, and the whole family of compromises this essay is about simply does not arise.

The cost in machinery #

Worth stating plainly, because it explains why flat picture planes dominate despite the edge stretch.

On a flat plane, straight lines stay straight, so a family of parallels is concurrent, so vanishing points exist, so a camera can be recovered from a picture, so cross-ratios can be checked, so measuring points work. Every construction and every check on this site depends on that first property.

On a curved surface, none of it survives. There are no vanishing points, no concurrency to test, no recovery, and no straightedge construction of any kind. A cylindrical panorama can be computed and cannot be constructed, and it cannot be audited by any of the methods that work on a flat picture.

That asymmetry is the practical reason flat is the default even where it is geometrically awkward. It is not only that flat surfaces are what paper and screens are; it is that flatness is what makes the geometry tractable, and the moment it is given up the subject becomes a matter of computation with no constructions and no checks available.

Curved screens, and what they are for #

The most-encountered curved picture surface today is a cinema screen, and its curvature is doing exactly what this essay describes.

A flat screen viewed from a seat off to one side presents its far edge at a steep angle, so that part of the picture is foreshortened for that viewer and its correct viewing distance is wrong by a large factor. Curving the screen toward the audience reduces the variation: every part of the screen faces more nearly toward the seating, and the range of effective viewing geometries across the auditorium narrows.

That is a different motivation from the panoramic one. A panoramic projection curves the picture surface used to make the image; a curved screen curves the surface the image is presented on, while the image itself is usually a rectilinear projection. The two can be combined — projecting a cylindrical projection onto a cylindrical screen restores the geometry exactly for a viewer at the axis — and large-format cinema formats do precisely that.

The same reasoning drives dome projection and head-mounted displays, and it ends at the limit this essay named: the ideal picture surface is a sphere centred on the eye, which cannot be printed and can be occupied. Every practical curved screen is an approach toward that surface from the flat end, and how far it goes is a compromise with the fact that an audience is not one person at one point.

Which is the same compromise an ordinary picture makes with its own correct viewpoint, scaled up to a room.

What this became #

This essay was the foundation phase’s one look at a curved picture surface, and it described a comparison rather than measuring one: a plane beside a cylinder, with the observation that neither is the distorted one and no number behind it.

The expansion phase turned that into a field. Six surfaces are now computed as maps from a direction to a mark — plane, cylinder, stereographic, equidistant and equal-area fisheye, equirectangular — and each is measured on the three things a picture surface can do to the world. How far it bends a straight line, as a fraction of that line’s own chord. How far it is from preserving shape, as the worst departure of a right angle and the ratio of the two magnifications. And how its area scale runs across the field.

Two of the resulting numbers are exact zeros and the rest are curves that grow. Stereographic preserves every angle and magnifies both directions equally, to the noise floor of the measurement. The equal-area fisheye holds its area scale at 1.0000 across the whole field. Nothing preserves straightness except the plane, and nothing is both straight and conformal — the corner of the plot where such a surface would sit is empty, and Beltrami’s theorem says it has to be.

The claim in this essay is therefore now a measurement with a table behind it rather than an even-handed remark.