The principal point is not the centre

The principal point is where the optical axis meets the sensor. It is one of the four intrinsic parameters of a camera, it appears in every formula on this site that has a camera in it, and almost every practical treatment of the subject fills it in with the middle of the picture and moves on.

That is right often enough to be tempting and wrong often enough to matter. This essay is about the two cases where it is wrong, what the geometry does in them, and what the assumption costs.

The problem the shift lens solves #

Photograph a tall building from across the street. A level camera puts the horizon halfway up the frame, and the building’s top goes off the top edge.

The obvious fix is to tilt the camera up. That works, and it changes the geometry: the vertical direction is no longer parallel to the picture plane, so it acquires a finite vanishing point, and the building’s sides converge toward it. The picture is a correct three-point perspective, and the building leans back.

The professional fix is not to tilt. It is to keep the camera level and move the lens upward relative to the sensor — a shift lens, or a view camera’s front rise. The picture plane stays vertical, so the vertical direction stays parallel to it and its vanishing point stays at infinity, and the building’s sides stay parallel.

What has changed is that the optical axis no longer meets the sensor at the middle of it. The principal point has moved.

Two things that are exactly true #

The figure measures both halves of that, because the second is the counter-intuitive one.

Verticals stay parallel through a shift. The spread of the four verticals’ angles is 0e+0° before the shift and 0e+0° after it, and 4.55° for the tilted camera that frames the same view. Zero, not nearly zero: the picture plane’s orientation has not changed, so a direction parallel to it is still parallel to it.

Every point moves by exactly the shift. Not approximately, and not by an amount that depends on where it is. Shifting the principal point by 95 px translates the whole picture by 95 px, because the projection formula is $u = c_x + fX/Z$, and $c_x$ enters additively.

The second one is why a shift is such a clean operation. It is a translation of the image, which is the least destructive thing that can be done to a picture — it preserves every angle, every length, every ratio, every vanishing point and every cross-ratio. Nothing about the picture’s geometry changes except which part of the projected world the sensor happens to be looking at.

That is a satisfying way to state what a view camera’s rise does: it slides the sensor around inside a much larger picture the lens is already forming, without moving the camera or the picture.

And one thing that is not #

The shift has a cost and it is not a geometric one.

The lens has to project a circle large enough to cover the sensor in its shifted position, which means it has to cover a much larger circle than a non-shifting lens of the same focal length. That is why shift lenses are large, expensive, and available only in a few focal lengths, and why the shift range is limited — run out of image circle and the corners go dark.

The same limit explains the cheap version of the trick, which is to photograph level with a wider lens and crop the bottom off. Geometrically that is identical to a shift: the lens forms the same picture, and a crop chooses a different part of it. The cost is resolution rather than money.

And the cropped case is why this essay matters to anyone who has never held a shift lens. Every crop that is not centred moves the principal point off the middle of the frame, and crops are universal. A picture that has been cropped to a different aspect ratio, straightened, or had its top trimmed has a principal point somewhere other than its centre, and nothing about the file says where.

The rectification that people reach for instead #

There is a software route to the same result and it is worth comparing, because it is what almost everybody actually does.

Photograph the building tilted, then apply a homography that sends the vertical vanishing point back to infinity. The verticals become parallel; the picture looks like a shifted one; nothing has been bought or carried.

Geometrically this is exact. The map that does it is a projective transformation of the picture plane, and a projective map of a projection is a projection — the corrected picture is a perfectly good perspective picture of the same scene from the same point, taken on a differently oriented picture plane.

The cost is in pixels rather than in geometry, and it is severe in a specific place. Sending the vertical vanishing point to infinity means stretching the part of the frame nearest to it, which is the top — where the building’s top was, and where the least information is, because that part of the building was imaged smallest. So the correction is exact in geometry and lossy exactly where the subject needed it most.

That trade is why architectural photographers still own shift lenses in an era when the software correction is one click. The click is free and it spends resolution at the top of the frame; the lens costs money and spends none.

There is a second, quieter cost. The rectified picture’s principal point is not the middle of the rectified frame either — the homography moved it, along with everything else — so the rectified file has exactly the problem this essay is about, and now nothing at all records where the principal point went.

What assuming the centre costs #

Now the measurement.

The elementary route to a focal length uses two vanishing points of perpendicular horizontal directions:

$$f^2 = -(\mathbf{v}_1 - \mathbf{p})\cdot(\mathbf{v}_2 - \mathbf{p})$$

and it needs a principal point $\mathbf{p}$ supplied from outside. Every textbook supplies the middle of the frame.

The reason the error grows so quickly is visible in the formula. The right-hand side is a product of two vectors from the principal point to the two vanishing points, and vanishing points are typically far from the frame — hundreds or thousands of pixels out. An error of 95 px in $\mathbf{p}$ therefore perturbs a product of two large numbers, and the perturbation is first-order in the error while the quantities are large.

At 150 px of shift — a fifth of the frame’s width, which is an ordinary crop — the recovered focal length is 1.6% out. That propagates directly into every viewing distance, every field of view, and every metric measurement made from the picture.

Reading the size of the effect #

The curve rises steeply and it is worth having a feel for where on it a real picture sits.

A 3:2 frame cropped to 16:9 by taking the middle band leaves the principal point where it was, because the crop is symmetric. Cropped to 16:9 by taking the top band — which is what happens when a horizon is being placed — moves it by about an eighth of the frame height. Straightening a picture by a couple of degrees and cropping to the largest inscribed rectangle moves it by a smaller and quite unpredictable amount, in a direction that depends on which way the picture was rotated.

A shift lens at full extension moves it by around a fifth of the frame, which is the right-hand end of the figure’s sweep and the 1.6% quoted.

So the ordinary case is not the extreme one. Most cropped pictures have their principal point a few per cent of the frame off centre, and the focal-length error that follows is a few tenths of a per cent — below the level anyone would notice and above the level at which a claim about which lens was used can be settled.

The slider on the figure changes the lens rather than the shift, and it is worth watching because the answer is not intuitive. A longer lens is hurt more by a given shift in pixels, because its vanishing points sit further out and the product of the two distances is larger. Wide lenses, which are the ones people actually shift, are the more forgiving case.

The route that does not assume it #

This site’s own recovery does not have this error, and the reason is worth stating because it is a genuine argument for the more elaborate method.

With three mutually orthogonal directions there are three relations of the form above, one per pair, and they over-determine the principal point rather than requiring it. Subtracting the pair that share a vanishing point shows that $\mathbf{p}$ must lie on the altitude of the vanishing-point triangle from that vertex, and doing it three times gives three altitudes, which meet at the orthocentre.

So the recovery computes the principal point as the orthocentre of the triangle of the three vanishing points, and then computes the focal length from it. Nothing is assumed. The figure confirms that the honest route returns the true focal length exactly when the picture is shifted, at any shift.

The price of the three-point route is that it needs three finite vanishing points, which means the picture must contain something with three mutually perpendicular edge directions and the camera must be tilted enough that the vertical’s vanishing point is finite. A level picture of a building has only two, and the third relation is unavailable.

Which is an awkward pairing: the shifted picture, where assuming the centre is most costly, is exactly the picture whose vertical vanishing point is at infinity and whose principal point therefore cannot be recovered this way.

What to do in the awkward case #

The honest options, in order of preference.

Read the shift off the image circle. On a shifted photograph the vignetting is asymmetric — darker on the side the lens moved away from — and its centre is the principal point. That is a photometric measurement rather than a geometric one, and it is what a raw converter’s lens profile uses.

Use a third direction that is not vertical. Any three mutually perpendicular world directions will do. A rectangular object in the scene sitting at an angle to the building supplies them; so does a road meeting a wall at a corner.

Or fit the principal point along with everything else, on multiple views, which is what a full calibration does — and accept that it will be weakly determined, for the same reason the distortion coefficients are: moving the principal point and adjusting the other parameters compensates almost exactly.

And in every case, quote the assumption. A focal length recovered from two vanishing points and an assumed centre is a number with a condition attached, and the condition is “provided this frame was not shifted or cropped”. That sentence costs nothing to write and it is the difference between a measurement and a number.

Why the site’s figures put the principal point in the middle #

A fair objection: every camera on this site puts its principal point at the centre of its canvas unless something asks otherwise.

That is a convention rather than an assumption, and the distinction is that the convention is known to be true here — the figures are generated from cameras whose principal points are declared, so nothing is being assumed about them. It matters that the machinery permits any principal point, which it does, because the moment it did not the shift figure could not be drawn.

The real assumption on this site is elsewhere and it is worth naming while the subject is up. Every viewing-distance claim depends on the width the figure is actually displayed at, which the site cannot measure, and every figure quotes 160 mm as its assumed display width with the arithmetic given so a reader can redo it. That is the same discipline as this essay’s last recommendation, applied to the one number this site cannot know.

Where else the principal point hides #

It is worth listing the places on this site where the principal point is doing real work, because the list is longer than it looks and every entry inherits this essay’s caution.

The viewing distance. The distance a picture is correct from is the focal length scaled to the displayed width, so a focal length 1.6% out is a viewing distance 1.6% out. That one is forgiving — nobody stands to the nearest millimetre.

The field of view. Same arithmetic, same error, and it is the number a photographer would recognise, so it is the one most likely to be checked against a claim about which lens was used.

Every height and every length. Single-view metrology uses the horizon, which is the join of two vanishing points, and the vertical vanishing point. Neither depends on the principal point — which is a small mercy — but the focal length does, and any measurement converting an angle into a length uses it.

The centre of the radial distortion, which is the subject of the section below.

And the anamorph. The printable sheet is the site’s one unconditional viewing-distance claim, and it is unconditional precisely because it fixes the display width by printing in millimetres. Its correctness still depends on the principal point being where the construction says, which on a generated figure it is by declaration.

Five uses, one parameter, and a convention that is right for uncropped frames from centred lenses and for nothing else.

The connection to distortion #

One more consequence, because it ties this essay to the field’s first one.

Radial distortion is radial about the principal point, not about the middle of the frame. So a distortion correction applied with the wrong centre does not correct the picture; it applies a slightly wrong radial map about a slightly wrong point, and the result has a residual that looks like decentring — like the tangential terms $p_1$ and $p_2$ were needed.

That is a genuine source of confusion in practice. A fit that frees the tangential terms on a cropped picture will find them non-zero, and they will be describing the crop rather than the lens. Free the principal point instead and the tangential terms go back to nearly nothing.

Two parameters that mean quite different things and can absorb each other’s effects: the same disease as $k_1$ and $k_2$, one level up. The cure is the same too — solve for them together, and report the conditioning rather than the point estimate.