One conic calibrates the camera

The foundation phase of this site recovers a camera from the picture it drew. Three mutually perpendicular vanishing points are read off the drawn edges of a box, the orthocentre of the triangle they form is the principal point, and the focal length falls out of a construction involving that triangle’s altitudes. It returns the focal length the camera was built with to one part in $10^{15}$.

That construction is correct and it is unmotivated. It works, the derivation can be followed, and nothing in it says why an orthocentre should have anything to do with a lens.

There is a second route with no construction in it at all, and following it explains the first.

The conic #

The plane at infinity carries a conic on it — an imaginary one, with no real points — called the absolute conic. Its defining property is the one that makes it useful: every rigid motion of space fixes it.

That is a strong statement. Move the camera anywhere, point it anywhere, and the absolute conic is unmoved, because it is a property of space’s metric structure rather than of anything in space. So its image in a photograph depends on the camera’s internals and on nothing else — not on where the camera is, not on where it points, not on the scene.

That invariance is the whole reason calibration is possible. A quantity depending only on the internals can be measured from any picture, and pictures of different scenes from different places all measure the same thing.

Written out, the image of the absolute conic is

$$\omega = (\mathbf{K}\mathbf{K}^{\mathsf T})^{-1}$$

where $\mathbf{K}$ is the calibration matrix. For the pinhole this site builds — square pixels, no skew, the principal point where the camera says it is — that comes out as a circle of radius $f$ centred at the principal point, which is why it can be drawn.

The condition #

Two directions in the world are perpendicular. Their vanishing points are the images of their points at infinity, so both lie on the image of the plane at infinity, and their perpendicularity is a statement about the absolute conic: two perpendicular directions are conjugate with respect to it.

Conjugacy carries through the projection unchanged, so in the picture

$$\mathbf{v}^{\mathsf T}\,\omega\,\mathbf{u} = 0$$

for the vanishing points of any two perpendicular directions. One equation, one unknown when the principal point is known, and it solves in a line:

$$f^2 = -(\mathbf{v} - \mathbf{p})\cdot(\mathbf{u} - \mathbf{p})$$

No construction, no altitudes, no orthocentre. The focal length is the square root of minus the dot product of the two vanishing points about the principal point.

The minus sign is the whole content #

That expression has a sign in it and the sign is not decoration.

$f^2$ is positive, so the dot product must be negative: the two vanishing points have to lie on opposite sides of the principal point in the direction joining them. If they do not, no focal length makes those two image directions perpendicular, and the equation returns an imaginary $f$.

An imaginary focal length is a refusal, not a small error. A camera recovered from a pair that fails the test is not a slightly wrong camera; it is a statement that the pair cannot be the images of perpendicular directions under any pinhole with that principal point. The machinery refuses rather than taking the square root of a negative number and returning its magnitude, and the site’s gate checks both halves — a pair that straddles is accepted, a pair that does not is refused.

That refusal is the thing the orthocentre construction expresses awkwardly. The altitudes of the vanishing triangle meet inside the triangle for an acute triangle and outside it for an obtuse one, and the case distinction the construction has to make is this sign.

What conjugacy means, concretely #

The word conjugate is doing the work in the condition and it is worth unpacking, because it turns the algebra into a picture.

A conic sets up a correspondence between points and lines: the polar of a point is the line of points harmonically conjugate to it with respect to the conic, and this site met that correspondence two phases ago when the pole of the horizon turned out to be the image of a circle’s centre, at 2.5e-13 px.

Two points are conjugate when each lies on the other’s polar. So the condition $\mathbf{v}^{\mathsf T}\omega\mathbf{u} = 0$ says: the vanishing point of one direction lies on the polar of the vanishing point of the other, with respect to $\omega$.

That is a drawable statement. Given $\omega$ and one vanishing point, its polar is a line in the picture, and the vanishing point of every direction perpendicular to the first lies on it. The polar of $\mathbf{v}$ is the vanishing line of the plane perpendicular to $\mathbf{v}$'s direction — which is a fact about the picture that can be constructed with a straightedge once the conic is drawn.

For a square-pixel camera and a vanishing point at distance $d$ from the principal point, the polar is a line perpendicular to that direction at distance $f^2/d$ on the far side. So the horizon of the ground plane and the vertical vanishing point are pole and polar of each other, which is a relation between two things this site draws in almost every figure and had never connected.

The two agree #

The site’s own recovery, written four phases and one field before any of this, is compared against the conic route directly.

At the camera drawn above, the orthocentre construction returns 812.7691 px and $\mathbf{v}^{\mathsf T}\omega\mathbf{u} = 0$ returns 812.7691 px. The relative difference is under $10^{-15}$, which is the arithmetic floor, and the recovered camera’s own $\omega$ puts the pair in conjugate position with a residual of 5.9e-10 in focal-length units.

Two derivations written far apart, agreeing to the last bit, is the evidence that the conic is a re-description rather than a second method. Had they been two methods there would be a difference somewhere in the tenth digit — a different linearisation, a different ordering of operations — and there is not.

What the conic buys that the construction does not #

If it were only a tidier derivation it would be worth a paragraph rather than a rung. It is not.

A calibrated camera supplies every plane’s circular points at once.

The circular points of a plane are where that plane meets the absolute conic. So their images are where the plane’s vanishing line meets $\omega$ — one line-conic intersection per plane, needing no circle in that plane, no known angle in it, and no further information from the picture at all.

Calibrate once, from any three perpendicular directions anywhere in the picture, and every plane in it becomes metric. The ground, each wall, the roof, a tilted table: each has a vanishing line, and each vanishing line cut by $\omega$ gives that plane’s two points.

Checked directly: taking $\omega$ from the camera’s own focal length, intersecting it with the ground plane’s horizon, and running Laguerre’s formula on two ground lines returns 90.000000000° for a right angle, with a residue of zero, and no circle anywhere in the construction.

The counting, and why three directions is the right number #

$\omega$ is a symmetric 3×3 matrix up to scale: five degrees of freedom, matching a general calibration matrix’s five. Each pair of perpendicular directions gives one linear equation in its entries.

So five independent perpendicularity constraints determine a general camera, and fewer determine a constrained one. The usual constraints and their costs:

• zero skew removes one, leaving four;
• square pixels removes another, leaving three — $f$, $p_x$, $p_y$;
• the principal point at the centre removes two more, leaving one, which is $f$ alone.

Three mutually perpendicular directions give three pairs, hence three equations, which is exactly enough for the square-pixel case and is what the orthocentre construction uses. That is why a box works and a single rectangle does not: a box has three directions and a rectangle in one plane has two.

The accounting also says what a fourth direction would buy — nothing, unless it is not perpendicular to the others, in which case it buys a residual. And a residual is what turns a recovery into a measurement, which is the difference between a calibration that returns a number and one that returns a number with something to say about it.

Where the principal point comes in #

The focal formula above assumes the principal point is known, and this site has a whole essay on the fact that it usually is not — the principal point is not the centre of the image, and assuming it is costs a measurable amount.

The conic form says exactly what the assumption is doing. $\omega$ for a square-pixel camera has three unknowns: $f$, $p_x$, $p_y$. Each pair of perpendicular directions gives one linear equation in the entries of $\omega$. So three mutually perpendicular directions give three equations and three unknowns, and the system determines all three — which is what the orthocentre construction is doing when it uses the whole triangle rather than one pair.

Assuming the principal point is the image centre spends two of those equations on nothing and leaves one over-determined. The cost of the assumption is then measurable as the residual of the equations it did not need to make, and that is a considerably more informative thing to report than a focal length with no error bar.

Why it is a circle here and not in general #

The figure draws $\omega$ as a circle, and that is a property of the camera rather than of the conic.

For a camera with square pixels and no skew, $\mathbf{K}\mathbf{K}^{\mathsf T}$ is $f^2$ times the identity in its top-left block, so $\omega$ is a circle of radius $f$ about the principal point. Give the camera a non-square pixel aspect and $\omega$ becomes an ellipse whose axis ratio is the pixel aspect; give it skew and the ellipse tilts.

So the shape of $\omega$ is the camera’s internal geometry, drawn in the picture. A photograph with a known $\omega$ is a photograph carrying its own calibration on it, and the number of degrees of freedom in a conic — five — is exactly the number in a general $\mathbf{K}$.

That correspondence is the reason the object is worth having rather than the formula. Five numbers in a matrix nobody can picture become one conic that can be drawn on the photograph, and the constraints turn from algebra into incidences.

What it does not do #

Two limits.

It needs perpendicular directions in the scene, or something equivalent. The conic is a fact about the camera and the picture only measures it where the scene provides a constraint. A photograph of a featureless plane constrains nothing.

It says nothing about lens distortion. $\omega$ is the image of the absolute conic under a pinhole, and a real lens is not one. Every claim here survives with the distortion removed and none of it survives with the distortion present, which is what the lens field’s fitting essays are for.

One object, four fields #

It is worth listing where this conic has already been on this site under other names, because the list is the argument for the object.

The foundations field’s focal recovery is the conjugacy condition. The foundations field’s circular points are $\omega$ intersected with a plane’s vanishing line. The metrology field’s rectification is the metric structure those points supply. The sensor field’s pixel aspect is $\omega$ being an ellipse rather than a circle. And the manyviews field’s calibration is $\omega$ estimated from more constraints than it needs.

Five constructions, written in five phases, each with its own derivation and its own figure, and one object underneath. That is the same shape of finding as the previous phase’s — three thin rows turning out to be one subject at three group levels — and it arrives the same way: by asking what the constructions are rather than what they compute.

The practical dividend is the one already stated and it is worth repeating in this company. Any one of the five determines $\omega$; $\omega$ determines all five. So a picture with a single calibrated camera has every plane in it metric, every angle in every plane readable as a cross-ratio, and every rectification available without a further reference — and none of that is visible from any of the five constructions taken alone.

The short version #

The absolute conic is fixed by every rigid motion, so its image depends on the camera’s internals and on nothing else. For a square-pixel camera that image is a circle of radius $f$ about the principal point.

Two vanishing points of perpendicular directions must be conjugate with respect to it. Solving that gives $f^2 = -(\mathbf{v} - \mathbf{p})\cdot(\mathbf{u} - \mathbf{p})$, whose minus sign is a refusal rather than a convention, and whose answer agrees with this site’s orthocentre construction to the last bit.

And it buys something the construction never offered: the circular points of every plane in the picture, as the intersection of that plane’s vanishing line with the conic. Calibrate once and every plane becomes metric, with no circle in any of them.