Recovering the camera from the picture it drew

A projection that only ever runs forwards proves nothing about itself. Feed it a scene, get a picture; the picture looks like a picture; there is no way to tell from it whether the machinery did what it claimed.

So the machinery here runs backwards as well. A box is drawn from a camera with a known focal length. The camera is then set aside, and a second piece of code — which has never been shown the scene or the camera, and cannot be, because it lives in a file that imports neither — is handed the twelve drawn line segments and asked what camera produced them.

It gets the focal length back to about one part in 10¹⁵, and the principal point to two trillionths of a pixel.

The relation everything rests on #

The recovery uses one fact, and it is short enough to derive.

A camera maps a world direction d to a picture point. For intrinsics consisting of a focal length f and a principal point p, the vanishing point of direction d sits at p + f · (rotated d) scaled by its depth. Undo that: given a vanishing point v, the direction it came from is proportional to ((v − p)/f, 1) in camera coordinates.

Now take two world directions that are perpendicular. Their dot product is zero, so

$$\frac{(\mathbf{v}_1 - \mathbf{p})\cdot(\mathbf{v}_2 - \mathbf{p})}{f^2} + 1 = 0$$

which rearranges to

$$f^2 = -(\mathbf{v}_1 - \mathbf{p})\cdot(\mathbf{v}_2 - \mathbf{p})$$

That is the whole relation. Two orthogonal directions, their two vanishing points, the principal point, and the focal length: any three determine the fourth.

Two features of it are worth pausing on. The right-hand side must be negative for a focal length to exist, which means the principal point has to lie in a particular region relative to the two vanishing points; when it does not, no camera produces that picture from a rectangular object, and the recovery refuses rather than returning the square root of a negative number. And the relation says nothing about the scene’s size, position or distance — only about directions. A matchbox and a shipping container photographed with the same lens at the same orientation give the same vanishing points.

Why the orthocentre #

With three mutually orthogonal directions there are three such relations, one per pair, and they over-determine the principal point. The solution has a pleasant closed form: the principal point is the orthocentre of the triangle formed by the three vanishing points.

The reason is not obvious from the algebra and is short in geometry. Take the pair v₁, v₂. The relation says that the vector from p to v₁ and the vector from p to v₂ have a fixed negative dot product — and subtracting the two relations that share v₃ shows that p must lie on the line through v₃ perpendicular to v₁v₂. That line is the altitude from v₃. Doing the same for the other two pairs gives the other two altitudes, and the three altitudes of a triangle meet at the orthocentre.

So the classical fact that the altitudes are concurrent is doing real work here: it is what makes the three relations consistent, and their consistency is not automatic. Three vanishing points that do not come from three orthogonal directions still form a triangle and still have an orthocentre, but the three focal lengths derived from the three pairs will not agree.

That disagreement is the useful output. The recovery returns all three focal lengths and their spread, and the spread is a measurement of how far the drawn object is from being a rectangular box.

Where the vanishing points come from #

The recovery has to be given three vanishing points, and where they come from decides whether the exercise means anything.

Taking them from the camera would make the round trip an identity: the camera’s own numbers would go out and come back, and any error in the projection would cancel against itself. That is the shape of test that reports a perfect score and measures nothing, and this fleet has been caught by it before.

So they come from the drawn edges. Each bundle of four parallel edges is turned into four image lines, and their common point is found by least squares, with the residual reported. On a correctly projected box the residual runs at 10⁻¹³ px.

That residual is the first of the two numbers this figure produces, and it is the one that generalises. Given any drawing, it says whether the edges that are supposed to share a vanishing point actually do. A drawing where they do not has no vanishing point to feed the recovery, and reporting the least-squares meet of a set of lines that do not meet would be inventing a number.

What the round trip proves and what it does not #

The agreement is to 6 × 10⁻¹⁶ relative on the focal length, 2 × 10⁻¹² px on the principal point, and 1.3 × 10⁻¹⁶ across the three independent focal estimates. Those are good numbers and it is worth being precise about their meaning.

They prove that the forward projection and the backward recovery are consistent, which is a real thing to have proved. A sign error in the projection’s handling of the principal point, a transposed rotation, or a field-of-view-versus-focal-length confusion would all break the agreement, and each of those is an error that leaves the picture looking exactly like a picture.

They do not prove that the projection is a correct model of a camera. Two consistent implementations of the same wrong idea agree perfectly. That is why the site carries other checks that come at the geometry from a different direction — the vanishing point that must be a limit of marched points, at first order, the cross-ratio that must survive, the measuring-point construction that must land on the projected divisions.

And they say nothing about a photograph. A real lens has radial distortion, which bends straight lines; the recovery assumes straight lines stay straight, so on an uncorrected wide-angle frame it returns a focal length that is wrong by more than the distortion looks. Correcting distortion first is standard and is a separate problem.

The same machinery, used on a picture nobody calibrated #

The recovery becomes interesting when it is pointed at a drawing whose camera was never specified — which is every drawing made by hand.

A perspective drawing has three bundles of edges if it depicts a box. Those bundles have vanishing points, the vanishing points have a triangle, the triangle has an orthocentre, and the orthocentre and the three focal estimates say what camera the drawing implies. Four things can come out.

The three focal estimates agree. The drawing is a projection of a rectangular box, and the focal length says which lens.

They disagree. The drawing is a projection of something, but not of anything with three mutually perpendicular edge directions. The spread measures how far off.

The dot product comes out positive. No camera produces this picture from a rectangular object at all.

The bundles do not meet. The drawing is not a projection of anything with parallel edges, and the residuals say by how much.

Applied to the two-point cube construction that every book teaches, the answer is the third-most-interesting of those: the drawing is a perfectly good projection, of a box that is not a cube, and which box depends on a step the method leaves to judgement.

Single-view metrology #

The recovery is one instance of a larger practice: getting measurements out of one photograph, with no scene survey and no calibration target.

Once the intrinsics are known, a good deal follows. The horizon of the ground plane is fixed by two horizontal vanishing points. A vertical vanishing point plus a known height anywhere in the scene gives the height of anything else standing on the ground, from a cross-ratio along the vertical. Distances along the ground follow from the same construction applied in plan. The rectifying homography that flattens a photographed façade into an elevation is determined by the façade’s own vanishing points.

The practice has real uses — forensic reconstruction from a single frame, architectural survey from photographs, recovering the geometry of a demolished building from a photograph of it — and one persistent limit. Everything above needs at least one known length to convert the ratios into metres. A photograph fixes shape and never fixes scale, because scale is one of the things a projection destroys and no amount of cleverness recovers it from within the picture.

The reason this is the site’s central check #

Most of the checks here are internal: a residual small, a tolerance bracketed, two formulas agreeing. The round trip is different in kind, because the two halves are separated by an interface narrow enough to inspect.

lib/projective.js imports the drawing helpers and nothing else. It has no access to the camera, no access to the scene, and no way to obtain either. Everything it knows about a picture arrives as a list of image points. That separation is enforced by the module structure rather than by discipline, and it is what makes fifteen digits of agreement worth quoting rather than merely reassuring.

It is also the pattern this fleet keeps arriving at from different subjects. A crystallography site generates a pattern, forgets which group made it, rediscovers the group and compares. A mechanism site counts mobility by formula and measures it from a Jacobian’s rank. Here a camera draws a picture and the picture gives back the camera. In each case the value is not in the forward computation but in the fact that a second, differently-shaped computation had to arrive at the same answer with the first one’s workings hidden.

What it costs #

Very little, which is the last point worth making.

The forward projection is a dot product and a divide. The recovery is three least-squares fits of four lines each, an orthocentre, and three square roots. The whole round trip on a box runs in well under a millisecond, and it runs at build time on every figure that uses it, so a change to the camera that breaks the geometry stops the site from building rather than producing a picture that is wrong in a way nobody can see.

That is the argument for doing it at all. The failure mode being guarded against is not a crash. It is a figure that looks entirely convincing and depicts a scene no camera could photograph, which is the ordinary condition of illustrations in this subject and is invisible without a computation of exactly this kind.

The vertical vanishing point, and why photographs of buildings lean #

One consequence of the recovery deserves its own section, because it explains something everyone has seen and few can name.

A camera held level has its optical axis horizontal. The vertical direction is then parallel to the picture plane, its vanishing point is at infinity, and vertical world edges stay vertical in the picture. A building photographed that way has parallel sides.

Tilt the camera up to fit the top of the building in and the vertical direction is no longer parallel to the picture plane. Its vanishing point comes in from infinity, and the building’s sides converge toward it. The building leans back.

Nothing is wrong with the photograph — it is a correct projection, and the recovery returns three consistent focal lengths from it. What has changed is that the drawing now has three finite vanishing points instead of two, which is precisely what the one-, two-, three-point vocabulary is counting.

The professional fix is to keep the camera level and shift the lens upward instead, which moves the principal point off the centre of the frame without tilting the picture plane. Verticals stay vertical because the picture plane stays vertical. That is what a shift lens does and what a view camera’s front rise does, and it is the reason architectural photography owns specialised equipment for a problem that sounds like it should be solved in software.

It can be solved in software, at a cost the recovery makes explicit. Rectifying a tilted photograph means applying a homography that sends the vertical vanishing point back to infinity, and that homography stretches the top of the frame — which is where the least information is, because the building’s top was the smallest part of the image. The correction is exact in geometry and lossy in pixels.

Reading a photograph for its focal length #

A last practical use, and one that needs nothing but the picture.

Given a photograph containing anything rectangular — a building, a room, a box on a table — the three edge bundles give three vanishing points, and the relation above gives the focal length in pixels. Dividing by the image width gives the field of view, which is the number a photographer would recognise.

This works on photographs whose metadata has been stripped, on scans of prints, on frames from film. It is how the focal length of a historical photograph is established when no record of the camera survives, and it is how a claim that a particular lens was used can be checked.

It has one systematic weakness worth stating alongside it: the accuracy depends on how far the vanishing points are from the principal point, and a nearly-frontal view of a box puts one of them almost at infinity, where its position is poorly determined by a short bundle of nearly-parallel edges. The residuals reported by the fit say when this is happening, which is the reason they are reported at all.

The weakness has a rate, and it is worth knowing which way it runs. Writing the relation as $f^{2} = |\mathbf{v}_1 - \mathbf{p}|\,|\mathbf{v}_2 - \mathbf{p}|\,|\cos\psi|$, a relative error in $f$ is half the relative error in either distance — so a vanishing point twice as far away tolerates twice the absolute error in pixels. That sounds like a distant vanishing point is the safe one, and it is the opposite. A bundle of edges of drawn length $L$ locates a point at distance $D$ to about $\varepsilon D^{2}/(Ls)$, where $s$ is the spread across the bundle and $\varepsilon$ the error on a marked endpoint: the intersection angle falls as $1/D$ and the lever arm grows as $D$, so the absolute uncertainty grows as the square of the distance while the tolerance grows only linearly. The net is a relative error in $f$ that grows in proportion to $D$. Frontal is worse, and it gets worse at a stated rate rather than merely eventually.

Recovering the scene instead of the camera #

This essay recovers a camera from a picture it drew. The expansion phase asks the harder and much older question: what can be recovered about the scene?

The answer is unusually clean. Everything except one number. A photograph of a scene is a photograph of every scene similar to it and of nothing else — scale the world and the eye together by any factor and not a pixel of the picture moves, which the site now demonstrates rather than asserts, at 137× and a disagreement of a few parts in 10¹⁴ of a pixel.

So a single view supplies every ratio and no size. A height comes out of four points on a vertical and one cross-ratio, in units of the photographer’s eye height; a length on a plane comes out of a homography, in units of a reference rectangle’s width. One length imported from outside the picture turns every ratio into a measurement.

The relationship to the round trip here is direct. This essay’s recovery is exact because the picture determines the camera up to nothing at all — three vanishing points fix the focal length and the principal point outright. The scene’s recovery is exact up to one scalar, and the scalar is the one quantity a projection through a centre cannot encode.