A box leaves a line of cameras

The pixel that is not square handed a recovery a picture from a camera with two focal lengths and watched it come back with one. A 2-to-1 anamorphic returned a focal length 49.6 per cent out, a principal point far from the truth, and no complaint from any of the checks. That essay’s limit was stated in careful words: a single view of a box determines three camera parameters and cannot determine four. And it left it there, because its interest lay in how silently the recovery fails.

A limit of that kind has a shape, and that shape is worth drawing. If three equations bind four unknowns, then what a picture of a box leaves is not a wrong answer but a curve of answers: one for each value of the fourth unknown, every one of them exact. A recovery that assumes square pixels reports one point on the curve. The whole curve is what the picture holds. Measuring it answers questions that a single wrong point cannot: how far the other members lie from the truth, whether any of them can be ruled out on their own terms, and how much each kind of extra knowledge is worth in closing the curve.

Three right angles in a squeezed page #

The classical recovery rests on an elementary fact. Three mutually perpendicular directions in the world vanish at three points on the page, and for a camera with square pixels the principal point is the triangle’s orthocentre, the point where its three altitudes meet. Recovering the camera measured how well that works.

Now let each pixel be $a$ units wide for each unit of height. Divide every column by $a$ and the page becomes the page of a square-pixelled camera, with its vanishing points at $(x/a, y)$. Everything the classical recovery says holds on that squeezed page. So for any assumed $a$ there is a squeezed triangle, an orthocentre, and a focal length from the same right-angle equations. Then multiply the orthocentre’s column back by $a$ and the result is a camera with that width, that focal length and that principal point, which is exact.

That track is the picture’s own statement of what it knows. Take a camera with $f = 854$ px, pixels 1.33 wide, principal point at (345, 210), photographing a box. Its vanishing points are on the page at the positions they are at. Assume a width of 1.33 and the recovery returns the camera exactly, as it must. Assume 1.1 and it returns a centre at (217, 407) and a focal length of 985. Assume 1 and it returns (169, 539) and 1,039, the same report the square-pixel recovery gives. Assume 1.7 and it returns (601, 44) and 658. Every member satisfies all three right-angle conditions to the rounding of the arithmetic: they are all right.

The centre’s journey deserves a look. Over the widths a real lens might have, it travels from below the frame’s lower left to near its upper right, covering most of the frame’s width and more than its whole height. The true centre is one point on that path, and nothing about the path marks it out.

What the rest of the camera does along the line #

Those three numbers vary together, and as the width grows the focal length falls and the centre’s column grows, in the way the page’s geometry dictates.

Between a width of 1 and 1.7 the focal length falls by 37%, the column more than triples and the row falls to a twelfth. A person who has decided, on grounds unrelated to the picture, that the principal point is in the middle has done a great deal of work with that decision, and the figure makes that visible: a centred principal point selects one vertical slice.

It is worth reading the same figure as an account of the failure the earlier essay measured. A 2-to-1 anamorphic read as square lands 49.6 per cent short on its focal length. This figure’s camera is a 1.33 and reads 21.6 per cent long at a width of 1. The signs differ, because it is the box’s turn that sets where along the family the camera lies and how the family runs. The size of the error depends on the view as much as on the lens, and a single figure quoted for a given squeeze is a statement about one view. So the failure measured at 2 to 1 should be read as an example and not as a rate.

The family has ends #

A curve of exact answers looks like a free parameter, as though a picture could be read at any width. The arithmetic says otherwise, and that is the first new thing the family gives.

A focal length is real only if the orthocentre lies inside the triangle, which requires the triangle to be acute. Squeezing a page along its columns changes the triangle’s angles, and a sufficiently hard squeeze will make one of them obtuse. At that point the equation for the focal length’s square yields a negative number. No camera with that width can have drawn these vanishing points, at any focal length, whatever the principal point. So each picture of a box comes with an interval: the widths its triangle allows.

For the 1.33 camera here, the interval runs from 0.59 to 2.52. A claim that this lens has pixels 3 wide can be refuted on the spot, from this picture alone, using nothing but the right angles of a box. That is a different kind of thing from a measurement of the width: it is the picture rejecting a whole range of cameras without being able to name the one that took it.

How large the interval is depends entirely on the view, and the figure makes that plain. A view at yaw 15° and pitch 45° bounds a true width of 1 between 0.76 and 1.79, which is a useful result. Others have no upper limit at all up to a width of twenty, because squeezing their triangle keeps every angle acute at any strength the sweep tried. A view nothing excludes is possible, and it costs nothing to compute which is in hand. So a practitioner’s first check on a suspected anamorphic is available for the price of an orthocentre and six comparisons: for the widths the supplier claims, does the triangle stay acute? This check can only refute, never confirm. It is a necessary condition rather than a sufficient one, and it has the rare merit of needing no extra knowledge about the scene.

What the internal check says #

The recovery’s standard internal check is the agreement of its three focal lengths. Each pair of vanishing points gives its own estimate of the focal length, and a large disagreement should warn that something is wrong. The pixel that is not square found this silent at its wrong answer, and it is worth saying now why it had to be.

The three estimates $\sqrt{-(v_i - c)\cdot(v_j - c)}$ are equal exactly when $c$ is the orthocentre: subtract any two of the expressions and the difference is an altitude’s equation. So the check is a theorem about the orthocentre, not a test of the camera. It compares a quantity with itself. For exact points it must report zero at all widths, as the figure shows. For noisy points it reports some value, but that value concerns the noise and says nothing whatever about the width. Other checks on a recovery that are really consequences of its definition can be found in the same way, and the question to put to each is whether a wrong assumption could change its answer. For this one the answer is a clean no, since every member of the line passes.

The centre closes it, at a price #

If a true principal point is known, the family closes at once. Its track crosses the known column once. That is one equation in the width, and the picture gives the width.

The closure is exact: assume a column of 345 and the width comes back as 1.33 to eight digits, with a focal length of 854 and a row of 210. The row is a by-product, which makes it a check, since an assumed row was never used. So with an assumption about the column, a box does read the squeeze. Yet the price is plain in the slope. A column 20 px too far right reads 1.363 and 20 px too far left reads 1.297, or 0.0017 per pixel. That is mild, on its face. The difficulty is that a lens can be shifted or a picture cropped by a great deal more than 20 px with no visible trace, and that a fitted camera’s centre is often the least well-known thing about it. A cropped anamorphic frame, for instance, has a principal point nowhere near the frame’s centre, and a recovery that assumes otherwise reads a squeeze that is really the crop.

A mirror-image trap sits inside this. A practitioner who assumes a centred lens and a square pixel at the same time is assuming two things, against a picture that can only supply three of four numbers. It is a consistent, overdetermined system, and the leftover residual looks like a check on both. It is then tempting to read a small residual as a confirmation. In fact it confirms only the combination, and a slight error in one can be compensated by a slight error in the other, all along the line. The residual is the distance of that assumed point from the line, and nothing else. In the figure above, that is the difference between the assumed row and the row the family reports at that column. The arc every eye stands on is the same structure with the width held and the eye free: one more unknown than the equations, and a curve of exact answers.

What a picture from a shifted anamorphic looks like #

The most useful single observation may be how ordinary the members look. Any member of the family draws the box’s three vanishing points exactly, with all its right angles preserved. Draw a whole scene with one of those cameras and look at it: the camera on the line differs from the true one in focal length and in width and in where the frame’s centre falls, and the box itself is unchanged. The wrong member gets every right angle of a box exactly and gets other things wrong, such as the shape of a circle or the ratio of two lengths on a face. Those are what the box does not constrain.

In other words, the three right angles of a box and a pixel’s width are close to independent as evidence. A picture is under-determined precisely in the direction that alters what is not a right angle, and it stays exactly determined in every other. That shows up in practice: the same picture, read to the wrong width, is often perfectly useful for what it was taken for, such as finding walls and aligning a model, because the wrong camera has the right map on the faces of the box. It is much less useful for reading off a true aspect from what was photographed, because the aspect is what has been traded away, which is the point at which it starts to matter.

The size of what a reader can claim #

Because every member is exact, none of this depends on how well the box was read. But the noise matters in one place. Each member of the family is built from three vanishing points that a real picture knows only approximately. With a few long edges read to half a pixel, a vanishing point near the frame is good to a few pixels, and one a long way off can be good only to hundreds. The track then has a thickness. The family becomes a band, and a width that the exact points allowed may fall outside it, or one they ruled out may be accepted. Those bounds at 0.59 and 2.52 will wobble from picture to picture.

The ends of the family are the most fragile part of it. An orthocentre near a triangle’s edge changes fast with the positions of the points, so that an end measured from noisy points is noisy, and the check above should not be used near a boundary with any confidence. Well inside the interval, or well outside it, the verdict holds up. No band of the track is measured here, because nothing in the argument needed it. A square closes the line of cameras supplies the reading directly for the number that counts.

Why not simply fit four parameters #

Most calibration routines allow two focal lengths and fit them as a matter of course, so why does this family not appear to their users? Because they photograph a flat target from many positions, and each position gives two equations. Three views of a plane is six equations for four unknowns. The family is a fact about a single view of a box, and a routine fed many views never meets it. One picture of a box is the arrangement that creates it, and what a single view cannot give has always been stated for that arrangement. It is common wherever the camera is not available for calibration: any archive photograph, a film still, a rendering whose settings have been lost, or any picture a recovery is applied to after the fact. Those are the pictures most often read with a square pixel assumed. For them the honest output is the line itself, and reporting a single camera says more than the picture knows. An angle is a cross-ratio puts the underlying right-angle condition in another form, and it runs into the same count.

What a practitioner might do with the line is limited but real. Report the interval the triangle allows. Report what follows for the column and the focal length at each width of interest. If a lens’s known squeeze ratios are in a short list, as they generally are, compute the family’s camera at each of them and compare with what else is known. That is a modest result to deliver. It is nevertheless more honest than one exact-looking wrong number, and it can be delivered in a line.

Skew, a leaning wall and a breathing lens all leave the family #

The family assumes no skew. Allowing one would make it a two-parameter surface, and it was noted in the earlier essay that an anamorphic whose axis is not aligned with the rows produces exactly a skew.

It assumes the box’s three directions really are at right angles, with no building that is out of true. A wall leaning a degree, of the kind a plumb wall tells from a horizon, does not sit on the family, and a recovery that absorbs it moves along the line by some amount that no single picture can detect. The size of that effect is a separate measurement.

And it assumes a pure lens: the bowing of a wide lens or an anamorphic’s own breathing, which changes the squeeze with the focus, would be extra distinct effects, and neither is treated here.

Still open: which single extra fact closes it best #

The centre closes the family, but the centre is the thing least likely to be known. Other extra facts could do the same job without any reference to the camera’s centre, all from the scene: a square on one face, since a square’s diagonals are at right angles and two directions at 45° to a right angle are a new equation; a circle on a face, whose circular points supply the same kind of equation; or a second view of the same box, which has the same inner camera and three more right angles.

The measurement that settles it reads a true squeeze from each of them, with a stated number of edges read to half a pixel. It asks how the width comes back: which source is best, at what level of reading precision, and how the result depends on the box’s turn. It also asks what happens when the supplied fact is wrong, as when a rectangle is taken for a square, since an extra fact that has been silently assumed is the same kind of thing as a centred lens that has been silently assumed.

A picture that cannot say which camera took it has still said a great deal. Three right angles leave a line, the line has ends, and an honest reading reports the line.