A symmetric object is its own stereo pair

One shutter, two views turns on one fact and nothing else: a mark and its reflection are related by a reflection in a plane, and a pair of image points related that way has a skew-symmetric fundamental matrix, coincident epipoles, and joining lines that all pass through one point.

Nothing in that argument mentions glass. It is a statement about a plane and a reflection in it, and there is a very large class of objects that supply both without any mirror being present.

The mirror an object brings with it #

A bilaterally symmetric object is one whose points come in pairs, each the reflection of the other in a single plane. A face, a chair, a car, a violin, a lodge at the end of a drive, a moth, most of the fronts of most buildings. Photograph one, and the left half of the picture and the right half are two views of the same half-object, taken from two places — because a point and its twin are related by exactly the relation a mirror produces.

The second view is not hypothetical or approximate. The twin of a point is its reflection, so the pair of image points satisfies the same skew-symmetric relation, and everything read off a mirror pair can be read off a symmetric object.

Measured on a small gabled building with fourteen paired points, photographed once from three-quarters on, the fitted matrix is skew to 2.0 × 10⁻¹², the joining lines meet at one point to 1.9 × 10⁻¹² pixels, and the point is the vanishing point of the symmetry plane’s own normal.

Where the plane is, and where it is not #

There is one difference from a mirror, and it is the interesting one.

A mirror hangs somewhere, and the photographer can walk up to it with a tape. A symmetry plane has no surface, cannot be measured, and in many pictures is not even visible — the plane of a car’s symmetry passes through the middle of the car. So the plane’s distance is unavailable in a way the glass’s is not.

Its direction is not, and that is what makes the reading work. The epipole is the vanishing point of the plane’s normal, so marking the point where the joining lines meet and reading it as a direction gives the normal exactly. Measured here, the recovered normal is out by nought degrees to the limit of the arithmetic.

So one photograph of a symmetric object supplies the plane’s direction and leaves its distance free, and the free distance is a single number. That is the same shape of answer two views give shape and no size reaches by a different route, and it has the same consequence: the shape comes back and the size does not.

The shape comes back, and it comes back exactly #

Set the plane’s distance to any convenient value, unfold every pair — the ray to a point, and the ray to its twin reflected in the assumed plane — and the intersection is the point, at whatever scale the assumption implies.

Done that way on the fourteen pairs, every ratio of distances in the recovered object matches the true one to 2.6 × 10⁻¹⁴. Changing the assumed distance changes the recovered object’s size and nothing else, which is the statement that the ambiguity is one scale rather than a deformation.

The strength of the result deserves a plain statement, because it sounds too good. A single photograph of a symmetric object determines its three-dimensional shape completely, with no assumption about the object beyond its symmetry, no second exposure, and no calibration object in the scene. The camera’s focal length is needed — it is what turns the epipole into a direction — and recovering the camera is the standing account of how much a single view supplies toward that.

One length, still missing #

The missing scale is not a technicality and it is the same missing scale the whole subject keeps meeting.

The one thing a single view cannot give states it in its most general form: multiply the world by any factor and move the eye by the same factor, and the picture is unchanged. Symmetry does not touch that argument. A model of a house and a house draw the same photograph, and both are perfectly symmetric.

What symmetry does is remove the other ambiguity. Without it, one photograph of an unknown object leaves the shape free as well as the size — a whole one-parameter family of depths for every point. With it, the shape is fixed and one number remains. Five facts that close the same gap is the catalogue of what closes that last number, and every entry in it applies here unchanged: a measured length anywhere on the object will do.

The test that has to come first #

Everything above assumes the object is symmetric. Real objects are not, quite, and the reading has to be able to say so.

The instrument is the meeting point. Move the second half of the object off the exact reflection by half a per cent — a centimetre in two metres, which is builder’s tolerance on a house front — and the joining lines stop meeting: the best common point is missed by 12.7 pixels, in a frame 690 across. Two per cent opens it to 46.5 pixels, four per cent to 86.2.

The recovered shape degrades much more gently over the same range: 2.4 per cent of error at half a per cent of asymmetry, 8.3 at two per cent, 12.7 at four. So the test on the print is several times more sensitive than the damage it is warning about, which is the right way round for a test to be.

The matrix test behaves quite differently and is worth separating out. The fitted matrix’s departure from skew goes from 2.0 × 10⁻¹² at exact symmetry to 1.95 at half a per cent — it is all-or-nothing, because an unconstrained fit to eight-or-more correspondences will happily return a general matrix and a general matrix is not nearly skew. So the matrix says whether the object is symmetric and the concurrence says how nearly, and only the second is a measurement.

Why the matrix has two numbers in it rather than seven #

The skewness is not a curiosity of the fit and the reason is short enough to carry.

In a frame where the camera’s calibration has been divided out, a point and its reflection have image directions $\mathbf{a}$ and $\mathbf{b}$, and the reflection sends the world point $\mathbf{X}$ to $\mathbf{X} - 2(\mathbf{n}\cdot\mathbf{X} - d)\mathbf{n}$. The displacement between the two is along $\mathbf{n}$ whatever the point is, so the eye, the point and its twin are always coplanar with $\mathbf{n}$ — which says $\mathbf{a}$, $\mathbf{b}$ and $\mathbf{n}$ are linearly dependent, that is, $\mathbf{b}^{\mathsf{T}}[\mathbf{n}]_{\times}\mathbf{a} = 0$.

A cross-product matrix is skew by construction, and it carries a direction rather than a general matrix: two numbers after scale, against a general fundamental matrix’s seven. The epipole is $\mathbf{n}$ read as a point of the picture, both ways round, which is why the two epipoles coincide.

So a symmetric object’s pair is not a lucky special case of the general two-view geometry. It is the smallest two-view geometry there is, and every pair of twins is one linear condition on a two-parameter unknown. Two pairs determine it, a third is already a test, and fourteen are an over-determination by twelve — which is why the residuals above sit at the arithmetic floor rather than at a fitting error.

The symmetry plane draws a line, and the line is visible #

The plane itself images as a line, and that line is worth finding because it is where the whole construction becomes a straightedge operation.

Every point on the symmetry plane is its own twin, so its joining line is a point rather than a line and it contributes nothing. But the image of the plane is still a definite line of the picture, and it passes through the epipole — the plane contains the normal’s direction only in the degenerate sense, but it contains every direction perpendicular to the normal, and the line joining any two of its points is the image of a line in the plane. A ridge, a central door, a bonnet seam: any two of them fix the line.

What that gives is a check that costs nothing. The line of self-twinned features and the meeting point of the joining lines are two constructions with no arithmetic in common, and on a symmetric object the first passes through the second. On an object whose symmetry has been broken they come apart, and they come apart in a way that says which features are asymmetric rather than only that something is.

It also settles a question the previous section leaves. A point on the plane is a degenerate correspondence, but it is not useless: it is a point known to lie on a known plane, which for the reconstruction is one constraint rather than none. Once the plane’s distance is assumed, such a point’s depth follows immediately from its ray, without any twin at all.

The reading, as a person with a print would do it #

Nothing above needs a computer, and setting the recipe out plainly is the test of whether the geometry has been understood.

Mark two pairs of twinned features — the two ends of a gutter, the two outer corners of a facade. Draw the line joining each pair and continue both until they cross. That crossing is the epipole, and everything else follows from it: a third pair’s joining line must pass through it, which is the check; the direction from the lens through that point is the symmetry plane’s normal; and any feature whose twin is wanted lies on the line joining it to the crossing.

The one thing a straightedge does not give is the reconstruction, which needs the focal length and a little arithmetic. But the part that fails — whether the object is symmetric enough for any of this — is entirely a straightedge matter, and it is settled before a single number is computed.

Which point pairs with which #

The reading needs correspondences, and finding them is the part a person does without noticing and a machine finds hard.

The constraint that helps is the same one two matches are enough exploits. Every joining line passes through the epipole, so once two pairs are known the epipole is known, and from then on a point’s twin lies on the line joining it to that point. The search drops from the whole picture to one line, which is the ordinary epipolar reduction arriving from an unusual direction.

It also says where the reading is worst. A point near the epipole has a short joining line and an ill-determined direction, so its twin is hard to place; and a point on the symmetry plane is its own twin, contributing a degenerate correspondence with no information in it at all. The ridge of a roof, the keystone of an arch, the badge on a car’s grille — all of them are on the plane, all of them look like excellent marks, and none of them is a correspondence.

When the camera is on the plane #

There is one arrangement in which all of this collapses, and it is the arrangement most photographs of symmetric objects are actually taken in.

Stand square on — lens on the object’s own symmetry plane — and the two virtual views coincide with the real one. The baseline is twice the camera’s distance to the plane, which is zero; the epipole runs off to infinity; the joining lines become parallel; and the picture is symmetric about a vertical line with nothing to triangulate. A perfectly symmetric photograph of a perfectly symmetric object contains no depth information whatever.

That is the same degeneracy square to the camera is the worst mirror measures in its own terms, taken to its limit, and the quantity to watch is the same one: where the meeting point falls. A three-quarter view puts it a few hundred pixels off the frame and the reading is strong; a nearly-frontal view carries it thousands of pixels away and the reading weakens with it.

The practical rule follows and it is the opposite of what an architectural photographer is taught. For a picture that records a symmetric facade, stand square on. For a picture that measures one, stand well round to the side.

What symmetry is worth against a second photograph #

A second photograph is the obvious alternative and it is worth saying plainly where each wins.

A genuine pair of exposures has a baseline the photographer chooses, so the geometry can be made as strong as the scene allows. A symmetric object’s baseline is twice the distance from the lens to its symmetry plane, in the direction of the plane’s normal — the photographer chooses it by standing somewhere, and that is the whole of the choice, exactly as hanging a mirror is.

What symmetry has instead is that it costs nothing and cannot go stale. There is no second exposure to register, no camera motion to solve for, no risk that the scene moved between frames. And it applies to photographs already taken, by people who were not measuring anything — which is the case that matters for a building since demolished or a car in an archive.

It also composes. Two mirrors are three cameras counts what a photograph containing two mirrors holds; a symmetric object photographed in a mirror holds the same thing with one of the two planes invisible, and the same five-number accounting applies with one of the distances unmeasurable rather than merely unmeasured.

What this does not settle #

The object here is a set of paired points, and a real photograph is not.

Finding the pairs is the whole practical difficulty and nothing above measures it. The epipolar reduction narrows the search to a line, but only after two pairs are known, and the first two have to come from somewhere. On a facade with repeated windows the wrong pairing is a consistent one — every window has several plausible twins — and a wrong match of that kind is the failure a wrong match is not a small error measures, which is that least squares spreads it across every other point rather than isolating it.

The asymmetry model is a stretch of one half, which is one kind of departure among many. A door hung on one side, ivy on one wall, a dented wing: these are local rather than global, and a local break should be detectable as a few pairs missing a meeting point the rest agree on. That is a robust-fitting question and it has not been asked here.

And nothing is measured about noise. Every number above is arithmetic on exact marks. Half a per cent of asymmetry shows as 12.7 pixels, which is far above any plausible marking error, but the boundary between an asymmetry a photograph can see and one it cannot has not been drawn.

Still open: how much symmetry an object has to have #

The sweep above breaks a symmetry that was exact. The question underneath it is the opposite one, and it is the question a reader with a real photograph actually has: given an object that is approximately symmetric, how much of it has to be symmetric before the reading is worth making.

A measurement of it would take one object and vary the fraction of its points that are paired rather than the size of the break — a facade with a symmetric frame and an asymmetric middle, a car with a symmetric body and one wing mirror — fitting the meeting point to the paired points only, and asking how few pairs are needed before the recovered shape of the whole object, paired and unpaired alike, is worth more than nothing. The unpaired points are not lost: once the plane is known, every unpaired point still lies on a known ray, and its depth is the one thing missing. So the honest question is how the recovered shape’s error falls as the paired fraction rises, and whether there is a fraction below which the plane is so ill-determined that the paired points do worse than no assumption at all.

The short version #

A bilaterally symmetric object supplies a reflection in a plane, which is the whole content of a mirror pair. One photograph of a small symmetric building gives fourteen correspondences whose joining lines meet at a point to 1.9 × 10⁻¹² pixels, a matrix skew to 2.0 × 10⁻¹², the symmetry plane’s normal exactly, and every ratio of distances in the object to 2.6 × 10⁻¹⁴.

The size is absent, as it is from any single photograph and from any pair of them. And the reading has a prior question — whether the object is symmetric at all — which the same meeting point answers: half a per cent of departure opens it to 12.7 pixels, against 2.4 per cent of damage to the recovered shape.