One shutter, two views

A mirror is a second camera settles what a reflection is: reflect the scene and photograph it, or reflect the camera and photograph the scene, and the two routes agree to the last bit once one image axis is reversed. That essay is about the map. This one is about the photograph — because a picture with a mirror in it contains two views of the same objects, taken at the same instant, and everything the collection knows about pairs of views applies to it.

The applying turns out not to be routine. A mirror pair is a two-view problem with one constraint added, and the constraint is worth more than it looks.

Two views, one exposure #

Set a camera in front of a mirror, put some objects between the two, and open the shutter once. Every object that can be seen in the mirror appears in the picture twice: directly, and again as a reflection. Those two appearances are a correspondence in exactly the sense a point is a line over there uses the word — two image points that are pictures of one world point.

The second view’s camera is the real one reflected in the mirror plane. It sits behind the glass, it looks back out, and it is not a camera any factory on this site will build, because a reflection reverses handedness and camera() constructs a right-handed basis. Nothing here builds one. The second view is produced by reflecting the scene and photographing it with the real camera, which is what the real camera does anyway.

The fundamental matrix is skew #

Fit a fundamental matrix to those correspondences with the eight-point algorithm, giving it nothing but the pairs of marks. What comes back satisfies $\mathbf{F} = -\mathbf{F}^{\mathsf{T}}$ to within 2 × 10⁻¹² of its own size.

A general fundamental matrix has no symmetry at all. Its nine entries are defined up to a scale and constrained to have vanishing determinant, which leaves seven free numbers, and eight correspondences are what it takes to pin them down. A skew-symmetric three-by-three has three entries and the other six are copies with their signs turned round; up to scale that is two numbers, and the rank condition comes for nothing, since every odd-sized skew matrix is singular by construction.

The control is the half worth having. Skewness here is a ratio of norms, and a routine that returned a small number for everything would pass the mirror test in silence. Photographing the same marks from a camera moved half a metre sideways and running the identical fit gives 2.00 — which is to say the matrix and its transpose share nothing, which is what an ordinary pair looks like.

Both epipoles are the same point #

An epipole is the image of the other camera’s centre. In a two-view pair there are two of them and they are generally in different places on the two pictures. Here both pictures are the same picture, and both epipoles are the same point, because $\mathbf{F}\mathbf{e} = 0$ and $\mathbf{F}^{\mathsf{T}}\mathbf{e} = -\mathbf{F}\mathbf{e}$ have the same solution.

That is a genuine collapse rather than a coincidence of notation, and it is the same kind of collapse perpendicular is a pairing records for an involution: a map that is its own inverse has fewer distinct outputs than a map that is not, and the count of them is the useful statement.

That point has a name in the room rather than in the algebra. The other camera’s centre is the real centre reflected in the mirror, so the epipole is the place in the photograph where the camera would see its own lens. A photographer standing square to a mirror is looking at the epipole of the pair being taken.

It is also the vanishing point of the mirror’s normal direction, in the sense where parallel lines meet fixed the term, and that is not a second fact. The line from the eye to the reflected eye runs perpendicular to the mirror, so the image of the reflected eye lies along that direction from the principal point — which is what a vanishing point is. When the mirror is a horizontal water surface the normal is vertical and the epipole is the nadir, which is where the second half of this row goes.

Where the other five numbers went #

Seven free numbers become two, and it is worth saying which two survive and what became of the other five, because the accounting names exactly what a mirror pair can and cannot report.

A general fundamental matrix carries the relative pose of two cameras up to scale: three numbers for the rotation between them and two for the direction of the baseline, which is seven once the calibration is folded in. A mirror pair has no free rotation — the second camera is the first reflected, and a reflection in a plane is completely determined by that plane’s normal. And the baseline’s direction is that same normal.

So the two numbers the skew matrix carries are the mirror’s normal direction, and nothing else. Two numbers name a direction; a direction names a reflection; a reflection names the second camera. The five that vanished were never free.

Three consequences follow, and the third is the one that decides what a photograph is worth.

The mirror’s distance is invisible. It sets the baseline’s length, and a fundamental matrix never carries a length — which is why the recovery here returns a direction and needs a tape measure for the rest, exactly as the closing section says.

And so is the mirror’s position and extent. Two photographs taken in two different mirrors hung parallel to each other, anywhere in the room, at any size, have the same fundamental matrix. The matrix is a fact about an orientation and about nothing else in the arrangement.

Which makes the epipole a compass rather than a landmark. It reports which way the glass faces, in the camera’s own frame, to whatever precision the marks allow — and reports it from a photograph in which the mirror’s edges may be entirely out of frame. A reflection in a shop window, half seen, with no visible frame and no known geometry, still names its own normal, and the whole of that statement is two numbers read off a point.

Every joining line goes through it #

The consequence a reader can use needs no arithmetic at all. Write the epipolar constraint for a skew matrix and it collapses:

$$\mathbf{x}'^{\mathsf{T}}[\mathbf{a}]_{\times}\mathbf{x} = \mathbf{a}\cdot(\mathbf{x}\times\mathbf{x}') = 0$$

The cross product of two homogeneous image points is the line joining them. So the condition on a correspondence is that the epipole lies on the line through the mark and its reflection — and since that holds for every correspondence at once, all of those lines meet at one point.

Measured over sixteen correspondences in the arrangement above, the worst line misses the epipole by 1.4 × 10⁻¹² pixels. That is arithmetic rather than a fit: no least squares is run anywhere in the statement, and the number is what floating point leaves behind.

This is a straightedge construction in the sense the polar with a straightedge means it. Two marks, a ruler, and the epipole is on the paper. Nothing is measured, no length and no angle enters, and the whole of it therefore survives being carried out on a photograph rather than on the scene — which is the property what a projection destroys spends its length establishing.

Why the epipole is where the lens is #

The identification is worth slowing down over, because it is the one statement in this essay that a reader can check by standing in a room rather than by trusting a fit.

An epipole is where one camera’s centre appears in the other camera’s picture. The second camera here is the first one reflected, so the epipole is the image of a point that sits behind the glass, exactly as far behind it as the camera is in front. A photographer looking into a mirror sees a camera there. That camera is the second view’s centre, and the point in the photograph where its lens appears is the epipole.

The same point read as a direction gives the other name. The segment from the real centre to the reflected centre is perpendicular to the mirror, so it runs along the mirror’s normal; the image of a point along a direction from the eye is the vanishing point of that direction; therefore the epipole is the vanishing point of the mirror’s normal. The two descriptions are one fact seen from either end, and the figure above measures them against each other rather than deducing one from the other.

There is a pleasing consequence for a mirror on a wall. A vertical mirror’s normal is horizontal, so the epipole is on the horizon; a mirror lying flat — a puddle, a polished floor, still water — has a vertical normal, and the epipole is the nadir. That second case is the one the landscape manuals have been drawing for two centuries without naming it, and it is where the refraction half of this row picks the thread up.

What the pair gives, and what it does not #

A two-view reconstruction from two photographs returns a shape and no size, which is the result two views give shape and no size measures. A mirror pair inherits that exactly, and the qualifier has an unusually concrete name.

The two centres are the camera and its reflection, so the baseline runs along the mirror’s normal and its length is twice the distance from the camera to the glass. Scale the whole arrangement — room, mirror, objects, camera — by any factor and not a pixel of the photograph moves, which is the same statement the one thing a single view cannot give makes about a single picture. What is different is that the free parameter is a distance somebody can walk over and measure with a tape.

That makes the closure available here a length, in the sense the metrology field uses the word. It is worth saying which length: not a length in the scene, which is what a scale bar supplies, but a length between the camera and a surface — the same closure as the camera’s own height, arrived at sideways.

Triangulating across the glass #

With the mirror’s plane known, the triangulation is a two-view triangulation whose second pose is exact rather than recovered. The ray to the direct image is a ray of the real camera. The ray to the reflected image is the same camera’s ray with the light’s history undone: it left the object, met the mirror, and arrived, so reflecting that ray in the plane gives a ray from the reflected centre through the object.

Two rays, one intersection, and the gap between them — the skew distance two rays that do not meet is named after — is 4.3 × 10⁻¹⁵ metres at worst over sixteen marks — which says the arrangement is consistent rather than that the answer is good, and the difference between those two statements is the subject of the third rung of this ladder.

Why the constraint is not a curiosity #

It is tempting to file “the fundamental matrix is skew” as an elegance. It is not, and the reason is the one this collection returns to whenever a count of unknowns falls: a constraint is worth an amount of evidence, and evidence is the thing in short supply on a real photograph.

Seven unknowns need eight correspondences and produce an answer whose error is spread over eight directions. Two unknowns need two, and every further correspondence is a check rather than a requirement. On marks read to four tenths of a pixel — which is careful work on a print — the constrained fit puts the epipole about five times closer to the truth than the unconstrained one given the same marks. That comparison and its sweep are the next rung.

There is a second reason, and it is about what a reader can verify. Almost nothing in two-view geometry can be checked by eye. An epipolar line drawn on a print is a claim about a matrix somebody else fitted. The concurrence of the joining lines is not: it is visible, it is a ruler away, and a photograph in which those lines do not meet at a point is a photograph whose mirror is not flat.

The boundary, stated #

Everything above assumes the mirror is a plane, that the correspondences are correct, and that the reflection is of the object rather than of some other object that happens to look like it. Each of those fails in a recognisable way.

A curved mirror has no reflected centre at all — over twenty centimetres of a two-metre ball the lines of sight miss their own best-fitting point by 2.8 millimetres — so the joining lines have no common point and the fit reports a matrix that is not skew. A wrong match is a wrong match here as everywhere, and a wrong match is not a small error applies without modification. And a mirror seen at a glancing angle reflects a narrow band of the room, which is a limit on how many correspondences exist rather than on their quality.

A fourth limit is quieter, and it is about the photograph rather than the geometry. A mirror in an ordinary room is a rectangle a metre or so across, and the correspondences it produces are confined to whatever part of the scene it happens to reflect. That is a constraint on the arrangement of the evidence rather than on its quality, and it bites the way any narrow bundle bites: the marks are clustered, the design matrix is worse conditioned than the count of correspondences suggests, and a seventeenth mark inside the same small region buys very little. The complaint appears whenever a fit is handed data with no spread in it, and the ladder of assumptions is a ladder of conditioning is where the collection sets it out generally.

One more limit deserves its own sentence, because it is the limit that decides whether the arrangement is any use. The baseline runs along the mirror’s normal, so a mirror facing the camera squarely puts the second centre directly behind the first — which is the forward-motion degeneracy, and it is why the third essay of this ladder is about turning the mirror.

What is measured here #

Four numbers, and the fourth is the one that gives the other three their scale.

The fitted fundamental matrix is skew to 2.1 × 10⁻¹² of its own size, and a genuine second view scores 2.00 on the same instrument. The epipole comes back the same to 4.1 × 10⁻¹¹ pixels by five routes that share no arithmetic — two null spaces, an image of a world point, a vanishing point of a direction, and two lines met with a ruler. Every joining line passes through it to 1.4 × 10⁻¹² pixels. And the same construction run on a genuine pair of photographs gives lines whose pairwise intersections are scattered over eight hundred and twenty pixels, which is what says the first three numbers are a measurement rather than a property of the drawing.

The short version #

A photograph containing a mirror is a stereo pair taken in one exposure. Its fundamental matrix is skew-symmetric, which reduces seven unknowns to two; both of its epipoles are the same point; that point is where the camera sees its own lens, and equivalently the vanishing point of the mirror’s normal; and the line joining any mark to its reflection passes through it, to the arithmetic floor.

The pair gives a shape and no size, as every pair does. What it gives that an ordinary pair does not is a free parameter with a physical name — twice the distance from the camera to the glass — and a construction a reader can carry out on a print with a straightedge and two marks.