A wedge of glass turns the camera behind it

What survives a pane of glass is one of the cleanest results in this collection, and it is clean because it is exactly two-sided. A plane-parallel slab moves every point of a picture sideways and moves no direction at all: a ray leaves the second face exactly parallel to the way it met the first. So a photograph taken through a shop window has the vanishing points it would have had with no glass there, and the camera recovered from them is exactly the camera that took it — out of a picture in which nothing is where it was.

A pane gives a product before it gives two numbers turned the question round and asked what the displacement says about the pane, and found the thickness and index arriving multiplied together.

Both results lean on one word, parallel. The two faces of a pane are made to be parallel, and the exactness of the first result is the exactness of that geometry. This essay takes the smallest departure from it — two flat faces a couple of degrees apart, a thin wedge — and follows it through the same round trip: what the glass does to rays, what it does to the frame, and what the camera recovered through it turns out to be.

A wedge turns every ray, and not by one amount #

The method is the slab’s with one change. A camera looks through two flat glass faces; the first is perpendicular to the camera’s axis and the second is tilted by $\alpha$ about a horizontal line, so the glass thickens toward the bottom of the frame. Each ray from the camera meets the first face, bends into the glass by the vector form of Snell’s law, meets the tilted second face, bends out, and leaves. With $\alpha = 0$ this is the slab, and the figure above is that case, computed by the same function, returning every direction unchanged to the arithmetic floor.

At $\alpha = 2°$, in crown glass of index 1.52, a ray along the camera’s axis leaves 1.04° from where it arrived. That is the thin-prism value $(n - 1)\alpha$ to the second decimal place, and it is the familiar statement that a thin prism deviates light by an amount independent of how the light meets it.

The familiar statement is a small-angle approximation, and a camera’s field is not small. Forty degrees off the axis on one side the deviation is 1.54°; forty degrees off on the other it is 1.67°. The deviation is least near the axis — a prism’s minimum deviation, which for a wedge this thin sits within a couple of degrees of normal incidence — and grows away from it, faster toward the thick side of the glass. So a wedge does not turn a picture’s directions by one angle. It turns them by an angle that depends on where in the field each direction is.

The frame: no rotation absorbs it #

A single deviation for every direction would be harmless: it would be a rotation of the camera, which a recovery absorbs without noticing and which a tilted sensor is not a distortion showed changes nothing about whether a picture is a projection. The question is how much of the wedge’s effect is that rotation.

The figure takes a grid of 77 pixels across a 690 × 420 frame with a 50° field, finds the direction the wedge sends each pixel’s ray, and fits the rotation that best carries those directions back onto the pixels’ own rays — a least-squares rotation of directions, found by the quaternion method rather than by any fit that centres the bundle first, because centring a bundle of directions subtracts the mean direction and changes the answer.

The best rotation is 1.14°, a little more than the axial deviation because most of the frame is off the axis. It leaves 0.94 px root-mean-square across the frame and 2.32 px at the worst pixel, in the pattern the arrows show.

On this frame one pixel at the centre spans 0.077° of direction, so the rotation’s residual of 0.94 px is about 0.07° — roughly a fourteenth of the deviation the glass gives the axis. The rotation has taken up the great majority of what the wedge does to the frame. What it leaves is small, it is not zero, and the next question is whether anything with more freedom than a rotation can take it up.

A rotation is the special case, so the figure also fits the best homography — any projective map of the frame, which would absorb a rotation, a change of focal length, a moved principal point and a tilted picture plane together. It leaves 0.90 px root-mean-square and 1.87 px at worst. A homography with eight free numbers does barely better than a rotation with three. Whatever the wedge does to the frame beyond turning it is not a change of camera of any kind, because every change of pinhole camera is a homography — the fact a projection of a projection measured from the other side, where a photograph of a photograph is predicted by four marks to $10^{-13}$ px. A picture through a wedge is not a projection from the camera’s centre.

With the faces parallel the arrows vanish, and the same code handed a pure rotation in place of the glass — a map that turns every direction by one fixed angle — finds that rotation and leaves nothing, at $10^{-13}$ px. Handed a pure change of focal length, it finds that no rotation absorbs it and that a homography absorbs it completely, which is what a change of camera should do. So the residual is a property of the wedge, not of the fits that measure it.

What a rotation takes up, and what it cannot #

The frame’s residual is small because the frame is narrow. Every pixel’s direction lies within 29° of the camera’s axis, and across that range the wedge’s deviation changes by only a fraction of itself. A rotation of 1.14° takes up the average turn, and what it cannot take up is the variation of the turn from one part of the frame to another.

That variation, not the deviation itself, is what makes the picture something other than a projection. A wedge whose deviation did not vary with angle would be a rotation exactly, and the picture through it would be a pinhole picture of a turned camera. The thin-prism formula describes precisely that wedge — one number, $(n - 1)\alpha$, for every ray — which is why reasoning from the formula concludes that a thin wedge merely turns the view, and why that conclusion is nearly right for the frame. The variation the formula drops, from 1.04° on the axis to 1.67° forty degrees off it, is modest where a frame’s pixels are and large where a recovery’s vanishing points are.

Three vanishing points read where the glass bends most #

A camera is recovered from a picture most directly from three vanishing points. Recovering the camera reads the focal length and principal point out of the triangle the three mutually perpendicular directions’ vanishing points form — the principal point at the triangle’s orthocentre, the focal length from how far the triangle’s vertices sit from it — and the triangle a camera cannot move found that triangle self-polar with respect to the image of the absolute conic, which is why the recovery is exact on a pinhole picture.

That construction has a feature that matters here. Three mutually perpendicular directions cannot all be near the camera’s axis. Their direction cosines with the axis square-sum to one, so the smallest their largest angle can be is when the three are equal, at $\arccos(1/\sqrt{3}) = 54.74°$. Any three-vanishing-point recovery reads at least one direction at least 54.74° off the axis — well outside a 50° frame, whose corners are 29° off it, and exactly where a wedge bends light most.

For this camera the three axes’ directions lie 54.7°, 63.5° and 46.9° off its axis, and the wedge bends them by 1.58°, 3.39° and 1.45°, where it bends the axis itself by 1.04°. The vanishing points the recovery reads are displaced by those amounts, in directions that are not a common rotation, and the recovery does the only thing it can with three displaced vanishing points: it finds the pinhole camera that would have put them there.

That camera is turned 3.37° from the true one — three times the 1.14° that best fits the frame. Its focal length is 1.33% short and its principal point 16.6 px from where it should be. Used to predict the frame, it misplaces the pixels by 14.7 px root-mean-square, fifteen times the residual the best rotation left. The recovery has not failed in the sense of returning noise; it has returned a precise, self-consistent camera that describes the three directions it was given and describes the picture badly.

Three vanishing points leave nothing over to disagree #

It is worth asking why nothing in the recovery warns, because the answer points at the remedy.

Three vanishing points of perpendicular directions determine a pinhole camera’s focal length, principal point and orientation exactly, with nothing left over. One conic calibrates the camera states the same count in another form: each pair of perpendicular directions’ vanishing points must be conjugate with respect to one conic, three pairs give three conditions, and the conic a square-pixelled camera has carries exactly three free numbers — its centre, which is the principal point, and its size, which is the focal length. Three conditions, three unknowns, one answer, and no residual.

Five marks and the sixth found the general shape of that situation: a fit that is exactly determined agrees with its data perfectly by construction, so its agreement says nothing about whether the model was right, and the first extra mark is the first thing that can disagree. A camera recovered from three vanishing points through a wedge is in the position of a conic through five marks. It fits because it must. What exposes the wedge is anything beyond the three: a fourth direction whose vanishing point the recovered camera mispredicts, or the frame itself, which it mispredicts by 14.7 px.

Why the recovery turns further than the picture #

The factor of three is not an accident of this camera, and the sweep shows how little it depends on the glass.

At a 1° wedge the frame’s best rotation is 0.57° and the recovered camera’s turn 1.71°; at 2°, 1.14° and 3.37°; at 5°, 2.86° and 8.17°. Both grow very nearly in proportion to the wedge, and their ratio moves only from 3.03 at half a degree to 2.85 at five. The ratio is set by the difference between the directions the frame contains and the directions the recovery reads, and that difference is a fact about the camera and the scene’s axes, not about the glass.

Everything else scales the same way. At a 1° wedge the best rotation leaves 0.46 px root-mean-square and the best homography 0.45 px; the recovered camera’s focal length is 0.66% short, its principal point 8.7 px away, and it mispredicts the frame by 7.4 px — each close to half its value at 2°. That proportionality is what makes the estimates for thinner wedges further on arithmetic rather than guesswork, for as long as the wedge stays thin enough for it to hold.

Two things combine to produce it. The vanishing points are further off the axis than any pixel, so the wedge bends them more than it bends anything in the frame. And the recovery converts a displacement of three points into a rotation plus a change of focal length plus a moved principal point, spreading the error across all of them rather than leaving it where it arose. A displaced vanishing point is not a small error in a pixel; it is an error in the camera, and the principal point is not the centre measured how steeply the camera’s other numbers respond to a principal point placed wrongly.

Through a 5° wedge the pattern is the same at larger size: the axes bent by 3.97°, 8.19° and 3.63°, the recovered camera turned 8.17°, its focal length 3.44% short, its principal point 37.0 px away, the frame mispredicted by 35.4 px. Nothing about the recovery warns that anything is wrong. Three vanishing points always determine a camera; a wedge simply determines the wrong one.

The wedge’s own product #

The thin-prism deviation $(n - 1)\alpha$ is worth one more look, because it is the pane essay’s structure appearing again.

The pane’s displacement at small angles was $t(1 - 1/n)$ times the angle — thickness and index multiplied, separable only by the cubic term a wide fan of angles reaches. The wedge’s deviation near the axis is $(n - 1)\alpha$ — wedge angle and index multiplied — and a picture made near the axis cannot say which of the two it is looking at: a 2° wedge of index 1.52 and a 2.6° wedge of index 1.40 deviate an axial ray alike, by 1.04°. The separating information is in exactly the part the thin-prism formula drops, the growth of the deviation away from the axis. That is the pane’s lesson in another form. Two quantities that multiply in the leading term separate in the next one, and the next one lives far off the axis — which for a wedge happens to be the same place the recovery reads its vanishing points, so the directions that most mislead the recovery are also the ones that could identify the glass.

What this does and does not say about photographs through glass #

It does not say that recoveries through windows are wrong. A window pane is made to have parallel faces, and the exact result for a slab is the right model of one that does. How far from parallel any real window is, and so how large a wedge a real photograph meets, is a question about glass that nothing here measures. The effects scale very nearly in proportion to the wedge angle, so by that proportion a wedge of a tenth of a degree would turn a recovered camera by about a sixth of a degree — small, and not zero, and larger than the frame’s own rotation by the same factor of three.

It says where to look for a wedge’s effect. Not in the frame, where the best rotation or homography leaves under a pixel at a 2° wedge and would leave a twentieth of that at a tenth of a degree. In the vanishing points, which a recovery reads far off the axis where the glass bends most. A camera recovered through glass and compared with the frame it came from — its prediction against the picture — exposes the wedge at fifteen times the frame’s own residual.

And the model is directions only. The wedge here is a direction map fixed to the camera, which is what vanishing points and a frame’s rays depend on. The sideways displacement a thick piece of glass also gives every finite point, which the pane essays measured, is not included, and neither is the spread of colours a prism produces, which would give each colour its own recovered camera. A camera behind a curved port meets glass whose faces are nowhere parallel, and that is a different computation.

Still open: whether the wedge can be recovered with the camera #

This essay recovered a pinhole camera through a wedge and found it turned three times too far, with nothing left over to say so. The open step admits the wedge into the model: it recovers the camera and the wedge together, with the wedge’s angle and the direction of its thick edge as two extra unknowns, from four or more vanishing points and the frame’s straight lines, so that the fit finally has something left over. The question it measures is whether the wedge can be told from a rotation of the camera at all — since near the axis the two are nearly the same map — and how far off the axis the read directions must reach before the wedge’s angle is determined to a tenth of a degree, with the index held at a quoted value and then freed, to see whether $(n - 1)\alpha$ separates the way the pane’s product did.