A facade matched a column out comes forward, not back

A railing matched one along stands underground found that a stereo pair which matches every post of a railing to the next one along builds a perfect railing 2.2 times further away, and that what convicts it is not anything near it but what it stands on: its feet sink 1.32 metres into the ground. It ended with the case a real photograph offers more often than a railing — a facade of identical windows, matched one column out — and predicted that the windows would reconstruct as a facade 2.2 times further away and 2.2 times wider, with nothing beneath it to sink into. What the facade does have is corners, where the wall turns and the windows stop, and the question was what those are worth.

The prediction was wrong about the direction, and the reason is worth more than the correction. A facade’s windows are usually further apart than a pair of cameras, and a repeated structure whose period is longer than the baseline cannot be pushed further away by a one-period mistake at all. It can only be pulled nearer. And the corners turn out to hold the facade only through an assumption about the wall between them, which the pictures cannot supply.

Windows further apart than the eyes come forward #

The facade below stands twenty metres from the same pair as before — two cameras 1.1 metres apart, nine hundred pixels of focal length — and faces them squarely. It carries two storeys of five windows, 2.4 metres apart, and a metre and a half of plain wall at each end before its corners.

In the left picture the windows are 108 pixels apart and each one’s partner in the right picture sits 49.5 pixels to its left. A matcher that confuses a window with its neighbour one column to the right asks for a disparity of 49.5 minus 108, which is minus 58.5 pixels: the two rays meet behind the cameras. Four cameras fit and one can see called that refusal cheirality, and it removes the mistake the earlier essay predicted. The mistake that survives is the other neighbour, one column to the left, which asks for 49.5 plus 108 — a disparity of 157.5 pixels — and puts every window at 6.29 metres.

The wrong facade is the true one scaled 0.314 times about the left eye, for the same reason the wrong railing was scaled 2.2: each wrong window lies on its true window’s ray from the left camera, since the left mark never moved, and the disparity sets how far along that ray. It is perfectly consistent. Every window lies on both its rays to $7\times10^{-15}$ metres, the windows are evenly spaced 0.75 metres apart in a straight row, and a mismatch on its own line needs a third eye already established that every test two photographs can run is at its floor for a mistake that slides along the epipolar line. The pair has built a small facade 13.7 metres in front of the corners of the real one.

One column of windows has gone. A shift of one column leaves one end column without a partner in the other picture, the count the railing’s length entered through its ends by. It is a real witness, but a weak one on a facade whose ends are plain wall, since a matcher that finds nothing for one column simply leaves it out. Drag the plan’s period and the counterfeit slides along the left camera’s rays: windows 1.2 metres apart are built at 9.57 metres, windows 3.6 metres apart at 4.68, so the coarser the repeat against a baseline of 1.1 metres, the nearer the wrong facade stands and the smaller it is drawn.

The period over the baseline decides the direction #

The railing’s posts were 0.6 metres apart and the facade’s windows 2.4. That difference is the whole story, and it can be stated without any distance in it.

A structure at distance $Z$ with a period of $p$ metres repeats every $fp/Z$ pixels in both pictures, and its true disparity is $fB/Z$. Matching every element $k$ periods out changes the disparity by $k$ periods, so the wrong depth is $Z \cdot B/(B - kp)$. The distance cancels: whether a repeated structure can be pushed back, and how far, depends only on its period measured in baselines.

While the period is shorter than the baseline there is exactly one reading further away for each whole number of periods that fits inside the baseline — the railing, at $p/B = 0.55$, has one, at 2.2 times its distance — and one nearer reading for every whole number of periods in the other direction. Once the period exceeds the baseline, the further readings are gone and only the nearer ones remain: the facade’s windows can be misread at 0.314 of their distance, or 0.186, or less, and never further. Whole pixels cut space into shells and depth is a reciprocal described every depth a pair reports as a reciprocal of its disparity; the family here is the same reciprocal, stepped by a period instead of a pixel.

The practical reading is short. A matcher confused by a repeated structure with a period shorter than its baseline can make either mistake, and the further one is the one that looks like the scene’s background. A matcher confused by windows, columns, bays or balconies spaced wider than its baseline — which is nearly all of them for a hand-held pair — can make only the nearer mistake, and its false facade hangs in the air between the cameras and the building.

A nearer facade floats, and nothing under it can say so #

The railing was convicted by its feet. The facade’s wrong reading has a foot too, and it is in the wrong place, but not in a way the pair can see.

Carried 0.314 of the way along the rays from the left eye, every height closes in on the height of the eye. The upper windows at 4.5 metres come down to 2.51, the lower ones at 1.5 go up to 1.57, and the wall’s foot, where it meets the pavement, rises to 1.10 metres. A two-storey building becomes a block less than two metres tall floating at head height — which, like the sunken railing, is a relation to the ground that the true facade satisfies and the wrong one cannot.

The difference is that the railing’s feet were matched, because a post’s foot is a point on a vertical edge and repeats as the post does. A wall’s foot is a horizontal line, the join of the wall and the pavement, and the next section shows why a pair reads nothing from it. The wrong facade’s foot is floating, and there is no mark on it that the pair could carry to the ground to find out.

Only the vertical edges carry depth #

Everything the pair knows about depth comes from sliding a patch of one picture along a row of the other until it fits. What the patch holds decides whether the slide has one answer, several, or none.

A patch on a corner, where the wall stops and the sky beyond it begins, fits at one disparity only, the true one. A patch on a window fits wherever a window is, once a period, and the true disparity is one of those fits with nothing to mark it as better. A patch on the roofline fits equally well everywhere. The roof’s edge runs horizontally, which is along the pair’s rows, and a horizontal edge slid along itself looks the same at every offset; the figure’s mismatch for it does not vary at all across two hundred pixels. The same is true of every string course, every window sill and head, and the wall’s foot.

So of all the edges a facade has, only the vertical ones say how far away anything is — corners, drainpipes, the jambs of a door — and of those, only the ones that do not repeat. The windows’ own verticals repeat. What is left is the corners and whatever plain-wall features happen to be vertical. That is why the question the earlier essay asked is the right one, and why it has to be asked about lines rather than points.

Two corners hold the wall, by an assumption #

The corners are matched correctly and reconstructed at twenty metres. The windows are reconstructed at 6.3. The plain wall between them carries no marks, so it is not reconstructed at all, and nothing in the two pictures says the windows and the corners belong to one surface. Two rays that do not meet is the reminder that a pair’s evidence about each point is its own two rays: the windows satisfy theirs at 6.3 metres, the corners theirs at twenty, and a textureless wall between them is consistent with any depth at all.

What ties them is an assumption from outside the pictures: that a facade is flat. It is the same kind of import as the level ground that let one stone convict a railing, and it is usually safe. Given it, the two corners define the wall — two parallel vertical lines lie in exactly one plane — and every window can be tested against it. Carry each window’s left mark along its ray to the wall and into the right picture, and compare with the mark the matcher paired it with. An honest window lands on its mark. A window matched a column out lands exactly one period away, 108 pixels here, because a one-column shift is a one-period change of disparity and the wall knows the true disparity.

With both corners visible the test never fails, from windows 2.4 metres apart down to a repeat of five centimetres, 2.3 pixels in the picture. Each corner is read across itself at two dozen heights, so its disparity is known to a fraction of a pixel, and the windows lie between the corners: the wall at the windows is interpolated, and interpolation between two well-read lines carries no more than their own error. Half a period is the margin, and on any repeat a matcher could resolve at all, the corners’ plane is better than that.

One corner and a pipe: the lever is the facade #

The case the earlier essay worried about is the one with only one corner in view — a facade that runs out of the frame, or a building seen from its end — and a single mark on the plain stretch beside it: a drainpipe, the edge of a sign, a crack. A vertical line of that kind is as good as a corner for the pair, since it too is read across itself. What changes is the geometry of the two lines.

Two lines close together fix the wall’s depth well near them and its turn badly: the turn is their difference in depth divided by their distance apart, and a small distance divides a reading error into a large one. The windows are not between the lines but beyond them, up to ten metres further along the wall, so the turn’s error is multiplied by that lever when it reaches them. For windows the margin is 54 pixels and a pipe half a metre in from the corner is enough nine times in ten; a pipe twenty or thirty centimetres in, which is where drainpipes often run, is not enough at all. A fine repeat — the ribs of cladding, a louvred screen, tiles — has a margin of a few pixels, and the second line has to stand metres in from the corner before the wall’s turn is known well enough to carry it across the facade.

So the answer to how far from the corner a single mark can be has the opposite shape to what the question expected. A mark close to the corner is the weak witness, because it adds almost nothing the corner did not already say; a mark far from it is strong, because it fixes the wall’s turn. The best second line is the other corner, and failing that, the furthest vertical thing on the plain wall. A wrong match is not a small error found that one bad correspondence spreads its error across a whole fit; here one well-placed good one, far from the first, is what lets the fit reach the whole facade.

What the wall’s own relief spends #

The flat-wall assumption is the one the whole test rests on, and a real facade is not flat. Windows sit in reveals a quarter of a metre deep, bays project, a centre section may be set back several metres. Every one of those puts an honest window off the corners’ plane, and the test charges it for the difference.

An honest window set $r$ metres behind the corners’ plane is off by $fBr/(Z(Z+r))$ pixels of disparity, and it is cleared only while that is less than half a period. So relief raises the smallest period the corners can defend. A quarter-metre reveal costs 0.61 pixels and matters only for repeats finer than about a pixel and a quarter, which a matcher would not resolve anyway. A centre bay set back a metre needs repeats of 4.7 pixels or more; one set back four metres, 16.5 pixels. The measured points sit above the line because the corners’ own reading error spends a little of the margin too, and because the grid of periods is coarse.

Windows are safe: at 108 pixels the margin is ten times anything a facade’s relief costs. The repeats at risk are the fine ones on an articulated wall — the cladding on a recessed bay, the balusters of a set-back balcony — where the facade’s own shape is as large, in disparity, as the mistake. How flat is flat enough measured the same currency from the other side: there, a scene’s relief had to reach about ten times the marking error before a pair could use it; here, a wall’s relief has to stay under half a period before its flatness can be used to convict a shift. Both are tolerances in pixels of parallax, and both are about whether a shape is large or small against the thing it is being asked to reveal.

What the corners hold #

The earlier essay drew a structure matched one period out everywhere as the true structure scaled about the first eye, convicted only by a physical tie to something that was not scaled. The facade adds two things to that statement. The scaling can go either way, and for anything spaced wider than the baseline it goes towards the cameras. And the tie a facade offers — its corners — is not physical in the sense the ground was. The railing’s feet were matched points standing on a matched ground; the facade’s windows and corners are joined only by a wall the pair cannot see, and the tie holds exactly as well as the assumption that the wall is flat.

Given that assumption the corners are an excellent witness: two of them convict a shifted facade at every period a matcher resolves. One corner and a far vertical line are nearly as good. One corner and a near line are not, however carefully read. The one thing a single view cannot give is a reminder of how much of this subject runs on assumptions of that kind: a flat wall is to a facade what a level floor was to the railing, a fact about buildings that the photographs are allowed to use and cannot prove.

What was assumed #

The facade faces the pair squarely. A facade turned away from the baseline puts its window rows on slanting lines in the pictures, and every row except the one at the eyes’ height then crosses the epipolar lines. A match one column out on those rows lands off the epipolar line by an amount that grows with the row’s height above the eyes, and the pair catches it unaided. The case measured here is the one the pair cannot catch.

The wall between the corners is flat and unmarked. Marks on the plain wall — weathering, signage, a pipe — are what the pipe test uses; a wall with many of them fixes its own plane without the corners, and the assumption of flatness becomes a measurement.

Lines are read across themselves to a pixel, at two dozen heights. A corner half hidden by a tree, or a pipe read over a metre of its length, carries a fraction of that and moves every threshold here in proportion to the square root of the loss.

The matcher shifts every column by the same amount. A facade matched one column out on its upper storey and honestly on its lower breaks into two planes six metres apart, which is its own witness, and a different failure.

Still open: whether a raised third eye reads the floors as the pair read the columns #

The pair here has a horizontal baseline, so its rows run across the facade, its ambiguity is between columns, and its blind edges are horizontal. A third camera raised above one of the pair — a pole, an upper window, a drone — has a vertical baseline with each of the others, so its epipolar lines run up the facade. Horizontal edges then carry depth and vertical ones do not, and the facade’s storeys, which repeat every three metres, take the place of its columns.

The measurement that settles what that eye is worth raises a third camera by a stated height, matches the facade’s floors one storey out along its vertical epipolar lines, and asks two things. First, whether the family of wrong readings is the one drawn here with the floor height in place of the column spacing and the raise in place of the baseline — so that a raise shorter than a storey can only pull the facade nearer, and a raise longer than one can push it back. Second, and more useful, whether the roofline and the wall’s foot, which the level pair could not read at all, pin the facade’s depth for the raised eye the way the corners did for the pair, and whether the two baselines together leave any reading of a lattice of windows that satisfies every picture except the true one.