The bundle keeps the needles turned, if the wall has three rows

Split the track and the needles turn found that a point triangulated from several cameras is known worst along one direction — its error is a needle — and that the needle points along the rays only while the cameras are gathered to one side. Split them into two groups more than a right angle apart and every needle swings across. For a facade photographed to measure its flatness, that turn was the whole point: six cameras in two groups at sixty degrees either side of the wall’s normal measured each point’s depth to 2.83 millimetres, where six on a narrow arc managed 12.47.

Those numbers held the cameras’ poses fixed and known. The essay ended on the objection that matters in practice. In a real reconstruction the cameras are found from the same pictures, and two groups far apart are each well determined in themselves and poorly determined against each other, since every tie between them runs through marks both groups see. An error in where one group stands relative to the other moves every point the two groups triangulate together, and for two groups either side of a wall that motion is towards and away from the wall — exactly the error the split was meant to remove.

The measurement below runs the full adjustment, cameras and points together, as the track and the scene together first did for a courtyard, and the answer turns out to depend on something the objection did not mention: how the marks are laid out on the wall.

A quantity no frame can move #

A bundle adjustment’s answer is fixed only up to a choice of frame: seven numbers no picture can name measured the reconstruction staying exactly where it was in every picture when it is shifted, turned and scaled as a whole. So “each point’s depth” has no single uncertainty once the cameras are found; an uncertainty is quoted from something found the same point’s error ranging from four millimetres to a hundred depending on what the adjustment was held at.

A flatness survey does not need depth in that sense. It needs each point’s departure from the plane that best fits the whole wall, and that quantity is the same in every frame: a shift, a turn or a change of scale of the whole reconstruction moves every point of a flat wall along a plane, and the fitted plane takes it up. So the comparison is made in that currency. Thirty-nine marks are spread over a wall six metres wide — thirteen across, three rows up, from half a metre to two and a half — and photographed from eight metres by six cameras, every mark read to half a pixel. For each mark, the error of its departure from the wall’s plane is computed twice: with the six cameras’ poses known and each point triangulated alone, and with every pose and every point recovered together in one adjustment from the same marks.

For the two groups at sixty degrees the bars barely rise above the ticks. With the poses known, the marks along the middle row are known to 2.53 millimetres at the centre and 3.03 at the ends; with the poses found, to 2.57 and 3.13. Over all thirty-nine marks the root mean square goes from 2.70 millimetres to 2.79. The adjustment has had to find six cameras’ positions and orientations, thirty-six numbers, from the same marks that fix the wall, and the wall’s flatness has paid three per cent for it.

The slider shows the other arrangements, and none of them pays much either. The narrow arc of six across thirty degrees goes from 11.98 to 12.78 millimetres, the wide arc across ninety from 4.29 to 4.35, the two groups at thirty degrees from 4.37 to 4.74. The ranking the known poses gave survives intact.

Two checks on the currency #

Two properties of the flatness error make it the right thing to compare, and both were checked rather than assumed.

The first is that it does not depend on the frame. A bundle’s covariance has to be written in some frame, and the one used above is the frame that spreads the uncertainty as evenly as possible over every camera and point. Written instead in the frame of the first camera — the convention every two-view result quietly adopts, and the one an uncertainty is quoted from something found making the last camera look a hundred millimetres uncertain — the flatness error of every mark comes out the same to within a hundredth of a per cent. A wall’s departure from its own plane is a shape, and a shape does not care where it is held.

The second is that finding the cameras can only cost. With the poses known, each mark is triangulated from rays whose origins and directions are exact; with them found, the same rays carry the poses’ own uncertainty. On every mark, in every arrangement and layout measured, the found error is at least the known one. The bundle’s charge is therefore always a charge, and its size is the whole question.

Thirty-nine marks, and ten #

The objection is right in its mechanism and wrong in its size, and the reason is the number of ties. Every mark both groups see is an equation relating the two groups’ poses, and thirty-nine of them, spread over the wall, pin the groups to one another nearly as well as knowing their poses would. The test of that reading is to take the ties away.

With ten marks — five across, two rows — the two groups at sixty degrees are charged seventy-eight per cent, from 2.38 millimetres to 4.23. The wide arc, whose cameras each overlap their neighbours, is charged seven per cent, from 3.74 to 4.01, and now measures the wall better than the split groups do. The narrow arc is charged thirty-seven per cent and remains far behind both. With few ties, the objection comes true: the groups’ uncertainty against one another returns, and what the split bought is given back and then some.

The comparison with the wide arc is the useful one, because it is the arrangement a surveyor would otherwise choose. On thirty-nine marks the split groups beat it by a third, 2.79 millimetres against 4.35; on ten they lose to it, 4.23 against 4.01. The crossover is somewhere in between, and where exactly depends on a detail the ten-mark case changed without saying: it has fewer marks, and it also has fewer rows.

A third row, not more marks #

A mark count conflates two things — how many marks there are, and how they are spread over the wall — and the ten-mark wall has only two rows. The sweep below separates them.

The two curves for the split groups do not meet. With marks in two rows, adding columns helps slowly: six marks leave 4.95 millimetres, ten leave 4.23, twenty-six leave 3.44 and fifty still leave 3.14, sixteen per cent above what known poses would give. With marks in three rows the bundle’s charge is small from the start: nine marks leave 3.01, fifteen leave 2.86 — twelve per cent above known poses — and thirty-nine leave 2.79. Fifteen marks in three rows are worth more than fifty in two.

The third row need not sit anywhere in particular. With five columns, a middle row at 1.5 metres (the cameras’ height) leaves 2.86 millimetres, one at 1.2 metres leaves 2.91, one at 2.0 metres leaves 3.02; four rows that skip 1.5 altogether leave 2.94. What matters is that the marks do not lie on two horizontal lines. Even a single mark added at the middle of the ten-mark wall takes the charge from seventy-eight per cent to thirty-four. Why a third row is worth so much more than more columns was not derived here; the next figure shows what it removes.

Why sparse marks look flatter #

The dashed curve in the last figure runs the wrong way for a measurement: with the poses known, the wall looks flatter the fewer marks it carries — 2.41 millimetres at nine marks, 2.70 at thirty-nine. The marks themselves are not better measured when there are fewer of them. What changes is the plane they are measured against.

A departure is taken from the plane fitted through the marks, and a plane fitted through few marks follows each of them closely. Fit a plane through three marks and every departure is zero, whatever the marks’ errors: the plane passes through them. Through nine, the plane spends three of the nine numbers the marks supply, and a departure keeps on average two-thirds of its mark’s variance. Through thirty-nine, the plane spends three of thirty-nine and the departures keep more than nine-tenths — which is about the difference between 2.41 and 2.70 millimetres. The known-pose numbers at small counts are optimistic for this reason alone, and the same optimism is in the found numbers — which makes the bundle’s charge at ten marks, seventy-eight per cent on top of a flattering baseline, larger than it looks rather than smaller.

A survey that wants its flatness error to mean the marks’ own error should carry enough marks for the plane to be decided by the wall and not by any one mark; on this wall that is a few dozen, and the three rows the bundle needs are a sensible way to lay them out.

The shape the bundle leaves #

The flatness errors of different marks are not independent once the poses are found. Every mark is triangulated by the same six cameras, so an error in the cameras moves the marks together, and the covariance of their departures has shapes in it — combinations of marks that tend to be wrong together. The largest of them is the way the wall is most likely to be drawn wrong.

With ten marks in two rows the largest mode carries seventy-one per cent of the total flatness variance, and it is a twist: the top row’s left end forward and right end back, the bottom row the reverse, so that the wall reads as a gentle saddle, 11.3 millimetres along the mode. No plane can take a twist up, which is why it survives into the flatness error, and it is a single shape rather than scatter, which is why the ten-mark wall’s error is so much larger than its marks’ reading error. With twenty-six marks in two rows the twist still carries forty-three per cent. With three rows it falls to twenty-two per cent on fifteen marks and eight on thirty-nine, and the error left is spread over many shapes, as it is with the poses known.

The hero drawing at the head of this essay is the ten-mark twist; the slider on the figure above walks from it to the thirty-nine-mark wall. A surveyor checking a facade for flatness from a bundle adjustment with few marks should expect exactly this artefact, and should distrust a reconstructed wall that comes out as a saddle: the likeliest reading of such a result, with marks in two rows seen from two groups, is that the groups’ relative pose was loose and the wall is flat.

How far to open the groups #

The earlier essay found the turn of the needle happening at forty-five degrees and the depth getting better the further the groups were opened. The bundle might change where that stops.

It does not. Opening the groups pays at every angle measured, with the poses found as well as known and with ten marks as well as thirty-nine. With thirty-nine marks the bundle’s charge holds at eight per cent from fifteen degrees to thirty-five and then falls, to one per cent at seventy-five; with ten it rises to eighty per cent at fifty-five degrees and falls back to forty-five at seventy-five. But at every angle the found error is lower than at the angle before. The earlier essay’s advice stands — open the groups as far as the marks can still be matched — and the bundle only adds a condition about what the marks must be.

The upper limit in practice is the matching, not the geometry. At seventy-five degrees off the normal a wall is seen almost edge-on, marks are foreshortened to a quarter of their width, and a real matcher begins to lose them. None of the figures here models that; every mark is found in every picture.

What the objection got right #

The earlier essay’s objection was that finding the cameras would put back the error the split removed, because the groups are tied to one another only through the marks both see. The measurements say that it is right about the mechanism and that the size of the effect is set by the ties. With the wall’s marks in three rows or more, thirty-nine of them, the bundle charges the split groups three per cent and they stay four and a half times better than the narrow arc and a third better than the wide one. With marks in two rows, the charge is sixteen per cent even at fifty marks and seventy-eight at ten, and at ten the wide arc wins.

The spread a point gets found that a reconstruction is short of angular spread, not photographs; this finds the bundle’s version of the same statement. What ties two groups is not the number of marks they share but the spread of those marks over the wall — a second dimension, in this case a third row — just as what fixed a point was not the number of cameras but the spread of their directions. And the eighth held number bends the scene found that constraining a reconstruction with too little can distort its shape in a particular way; the twist here is the same kind of object, a single shape the data leave loose, and it is the first thing to look for in a wall that comes out curved.

What to put on a wall #

The practical reading is short and cheap to follow. A facade photographed for its flatness from two groups either side should carry marks in at least three rows — targets stuck to it, or the corners of windows, sills and lintels, which a real facade supplies in rows already — and a few dozen of them. With that, the bundle adjustment gives back almost exactly what known poses would, and the two groups beat any single arc by the margin split the track and the needles turn promised. With marks along two lines only — a string course and a cornice, say, the commonest features on a plain wall — the groups’ relative pose stays loose however many marks those lines carry, and a single arc that sweeps the whole ninety degrees is the safer arrangement.

What the covariances assume #

Every mark is found in every picture. The groups at sixty degrees see the wall obliquely, and a real matcher loses marks at that angle, especially fine ones. A mark seen by only one group ties nothing and is triangulated only by that group’s short baseline.

The cameras’ focal length is known. Every camera here shares one calibration, held fixed. A bundle that also adjusts the focal length adds a freedom that trades against the wall’s distance, and the groups’ relative placement would be looser still.

The marks are read with independent errors of half a pixel. Where the adjustment stops found the reading error setting the floor of every residual. All figures here are linearised covariances, exact for small errors; a mismatched mark is not a small error, and one mismatch among ten carries far more weight than one among thirty-nine.

The wall is flat and the flatness is measured against a plane. A survey of a wall that is meant to be curved would measure departures from a fitted cylinder, which takes up more shapes than a plane does. Whether it also takes up the twist is a separate question.

Still open: whether the twist can be read from the adjustment itself #

The twist is the bundle’s own weakest shape, and a bundle adjustment reports its covariance along with its answer. That raises a practical question with a measurement in it.

A surveyor who has reconstructed a wall and found it twisted by some millimetres has two readings available: the twist itself, and the uncertainty the adjustment attaches to the twist mode. The measurement that settles what they are worth reconstructs a wall that is truly flat and one that is truly twisted by a stated amount, both from ten marks in two rows and from fifteen in three, with the marks’ errors drawn at random, and asks how often the fitted twist exceeds twice its own reported uncertainty in each case. If a real twist of five millimetres is reported at three sigma from the three-row wall and at one from the two-row wall, then a single added row is also what lets the adjustment tell a surveyor whether to believe the shape it drew.