One camera known to be level ties the groups; a level phone picture does not read its centre

An uncalibrated picture ties the two groups, if the height of its centre is known took the flatness survey of a wall six metres wide — two groups of three cameras sixty degrees either side of its normal, every pose and mark found in one bundle adjustment — and asked what happens when the picture that ties the groups comes from a phone whose lens the adjustment must find. The wall’s twist, a saddle with opposite corners pushed out together, is the two groups pitching in opposite senses, and the groups alone report it only to ±5.05 millimetres. One extra picture ten degrees off square ties them to ±1.30 when its lens is known. With its focal length unknown it still does, and with the column of its centre unknown; with the height of its centre unknown it ties nothing, ±4.83, because a level picture of a flat wall cannot tell its own centre’s height from its own pitch.

That essay ended on a remedy that looked as if it should work. A level picture’s principal point lies on its horizon, the horizon is where the wall’s horizontal lines meet, and a picture’s drawn verticals — door jambs, the wall’s edges, the marks’ own columns — say whether it was held level. So add to the adjustment the condition that the extra picture is level, and the height of its centre should be read from the horizon.

It is not, near square, and the reason is arithmetic about where that horizon is. But the condition does something else instead, and the something else is better than the remedy it was meant to be.

Level against the wall, not against the frame #

First, what “level” can mean inside an adjustment. The adjustment has no gravity in it. Its frame is a gauge — here the first camera’s pose and one coordinate of the second’s, held fixed — and a camera “level” in that frame would be levelled against the first camera, which nothing measured. A condition that rotated freely with the frame would be no condition at all; one that did not would be information the survey never collected.

A leaning post is worth its lean made the same distinction for a street’s posts: a vertical read from the scene is a vertical only as far as the scene is plumb. So level is written against the wall itself: a camera is level when its optical axis and its horizontal axis are both square to the wall’s verticals, and the wall’s vertical is the mean of top mark minus bottom mark down the marks’ columns. That is what a picture’s drawn jambs license and what a phone’s tilt sensor reports, provided the wall is plumb — a proviso this essay comes back to. Each levelled camera adds two readings to the adjustment, the cosines of those two angles, weighed as a reading of stated precision: a camera known level to 0.1°, say, is a reading of each angle with a standard deviation of 0.1°.

A level picture’s horizon is out of reach #

Give the extra picture of the earlier essay — ten degrees off square, eight metres from the wall, its focal length and whole centre unknown — the condition that it is exactly level.

The height of its centre was read to ±73 px with the lens unknown. Exactly level, it is read to ±61. The tie needs it to about ten pixels — the earlier essay found a centre known beforehand to ten pixels kept the twist at ±1.44 mm — and sixty is nowhere near. The twist is reported to ±4.17 mm, a fifth of the way from no tie to a full one.

The reason is in where a level picture’s horizon is. The horizon at eye level is the rule that a level camera’s horizon passes through its principal point, and the rule is exact; what it needs is a horizon that can be found. The wall’s horizontal rows of marks run off towards a vanishing point at $f\cot$ of the turn from the picture’s centre — 4,200 px out of a 690 px frame at ten degrees off square, 8,500 px at five. Two rows of five marks converging that slowly fix the height of their meeting point poorly, and “the centre is on the horizon” only puts the centre on that poorly fixed line. The centre is read to ±29 px when the picture is exactly level and turned twenty degrees, ±18 at thirty, ±10.5 at forty-five and ±7 at sixty: the horizon comes into reach as the picture turns, and at forty-five degrees it is near enough to read the centre to the ten pixels the tie wants.

But a picture turned forty-five degrees off square has stopped being a tie. The earlier essays found that a calibrated picture loses its grip on the groups as it is turned towards one of them — ±1.30 mm at ten degrees, ±3.12 at forty-five — because it is becoming a member of the group it turns towards. The horizon reads the centre exactly where the centre no longer matters.

A level picture far from square ties through its pitch #

That is the remedy’s failure. The figures beside it point somewhere else.

Known level to 0.1° with its lens unknown, the extra picture reports the twist to ±4.49 mm at five degrees off square, ±4.39 at ten, ±3.60 at thirty, ±2.92 at forty-five and ±2.38 at sixty. At forty-five degrees that is better than a calibrated picture that is not known level, ±3.12, and at sixty it is better by more than half: ±2.38 against ±5.03. A calibrated picture known level is better still, and stays at ±1.23 to ±1.28 at every turn from square to sixty degrees. The level is not helping the picture read its centre; at sixty degrees the centre is read well anyway. It is tying the groups on its own.

The twist, as a bundle’s twist is tested by its own error and the earlier essays found it, is not a property of any camera: it is the two groups pitching in opposite senses while the marks they share barely notice. A camera that knows its own pitch against the wall’s verticals cannot take part in that motion. If it belongs to a group, its group cannot pitch; and a picture turned sixty degrees towards a group is one. The question to ask, then, is not whether the extra picture can be helped by being level, but whether any camera can.

One camera of one group #

One camera of one group — no extra picture at all — known level to 0.1° reports the twist to ±1.37 mm. That is the calibrated square-on picture’s ±1.30, near enough, from a reading a phone’s tilt sensor or a spirit level on the camera can supply. Known level to a quarter of a degree, ±1.68; to half a degree, ±2.36; to a degree, ±3.47; to two degrees, ±4.43, by which point it is nearly the groups alone. One camera in each group improves on that a little, ±1.31 at 0.1° and ±1.94 at half a degree, and all six levelled give ±1.27 and ±1.52.

The scale of those numbers is set by how well the survey knows its cameras’ pitch without help. Left to itself, the adjustment reads each camera’s pitch against the wall’s verticals only to ±1.15° at the outermost camera of a group and ±1.31° at the innermost — that loose pitch is the twist, seen from the cameras’ side. A level known to a degree is a second reading about as good as the survey’s own, and the twist improves from ±5.05 to ±3.47 mm, roughly what averaging two such readings would predict. A level known to a tenth of a degree is ten times better than the survey’s own reading, the pitch is held, and the twist with it.

It matters little which camera of a group is levelled. The outermost, sixty-five degrees off the wall’s normal, gives ±1.37 mm at a tenth of a degree; the innermost, at fifty-five, gives ±1.36. Each camera of a group shares nearly all its marks with the other two, and holding any one of them holds the three. Nor does it matter which group: the survey is symmetric, and the left group’s levelled camera does what the right group’s does, with the sign of everything it does reversed — which is what the next section turns to account.

The mechanism is the one the earlier essays found, run the other way. The two groups pitch against each other because nothing measures pitch: each camera’s marks are as well explained by a slightly different pitch and slightly different depths as by the true ones, and the two groups can share out the difference so that the marks’ depths form a saddle. A camera whose pitch is held cannot share it. Its group is held with it, through the marks its three cameras share, and the other group is held to the first through the marks all six share.

Against that, the phone picture ten degrees off square, known level to the same tenth of a degree, is at ±4.39. It is level too, and its level is worth almost nothing to it, because near square its unknown centre trades with the camera’s own height on the wall — a picture of a flat wall drawn from a little higher, with its centre moved down to match, is very nearly the same picture — and being level does not say how high the camera stood. The extra picture spends its level on its lens. A survey camera with a known lens spends its level on the twist.

A wall out of plumb is a false twist #

The level a phone reports is against gravity, and the adjustment uses it as level against the wall’s verticals. They are the same only if the wall is plumb. An old brick wall leaning half a degree is unremarkable — two and a half centimetres over three metres — and it makes every levelled camera’s reading wrong by half a degree of pitch against the wall.

With one camera levelled to a tenth of a degree, a half-degree lean puts a false twist of 2.1 mm into the survey — more than the ±1.37 the survey reports, so a flat wall that merely leans is called twisted. The adjustment reads the lean as one group pitched against the other, which is exactly what a twist is. Weighed more loosely the level is followed less and misleads less — 1.8 mm at half a degree’s weight, 0.5 at two degrees — but it also ties less, and the reported error rises to meet the bias.

With one camera in each group the false twist is zero, to the last digit of the arithmetic, at every weight. A lean pitches both levelled cameras’ readings the same way. The twist depends only on the difference between the two groups’ pitches, and a lean common to both changes the difference by nothing. The cameras here are placed mirror-wise, the outermost of each group, which makes the cancellation exact; any one camera from each side cancels most of it.

That is the practical instruction, and the reason for the second camera is not the small gain in precision. One levelled camera makes the survey hostage to the wall’s plumb; two, one on each side, make it indifferent to it.

What a surveyor now sees #

A four-millimetre twist is called real 86 times in 100 with a calibrated picture ten degrees off square, 75 with a camera in each group levelled to a quarter of a degree, 65 with one, and 12 with the groups alone. A flat wall is called twisted five times in a hundred by every arrangement whose bias is zero, as a test at twice its reported error should. With one levelled camera and a wall leaning a quarter of a degree it is called twisted 9 times in 100: the false twist is a millimetre, under the reported error, and it raises the false alarm by four in a hundred without anyone being told.

A quarter of a degree is a modest level — a careful spirit level on a tripod does better — and at a tenth the levelled arrangements match the calibrated picture outright. Split the track and the needles turn found the same shape in a survey along a street — a weakness that is a motion the views cannot hold against each other, repaired by anything that pins that one motion — and a level is the most direct pin the twist has. The calibrated square-on picture has one advantage they lack: it needs no claim about gravity or about the wall. The levelled cameras have one it lacks: they need no extra picture, no calibrated lens beyond the survey’s own, and nobody standing in the street in front of the wall.

What the reading leans on #

Level is read against the wall’s verticals. Everything above takes the levelled cameras to know their pitch and roll against the wall itself. A phone’s sensor knows them against gravity, and the two differ by the wall’s lean; the arrangement with one levelled camera in each group is the one that does not care.

The level reading is unbiased apart from the lean. A tilt sensor can carry an offset of its own, differing from one instrument to the next. If the same phone takes every levelled picture, its offset is common to them all and behaves exactly like a lean: one camera carries it into the twist, two on opposite sides cancel it.

The lenses of the survey cameras are known. The levelled cameras tie the groups because their pictures are already explained by their poses; a levelled camera whose lens the adjustment must also find would spend some of its level on its lens, as the phone picture did.

The marks are matched correctly. A wrong match is not a small error applies unchanged; a level reading is no defence against a coherent mismatch.

Every number is the adjustment’s own covariance at the true solution. That is the first-order description of a least-squares fit, and the false twist is that description’s bias for a stated error in the levels. Neither says what a solver started from a poor guess would find.

The number that was missing #

One square-on picture ties the two groups found the cheapest repair for the survey’s commonest failure, and the earlier essay found its condition: the picture must know the height of its own centre. This essay finds the condition was a symptom. What the survey lacks is any reading of pitch, and the square-on picture supplied one indirectly, by seeing the marks from a direction neither group sees them from. A level picture shows its rise on its horizon read a shift lens’s rise from a level camera’s horizon, which works when the horizon is in the frame; a survey square to a wall has its horizon thousands of pixels away, and the rise has nowhere to be read from. A tilt sensor reads pitch directly, and where the adjustment stops is the reminder that an adjustment can only be as good as the readings that pin what it cannot pin itself.

Still open: whether the wall’s own verticals can stand in for a level #

Everything here took the levelled cameras’ pitch from outside the pictures — a sensor or a spirit level — and treated the wall’s verticals only as the reference that pitch is measured against. But the wall’s verticals are in the pictures too. A door jamb or a building’s corner beside the wall is a long straight line, and in a camera pitched even slightly it converges with the others towards a vanishing point far below the frame, at a rate that is the camera’s pitch.

The measurement that settles what that is worth adds to each survey picture a few long vertical edges — a corner, two jambs — reads each picture’s pitch from how they converge, and asks whether those readings, entered in the adjustment as the sensor’s were, tie the groups as a tenth of a degree did; how long and how many the edges must be, since a pitch read from lines that converge four thousand pixels away is read slowly; and whether edges that are part of the wall itself, rather than of a building standing beside it, bring the wall’s own lean in with them and so cancel it in every camera at once.