Closing a loop mends its ends

A camera that walks away from where it started accumulates uncertainty about where it is. A camera that walks back and recognises where it started has, at that moment, a measurement no amount of careful walking could supply: the last picture and the first picture see the same wall, so the last camera’s position relative to the first is pinned by those shared points directly, not through the long chain of pictures in between.

That recognition is loop closure, and the usual account of what it does is tidy. The discrepancy at the closure is spread back round the loop, the error that had been growing along the walk becomes a bridge that is zero at both ends and largest in the middle, and the worst error is halved.

The account is easy to measure, because the uncertainty of every camera on a loop can be computed for the loop left open and for the loop closed, from the same pictures. Measured, it is right about the ends and wrong about the middle.

The walk #

The ring is a set of facades 13 m from its centre, give or take two, with points scattered over them up to five metres high. Twenty-four cameras stand on a circle 8 m out, fifteen degrees apart, each looking straight out at the wall in front of it through a 100° field. A point is kept if at least three cameras see it, which is what it takes to carry a scale from one pair of views to the next.

Twenty-two of those points are seen both by one of the first three cameras and by one of the last three. They are what closes the loop. Left open, those points are treated as new points when the last cameras see them — the walk does not know it has returned. Closed, they keep their identity.

Nothing is solved. The question is not what one solver found but what the pictures can determine, so each figure is the linearised covariance of every camera and every point at the true configuration, with the marks’ error set at a standard deviation of $1/\sqrt{12}$ px in each coordinate — what rounding to a whole pixel produces.

The covariance is read in one gauge, and the choice is not arbitrary. An uncertainty is quoted from something found that holding the first camera and a baseline gives uncertainties that are zero at the start and grow away from it, and that this is the wrong report for most questions and exactly the right one for a single question: how far is each camera from where the walk began. That is the question a loop closure is meant to answer, so it is the gauge used here throughout.

The worst camera is not the last one #

The first figure already contradicts half the usual account before the loop is closed.

A walk that drifts is expected to be worst at its end. This walk is worst across the ring, at camera 14, whose centre is 520 mm from certain relative to the start. The last camera, number 24, is 332 mm.

The reason is that on a loop the end of the walk is physically beside its start. An error in orientation made early in the walk — a small rotation of everything after it about the first camera — moves each later camera by an amount proportional to its distance from the first camera, and the camera furthest from the first is the one across the ring, not the one at the end of the list. Position error on a loop is a lever, and the lever is longest halfway round. That is the explanation that suggests itself, and it is only half right: drawn as ellipses rather than spheres, the far cameras’ uncertainty points along the line to the start rather than across it, which is a length carried round the walk rather than a turn — a loop’s far side is a length measures the difference, and what it takes to reach it.

So “the worst error is halved” has to say which error. The error at the end of an open walk, which is what drift is usually described as, is not the worst error on a loop.

Closing it #

Recognising the twenty-two shared points changes the picture at once, and unevenly.

The last camera falls from 332 mm to 51. The worst camera, which is now number 13 rather than 14, falls from 520 mm to 418.

Drawn camera by camera, the two curves are the same shape for most of the walk and part company only near its end. The open curve rises across the first half of the ring, peaks opposite the start and comes down again as the walk returns — not to zero, because nothing has told it the walk has returned, but to 332 mm. The closed curve follows it closely until about camera 16, then turns down much more steeply and arrives at the last camera at 51 mm.

The last camera gains a factor of 6.5. The worst camera gains 20 per cent.

The bridge account predicts something in between for the middle, and it is worth seeing why. If the far side of the ring were reached by two independent routes of equal length — clockwise from the start and anticlockwise from the start — and each route alone gave the open walk’s uncertainty there, combining them would divide the variance by two and the standard deviation by $\sqrt{2}$, a gain of 29 per cent. The measured gain is 20. The two routes are not independent: both begin at the same first cameras, both carry scale through the same few pictures near the closure, and the anticlockwise route to the far side is simply the open walk’s clockwise route read backward for most of its length. A closure adds a second route only where the first route was long.

Where the gain goes #

Dividing the open uncertainty by the closed one, camera by camera, shows the shape directly.

The gain is 1.16 at cameras 2 to 5, rises to 1.25 at the middle of the ring, passes 1.5 at camera 19 and reaches 6.47 at the last camera. Averaged over the first half of the walk it is 1.18; over the second half, 2.10.

The cameras near the start barely gain because they were never badly placed relative to it. Two or three pictures connect them to the first camera, and the closure adds a route through twenty-odd pictures, which carries almost no information they did not already have. The cameras near the end gain enormously because, left open, their only route to the start was the long way round; closed, they have a route of two or three pictures through the shared points, and that short route dominates. The far side is roughly equally far from the start by either route, so it gains the modest amount two long routes give over one.

So the closure is not a correction spread evenly round the loop. It is a new short connection placed at one point of the loop, and it helps each camera in proportion to how much shorter it makes that camera’s best route back to the start.

How few recognised points it takes #

A closure depends on recognising shared points, and recognition is the hard part in practice. So it matters how many are needed.

Recognising two of the twenty-two shared points cuts the last camera’s uncertainty from 332 mm to 122. Five cut it to 72, ten to 57 and all twenty-two to 51. Most of the gain at the end of the walk comes from the first few recognised points, and the rest refine it.

The worst camera behaves differently. Two recognised points take it from 520 mm to 502, five to 488, ten to 470 and all twenty-two to 418 — a steady, slow improvement that has not flattened by the time every shared point is used.

The two rates have the same cause as the uneven gain. The last camera’s problem is that it has no short route to the start, and two shared points supply one; after that, more points make the short route stronger, which helps less. The far side’s problem is that every route to it is long, and more shared points only make the closure’s contribution a little firmer, adding a little to a route that was never going to be short.

The practical reading is that a loop closure recognised from a handful of points is nearly as good as a thorough one for the cameras near the closure, and nowhere near as good for the cameras far from it — which are the cameras that were worst to begin with.

A closure is an adjustment’s to make #

A chain cannot close a loop at all, and it is worth being exact about why, because the closed curve above is not something a chain could have produced by trying harder.

A chain and an adjustment described the chain’s two real weaknesses: its errors are frozen once a link is computed, and it has no mechanism to notice that the last picture sees what the first one saw. The second is not an omission that a chain could repair with an extra step. A chain places each camera from the one before it and nothing else, so learning at camera 24 that it sees camera 1’s wall gives the chain a discrepancy and no way to act on it; every camera between them was placed before the discrepancy existed.

An adjustment is what acts on it. The track and the scene together solved for every camera and every point at once against every observation, and a recognised point seen at both ends is simply more observations in that one system: the same point, constraining camera 1 and camera 24 through the same three coordinates. The closed covariance in these figures is the covariance of that system. It exists because the adjustment exists, and it is what the adjustment can do with the closure, not what any particular solver did.

That also says what the closure cannot do. The adjustment is still determined only up to the seven numbers no picture can name: a loop that returns to its start is no more placed in the world, turned in it or sized than a walk that does not. Every figure here holds the first camera to remove those seven, and the closure changes nothing about them. What it changes is the shape of the track relative to that held camera.

A closure is not control, and not more pictures #

Two other ways of improving a walk are easy to confuse with a closure, and they are not the same thing.

Control fixes positions from outside. The eighth held number bends the scene held surveyed coordinates at points of a courtyard and found that seven of them choose a frame and more than seven make claims about the shape. A surveyed point on the far side of this ring would pin the cameras near it directly, at whatever accuracy the survey had. A closure pins nothing from outside; it only tells the adjustment that two parts of its own reconstruction are the same place. It cannot make the far side accurate, only consistent with the near side.

More pictures fill in the same walk. Another picture of the same sweep found that adding views between existing ones, across the same arc, left a reconstruction no better, because what it lacked was angular spread rather than photographs. A closure is not more of the same walk. It is a connection between two parts of the walk that the walk itself had placed as far apart as they could be in the sequence and as close as they could be in space — which is exactly why it helps the ends, where that contradiction between sequence and space was largest, and not the middle, where there was none.

What a closure does not repair #

Systematic error. Every figure here is a covariance: it describes how the reading error in the marks spreads into the cameras, and nothing else. A scale chain leans rather than wanders found that along a chain of pairs the random error largely cancels and a systematic lean accumulates instead. A lean is not in a covariance. What an adjustment does with the discrepancy a lean leaves at the closure — whether it spreads it round the loop as a bridge, or concentrates it where the constraints are weakest — is a question about a solve on biased data, and no solve was run.

The far side, by recognition at the start. A closure is a connection placed where the loop begins and ends. The cameras it helps least are the ones furthest from that place. A walk that must be accurate at its far side needs a connection there — a view across the ring, a second pass, a surveyed point — and no amount of recognition at the start supplies it.

The gauge. The uncertainties here answer how far each camera is from the first one, which is the question a closure is usually asked. In a gauge that held nothing — the inner constraints of the essay on quoted uncertainties — the cameras near the start would not read near zero, the open curve would not rise from nothing, and the gain would be distributed differently. That reading is not drawn.

Why the account was tidy #

The bridge picture comes from a one-dimensional random walk: a sum of independent steps, pinned to zero at both ends. For that object it is exactly right. The variance at the middle of a pinned walk is a quarter of the open walk’s variance at its end, so the worst error is halved, and it is in the middle.

A walk round a ring of walls differs from that object in two ways, and both showed up. The quantity that goes wrong is a position in the plane, not a number on a line, and it is governed by what the walk carries from its start to its far side — mostly, it turns out, its scale — so the open walk is not worst at its end. And the steps are not independent, because every picture shares points with its neighbours and a scale is carried through the points two pairs share, so two routes round the ring are not two independent measurements of the far side.

The account is a good description of drift along a line with a return to its start. It is a poor description of a loop, and the difference is large exactly where it matters: the account promises the most improvement where the measurement finds the least.

What this does not settle #

One loop. Twenty-four cameras on one ring, one field of view, one rule for keeping points and one seed. Other loop sizes were attempted with the same scene and the same rule, and at thirty cameras the rule left one camera with nothing it could determine, so the dependence on loop size is not measured.

Linearised. The covariance is first-order at the true configuration. A closure that corrects an error of many decimetres is not a small perturbation, and the true distribution of errors after an adjustment would differ from the linearised one by an amount not measured here.

Rounding. As throughout this field, the marks’ error is rounding to a grid.

Still open: what a connection across the ring buys #

The closure mends the ends because it adds a short route where the walk begins and ends. The obvious experiment follows from that reading: add a short route somewhere else instead.

A second connection across the ring — one camera on the far side and one near the start seeing a common landmark in the middle of the courtyard, say — would give the far side its own short route to the start. The question that leaves is whether one such connection helps the worst camera more than the whole closure did, how the gain from a cross-connection compares with the gain from a closure camera by camera, and whether a single well-placed connection across the ring and a closure together bring every camera within some fixed factor of the start — which is the difference between a walk that needs to come home to be trustworthy and one that needs to be seen from across the middle.