A third eye sees the water the pair cannot

The stick a stereo pair puts back photographed a straight metre of stick leaning thirty degrees into still water, from a pair of eyes 65 mm apart standing 1.6 m above the surface, and found that the pair reconstructs a stick that is not there. With the eyes side by side the two rays to every point of the stick meet exactly, and they meet at the stick’s sagittal image — a curve 715 mm long, where the real stick is a metre. The residual does not warn and a fit weighted by the miss trusts only the surface then tried every way of asking the pair itself whether anything was wrong, and every one of them said no, loudest where the answer was worst.

The last of those essays ended by naming the one thing a pair cannot do for itself. Its two rays agree because both eyes belong to one family of planes the refraction is symmetric about; an eye that belongs to neither family would see each point through rays that no member of the pair’s family would produce. A third eye placed above the pair, it proposed, should turn the pair’s perfect agreement into a measurable disagreement — and a rig could then detect that it is looking through water without being told.

It does. The disagreement is there, and it is large enough to see. What the rig then does with it matters more than whether it is there, and the obvious thing to do with it turns out to be the worst.

The third ray misses by pixels #

The hero figure keeps the pair where it was, reconstructs the stick from it as before, and adds a third eye raised above the pair’s middle — twenty-five, fifty, a hundred and two hundred millimetres. For each point on the real stick, the third eye’s ray is traced through the surface exactly as the pair’s were, and the figure plots how far that ray passes from the point the pair put back.

The miss is exactly zero where the stick enters the water, since a ray to a point on the surface does not bend, and it grows steadily down the stick. At the tip, with the third eye raised a hundred millimetres, it is 2.60 px in a camera with a focal length of 900 px; raised fifty, 1.35 px; raised two hundred, 4.81 px. It grows nearly in proportion to how far the third eye stands from the pair.

That is the disagreement the continuation predicted, and it is comfortably above the reading error of any ordinary matcher. The pair’s rays meet to the last bit and its reconstruction is 285 mm short; a third camera ten centimetres above one of them sees, at the tip of that reconstruction, a discrepancy between two and three pixels. The pair had no way to see that, because the question it can ask — do my two rays meet? — has the answer yes.

Which way the third eye stands decides it #

Raising the third eye was a choice. The figure below carries it round the line of sight at a fixed hundred millimetres from the pair’s middle.

Along the pair’s own line — a third eye beside the other two, level with them — it barely disagrees: 0.032 px. Raised it disagrees by 2.60 px, lowered by 2.99, and in between the miss rises smoothly from one to the other.

That small level miss corrects something the earlier essay said. It described the level pair’s agreement as a property of the family its eyes belong to — both in the horizontal plane across the line of sight — and a third level eye would then agree too. It does not quite. The pair’s two rays meet because the two eyes stand symmetrically about the vertical plane containing the stick: by that symmetry, their two rays cross in that plane. A third eye on the same line but not symmetric about that plane breaks the symmetry and misses, weakly. The family is necessary; the symmetry is what makes the agreement exact. A straight stick in water is a kink and a curve found the same symmetry at work in a single photograph: a stick leaning straight toward or away from the eye is drawn as one straight line, its kink and curve both hidden by the plane it shares with the eye. The third eye should go where the symmetry cannot help it, above or below the pair, and the figure’s unequal peaks — larger below than above — come from the line of sight looking down at the water, so that a lowered eye sees each point at a steeper angle to the surface than a raised one.

Asked to agree, the three rays hide it #

A third camera is not usually added to measure one ray’s distance from another pair’s reconstruction. It is added to triangulate: every point is put where all three rays come closest, and the rays’ residual from that point is the rig’s statement of how well its views agree. That is the natural thing to do with a third eye, and it is what the figure below does, with the three rays read as if the water were not there.

The two curves separate at once. The third ray’s miss of the pair’s point keeps growing with the offset, to 8.23 px at 400 mm. The three rays’ miss from the point they are jointly nearest does not: it rises to 0.39 px at a hundred millimetres, peaks at 0.41, and then falls. The rig that triangulates all three sees almost none of what the third eye saw.

The reason is in the shape of the pair’s evidence. The pair’s two rays are sixty-five millimetres apart and the stick is more than two metres away, so their rays are nearly parallel. They fix where a point lies across the line of sight very well and how far along it very poorly. The third ray arrives from a noticeably different direction. The cheapest way to reconcile the three is to slide the common point along the pair’s rays — in depth — until it nearly lies on the third, which costs the pair’s rays almost nothing and absorbs most of the third ray’s disagreement.

That slide is not free, and the numbers under the lower curve show its price. The stick the three rays agree on is 715 mm long with the third eye a millimetre above the pair, 574 mm at fifty, 493 at a hundred and 469 at two hundred. The pair was 285 mm short of the metre; with the third eye added and read as air, it is 507 mm short. The third view, which saw the problem clearly, has been used to make the answer worse, and the residual that should have reported the problem reports four tenths of a pixel.

A third ray is worth what its picture is worth found in air that a third ray improves a point exactly as much as its own picture’s precision allows. Through water the same arithmetic works against the rig. The third ray is not a noisier reading of the same point; it is an exact reading of a different point — a different focal image of the stick — and least squares averages the two images into a third that is neither. The midpoint is a choice of ruler found that where two rays miss, the point chosen between them depends on how distance is measured; here the choice is made between two real images, and no ruler picks the stick.

A test that flags the water #

The residual is small, but it is not zero, and a rig can still ask whether it is larger than it would be without water. The figure runs that test as a rig would have to: compare the three rays’ consistency, read as air, against the distribution of the same statistic for the same rig looking at a dry stick with the same reading error, and flag the water when it is worse than ninety-five per cent of dry readings.

Read to a quarter of a pixel, the test flags the water in thirteen per cent of trials with the third eye ten millimetres up, three quarters at twenty-five, and every trial from fifty millimetres on. Read to half a pixel it flags half the trials at fifty millimetres and four fifths at a hundred. Read to a pixel it flags no more than a quarter of them anywhere, against the five per cent the test flags by chance.

And raising the third eye stops helping at about a hundred millimetres. Beyond that the power at half a pixel is the same seventy-nine per cent at two hundred as at one hundred. That is the absorption again: raising the eye increases what it sees, but the three-ray residual grows no further because the extra disagreement goes into depth. A rig that wants to detect water by consistency alone should buy a better matcher, not a taller mast.

This is the answer the continuation asked for, and it is more qualified than hoped. A rig can detect that it is looking through water without being told, provided it reads its marks to a few tenths of a pixel and places its third eye off the pair’s line by a few centimetres. It detects the water by a residual of a few tenths of a pixel, well below the several pixels the third eye actually sees.

Told there is a surface, three eyes find its index #

The absorption has a remedy, and it is to stop reading the rays as air. Suppose the rig is told only that there is a flat surface where the water is — which a rig looking into a pool, a tank or an aquarium can see for itself — but not what the water’s refractive index is. Each eye’s ray is then bent at the surface by an assumed index before the three are triangulated, and the assumed index can be varied.

The two curves in the figure are the whole of the case for putting the third eye somewhere the pair’s symmetry cannot reach. With the third eye on the pair’s own line, the three rays’ miss stays below a hundredth of a pixel at every index from air to 1.5: such a rig cannot tell one index from another, which is the level pair’s blindness inherited. With the third eye raised a hundred millimetres, the miss falls to exactly zero at 1.3330 — the index of water — and rises on either side. At that index the three rays meet at every point of the stick, and the stick they meet on is a thousand millimetres long.

Every earlier essay in this sequence found the pair unable to learn from itself. A point under water has two depths explained why: the rays a submerged point sends to an eye pass through two focal lines, and two eyes side by side read one of them while two eyes one above the other read the other. Three eyes that include both directions read both, and only the true index makes both readings the same point. The disagreement the third eye saw is not an error to be averaged away; it is a measurement of the refraction, and the index is what it measures.

What a pixel costs the recovered stick #

Recovering the index from three rays is a fit, and a fit to noisy readings has a precision. The last figure measures it.

With every direction misread by a quarter of a pixel and the third eye raised a hundred millimetres, the fitted index is 1.336 ± 0.059 and the stick comes back at 1,007 ± 93 mm. At half a pixel the scatter in length roughly doubles, to about ±220 mm. Raising the third eye beyond a hundred millimetres buys little: ±85 mm at two hundred, ±90 at four hundred. The three rays’ disagreement, the thing the index is fitted to, grows more slowly than the offset, for the same reason the detection test stopped improving.

Those scatters are large, and they should be stated beside the alternative. The pair alone is 285 mm short with no scatter at all: its error is systematic, and more readings of the same kind would never reduce it. The three eyes with a fitted index are right on average and scattered by nine centimetres at a quarter of a pixel, from twelve points on one stick. Every additional point on the stick, or another stick in the same water, adds to the evidence for the one index they share, while it would add nothing at all to the pair’s certainty about a wrong answer. A systematic error of 285 mm has been exchanged for a random one that averages away.

What a third eye is for, under water #

The earlier essays in this sequence set out a rig that could not see its own mistake. The third eye’s contribution comes in three parts, and they point in different directions.

It sees the mistake: its ray misses the pair’s reconstruction by pixels, in proportion to how far off the pair’s line it stands. Used as a third triangulating view without a model of the water, it conceals the mistake and deepens it, sliding the reconstruction in depth toward a different focal image and reporting a residual of a few tenths of a pixel. And used with a model that has one unknown — the index of a surface the rig can see — it measures the mistake and removes it.

The practical rule that follows is not about cameras. A rig that suspects it is looking through something should not ask whether its views agree; it should ask what its views would have to be looking through for them to agree, and fit that. A picture through water has no viewpoint established that no single centre explains a refracted picture. The index is the extra number that makes the pictures explicable, and three suitably placed eyes are enough to find it. What a ray does at a surface began this sequence with the one fact everything since has rested on — that a pool looks three quarters as deep as it is only when seen straight down — and the index is that fact’s single number, recovered from the pictures instead of assumed.

What was assumed #

The surface is flat, still and known. Every figure places it exactly at a known height. A rig must find the surface too, from its edge or from floating marks, and an error in its height is a second unknown that would compete with the index — nothing here measures how well the two separate.

One stick, in the eyes’ symmetric plane. The stick leans straight away from the pair, in the vertical plane that makes the level pair exact. A stick across the line of sight gives the pair a nonzero miss of its own, as the earlier essays found, and the third eye then adds to a disagreement that already exists.

Directions are misread independently. Real matchers make correlated errors between neighbouring points, and those would inflate the index’s scatter beyond what twelve independent points suggest.

Still open: whether the surface’s height separates from its index #

The index fit here was given the surface. A rig looking into a tank sees the surface’s rim and can place it, but one looking into a pond, or at a river bed through moving water, has to find the surface from the same pictures it is trying to correct. Raising the surface and lowering the index do similar things to a ray: both change where a bent ray goes as it deepens, and a pair of eyes cannot tell them apart any better than it could tell the index from air.

Three eyes might. The index changes how sharply rays bend at every angle of incidence, while the surface’s height changes where the bend happens, and rays arriving at the surface at different angles — from a raised eye and a level pair, looking at points at different depths — weigh those two differently. The measurement that settles it fits the index and the surface’s height together from the same three eyes and twelve points, reads the directions to a quarter of a pixel, and asks whether the two parameters come back separately or trade off along a ridge — and if they trade off, whether a second stick at a different distance, seen through the same surface at a different angle, breaks the trade.