A picture with nothing straight in it

Every construction in this collection so far has started by assuming the horizon. The measuring point needs it. The distance point is on it. Transferring a height across a room runs a line to it. Rectifying a façade uses it twice. It is the single most useful line in a picture, and each of those essays gets it the same way: find two edges that are parallel in the room, extend them, and take the crossing.

That works for architecture and for nothing else. A photograph of a crowd on a beach has no parallel edges. Neither has a hillside, a stand of trees, a curved façade, a market square full of people, or most photographs anybody has ever taken.

Such a picture is not without structure. It has repetition — several things of about the same size — and repetition is enough.

Two of a height give a point #

Take two objects of the same height standing on the same ground plane — two people, two bollards, two chairs.

In the room, the segment joining the two feet and the segment joining the two heads are parallel: one is the other displaced straight up by the common height. Parallel lines in the room have one vanishing point in the picture, so the drawn line through the feet and the drawn line through the heads meet there.

Both lines are horizontal in the room, so that meeting point is on the horizon.

Nothing in that argument mentions how tall the two objects are, how far apart they stand, or what camera took the picture. It mentions only that the two heights are equal, and equality is a thing repeated objects supply for free.

Measured, the six points four uprights produce sit on the camera’s own horizon to about six parts in a hundred trillion of a pixel — which is to say the construction is not an approximation at all. It is the same kind of statement as the constructions in carrying a height across the room: joins and meets only, so it survives whatever the projection did.

And two of a height give only a point #

A point is not a line, and the horizon is a line.

With exactly two objects there is one meeting point and nothing to say which way the horizon runs through it. In a picture taken with the camera not rolled the horizon is level on the paper and one point is enough — but “the camera was not rolled” is an assumption about how the photograph was taken, and the whole appeal of this construction is that it assumes nothing.

Three objects give three pairs and therefore three points, which is a line and a check. Four give six.

This is a refusal rather than a degradation, and the machinery here treats it as one: handed two uprights it returns the one point and declines to fit a line through it, because a line through one point is a choice and not a measurement.

The arrangement that looks best and gives nothing #

The second refusal is the one worth remembering, because it is the arrangement a photographer naturally produces.

Line three people up abreast, all at the same distance from the camera, and photograph them. Their feet lie along a line parallel to the picture plane; so do their heads. A world direction parallel to the picture plane has its vanishing point at infinity — the drawn lines are parallel on the paper, and parallel lines do not meet anywhere a draughtsman can mark.

In the case drawn above the three meeting points land thousands of picture widths outside the frame, and as the three uprights line up exactly they go to infinity.

So the construction wants the repeated objects spread in depth, and a row of them across the frame is the single least useful arrangement. That is the opposite of the intuition, which says a nice orderly row ought to be the easy case. What makes the case easy for a person looking at the picture — everything the same size, obviously equal — is precisely what makes the construction degenerate.

This is the same trap in a different costume as the one recorded on this site about sampling a surface along its own axis, and the same one as choosing four rectification points at evenly spaced indices through a grid and getting four collinear points. In each of them a configuration that is tidy is a configuration that is degenerate, and tidiness is what an author reaches for when constructing an example.

Which pairs to trust, and why the worst one looks the best #

There is a practical consequence of the abreast refusal that is worth separating out, because it changes which part of a photograph a reader should look at.

The quality of a pair is decided by the angle between the two lines being crossed — the line of feet and the line of heads. Two uprights well separated in depth make those lines cross steeply, and their meeting point is pinned down. Two uprights at nearly the same distance make them cross at a hair’s breadth, and the meeting point slides along the horizon under any wobble at all.

That is the ordinary business of a badly conditioned intersection, and it has a consequence that is not ordinary: the pairs that contribute least are the ones whose two members look most obviously alike. Two people at the same distance draw at the same size, so a reader can see at a glance that they are the same height and will reach for them first. Two people at four metres and thirteen metres draw at wildly different sizes, so the equality is an inference rather than an observation — and that is the pair that fixes the horizon.

The rule that falls out is short. Pick the pairs that are furthest apart in depth, and treat a pair drawn at the same size as no evidence at all. In the picture drawn above, the pair that does most of the work is the nearest upright with the furthest one, and the three points contributed by pairs at similar depths are the three that sit far out to the side, where the fit is right to give them little weight.

It also explains a mistake that is easy to make with a large crowd. Adding more people does not help if they are all standing in a row: a hundred uprights abreast are a hundred uprights at one depth, giving several thousand meeting points, every one of them off the paper. The information is not in the count. It is in the spread.

What happens when the objects are not equal #

People are not all the same height, and that is the whole difficulty with using them.

The construction takes “equal” as exact. Given uprights whose heights vary, each pair still produces a meeting point, but the points no longer lie on one line: they scatter about the horizon, and a line fitted through them lands somewhere near it.

How near is the question a reader needs answered, and it has a number.

On the picture drawn — a 690-pixel-wide frame, a 46° field, six uprights spread from four metres to thirteen — a spread of two centimetres in the heights puts the found horizon about two pixels from the true one. A spread of five centimetres puts it about five and a half. A spread of seven centimetres, which is roughly the standard deviation of adult human height, puts it about eight and a half pixels out, and twelve centimetres puts it eighteen.

Eight and a half pixels on a 690-pixel picture is a little over one per cent of the width. It is worth converting that into the quantity a reader cares about, which is not pixels.

The horizon is at eye level. Its height in the picture, compared with the drawn height of anything standing on the ground, gives the camera’s own eye height — that is the content of the horizon is at eye level. At this focal length, an eight-pixel error in the horizon on a person standing at seven metres corresponds to an eye height wrong by something like seven centimetres. That is not nothing and it is not much: it is the difference between a camera held at the eye of a tall person and the eye of a short one.

So the honest statement is that a crowd gives the horizon to within a few centimetres of eye height, and a colonnade — where the repeated objects genuinely are equal — gives it exactly.

Fitting a line, and which fit #

Six points that are supposed to be on a line, and are not quite, have to be fitted, and the choice of fit is not a formality here.

The obvious fit — least squares of the vertical offsets, y against x — treats the horizontal position of every point as exact. The points this construction produces do not deserve that: a pair of uprights nearly abreast contributes a point a long way out to the side, whose position along the horizon is enormously uncertain and whose height is the only part worth trusting.

The fit used is total least squares, which minimises perpendicular distance and has no preferred axis. It is the difference between a line that is dragged sideways by a distant outlier and one that is not.

There is a better answer still, which this essay does not take: weight each point by how badly conditioned its pair is, since a pair spread in depth deserves more say than a pair nearly abreast. That is a real improvement and it needs a model of where the drawing error comes from, which is a different subject.

Other things that repeat #

Uprights of equal height are the clearest case and not the only one.

Equal spacing along a line. A row of equally spaced posts, fence pales, sleepers or window mullions is a repeated interval rather than a repeated height, and a repeated interval on a straight line in the room gives that line’s vanishing point directly — which is on the horizon if the line is horizontal. That is the construction in the bay repeated by a straightedge run backwards.

Any repeated shape on the ground. Paving slabs, tiles, a repeated motif in a carpet: two corresponding points of two copies are a pair of points related by a translation in the world, and two such pairs give the translation’s vanishing point.

A person walking. The same person photographed twice in one frame — or once, with their reflection — is a pair of equal uprights that happen to be the same object.

The check that comes free #

Four uprights give six points. Three of those six are enough to determine a line, so the other three are a test.

That is worth saying plainly because it is the difference between a construction and a measurement. A construction with exactly enough information produces an answer and no evidence. A construction with more than enough produces an answer and a residual, and the residual is the only thing that can tell a reader the heights were not equal after all.

The residual here is the root-mean-square perpendicular distance of the meeting points from the fitted line. At perfectly equal heights it is arithmetic noise. At a human spread it is tens of pixels — much larger than the error in the horizon itself, because the points scatter along the line as well as across it.

That asymmetry is useful. A large residual with a horizon that is nevertheless nearly right is the signature of unequal heights, which is a thing that averages out. A large residual with a horizon that is badly wrong would mean something else — objects not on one plane, or not upright — and the two can be told apart.

What this does and does not unlock #

With the horizon in hand, a photograph of a crowd becomes a measurable object. Heights compare against heights by cross-ratio. A plane in the picture can be rectified once four points on it are identified. The camera’s eye height comes out. Everything in a height from one photograph applies.

What it does not give is the rest of the calibration. The horizon is the ground plane’s vanishing line and that is affine information: it buys midpoints, ratios along a line, parallelism. It does not buy angles or the focal length, which need something more — a second vanishing direction known to be perpendicular, or a circle in the scene, or the machinery in what one picture determines.

The shape of the finding #

The horizon is usually presented as something a picture either shows or does not — the sea, the edge of a field, the line where the two sets of parallel edges converge. It is better thought of as something a picture encodes, and the encoding survives the removal of every straight line in the scene.

What it does not survive is the removal of repetition. A photograph of one object, of any shape, from one viewpoint, carries no horizon at all: there is nothing in it to compare. A photograph of two of anything carries a point. Three carry a line.

That is a cleaner statement of the requirement than “the picture must contain parallel edges”, and it covers the parallel-edge case as a special instance — two parallel edges are two repeated things, and their ends are two pairs of corresponding points.