The plan hidden in the photograph
The ground is a plane. It is the plane the viewer is standing on, the plane most of the interesting things in a photograph are sitting on, and the plane whose projection produces the convergence that makes a picture look like a picture. It is also, geometrically, exactly the same kind of object as the wall of a building — and so exactly the same operation flattens it.
Doing that turns a photograph into a plan. Four points on the ground whose plan positions are known, one homography, and the photograph becomes a map: distances between marks, the shape of a footprint, the position of a vehicle relative to a kerb, all of it available with a ruler.
One operation, two subjects
The literature teaches these separately and the separation is entirely historical. Rectifying a façade is called photo-rectification and belongs to architectural survey; rectifying the ground is called making an orthophoto or plan rectification and belongs to accident reconstruction and to aerial mapping. They are one function with a different plane passed in.
The implementation on this site makes that explicit: the same generator produces both figures, and the plane is an option. That is not a tidiness measure. Two implementations of one operation drift, and when they drift the drift shows up as a discrepancy between two answers that ought to be the same, in a field where nobody has a second view to check against.
What differs between the two is only which control points are available and how they are conditioned. A façade usually offers a window or a door — a compact, well-proportioned rectangle near the middle of the frame. The ground usually offers a road marking, a paving pattern, or four corners of a car park, which are typically spread across the whole depth of the picture and therefore very unevenly imaged: the near pair huge and well separated, the far pair small and close together.
That asymmetry is the ground case’s whole practical character. The far control points carry most of the information about the map and are the ones placed least precisely.
The horizon comes out, rather than going in
There is a check available on a ground rectification that has no equivalent on a façade, and it is the strongest one in this field.
The homography maps image points to plan points. It is built from four correspondences and knows nothing about the horizon. But the ground plane’s line at infinity — the set of directions that never arrive anywhere — has an image in the photograph, and that image is the horizon. So if the map is right, points marching out along the ground must have images that approach the horizon.
Approach it, and not merely be near it. “Near the horizon” is satisfied by any line drawn near the horizon, which is exactly the failure mode a plausible-looking-but-wrong rectification has. The claim that has content is that it is a limit, and a limit has a rate.
Measured: a ground point ten thousand units away images 0.102 px off the computed horizon. The same point at a million units images 0.00102 px off it. A hundred times the distance, a hundred times closer — a factor of 100.0, which is first-order convergence and is what a genuine line at infinity does.
This is the same shape of check the site uses on vanishing points, where a closed-form vanishing point is verified by marching a point out along its direction and requiring the error to fall by ten for ten times the distance. A closed form with the wrong axis or sign can sit near the right place; it cannot converge to it at the right rate. Being close is a coincidence available to many wrong answers. Being a limit is not.
Four points that are not four corners
A façade offers a window. The ground often offers nothing rectangular at all, and the method still works, because what it needs is four points whose plan positions are known — not four corners of anything.
That distinction opens up sources a rectangle-shaped search would miss. Four road studs at surveyed positions. Four corners of a manhole cover of a standard size. The two ends of two parallel road markings whose separation is specified. The four wheel contact patches of a vehicle whose track and wheelbase are in a manufacturer’s data sheet — which is the accident-reconstruction standard, and it works because a car’s four contact patches really are four coplanar points at known relative positions, sitting on the very plane being measured.
The last one repays a second look, because it is doing something the others are not. The vehicle is both the subject of the measurement and the instrument that calibrates it. Its position is what is wanted; its dimensions are what makes the plan possible. Nothing about that is circular — the dimensions come from outside the photograph and the position comes from inside it — but it does mean that a misidentified model of car produces a plan that is wrong everywhere and internally consistent.
There is one more requirement, easy to state and easy to violate: the four points must be in general position. No three of them collinear, and the quadrilateral they form should be as large and as square-on as the scene allows. Four points along a kerb are four points on a line and determine nothing at all — the solve is singular and this site’s Gaussian elimination refuses rather than returning a plausible matrix, in the way a camera refuses a point behind the eye, which is the behaviour to want. Four points forming a long thin sliver do not fail; they produce a map whose sensitivity to a one-pixel error is enormous, and nothing complains.
What the plan is in units of
A plan produced this way is in units of whatever the control points were given in.
If the four control points are given real plan positions in metres — surveyed corners, a road width from a specification, the wheelbase of a vehicle in the frame — the plan is in metres and every measurement on it is a measurement. If they are given as an abstract square, the plan is in units of that square’s side and every measurement is a ratio.
This is the same one-degree-of-freedom situation as everywhere else in this field, and it is worth restating in its ground-plane form because the ground offers an unusually convenient reference. Standard road markings are specified: a dashed centre line in most jurisdictions has a defined mark and gap length. Paving slabs are made to sizes. A parking bay is a legislated rectangle. A photograph of a road often contains its own scale bar and the person taking it did not know.
What no photograph contains is a scale that comes from the picture. The demonstration is the same one that applies to every measurement here, and it does not become less true because the ground is convenient.
Where a ground rectification goes wrong
Three failures are specific to this plane and worth naming, because all three produce a plan that looks entirely reasonable.
The ground is not flat. A road has a camber; a field has a slope; a pavement rises to a kerb. The rectification assumes one plane, so everything off it is displaced radially away from the point below the camera, by an amount proportional to its height above the plane and to its distance from that point. A vehicle roof rectified as though it were on the road appears displaced by a metre or more. This is the same failure as the balcony on a façade and it is more common here, because roofs and heads are much further off the ground plane than a balcony is off a wall.
The control points are in the wrong plane. The four corners of a building’s footprint are on the ground; the four corners of its roof are not. A rectification built on the wrong four produces a plan of a plane at roof height, correctly, and labels it the ground.
The far control points are badly conditioned. In an oblique photograph the far pair of control points may be separated by twenty pixels where the near pair are separated by four hundred. A one-pixel error in the far pair is a five per cent error in their separation, and it propagates into the whole depth scale of the plan. The near end of the resulting plan is accurate and the far end is not, and nothing in the plan indicates which is which.
That last one has a partial cure and it is the same one as everywhere: use more than four points. Five or more control points over-determine the homography, the least-squares fit has a residual, and the residual is a number that says how consistent the control points are with being four points of one photographed plane. Four points always fit exactly and therefore never complain.
How much of the picture is worth rectifying
There is a question a rectified plan raises immediately and nobody asks: how far out should it be drawn?
A ground rectification has no natural boundary. The homography maps the whole image plane, so a plan can be drawn out to any distance simply by mapping more of the photograph — and the further out it is drawn, the fewer picture pixels went into it. Somewhere the plan stops being a measurement and becomes an extrapolation, and the map itself gives no sign of where.
The number that decides it is available and is worth computing rather than eyeballing. A patch of ground at distance D images with an area proportional to 1/D³ for a camera looking obliquely along it — one factor of 1/D² from the ordinary inverse-square of a projection, and one more from the foreshortening as the ground turns edge-on. So the picture’s information density on the ground falls as the cube of the distance.
Put numbers on it. If a square metre of ground at 5 m occupies 400 pixels of the photograph, the same square metre at 25 m occupies 400 divided by 5³, which is a little over 3 pixels. At 50 m it is under half a pixel. A plan drawn to 50 m from a photograph taken at eye level is, in its outer half, an interpolation between a handful of samples, and it will look exactly as crisp as the near half because the rectification stretches those samples to fill the space.
That is the ground plane’s version of the polar redundancy in an equirectangular file, and it has the same remedy: know where the information is and do not present a re-projection as though it were uniform. A plan produced this way ought to carry a distance beyond which it is not evidence, and the cube law is what sets it.
The plan and the picture are inverses
Everything about this operation is reversible, and the reversibility is what makes it a projection rather than a repair.
The homography that takes the picture to the plan has an inverse that takes the plan back to the picture. So a plan drawn by a surveyor can be projected into a photograph taken from a known viewpoint, and a proposed building can be drawn into a photograph of the site it will occupy, and both are the same eight numbers used the other way round.
That is the operation an architect’s visualisation performs, and it is the operation this site performs every time it projects a scene. The difference between “make a plan from a photograph” and “put a building into a photograph” is a matrix inverse, and it is not obvious from the outside that they are the same task.
The one thing the inverse cannot supply is what the plan does not contain. A plan has no heights, so projecting it back gives a picture of a flat world — footprints on the ground, correctly placed, with nothing standing up. Heights have to come from somewhere else, and on a single view they come from the cross-ratio construction that measures them one at a time.
The plan a painter already had
There is a five-hundred-year-old version of this operation and it runs the other way, which makes it a useful thing to hold beside the modern one.
Alberti’s construction, and the distance-point method that follows it, both start from a plan: a floor of square braccia, laid out on the ground, whose transversals are then found in the picture. The painter has the plan and wants the picture. The rectification here has the picture and wants the plan. Between them they are the two directions of one homography, and the fifteenth century had one of them exactly.
What the painter did not have — and this is the point of putting the two side by side — is any way to check. Alberti’s construction produces a pavement of squares because it was built to; there is no measurement in it, and the only test available is whether the result looks right. A modern rectification produces a plan and can be handed lengths it was never given, which come back correct or do not.
The difference is not one of mathematical sophistication. The projective content of the two operations is identical, and this site’s checks confirm that Alberti’s transversals and a pinhole camera’s agree to 6 × 10⁻¹⁴ px. The difference is that one direction of the map can be verified against a world that exists and the other cannot, because the painter’s floor was never laid.
Aerial photographs, and why they are easier
The whole difficulty above is a consequence of the camera being near the ground and looking along it. A camera looking straight down has none of it.
In a vertical aerial photograph the ground plane is parallel to the picture plane, so the homography reduces to a similarity — a scale, a rotation and a translation — and the photograph is already a plan up to one number. The horizon is nowhere in the frame, the control points are evenly conditioned across the whole image, and the depth-scale problem does not exist because there is no depth in the picture.
That is why aerial mapping was a working technology decades before anybody rectified a street photograph, and it is worth stating because it locates precisely what this field is about. Single-view metrology from an oblique photograph is difficult for exactly one reason: the plane being measured is strongly foreshortened, so the picture spends most of its pixels on the near end and a handful on the far.
Everything else — the algebra, the invariants, the exactness of the homography — is the same in both cases and is not where the difficulty lives.