A texture reaches the horizon as a rate

A point at infinity is an ordinary vertex put a ground triangle through the projection matrix with its two far corners written as directions — a zero in the fourth slot — and watched it reach the horizon, where the same triangle drawn out to ten kilometres stops a fifth of a pixel short. It placed the vertices and stopped. A ground is rarely drawn bare, and the moment it carries a texture there is a problem the vertex positions did not have: every vertex of a triangle carries its attributes, and a far corner at infinity has no texture coordinate to carry.

For a repeating ground texture — one-metre checks, say — the coordinate of a point is the number of checks between it and some origin, and the point at the end of a direction is infinitely many checks away. A texture does not interpolate on the page derived the only interpolation that is right under perspective: carry each attribute divided by the vertex’s $w$, carry one over $w$ beside it, interpolate both straight across the page, and divide at the end. At a vertex with $w = 0$ the attribute is infinite and so is one over $w$, and the recipe asks for their ratio at the only vertex where neither is a number.

The question this essay settles is whether the recipe can be written without them.

A rate and a weight #

The answer is already in the vertex. A direction $\mathbf{d}$ written as $(\mathbf{d}, 0)$ is the limit of the point $(\mathbf{P} + \mathbf{d}/w,\ 1)$, rescaled to $(w\mathbf{P} + \mathbf{d},\ w)$, as $w$ goes to zero. A texture coordinate that is linear on the ground, $u(\mathbf{X}) = u_0 + \mathbf{g}\cdot(\mathbf{X} - \mathbf{P})$, can be rescaled the same way: carry the pair $(w\,u,\ w)$ instead of $u$, and at the limit it becomes $(\mathbf{g}\cdot\mathbf{d},\ 0)$. The first number is a rate — how many checks the texture gains per metre along the direction — and the second is a weight, 1 at every point and 0 at every direction. The texture coordinate anywhere is the first number divided by the second.

Both numbers are linear in the vertex’s homogeneous coordinates, so both interpolate under perspective by exactly the recipe that already exists: divide each by the clip-space $w$, interpolate across the page, and divide the two interpolated quantities by each other at the end. The weight takes the place of the “one” in one over $w$. At the near vertex the pair is $(u_0, 1)$ and the recipe is the familiar one. At each far vertex it is (rate, 0), and the clip-space $w$ it is divided by is the direction’s depth rate — 0.887 and 0.438 for the two ground directions here — which is an ordinary positive number. Nothing is infinite anywhere, and the infinity the texture really has is produced where it belongs: at the horizon, where the interpolated weight reaches zero.

A graphics processor already interpolates every attribute over $w$ and divides by the interpolated one over $w$ before handing it on, so the pair is carried as two ordinary attributes and the division of one by the other happens once per pixel after that. The hardware’s own division by one over $w$ cancels out of the ratio.

The hero picture is the result. Every check is drawn exactly, through the map the pair defines rather than sampled, until checks become thinner than the ink; beyond that the band is left in their average tone, because there every row of the picture holds more than one check. The check lines run straight to their vanishing points, where parallel lines meet, the rows close up toward the horizon, and the ground reaches the horizon line with no gap.

The texture holds its coordinate to half a pixel from the horizon #

Exact is a claim, and the measurement is simple to state: walk down the principal column from the triangle’s near vertex, 204 px below the horizon, to half a pixel short of it, and at each row compare the interpolated coordinate with the coordinate of the ground point that pixel actually sees.

In double precision the rate-and-weight interpolation is off by $3 \times 10^{-15}$ checks 200 px below the horizon and $6 \times 10^{-10}$ half a pixel below it. The growth is the arithmetic’s rounding and nothing else: the coordinate itself grows from about 5 checks to 2,443 over the same rows, because half a pixel below the horizon the column is looking 2.4 km along the ground. Relative to the coordinate the error never exceeds a part in $10^{12}$.

Single precision is the format a graphics processor interpolates in, and the same arithmetic carried in it — every product, sum and division rounded to single — is off by $2 \times 10^{-8}$ checks at the near vertex and $2 \times 10^{-4}$ at the last row. That is a part in ten million of the coordinate, which is what single precision holds, and it is the answer to the precision question the earlier essay left open: a direction written as a zero in the fourth slot loses nothing in the divide beyond the digits the format has. The vertex at infinity is not an ill-conditioned special case. It is the best-conditioned vertex in the triangle, because its $w$ is a depth rate of order one rather than a depth of thousands of metres. All of it is drawn through the matrix with no far plane, which one plane is nearly free found costs the depth buffer almost nothing.

The third curve is what happens to an attribute that was never given its weight.

A value at infinity draws the ground backwards #

The obvious thing to put at a far corner, if the corner is treated as an ordinary vertex, is the texture coordinate of some far stand-in point — the point twenty metres along the direction, say — with the implicit weight of 1 every value has. The interpolation is then the textbook one, and it divides that value by the vertex’s $w$.

The vertex’s $w$ is the direction’s depth rate: 0.887 for one ground direction and 0.438 for the other. An interpolation that treats that number as a depth — the error dividing depth by eye found a hand making, made here by arithmetic — is being told that the far corners are 0.89 m and 0.44 m from the eye, while the near corner is 5.93 m away. It therefore draws the texture of a triangle whose far edge is close and whose near vertex is distant. The checks crowd together at the bottom of the picture, where the ground is actually nearest, and swell toward the horizon, where it is actually farthest.

That is reverse perspective, produced by a machine rather than a painter, and the resemblance is exact rather than loose. An inverse perspective is a leaning plane found that a divergent icon’s table is the correct picture of a plane tilted the other way; the texture here is the correct picture of a ground whose depths have been inverted in their order, which is what feeding a rate into a slot that expects a depth does. The error down the principal column is 6 checks near the vertex, 102 at 10 px below the horizon, and 2,428 at the last row, and it is not a matter of choosing the stand-in better. A stand-in a kilometre out makes the checks finer but leaves the depth order just as inverted, because the stand-in changes only the value, never the $w$ it is divided by.

The failure is worth having in view because it is what a renderer does by default. A mesh format with a vertex position and a texture coordinate has no field for the weight, so a vertex at infinity written into it silently carries a weight of 1. The position is fine — the matrix draws it at the vanishing point — and the texture comes out drawn from the wrong side.

A clipper has to carry the weight too #

The triangle so far had its near vertex in front of the eye. A ground that starts beneath the viewer has a vertex behind it, and such a triangle is clipped at the near plane before the divide, which what happens behind the eye showed is not optional. The clipper cuts each edge that crosses the near plane at a fraction $t$ of its length in clip space and makes a new vertex there, mixing the endpoints’ coordinates and attributes with that fraction.

When one endpoint is a point and the other a direction, the new vertex is a mixture of a value and a rate. Mixed as the pair, it is an ordinary point with weight $1 - t$, and its texture coordinate is the mixed value over that weight. Mixed as the attribute alone, with the new vertex given a weight of 1 because every vertex a clipper writes is a point, it carries a quantity that is partly a count of checks and partly a count of checks per metre, divided by nothing.

The picture is the tell. With the weight dropped, the clipped polygon’s two triangles each carry a self-consistent but different map from page to texture, and the checks break along their shared edge — the lines of the checker bend where they cross it, and the rows on the two sides no longer meet. Down the principal column the texture is off by 0.7 checks at the bottom of the frame, 38 checks 20 px below the horizon and 1,693 at the last row. With the weight carried, the same clip and the same fan give a texture off by $10^{-14}$ checks at the bottom of the frame and $5 \times 10^{-10}$ at the last row, and the seam cannot be found, because both triangles now carry the same map.

A clip with its weight depends on nothing a renderer is free to choose. The polygon the near plane leaves has four vertices, and it can be split into two triangles two ways; with the weight carried the two splits agree to $3 \times 10^{-8}$ checks down the principal column — rounding on a coordinate of thousands — and without it they differ by up to 406 checks, since each split puts its diagonal, and its break, somewhere else. So the weight is not an optional refinement for skies and grounds. Once a vertex at infinity exists in a scene, it is part of the attribute, and every stage that touches the attribute — interpolation and clipping alike — has to carry it or produce a picture that depends on bookkeeping.

A pixel’s reach grows as the square along and linearly across #

With the coordinate exact everywhere, the remaining question the earlier essay posed is what the texture’s density on the page does as it approaches the horizon. It is a question about the map rather than about any texture, and the map answers it with two clean laws.

Take the rate of change of the texture coordinate with respect to a step of one pixel, at each row of the principal column. Down the column, along the view, it is $1224.5/y^2$ checks per pixel for a pixel $y$ rows below the horizon. Across the column it is $1.637/y$. Each constant holds from 200 px below the horizon to half a pixel below it to a part in a hundred thousand, and on logarithmic axes the two curves are straight lines of slope $-2.000$ and $-1.000$.

Both laws come from the one fact the earlier essay measured about the ground: a point on it sits below the horizon at eye level by a constant over its depth, so the depth a row sees is a constant over $y$. A step of one row changes that depth by the derivative of a constant over $y$ — the constant over $y^2$ — which is the along-view law. A step of one column moves across the ground by the depth over the focal length, a constant over $y$, which is the across law. The first is foreshortening, the second is diminution, and the horizon is where both run off to infinity at different rates. It is the same reciprocal that depth is a reciprocal found a disparity measured in, differentiated once.

The ratio of the two is the elongation of the patch of ground a pixel sees: $748/y$. At the triangle’s near vertex it is 3.7 to one. Half a pixel below the horizon it is 1,496 to one — a pixel there sees a strip of ground 3.3 checks wide and 4,898 checks long. Along the principal column a line of constant across-coordinate runs straight down the page to the horizon — elsewhere it leans, which is the angle a tilted span walks a staircase priced — so the two directions do not mix there; the measured cross terms are below a millionth of the along term, which is why the column was chosen.

The same laws say what “density on the page” means close to the horizon. Rows of checks arrive at a rate of $1224.5/y^2$ per pixel, so between any row and the horizon there are infinitely many of them, packed into a finite number of pixels: between 50 and 40 px below the horizon there are 6.1 rows of checks, between 5 and 4 px there are 61, and between one pixel and half a pixel there are 1,224. The texture on the page does not end at the horizon; it accumulates there.

Where the checks outnumber the pixels #

The half-check line in the previous figure is where the map starts putting more than one check into each pixel along one of the two directions, and because the two laws have different powers the two directions cross it at very different heights.

For one-metre checks, a pixel spans more than half a check along the view from 49.5 px below the horizon, and across the view only from 3.27 px below it. Of the 204 rows the triangle occupies on that column, 49 hold more than half a check along the view and three hold more than half a check across it. The first row is $\sqrt{2 \times 1224.5 / p}$ for a period of $p$ metres, so it moves as one over the square root of the check size: 99.0 px for quarter-metre checks, 24.7 px for four-metre ones. The second is $2 \times 1.637 / p$ and moves as one over the size itself.

The reason the two heights are so different is the elongation. Across the view, detail survives to within a few pixels of the horizon; along it, detail is lost across a quarter of the triangle’s height. A texture of planks running away from the eye — detail across the view and none along it — keeps its planks legible on the page to about three pixels below the horizon. A texture of stripes running across the view — detail along it and none across — is already more than half a stripe per pixel fifty pixels down.

Nothing in this measurement depends on the rate-and-weight arithmetic or on the vertex at infinity. It is a property of the map from page to ground, which is the same map whether the triangle stops at ten kilometres or reaches the horizon. What a renderer does with a pixel that sees 4,898 checks along its length is a question about sampling and filtering, which is a different subject from the map; the map’s part is to say how much ground each pixel sees and in what shape, and that part is exact.

Why the weight belongs to the attribute and not to the vertex #

The arithmetic above could be read as a trick for skies. It is more general than that, and the generality is what makes the clipper’s failure unsurprising.

An attribute that is linear on the ground is a linear function of the homogeneous point, not of the Cartesian one. The texture coordinate $u$ at a point $(\mathbf{X}, 1)$ becomes, at the rescaled point $(w\mathbf{X}, w)$, the pair $(w\,u, w)$, and every operation the pipeline performs on homogeneous points — mixing them in clip space, dividing through at the end — is correct for the pair exactly when it is correct for the points. That is why the pair is the right thing to clip: the clipper mixes four-component positions, and the pair is two more components of the same kind.

For an ordinary triangle whose vertices all have weight 1, the weight is invisible. It is the “one” in one over $w$, and every renderer carries it without naming it. It becomes visible only when some vertex has weight zero, and then any stage that assumed it could substitute the number 1 is wrong in a way that depends on the scene. The divide is postponed found that the pipeline’s single divide is what lets directions be ordinary vertices; this is the same observation made about attributes. The divide of an attribute by its weight is postponed to the very end for the same reason the divide of a position by its $w$ is, and a stage that performs it early — by assuming the weight is 1 — has done the division in the wrong place.

What these measurements leave alone #

Other attributes. Every figure uses a texture coordinate, which is linear on the ground. A normal, a colour or a light direction is not linear on the ground in general, and whether a rate at a direction is the right thing to carry for them was not measured.

A triangle made only of directions. The triangles here all have at least one point. A triangle whose three vertices are all at $w = 0$ has a weight of zero everywhere, and the pair has nothing to divide by; that case is the continuation below.

Filtering. The half-check rows say where a pixel’s footprint exceeds the texture’s detail. How that footprint should be averaged, and what a finite texture image does once a pixel covers all of it, belongs to sampling and was not measured.

Hardware. The single-precision curve rounds every operation of the interpolation to single precision on the CPU. A graphics processor’s interpolators and dividers are implemented in their own way and may round differently in the last place; the measurement says what the format allows, not what any device does.

Still open: a triangle with no point in it #

A sky drawn as a dome of triangles has every vertex at infinity. Each vertex is then a direction with weight zero, the interpolated weight is zero at every pixel, and the rate-and-weight pair has nothing to divide by — the triangle is a piece of the plane at infinity, and on that plane there are no distances for a rate to be measured along.

The natural attribute for such a triangle is the direction itself, since that is what a sky texture is looked up by. The direction is linear in the vertex’s homogeneous coordinates in the same way a texture coordinate is, so interpolating it over $w$ should return, at every pixel, the direction that pixel’s ray points in, up to its length. The measurement that would settle it draws a sky triangle spanning a wide part of the view, compares the interpolated direction at each pixel with the ray’s own, and asks two things: whether the agreement is exact to rounding, as the linearity argument predicts, and how much a sky texture looked up by the interpolated direction’s angle — rather than by the direction — costs across the triangle, since an angle is not linear in anything the pipeline carries.