A texture does not interpolate on the page

A surface being drawn carries things that vary across it. Where in a texture each point sits, which way the surface faces, what colour was assigned at each corner. A renderer walks the drawn shape pixel by pixel and has to know, at each pixel, what those quantities are.

The obvious way to do it is to step them by a constant amount per pixel, which takes an addition and no divisions. It is also wrong, and the size of the wrongness is not small: for a surface receding from 2 m to 20 m, the worst error is 0.52 of the whole range of whatever is being interpolated. Not five per cent. Fifty-two.

The change of parameter #

A straight world segment images as a straight page segment. That much is a projection’s defining property and this site has leaned on it since its foundation.

What does not survive is the parametrisation. Walk the world segment at a constant rate and the drawn point does not move at a constant rate; walk the drawn segment at a constant rate and the world point does not. The relation between the two is one line, and every result in this essay is a consequence of it:

$$s = \frac{t/a}{(1-t)/b + t/a}, \qquad t = \frac{a\,s}{b(1-s) + a\,s}$$

with $a$ the depth at one end, $b$ the depth at the other, $t$ the fraction along the world segment and $s$ the fraction along the drawn one.

The direction is worth reading off, because it is the opposite of most people’s first guess. At $s = \tfrac12$ — the middle of the drawn edge — the world parameter is $a/(a+b)$, which for a receding edge is less than a half. The middle of the drawn edge is nearer than the middle of the real one. For an edge from 2 m to 20 m, the page’s midpoint is only 9.1% of the way along the world segment, and it sits at world depth 3.64 m.

3.64 m is $2 \cdot 2 \cdot 20/22$: the harmonic mean of the two depths. That is the same quantity the depth buffer’s midpoint turned out to be, for the same underlying reason — something linear in $1/z$ is being halved.

The closed form, and where its peak is #

The error made by assuming $t = s$ is $|t(s) - s|$. Differentiating with $k = b/a$:

$$\frac{\mathrm{d}t}{\mathrm{d}s} = \frac{k}{\big(k - s(k-1)\big)^2}$$

Setting that to one gives $s^\star = \sqrt{k}/(\sqrt{k}+1)$, and there

$$\big|t - s\big|_{\max} = \frac{\sqrt{k}-1}{\sqrt{k}+1}$$

The maximum depends on the depth ratio and on nothing else. Not on the focal length, not on the orientation of the surface in the picture, not on how large the surface is drawn. Two surfaces spanning the same depth ratio make the same worst error whatever else is different about them.

At a depth ratio of four the error is exactly one third, which is a pleasant enough coincidence to be worth checking: $(\sqrt4 - 1)/(\sqrt4+1) = 1/3$, so yes, exactly.

The error is a hyperbolic tangent of the log ratio #

The maximum $(\sqrt{k}-1)/(\sqrt{k}+1)$ has a second form that makes every number in the table above follow from one another, and it is worth writing because it explains the shape of the whole subject.

Put $k = e^{L}$, so $L = \ln k$ is the depth ratio measured in logarithms. Then

$$\frac{\sqrt{k}-1}{\sqrt{k}+1} = \frac{e^{L/2}-1}{e^{L/2}+1} = \tanh\frac{L}{4}.$$

Checked against the table: $\tanh(\ln 2 / 4) = 0.1716$, $\tanh(\ln 4/4) = 0.3333$, $\tanh(\ln 10/4) = 0.5195$, $\tanh(\ln 20/4) = 0.6345$. All four, exactly.

Three things follow that the square-root form hides.

The natural variable is the logarithm of the depth ratio. Going from 2 : 1 to 4 : 1 and from 4 : 1 to 8 : 1 are the same step in $L$, and the errors are 0.172, 0.333 and 0.462 — increments of 0.161 and 0.129, falling because a tanh flattens. So the error is not “proportional to how deep the surface is” in any useful sense; it is proportional to $L$ while $L$ is small and saturates while it is large.

And it saturates at 1 without reaching it, which is a different shape of limit from the ones the depth buffer’s precision runs into — that one degrades without bound as the square of the distance, while this one has a ceiling it approaches and never crosses. Two artefacts of the same postponed divide, one unbounded and one saturating, and the difference is whether the quantity being spoiled is a difference or a ratio. A surface spanning 100 : 1 has a worst error of 0.818, one spanning 10,000 : 1 has 0.980, and no surface has 1. The interpolated quantity can be wrong by almost the whole of its range and never by the whole of it — which is the honest ceiling on a failure that is usually described as unbounded.

The subdivision law is the same statement. Cutting a surface into $n$ pieces of equal ratio divides $L$ by $n$, so each piece’s error is $\tanh(L/4n)$ — which is the expression the section below quotes, derived rather than fitted. And for large $n$, $\tanh x \approx x$, so the error falls as $\ln k / 4n$: the linear-in-$1/n$ behaviour, with the constant being the log ratio and not the ratio.

That the answer is a tanh of a logarithm is also a small piece of evidence about what kind of quantity this is. A tanh of half a log-ratio is exactly the form a rapidity takes, and it appears here for the ordinary reason: the change of parameter is a Möbius map, Möbius maps compose by adding log-ratios, and the largest departure from the identity is a function of that sum alone. Which is why the same expression turns up wherever a projective map’s departure is being measured and why the depth ratio is the only thing in it.

The fix, and why the pipeline was already carrying it #

The correction is to interpolate two quantities that are linear in screen position, and divide at the very end:

$$u(s) = \frac{(1-s)\,u_0/a + s\,u_1/b}{(1-s)/a + s/b}$$

Both the numerator and the denominator are linear in $s$, so both can be stepped by a constant amount per pixel. One division per pixel buys exactness — the gate measures the residual at $10^{-16}$, which is the noise floor.

Why those two? Because $1/z$ is what a projection makes linear across the page. That is the same fact that the depth buffer stores an affine function of $1/z$, that the horizon is the image of the ground plane’s line at infinity, and that a picture with no size–distance signal cannot be read for depth. The reciprocal of depth is the quantity a perspective picture is linear in, and every convenience in this pipeline is somebody exploiting that.

Which is the third reason the divide is postponed. Clipping needs the fourth coordinate’s sign; edge interpolation needs its linearity; and this needs its value, at the very last step, per pixel. The first essay of this field sets out all three together, and this is the one that reaches furthest down the pipeline.

The control, and why the bug survives testing #

Set $a = b$. The depth ratio is 1, the closed form gives $(\sqrt1 - 1)/(\sqrt1+1) = 0$, and the change of parameter is the identity: $t = s$ exactly.

So a renderer with no perspective correction at all draws a fronto-parallel surface perfectly. A wall square to the camera, a heads-up display, a piece of user interface, a billboard turned to face the viewer — all exact, to one part in $10^{16}$.

That is not a footnote. It is the reason the bug is famous rather than obvious, and it has a distinctive signature: a system looks entirely correct until somebody turns a surface away from the camera, and then it is dramatically wrong. There is no gentle degradation, because the error is governed by the depth ratio and a surface square to the camera has a depth ratio of exactly one.

The gate asserts this as a control rather than mentioning it, and the assertion had to be weakened once for an honest reason: at $a = b$ the two routes are the same function, and what separates them numerically is the rounding of a division that cancels algebraically — one ulp, $1.1 \times 10^{-16}$. Writing === 0 passed on the first draft only because the sweep happened to miss the value that rounds, which is a check of the sample list rather than of the claim.

The same error, made by a person #

This site already has an essay about a human being making exactly this mistake, and it was written three phases before there was any machine in the subject.

Dividing depth by eye measures what happens when somebody spaces the transversals of a receding pavement by judgement rather than by construction. The commonest by-eye method is to space them at a constant ratio, and the essay’s finding is that a constant ratio is not what perspective does — the drift accumulates and the depths implied by the drawn spacings run away from the depths a camera would produce.

The very simplest by-eye method is to space them equally on the page, and that is screen-space interpolation exactly. Same assumption, same error, same closed form. The only difference is that a person doing it produces a slightly wrong drawing of a pavement, and a renderer doing it produces a texture that swims.

There is a genuinely interesting difference in what the two mistakes reveal. The person’s version was, historically, a method — an attempt to get perspective without computing it, and one that is easy to defend because the picture looks nearly right. The machine’s version is a shortcut for speed, and it looks nearly right for the same reason. In both cases the defence is aesthetic and the refutation is a number.

Subdivision, and why it nearly works #

Before the divide-at-the-end correction became universal, the standard workaround was subdivision: cut the surface into smaller pieces, interpolate linearly across each one, and rely on each piece spanning a small depth ratio.

It is worth pricing, because the closed form gives the answer immediately and the answer explains both why the technique was used and why it was abandoned.

Splitting a surface spanning ratio $k$ into $n$ pieces of equal depth ratio gives each piece a ratio of $k^{1/n}$, so the worst error per piece falls as

$$\frac{k^{1/2n} - 1}{k^{1/2n} + 1}$$

For $k = 10$: one piece has an error of 0.519, two pieces 0.290, four pieces 0.152, eight 0.077, sixteen 0.039. Halving the error costs a doubling of the geometry, for ever.

That is the classic shape of a workaround that is good enough at first and never good enough later. Four subdivisions get the error below a sixth, which on a coarse display is invisible; getting it below a hundredth needs about sixty-four, which is sixty-four times the vertex work to avoid one division per pixel.

Where the cuts are put matters as much as how many there are, and “equal pieces” is ambiguous in exactly the way this essay is about. Splitting a surface running from $a$ to $b = ka$ into $n$ pieces of equal depth gives a first piece of ratio $1 + (k-1)/n$ — for $k = 10$ and four pieces, a ratio of 3.25 and a worst error of 0.286. Splitting it into pieces of equal depth ratio gives every piece $k^{1/n} = 1.78$ and a worst error of 0.143. Same piece count, half the error, and the difference is entirely which parameter the cuts were made in.

The asymptotics say the gap widens rather than closing. For large $n$ the equal-depth split’s error is about $(k-1)/4n$ and the equal-ratio split’s is about $\ln k / 4n$: both fall as $1/n$, and the constants are 2.25 against 0.58 at $k = 10$, and 24.8 against 1.15 at $k = 100$. So the deeper the surface, the more the choice of cut matters — a factor of four at ten to one and a factor of twenty at a hundred to one.

Which is worth pausing on, because the correct cut is not the intuitive one. Equal-ratio pieces are equally spaced in $\log z$, so a subdivided surface should be cut densely near the camera and sparsely far away — the opposite of the instinct to put more geometry where more of the screen is being covered. And it is a second instance of the essay’s own subject: choosing where to cut in world depth rather than in the parameter the projection is linear in is the same mistake as interpolating in the parameter the page is linear in, made one level up. Splitting a surface equally in the world to fix a screen-space error is fixing one non-uniformity with another.

And subdivision has a defect the arithmetic does not show. The error does not vanish at the joins — it vanishes at the ends of each piece, so the reconstructed function is continuous and its slope is not. A subdivided surface has a kink at every join, and a moving camera sweeps those kinks across the surface. The visible result is not blur but a faint ripple, which is harder to attribute and easier to argue about than the swimming it replaced.

What is being interpolated does not matter #

One thing worth saying plainly, because the standard name for this — perspective-correct texture mapping — makes it sound like a fact about textures.

Nothing in the derivation mentions what is being carried. It is a statement about the change of parameter between a world segment and its image, so it applies identically to a texture coordinate, a surface normal, a per-vertex colour, a depth value, a light intensity, or any other quantity that varies linearly across the real surface.

That generality is why the fourth coordinate is worth carrying rather than special-casing. One correction, applied to every varying quantity, at the point where they are all being stepped anyway.

And it is why the error’s signature is so recognisable when it appears: everything a slanted surface carries is wrong together, in the same direction, by the same amount — so the surface does not look noisy, it looks like it is sliding.

The barycentric version, and the one thing it adds #

Everything so far is about an edge. A surface is a triangle, and the correction across a triangle is the same statement in three terms rather than two:

$$u = \frac{\sum_i \lambda_i\,u_i / z_i}{\sum_i \lambda_i / z_i}$$

with $\lambda_i$ the barycentric coordinates measured on the page. Set two of the three to zero and it collapses to the edge formula, which is the check that the generalisation is the same object.

One thing is worth extracting from the three-term version that the two-term version hides. The denominator $\sum \lambda_i / z_i$ is the interpolated value of $1/z$ — so a renderer computing perspective-correct anything is, as a side effect, computing the depth at that pixel, in exactly the form the depth buffer wants it. The reciprocal depth is not an extra quantity carried for the correction; it is the same number the depth test needs, and the correction is what makes it available.

That is a satisfying piece of economy and it also explains a historical accident. Early hardware that interpolated depth linearly in screen space and textures linearly in screen space was internally consistent and wrong about both, in the same way, for the same reason — and fixing either one required the other, because they share the divisor.

Why the depth ratio is the whole story #

It is worth ending on the one substantive surprise here, which is how little the error depends on.

Not on the focal length: a wide lens and a long one make the same error on a surface spanning the same depth ratio. Not on the field position: a slanted surface at the edge of the frame and one in the middle behave identically. Not on the size of the drawn shape: a slanted surface filling the frame and one twenty pixels across make the same fractional error.

The depth ratio, and only the depth ratio. Which is a statement that this is a projective fact rather than an imaging one — it survives any change of intrinsics, any change of viewpoint that preserves the two depths, and any change of what is being carried across the surface. That is the same kind of statement as the cross-ratio surviving a projection, and it is worth having for the same reason: something that depends on almost nothing is something that can be relied on.

The quantity that does interpolate linearly across the page is the reciprocal, which is what depth is in a picture, and the whole correction is the observation that the texture coordinate has to be carried in that currency rather than in the one the drawing is measured in.