The divide is postponed, not avoided

Every projection on this site so far has been one line of arithmetic. A world point goes into the camera’s own coordinates, and then

$$u = c_x + f\,\frac{x}{z}, \qquad v = c_y + f\,\frac{y}{z}$$

and that is the whole of it. Two divisions, and the picture exists.

A renderer does not do that. It multiplies the point by a four-by-four matrix, gets four numbers back, and divides by the fourth one somewhere further down the line — after a clipper has looked at all four, and in the case of a textured surface, not until the very last step of drawing each pixel. The divide is not skipped. It is postponed, and the interval between the multiply and the divide is where four of this field’s five essays live.

Why this needs establishing rather than assuming #

It would be easy to treat this as a formality. The projection matrix is in every graphics text; its derivation is standard; nobody doubts that it produces a perspective picture.

But this site’s whole method is that a figure is projected from a stated camera rather than constructed by a hand that knew what the answer should look like, and a second projector introduced without comparison is a second hand. If the matrix pipeline and the site’s own camera differ anywhere — a sign, a convention, a half-pixel — then every measurement in the three fields that follow is a measurement of a camera that appears nowhere else on the site, and the comparisons with the earlier fields would be comparisons of two different things.

So the matrix is built here in the site’s own camera coordinates, and the agreement is checked and printed rather than asserted in prose. The site’s gate for this field runs that comparison at seven fields of view from 18° to 130°, and the worst disagreement anywhere is 5.7 × 10⁻¹⁴ px.

The matrix, with nothing hidden in it #

The usual presentation builds the projection matrix from a field of view, an aspect ratio, and a pair of clipping planes. That is a perfectly good recipe and it obscures the one thing worth seeing, which is that every entry is a focal length or a principal point. Written in the coordinates this site’s camera already uses — $x$ right, $y$ down the page, $z$ along the view — the matrix is

$$M = \begin{pmatrix} 2f/W & 0 & 2c_x/W - 1 & 0 \\ 0 & -2f/H & 1 - 2c_y/H & 0 \\ 0 & 0 & A & B \\ 0 & 0 & 1 & 0\end{pmatrix}$$

with $A$ and $B$ the two numbers that map the near and far planes onto the ends of the depth range. Multiply a world point written as $(x, y, z, 1)$ and four numbers come out, conventionally called $x_c, y_c, z_c, w_c$ — clip coordinates.

The fourth of them is $z$. That is the entire trick: the bottom row of the matrix is $(0, 0, 1, 0)$, so $w$ is the depth, carried forward untouched. Everything a graphics text says about homogeneous coordinates and the projective plane is true and is not needed to see what is happening here. The matrix has arranged for the depth to survive the multiplication, so that the division by it can happen when somebody is ready.

The first two rows are the pinhole with the divide taken out and a rescaling to the range $-1 \ldots 1$ applied. The second row is negated because normalised device coordinates put $y$ upward and this site’s images put it down the page — two conventions, one sign, and no content.

The first reason to wait: linearity #

Clip coordinates are linear in the world point, and the pixel coordinates are not.

That distinction is the whole justification for the postponement, and it is checkable rather than rhetorical. Take two world points, take their midpoint, and run all three through the matrix: the midpoint’s clip coordinates are the average of the other two’s, to $2 \times 10^{-16}$. Then take the same three points’ pixels: the midpoint’s pixel is 22.1 px away from the average of the other two.

The site has been saying this since its foundation, from the other side. A projection destroys the ratio in which a point divides a segment — that is the reason a perspective picture cannot be measured with a ruler, and the reason the cross-ratio is worth as much as it is. What the matrix does is delay the destruction. Before the divide, everything is still affine and midpoints are still midpoints; after it, they are not.

Anything a renderer wants to do by interpolation — clipping a segment at a plane, walking a triangle’s edge, blending a vertex attribute — is easier on the side of the divide where interpolation is honest. That is not a small convenience. It is the difference between a linear equation and a rational one at every step of the pipeline.

The second reason to wait: points behind the eye #

This site’s camera refuses to project a point at or behind the eye plane, and the reason is written beside it: returning nothing is “a refusal rather than a wrapped-around point appearing somewhere plausible in the picture.”

That refusal is not fussiness. Divide $x_c$ and $y_c$ by a negative $w$ and both signs flip, so a point behind the eye lands through the principal point on the far side of the frame, at a perfectly ordinary-looking position with nothing whatever to mark it as impossible. A segment with one end in front and one behind is then drawn as a line running the wrong way across the whole picture — straight, inside the frame, and reversed.

A clipper is what makes that impossible, and it can only work before the divide, because before the divide both ends of the segment are still finite numbers with a sign that says which side of the eye they are on. After the divide, the far endpoint has already become nonsense and there is nothing left to interpolate along.

The next essay is about that picture and nothing else. What matters here is that the reason for the postponement and the reason for the clipper are the same reason, stated twice: the fourth coordinate is the only thing that knows which side of the eye a point is on, and dividing by it throws that knowledge away.

The third reason to wait: what a surface carries #

The third reason is the one this field’s last essay is about, and it is worth naming now because it explains why $w$ is carried past the clipper and all the way into the drawing of individual pixels.

A surface being drawn carries quantities that vary across it — where in a texture each point sits, which way the surface faces, what colour it was assigned at each corner. Interpolating those linearly across the page is wrong, and the size of the error is a closed form: for a surface receding from depth $a$ to depth $b$, the worst departure is

$$\frac{\sqrt{k}-1}{\sqrt{k}+1}, \qquad k = b/a$$

which at a depth ratio of ten is 0.52 — more than half the whole range of whatever is being interpolated.

The fix is to interpolate $u/w$ and $1/w$ across the page, both of which really are linear there, and divide at the very end. So $w$ has to still exist at the point where individual pixels are being written, which is about as late as a postponement can be taken.

The depth row, which is the only part with content of its own #

The first two rows of the matrix are the pinhole rearranged. The third row is not — it is a decision, and a consequential one.

It maps camera depth onto a value that will be stored and compared, and the mapping chosen is

$$z_{\text{ndc}} = A + \frac{B}{z}$$

with $A = (f + n)/(f - n)$ and $B = -2fn/(f-n)$ for near plane $n$ and far plane $f$. That is affine in $1/z$, not in $z$, and that single choice is why a depth buffer spends ninety per cent of its precision inside the first metre.

The choice is not arbitrary, and it follows from the postponement itself. Whatever goes in the third row has to survive the same divide by $w = z$ that the first two rows do, and a value that is affine in $1/z$ before the divide is exactly a value that is affine in $z$ after it — which is what a matrix can produce. A depth that was linear in $z$ after the divide would need a matrix entry that was quadratic in $z$ before it, and a matrix does not have one.

So the depth buffer’s famous lopsidedness is a consequence of a decision made three steps earlier for a completely different reason. That is worth stating plainly because it is the shape of most of the findings in this field: the machine’s departures from the ideal projection are not carelessness, they are the price of doing the projection in a particular order.

The near and far planes are not a rendering detail #

A pinhole camera has no near plane and no far plane. Every point in front of the eye has an image, and points at any distance have perfectly good ones — the whole metrology field depends on that, because a measurement made from a picture cannot have a nearest measurable object.

The matrix has two, and they are the price of putting the depth in a matrix row at all. Fixing $A$ and $B$ requires naming two distances, and every consequence in this field’s third essay follows from which two.

The near plane also has a geometric meaning worth stating rather than treating as a parameter. It is where the clipper cuts, so it is the closest anything can be and still be drawn — and setting it too small is the commonest way to ruin a depth buffer. Setting it too large cuts holes in things the camera is nearly touching. Neither has an obviously right answer and the trade is quantified in the essay on what precision is left.

What the matrix is not #

Two things this presentation deliberately avoids, because both are the sort of claim that sounds deep and is not.

It is not “perspective is a linear transformation”. Perspective is not linear and the matrix does not make it so; the matrix is a linear step followed by a nonlinear one, and the nonlinearity is exactly where it always was.

And it is not a statement about projective geometry that this site needs. Homogeneous coordinates are a genuinely beautiful idea and their power here is entirely practical: they let one matrix represent a projection, they make a point at infinity a value rather than a special case, and they let the divide be deferred. The site already uses the second of those constantly — a vanishing point is the image of a point at infinity, and this site’s projective machinery has represented lines as three numbers since the first commit for exactly that reason.

The check that matters, and the one that would not #

Comparing the matrix pipeline with the pinhole is worth something precisely because the two are written independently — one divides on the spot and one carries four coordinates through a clipper — and it would be worth nothing if the pipeline had been written by expanding the pinhole’s formula.

There is a second check with the same shape and more content, and it is the site’s own. Draw the box with the matrix, throw the camera away, and recover it from the drawn edges: three bundles of parallel edges give three vanishing points, and three mutually orthogonal directions determine the focal length and the principal point. The recovery is shown the picture and never the camera.

It does survive, at the same precision the pinhole does. That is the licence the rest of this field runs on: every departure measured in the next four essays is a departure from this camera, which is the one the whole site is built on.

What the postponement actually buys #

Set the three reasons side by side and one thing is common to all of them.

Clipping needs the depth’s sign, which the divide destroys. Interpolation across an edge needs linearity, which the divide destroys. Interpolation across a surface needs the depth itself, which the divide consumes.

Each of the three wants the same thing — the depth, still present, still separate — and the matrix’s fourth coordinate is what supplies it to all three from one multiplication. That is a genuinely good piece of engineering and it is worth admiring for the right reason: not because a 4×4 matrix is elegant, but because three unrelated problems turn out to have one prerequisite and it can be met once.

The cost is that the depth then has to be stored in some finite number of bits, mapped by a row of a matrix that could not be anything but affine in $1/z$, and compared against other depths stored the same way. Which is the next-but-one essay, and is where this field stops being an appreciation and starts being a measurement.