A pixel is not a point

The last step of the pipeline is arithmetic that looks like nothing. Normalised device coordinates run $-1$ to $+1$; the viewport is $W$ pixels wide; so

$$u = \frac{(x_{\text{ndc}} + 1)\,W}{2}$$

and the point has landed on the picture. There is no geometry left in this step, no projection, no divide. It is a change of units.

It is also where two independent half-pixel mistakes live, and this site is in an unusual position to say what either one costs — because it does not merely draw pictures, it recovers cameras from them. A half pixel is not a quantity a picture can be inspected for. It is a quantity a recovery can name, and the two mistakes turn out to be named differently.

What a pixel actually is #

A pixel is not a point. It is a small square area of the picture, and everything about it that matters here follows from that being taken literally.

A picture $W$ pixels wide covers the interval from 0 to $W$, and pixel $i$ occupies $[i, i+1)$. Its centre — the place a sample taken to represent it should be taken — is at $i + \tfrac12$. Its corner is at $i$.

Those two differ by half a pixel and both are perfectly reasonable-looking things to write. i is what a loop index supplies. i + 0.5 requires knowing why. So the mistake is not exotic: it is the default that arises from writing the obvious loop.

The first mistake: sampling at the corner #

Take the correct pipeline and change one thing — treat the pixel’s corner as its sample point rather than its centre. Every mark in the picture moves by $-\tfrac12$ px in $x$ and $-\tfrac12$ px in $y$, so by $\sqrt{1/2} = 0.7071$ px along the diagonal.

Two facts follow and the gate asserts both.

Every mark moves by the same amount. The spread across the eight vertices of the test box is 0.0 px exactly — not small, zero — because the operation is a translation and a translation moves everything equally. That is worth checking rather than assuming, because a change that moved marks by different amounts would be a distortion and would need a completely different account.

And a recovery reads it as a principal point error. Run the site’s round trip on the shifted picture: three bundles of parallel edges, three vanishing points, and the focal length and principal point that follow. The focal length comes back at 898.755727 px against a true 898.755727 — identical to six decimal places, and in fact to the last bit. The principal point comes back 0.707107 px away.

That is not a coincidence and it is not an approximation. A translation of the image plane is exactly a translation of the principal point: the picture is a perfectly good projection, of a camera whose principal point is half a pixel from where the renderer believes it is.

The second mistake: mapping onto W − 1 pixels #

The second error is quieter and it is the more common one, because it comes from a genuine ambiguity about what “the width of the picture” means.

A picture $W$ pixels wide has pixel indices $0$ to $W-1$. So the mapping from NDC to pixels is sometimes written to send $-1 \mapsto 0$ and $+1 \mapsto W - 1$, which puts the extreme device coordinates at the centres of the outermost pixels rather than at the outer edges of the picture.

That is not a translation. It is a scaling, by $(W-1)/W$, and at $W = 690$ that is 0.99855 — a shortfall of 0.145%.

And a recovery names it differently. The focal length comes back short by exactly that factor: 1.30 px on a focal length of 899. The principal point does not move at all, because the scaling is about the picture’s centre, which is where the principal point already is.

So the two half-pixel mistakes are orthogonal in exactly the sense a calibration cares about:

Two errors of the same size in pixels, one affecting each of the two things a camera recovery returns. Neither is visible in the picture.

Is 1.3 px a lot? #

It depends entirely on what the picture is for, and this site is one of the few places where the question has both answers written down.

As a picture, it is nothing. A misplaced principal point of 0.7 px shifts everything by less than the width of a hairline. No viewer will ever see it, and no figure on this site would look different.

As a measurement, it is not nothing. Every essay in the metrology field reads a scene out of a picture, and every one of them is sensitive to the intrinsics. A height from a photograph is a cross-ratio along a vertical, which the principal point enters through the horizon. The plumb-line fit recovers a lens’s distortion by minimising the sag of lines that ought to be straight, and it is measuring sags of a few pixels — so a systematic half-pixel bias is a substantial fraction of the signal.

And it is a systematic error, which is the important word. Random error at half a pixel averages away over many measurements. A convention error does not: it is the same half pixel in the same direction in every picture the pipeline ever produces, and averaging a thousand of them gives the same wrong answer with a smaller standard error.

Why the recovery is the right instrument for this #

There is a more direct way to detect a half-pixel error: render a scene twice with the two conventions and subtract. That works, it takes one line, and it reports almost nothing — that the pictures differ by half a pixel, which was already known because the difference was introduced deliberately.

Recovering the camera reports something that was never put in: what kind of camera the wrong picture is a picture of. The answer — a real camera, with an intact focal length and a displaced principal point — is not obvious in advance, and it is the answer that decides what to do about it.

If a half-pixel convention error produced a picture that was not a projection of anything, the fix would be urgent and detectable. It produces a picture that is a perfect projection of a slightly different camera, which means:

• no residual anywhere reports it — the drawn edges are exactly concurrent;
• no consistency check reports it — the three independent focal estimates agree to 1e-16;
• and any calibration performed on such pictures will recover the shifted principal point, faithfully, and be right about the camera it was actually shown.

That last one is the interesting case, and it is genuinely benign: a system that both renders and calibrates with the same wrong convention is internally consistent and its measurements are correct. The error only bites when a picture made under one convention is measured under another — which is exactly what happens whenever an image passes between two pieces of software.

The convention this site uses, and why it is stated #

This site’s camera puts the principal point at $(W/2, H/2)$ by default and projects to continuous coordinates, and the pipeline maps NDC 0 to $W/2$ — the boundary between the two middle pixels of an even-width picture, not the centre of any pixel.

That is the centre convention and it is the one that makes the matrix pipeline agree with the pinhole to $5.7 \times 10^{-14}$ px. The corner convention is implemented too, because an essay measuring a wrong convention needs the wrong convention to exist, and a claim about what a mistake costs is worthless if the mistake was never actually made.

The general form of that is a habit this whole site runs on: an assertion that has never rejected anything proves nothing, so every gate here ends by feeding its machinery input it must refuse. The corner convention is that input, for this claim.

Why the two errors do not simply add #

A pipeline could easily make both mistakes at once, and it is worth working out what a recovery would then say, because the answer is not the sum of the two rows in the table.

The corner convention translates; the $W-1$ convention scales about the picture’s centre. Applying a scale and then a translation gives a map whose fixed point is not the picture’s centre, so the composite is a scaling about a slightly different point — which a recovery reports as both a shortened focal length and a displaced principal point, in a proportion that depends on the order the two were applied in.

So the two errors are separable in principle and entangled in practice. A calibration performed on pictures from such a pipeline recovers a self-consistent camera; comparing that camera against the one the renderer believes it has gives a discrepancy in both parameters; and there is no way, from the discrepancy alone, to say which of the two mistakes was made or whether both were.

That matters because the natural debugging move — measure the discrepancy and correct it — works and teaches nothing. The camera is corrected, the pipeline still has two bugs in it, and the next picture rendered at a different resolution is wrong again, because one of the two errors scales with $W$ and the other does not.

That is the diagnostic. Render the same scene at two resolutions and recover the camera from each. A corner-convention error gives a principal-point offset of 0.707 px at both, in absolute terms. A $W-1$ error gives a focal-length shortfall of one part in $W$ — half as large, in relative terms, at twice the resolution. One is constant in pixels and one is constant in fractions, and comparing two resolutions separates them.

The half pixel and the pixel grid are different subjects #

One boundary worth drawing, because the two get conflated and only one of them is this site’s.

Where a sample sits is geometry: it is a statement about which point of the continuous picture a stored value represents, and it has an exactly right answer that this essay computes.

What happens when a continuous picture is reduced to samples is not geometry. Aliasing, filtering, reconstruction and the whole sampling literature are a different subject with different mathematics, and this site takes no position on any of it. The fleet’s register of which site owns which argument records the boundary, and the test that separates the two is this: every argument here survives with every sample taken exactly, and a sampling argument does not.

The distinction is easy to hold onto by asking what the answer depends on. A half-pixel offset is decided by the coordinate convention alone and is the same for every scene. Everything in the sampling literature depends on what is being sampled.

The general lesson, which is about units #

Both mistakes in this essay are unit errors in the strict sense: a quantity measured from one origin used as though it were measured from another, and a quantity counting intervals used as though it counted endpoints.

That second one — the fencepost — is the more instructive, because it recurs everywhere and is genuinely ambiguous rather than merely careless. A picture 690 px wide has 690 pixels and 691 pixel boundaries, and both counts are the right answer to some question. NDC $\pm 1$ maps to the boundaries; pixel indices 0 to 689 count the pixels. Choosing wrong is not a slip, it is answering the wrong one of two reasonable questions.

The reason it matters here rather than being a curiosity is the one this whole site is built on: a picture does not carry the conditions under which it is a picture of anything. It does not say what its focal length was, where its principal point is, or which of two pixel conventions its coordinates are in. All of that has to be stated separately — and when it is not, the picture goes on looking exactly right while the measurements made from it drift by an amount nobody can see.