A pixel is not a point

The last step of the pipeline is arithmetic that looks like nothing. Normalised device coordinates run $-1$ to $+1$; the viewport is $W$ pixels wide; so

$$u = \frac{(x_{\text{ndc}} + 1)\,W}{2}$$

and the point has landed on the picture. There is no geometry left in this step, no projection, no divide. It is a change of units.

It is also where two independent half-pixel mistakes live, and this site is in an unusual position to say what either one costs — because it does not merely draw pictures, it recovers cameras from them. A half pixel is not a quantity a picture can be inspected for. It is a quantity a recovery can name, and the two mistakes turn out to be named differently.

What a pixel actually is #

A pixel is not a point. It is a small square area of the picture, and everything about it that matters here follows from that being taken literally.

A picture $W$ pixels wide covers the interval from 0 to $W$, and pixel $i$ occupies $[i, i+1)$. Its centre — the place a sample taken to represent it should be taken — is at $i + \tfrac12$. Its corner is at $i$.

Those two differ by half a pixel and both are perfectly reasonable-looking things to write. i is what a loop index supplies. i + 0.5 requires knowing why. So the mistake is not exotic: it is the default that arises from writing the obvious loop.

The first mistake: sampling at the corner #

Take the correct pipeline and change one thing — treat the pixel’s corner as its sample point rather than its centre. Every mark in the picture moves by $-\tfrac12$ px in $x$ and $-\tfrac12$ px in $y$, so by $\sqrt{1/2} = 0.7071$ px along the diagonal.

Two facts follow and the gate asserts both.

Every mark moves by the same amount. The spread across the eight vertices of the test box is 0.0 px exactly — not small, zero — because the operation is a translation and a translation moves everything equally. That is worth checking rather than assuming, because a change that moved marks by different amounts would be a distortion and would need a completely different account.

And a recovery reads it as a principal point error. Run the site’s round trip on the shifted picture: three bundles of parallel edges, three vanishing points, and the focal length and principal point that follow. The focal length comes back at 898.755727 px against a true 898.755727 — identical to six decimal places, and in fact to the last bit. The principal point comes back 0.707107 px away.

That is not a coincidence and it is not an approximation. A translation of the image plane is exactly a translation of the principal point: the picture is a perfectly good projection, of a camera whose principal point is half a pixel from where the renderer believes it is.

The second mistake: mapping onto W − 1 pixels #

The second error is quieter and it is the more common one, because it comes from a genuine ambiguity about what “the width of the picture” means.

A picture $W$ pixels wide has pixel indices $0$ to $W-1$. So the mapping from NDC to pixels is sometimes written to send $-1 \mapsto 0$ and $+1 \mapsto W - 1$, which puts the extreme device coordinates at the centres of the outermost pixels rather than at the outer edges of the picture.

That is not a translation. It is a scaling, by $(W-1)/W$, and at $W = 690$ that is 0.99855 — a shortfall of 0.145%.

And a recovery names it differently. The focal length comes back short by exactly that factor: 1.30 px on a focal length of 899. The principal point does not move at all, because the scaling is about the picture’s centre, which is where the principal point already is.

So the two half-pixel mistakes are orthogonal in exactly the sense a calibration cares about:

Two errors of the same size in pixels, one affecting each of the two things a camera recovery returns. Neither is visible in the picture.

Is 1.3 px a lot? #

It depends entirely on what the picture is for, and this site is one of the few places where the question has both answers written down.

As a picture, it is nothing. A misplaced principal point of 0.7 px shifts everything by less than the width of a hairline. No viewer will ever see it, and no figure on this site would look different.

As a measurement, it is not nothing. Every essay in the metrology field reads a scene out of a picture, and every one of them is sensitive to the intrinsics. A height from a photograph is a cross-ratio along a vertical, which the principal point enters through the horizon. The plumb-line fit recovers a lens’s distortion by minimising the sag of lines that ought to be straight, and it is measuring sags of a few pixels — so a systematic half-pixel bias is a substantial fraction of the signal.

And it is a systematic error, which is the important word. Random error at half a pixel averages away over many measurements. A convention error does not: it is the same half pixel in the same direction in every picture the pipeline ever produces, and averaging a thousand of them gives the same wrong answer with a smaller standard error.

Why the recovery is the right instrument for this #

There is a more direct way to detect a half-pixel error: render a scene twice with the two conventions and subtract. That works, it takes one line, and it reports almost nothing — that the pictures differ by half a pixel, which was already known because the difference was introduced deliberately.

Recovering the camera reports something that was never put in: what kind of camera the wrong picture is a picture of. The answer — a real camera, with an intact focal length and a displaced principal point — is not obvious in advance, and it is the answer that decides what to do about it.

If a half-pixel convention error produced a picture that was not a projection of anything, the fix would be urgent and detectable. It produces a picture that is a perfect projection of a slightly different camera, which means:

• no residual anywhere reports it — the drawn edges are exactly concurrent;
• no consistency check reports it — the three independent focal estimates agree to 1e-16;
• and any calibration performed on such pictures will recover the shifted principal point, faithfully, and be right about the camera it was actually shown.

That last one is the interesting case, and it is genuinely benign: a system that both renders and calibrates with the same wrong convention is internally consistent and its measurements are correct. The error only bites when a picture made under one convention is measured under another — which is exactly what happens whenever an image passes between two pieces of software.

The convention this site uses, and why it is stated #

This site’s camera puts the principal point at $(W/2, H/2)$ by default and projects to continuous coordinates, and the pipeline maps NDC 0 to $W/2$ — the boundary between the two middle pixels of an even-width picture, not the centre of any pixel.

That is the centre convention and it is the one that makes the matrix pipeline agree with the pinhole to $5.7 \times 10^{-14}$ px. The corner convention is implemented too, because an essay measuring a wrong convention needs the wrong convention to exist, and a claim about what a mistake costs is worthless if the mistake was never actually made.

The general form of that is a habit this whole site runs on: an assertion that has never rejected anything proves nothing, so every gate here ends by feeding its machinery input it must refuse. The corner convention is that input, for this claim.

Why the two errors do not simply add #

A pipeline could easily make both mistakes at once, and it is worth working out what a recovery would then say, because the answer is not the sum of the two rows in the table.

The corner convention translates; the $W-1$ convention scales about the picture’s centre. Applying a scale and then a translation gives a map whose fixed point is not the picture’s centre, so the composite is a scaling about a slightly different point — which a recovery reports as both a shortened focal length and a displaced principal point, in a proportion that depends on the order the two were applied in.

So the two errors are separable in principle and entangled in practice. A calibration performed on pictures from such a pipeline recovers a self-consistent camera; comparing that camera against the one the renderer believes it has gives a discrepancy in both parameters; and there is no way, from the discrepancy alone, to say which of the two mistakes was made or whether both were.

That matters because the natural debugging move — measure the discrepancy and correct it — works and teaches nothing. The camera is corrected, the pipeline still has two bugs in it, and the next picture rendered at a different resolution is wrong again, because one of the two errors scales with $W$ and the other does not.

A resolution sweep is the natural thing to try here and it does not work, for a reason worth setting out, because the reason says what the right sweep is.

The sweep that separates them is the lens, not the resolution #

Rendering at two resolutions looks like the obvious way to tell the two errors apart, and the arithmetic says it separates nothing at all.

The corner error is half a pixel in each axis, so it is 0.7071 px at every resolution — constant in pixels by construction. The $W-1$ error scales the picture by $(W-1)/W$, so the focal length is short by $f/W$ pixels. And for a fixed field of view $f = (W/2)/\tan(\theta/2)$, so

$$\frac{f}{W} = \frac{1}{2\tan(\theta/2)}$$

which has no $W$ in it. Both errors are constant in absolute pixels and both are one part in $W$ in relative terms, at every resolution, so doubling the picture’s width moves neither of them relative to the other. A two-resolution comparison reports the same discrepancy twice.

What does separate them is the field of view, and the expression above is why. The principal-point offset is 0.7071 px whatever the lens; the focal-length shortfall is $1/(2\tan(\theta/2))$ px, which is 0.50 px on a 90° lens, 1.30 on the 42° lens this essay measures, and 3.80 px on a 15° one. Rendering the same scene at two very different fields of view and recovering the camera from each therefore gives one quantity that moves by a factor of seven and one that does not move at all.

That is a better diagnostic than the resolution sweep in two ways. It is stronger, because a factor of seven is unmistakable where a constant ratio is not. And it needs no assumption about which parameter each error affects — it reads the dependence rather than the identity, which is the same preference the collection’s conditioning arguments show for a sweep over a single reading.

It also says something about the entanglement the section above describes. A pipeline with both errors reports a principal-point offset and a focal-length shortfall together, and at one lens setting there is no way to apportion them. Sweep the lens and the apportionment falls out: whatever part of the discrepancy scales as $\cot(\theta/2)$ is the $W-1$ error and whatever part stays at 0.7071 px is the corner error. Two unknowns, two behaviours, one sweep.

Two cautions, though, and they are the reason this remains a diagnostic for a pipeline rather than for a picture.

Neither error is detectable from a single picture at all. A displaced principal point is exactly what a shifted or cropped frame has, and a shortened focal length is exactly what a slightly different lens has. Both wrong pictures are perfect projections of real cameras, so the discrepancy is only visible against the camera the renderer declared — which is a fact about software rather than about the image, and is why this essay’s instrument is a round trip rather than an inspection.

A third convention exists and is worth ruling out before either of these. Some pipelines flip the vertical axis between the projection and the viewport, which is a reflection rather than a translation or a scale — and a recovery reads a reflection as a camera whose down vector points the wrong way, producing a mirrored reconstruction that reprojects perfectly. That is the failure a repeated singular value produces in the two-view field by a different route, and it is worth checking first because it is the one error of the three that a viewer can sometimes see.

And the sweep assumes the field of view is the thing being changed. Changing the focal length while holding the sensor fixed is the same sweep; changing the sensor while holding the focal length is not, because it changes $W$ and $\theta$ together in a way that leaves $f/W$ alone — which is the format cancelling out arriving one field over. The quantity to vary is the angle, and a pipeline that varies it by changing the “sensor size” input will see nothing move.

The half pixel and the pixel grid are different subjects #

One boundary worth drawing, because the two get conflated and only one of them is this site’s.

Where a sample sits is geometry: it is a statement about which point of the continuous picture a stored value represents, and it has an exactly right answer that this essay computes.

What happens when a continuous picture is reduced to samples is not geometry. Aliasing, filtering, reconstruction and the whole sampling literature are a different subject with different mathematics, and this site takes no position on any of it. The fleet’s register of which site owns which argument records the boundary, and the test that separates the two is this: every argument here survives with every sample taken exactly, and a sampling argument does not.

The distinction is easy to hold onto by asking what the answer depends on. A half-pixel offset is decided by the coordinate convention alone and is the same for every scene. Everything in the sampling literature depends on what is being sampled.

The general lesson, which is about units #

Both mistakes in this essay are unit errors in the strict sense: a quantity measured from one origin used as though it were measured from another, and a quantity counting intervals used as though it counted endpoints.

That second one — the fencepost — is the more instructive, because it recurs everywhere and is genuinely ambiguous rather than merely careless. A picture 690 px wide has 690 pixels and 691 pixel boundaries, and both counts are the right answer to some question. NDC $\pm 1$ maps to the boundaries; pixel indices 0 to 689 count the pixels. Choosing wrong is not a slip, it is answering the wrong one of two reasonable questions.

The reason it matters here rather than being a curiosity is the one this whole site is built on: a picture does not carry the conditions under which it is a picture of anything. It does not say what its focal length was, where its principal point is, or which of two pixel conventions its coordinates are in. All of that has to be stated separately — and when it is not, the picture goes on looking exactly right while the measurements made from it drift by an amount nobody can see. A half pixel is the smallest quantity this collection has found worth measuring, and it is measurable for the same reason every larger one is: because a recovery reads the picture and reports a camera, and a camera can be compared against the one that was declared.

It is the same gap recovering the camera closes from the other side. That recovery returns intrinsics to fifteen digits from a drawn box, and every one of those digits is in the coordinate convention the marks were made in — so a half-pixel disagreement about where a pixel’s centre lies is a half-pixel disagreement about the answer, exactly and permanently.