A tile is an off-centre frustum

A picture too large to render at once is rendered in pieces and the pieces are joined. There are two obvious ways to do it and one of them is exact.

Which of the two is used decides whether the join is invisible or merely inconspicuous, and the difference does not show up in any single tile.

The exact way is one line #

The direction a pixel u looks along has x/z = (u − pₓ)/f. That is the whole of the camera’s lateral geometry, and it is a statement about a pixel rather than about a picture — so a tile’s sides are the frustum’s sides evaluated at the tile’s own pixel bounds:

$$\text{left} = \text{near}\cdot\frac{x_0 - p_x}{f}, \qquad \text{right} = \text{near}\cdot\frac{x_1 - p_x}{f}$$

and the same for top and bottom. Same eye, same focal length, same near plane; a different window.

The tiles then reassemble to the last bit. Over a scene of points spread across the frame, the largest disagreement between a tile’s answer and the whole picture’s is about a part in ten thousand million million of a pixel, which is two floating-point routes to one number.

That exactness is not a numerical accident and it is worth saying why. The composition of the tile’s matrix with the tile’s viewport is the same affine map of the same ray as the composition of the full matrix with the full viewport. The two matrices differ; the two viewports differ; the products do not.

The other way is a rotation #

The alternative is the one a person reaches for without a frustum in mind: keep the eye where it is, aim the optical axis at the centre of each tile in turn, and give each camera a field of view wide enough to cover its tile.

Every step of that is defensible. The eye has not moved, so nothing about the scene’s perspective has changed. The field of view is computed from the tile’s own angular extent rather than guessed. And the tiles are the right size, so they abut.

It is a different picture, and by a lot. The worst displacement across the same scene is about twelve pixels at three tiles by three, with a median of about three, on a picture six hundred and ninety pixels wide. Points near a tile’s centre are fine and points near a corner are not.

The reason is the one this site has already stated in another vocabulary: a rotation is not a crop. Straightening does not move the eye is about the same fact from the other end, where a keystone correction is a homography between two picture planes rather than a change of viewpoint. Rotating a camera and cropping its picture are related by a homography, and a homography is the identity only when the rotation is zero.

What the two constructions differ by, exactly #

The difference between the two pictures has a name, and having the name makes the size of the error predictable rather than measured.

Two pinhole cameras at the same eye, differing by a rotation R, produce pictures related by the homography K·R·K⁻¹ — where K carries the focal length and principal point. That is the standard result and this site uses it elsewhere: a keystone correction is one of these, and the panorama field is built on them.

So the rotated tile’s picture is the correct tile’s picture with a homography applied, and the homography is the identity only when R is. Near the fixed point of that homography — the direction along which the rotation acts trivially, which is the tile’s centre — the displacement is second order in the angular offset, which is why the middle of every tile is clean. At the tile’s edge the offset is the tile’s own half-angle and the displacement is proportional to it times the rotation.

Both factors are the reason subdividing fails to help in proportion. Halving the tile halves its half-angle, which halves the displacement — and it also halves the tile, so the ratio is unchanged. The two shrink together because they are the same quantity.

Subdividing does not fix it #

The natural response is that the tiles are too big. Make them smaller and the rotation per tile is smaller and the error goes away.

The absolute error does fall. From two tiles across to twenty-four, the worst displacement drops from about twenty-five pixels to about three.

As a fraction of a tile it does not move at all: between about seven and eleven per cent, everywhere from two by two to twenty-four by twenty-four. So the error falls only as fast as the tile does, and reaching half a pixel needs tiles a handful of pixels across — which is thousands of renders of the same scene, each one with its own camera setup, to avoid a construction that was one line of arithmetic.

This is the collection’s own convergence gate in miniature, and it is why that gate exists. “The error gets smaller under subdivision” is true and is a statement about the units. Whether a quantity is converging is a question about the ratio it is being measured in, and a number that shrinks in pixels while holding steady in tiles has not been fixed by anything.

What tiling is for #

Three things want a picture in pieces, and the construction is the same for all of them, which is part of why it is worth getting right once.

A picture too large for one pass. A print at three hundred dots per inch on a sheet a metre across is tens of thousands of pixels on a side, and the memory a renderer needs grows with the area. Cutting it into tiles trades passes for memory and nothing else — provided the tiles are exact, because a print is the one medium where a seam is permanent.

A picture split across machines. The same arithmetic, with the tiles rendered in different places. Here the exactness matters for a second reason: the machines have to agree without communicating, and they do, because each one is computing a crop of a picture that is fully determined by numbers all of them have.

A picture split across a wall of displays. Several projectors or panels showing one image, each with its own frustum. This is the case where the naive method is most tempting, because each projector really is aimed somewhere different — and it is still wrong, because the projectors’ lenses are aimed differently and the virtual camera should not be. The eye the picture is drawn for is one eye, in one place, and the wall is one window cut into several pieces.

The third case is the one where the error is most visible, since a wall of displays puts its seams in the middle of the picture rather than at the edges. A line crossing a seam between two naively rendered panels kinks by a few pixels, and the kink is in the part of the image a viewer is looking at.

Where the error is, and why it is invisible #

The displacement is zero at every tile’s centre, by construction: the tile’s camera is aimed along the ray through that point.

It grows toward the tile’s edges, and it grows with how far off-axis the tile is — so the worst place in the whole picture is the outer corner of a corner tile, where a large rotation and a large angular offset compound. In the middle tile of a nine-tile grid the error is small everywhere, because the rotation is zero there and the tile is only a crop.

Which makes the failure hard to see in the ordinary way. Look at any single tile and it is a perfectly good picture — straight lines are straight, the perspective is consistent, nothing is distorted. The defect appears only at the seams, as a discontinuity where two tiles disagree about where a line crosses their shared edge, and even then it appears most strongly at the corners of the picture where a viewer is least likely to be reading detail.

A tiled render assembled this way looks right, has straight lines that kink slightly at tile boundaries, and gets worse toward the edges of the frame. Every one of those symptoms would be read as an artefact of something else.

What the correct construction is not doing #

It is worth being clear that the exact tiling is not a trick or a special case. It is what happens when the camera is left alone.

A tile’s frustum has all four sides on the same side of the axis for a corner tile — a frustum entirely off-axis, which four numbers and a window notes is perfectly legitimate and has no field of view in the usual sense. There is no camera “aimed at” that tile; there is a camera aimed where it always was, looking through a window that is nowhere near the middle of its own view.

That is exactly the situation a shift lens produces, and the principal point is not the centre measures what happens when a recovery assumes otherwise: on a fifth of a frame’s shift, a focal length computed from two vanishing points with the principal point taken as the image centre is out by one and a half per cent. A tile of a picture is a shifted frame, and a tile’s principal point is outside its own rectangle for every tile except the middle one.

Tiles that overlap, and tiles that do not abut #

Nothing in the construction requires the tiles to tile. The sides are read at whatever pixel bounds are asked for, so overlapping tiles, tiles with gaps, and tiles of different sizes all work and all agree with the whole picture where they cover it.

That is more useful than it sounds and it is the test worth running on any implementation. Ask for two tiles that overlap by a strip, render both, and compare them on the strip: an exact construction agrees to the last bit, and a construction with a rotation in it disagrees by the same order as everything else. The overlap is a self-check that needs no reference picture at all.

It also disposes of a small confusion about resolution. A tile can be rendered at a different pixel density from its neighbours — twice as many samples across the same angular extent — and the geometry is unchanged, because the sides depend on the tile’s bounds in the original picture’s pixel coordinates and not on how finely the tile is drawn. A high-resolution inset of a low-resolution picture is a tile like any other, and it lines up.

The check that makes it a claim #

Both halves are asserted, because either alone is misleading.

The frustum tiling must be exact — otherwise the construction is merely close, and there would be no reason to prefer it to the other. And the rotated tiling must be wrong by a real amount — otherwise the essay is a warning against a mistake that does not matter.

The second assertion caught the only bug in the construction, and it caught it by being too generous. The first version reported forty pixels of error at every subdivision, from two tiles to twenty-four, with the worst point always the same one. It was a point outside the picture: the tile assignment clamped an off-frame point into the edge tile, and then measured a point half a picture from that tile’s centre and blamed the tile. A tile owns the points inside it, and a point outside the picture is in no tile at all.

The tell was the flatness. An error that does not respond at all to the thing it is supposed to depend on is usually not measuring that thing.

What each tile costs to set up #

There is a practical asymmetry between the two methods that is easy to miss and points the same way.

The correct method needs, per tile, four numbers: the tile’s pixel bounds run through one subtraction and one division each. Nothing is rotated, nothing is normalised, no basis is built, and the near and far planes are the ones the whole picture had.

The rotated method needs a direction, a basis about that direction, a focal length derived from the tile’s angular extent, and a decision about which way is up in the tile — and that last one is where its own second bug lived. The basis has to be built the way the rest of the collection builds one, with the down vector as forward × right; written the other way round it produces a basis that is perfectly consistent, mirrors every tile horizontally, and passes every assertion about relative positions. It reported error figures four times too large until the sign was found.

So the wrong method is also the more elaborate one, which is the usual shape of this kind of mistake. It is not a shortcut that trades accuracy for simplicity. It is a longer construction that arrives somewhere else.

One more thing the exact construction gives free #

Because a tile is an ordinary frustum, everything the collection knows about frusta applies to it unchanged — and one consequence is worth stating because it is the kind of thing that gets rediscovered awkwardly.

A tile’s near and far planes need not match its neighbours’. They are not camera parameters, so changing them moves nothing in the picture; what they change is what each tile clips and how well each tile orders its own depths. A tile covering only distant scenery can push its near plane out and buy back depth precision that the whole-picture render had to spend on the foreground.

That is a real saving and it is available only because the planes are separable from the camera. In the rotated-camera method it would be available too, and it would be sitting on top of a picture that was already wrong, which is the general reason to fix the geometry before optimising anything.

And the sun-lit version of the same mistake #

The rotated-tile construction is the rendering form of a much older error, and naming it makes the pattern portable.

A panorama assembled from photographs taken by turning a camera on a tripod has exactly this structure — each frame is a rotation, not a crop — and the frames cannot be laid side by side on a flat sheet without warping. They have to be reprojected, which is what panorama software does, and the reprojection is the homography above. Anyone who has tried to butt two turned photographs together has met the error in the figures.

The difference is that in the photographic case the rotation is forced: a camera on a tripod cannot take an off-centre frustum, because a real lens’s principal point is where the lens puts it. In the rendered case it is not forced, and the matrix will take any four sides it is given.

Which is the small moral of the whole essay. The exact construction is available because a synthetic camera has a freedom a physical one does not, and the mistake is reaching for the physical camera’s constraint when nothing is imposing it.