A shadow map's texels land by two distances and two cosines

A shadow is a second projection found that the construction which puts a shadow on the ground is the construction which puts the scene on the picture, with the lamp standing where the eye was. A renderer takes that literally, and makes the lamp the second eye of every frame. To decide whether a point is in shadow it first draws the scene from the lamp — a whole second picture, with its own frustum of four numbers and a window — and stores in each of its pixels, its texels, the depth of the nearest surface. Then, for every pixel of the eye’s picture, it finds where that point would land in the lamp’s picture and compares its depth with the one stored there.

The comparison happens one texel at a time, so a texel is the smallest piece of shadow the renderer can place. How large a piece that is, on the eye’s screen, decides whether a shadow’s edge is crisp or a staircase. And that is a question about two projections and a surface, which needs no raster to answer.

A texel reaches the screen through a plane #

Near any point, a surface is a plane, and the map from the lamp’s picture onto a plane and back out into the eye’s picture is a homography — the map two marks off a known plane find the other eye built between two ordinary photographs of a plane. A homography’s local behaviour at a point is a two-by-two matrix: how far, and in which direction, a step of one texel in the lamp’s picture moves the point’s image in the eye’s picture. Its determinant is the number of the eye’s square pixels that one texel covers.

That determinant has a closed form, and it is worth building from its two halves. A texel is a small solid angle seen from the lamp: one over the lamp’s focal length squared, in texels, reduced by the cube of the cosine of its angle off the lamp’s axis. On a surface at distance $d_L$, whose normal makes an angle $\theta_L$ with the lamp’s ray, that solid angle covers an area $d_L^2/\cos\theta_L$ times as large. A wall does not get darker measured exactly this footprint for one camera: the patch a pixel sees grows as the square of the distance and as one over the cosine of the tilt. The eye then sees that patch across its own distance $d_E$ and at its own angle $\theta_E$, and divides it by its own pixel’s solid angle. Put together,

$$\text{pixels per texel} = \left(\frac{f_E}{f_L}\right)^{2}\left(\frac{d_L}{d_E}\right)^{2}\frac{\cos\theta_E}{\cos\theta_L}\cdot\frac{\cos^3\alpha_L}{\cos^3\alpha_E},$$

with $f$ the two focal lengths in pixels and texels and $\alpha$ each ray’s angle off its own camera’s axis. The first figure measures the determinant directly, by carrying a thousandth of a texel’s step from the lamp’s picture onto the floor and into the eye’s picture, and it agrees with the formula to four parts in a billion at every point of every lamp.

So the size of a texel on the screen is two focal lengths, two distances and two cosines. The focal lengths are fixed per lamp and per eye. Everything that varies across a picture is in the other two ratios.

Where the lamp stands #

The three lamps in the first figure hold the map fixed at 1024 texels across and move only the lamp.

Beside the eye, with a 60° field against the eye’s 50°, every texel lands at 0.79 px, 5 m out, 10 m out and 20 m out. The two distances are nearly equal at every point, and so are the two cosines, because the lamp and the eye see each point along nearly the same ray; what is left is the ratio of focal lengths, 739.9 px against 886.8 texels, a little reduced by the angles off axis. A lamp at the eye takes a picture at almost the eye’s own resolution, and its texels arrive at almost a pixel everywhere.

Thirty metres overhead, with a wide 100° field so that it covers the floor, the distance from the lamp hardly changes along the floor while the distance from the eye grows from a few metres to tens. The texel is 5.77 px at 5 m, 2.09 at 10, 0.75 at 20: it shrinks as the eye’s distance grows, and near the eye it is large because the lamp is far and the eye is close.

Forty metres ahead, 5 m up and facing back toward the eye, is the arrangement renderers dread. Near the eye the floor is close to the eye and nearly 40 m from the lamp, so the distance ratio is at its worst exactly where the eye looks hardest, and the lamp’s rays arrive there at a shallow angle, so its cosine is small as well. The texel is 8.69 px at 5 m, 2.52 at 10, 0.51 at 20.

Taking the worst texel apart #

The closed form is a product, so a texel’s size can be accounted for factor by factor, and at 5 m out the accounts for the three lamps differ in kind.

For the lamp beside the eye, the focal lengths contribute 0.834 — the eye’s 739.9 px against the lamp’s 886.8 texels — the distances 1.01, the cosines 0.95 and the angles off axis 0.98. Every factor but the first is within a few per cent of one, and the product is 0.79 px. That lamp is nearly the control: move it onto the eye and give it the eye’s focal length, and every factor is exactly one, the homography between the two pictures is the identity, and every texel lands on exactly one pixel.

For the lamp overhead, the distance ratio dominates. The floor 5 m out is 31.62 m from the lamp and 5.28 m from the eye, a factor of 5.99 in width. The cosines pull the other way, 0.583, because the lamp looks almost straight down on the floor while the eye sees it at a slant, with a cosine of 0.322 — the eye’s own foreshortening squeezes the texel’s image. And the focal lengths add 1.72, which is the price of the lamp’s wide field: a focal length is not an angle, and spreading 1024 texels across 100° rather than 60° gives the lamp a focal length of 429.6 texels instead of 886.8, making every texel 2.06 times as wide before any distance enters. With the angles off axis, 0.96, the product is 5.77 px. By 20 m out the distance ratio has fallen to 1.52 and the eye’s slant to a cosine of 0.085, and the texel is 0.75 px.

For the lamp facing back, the distance ratio is larger still, 6.70 — 35.36 m from the lamp against 5.28 m from the eye — and the cosines no longer help. The lamp’s rays meet the near floor at a cosine of only 0.141, shallower than the eye’s 0.322, so the cosine factor is 1.51 and multiplies the distance instead of offsetting it. With the focal lengths at 0.834 and the angles at 1.03, the product is 8.69 px. At 10 m the same lamp’s distance factor is 3.00 and its cosine factor 1.01, and the texel is 2.52 px; at 20 m the lamp is nearly as close as the eye and its rays are steeper than the eye’s, and the texel is 0.51 px.

So near the eye the facing lamp is worse than the overhead one for two separate reasons, and the factors say how much of each: 12 per cent more distance ratio, and a cosine factor that has turned from a 0.58 saving into a 1.51 penalty.

The grid, drawn where the eye sees it #

The numbers say how large a texel is at a point. Drawing the map’s texel grid onto the floor, in the eye’s own picture, shows the whole distribution at once.

From the lamp facing back, a cell of 64 texels spans about 409 px of the eye’s picture 6 m out — more than half its width — and 32.4 px at 20 m. The cells are wedges, broad toward the eye and narrow toward the lamp, and each texel inside them is not square on the screen but stretched, 1.9 to 1 near and 2.9 to 1 further out. The grid is the lamp’s picture of the floor, seen from the wrong end.

From beside the eye the same map lays an almost regular grid over the floor: 50.6 px a cell at 6 m and 50.5 px at 20 m, each texel stretched 1.2 to 1. The lamp’s picture and the eye’s are nearly the same picture, and the grid between them is nearly the identity. It is the same 1024 texels in both figures. What changed is only where they were taken from.

The cosines #

The distances depend on where a point is. The cosines depend on which way the surface through it faces, and that makes them behave differently.

Hold one point fixed, 10 m out and 1 m above the floor, and turn a surface through it about the vertical, with the lamp off to one side. The distances do not change as it turns; only the two cosines do. Turned to face the lamp and the eye as nearly as it can at once, one texel spans 0.74 px. Turned to within a degree of edge-on to the lamp, the lamp’s rays skim the surface and a texel’s worth of them spreads a long way across it: 5.46 px, stretched 56 to 1 along the direction toward the lamp. Turned to within a degree of edge-on to the eye, the eye sees the surface almost end-on and a texel is crushed into 0.10 px.

A stretch of 56 to 1 means the texel does not arrive as a larger square but as a sliver, and the sliver has a direction: its long side runs across the surface toward the lamp, because that is the direction in which a grazing ray travels furthest for each step it takes across the lamp’s picture. A shadow edge running that way is drawn by a few texels spread thin along it; the same edge running across it is drawn by many, packed close.

The measured area follows $\cos\theta_E/\cos\theta_L$ at every orientation, including a quarter of a degree from each edge-on position. As $\cos\theta_L$ goes to zero the texel’s area grows without bound, and as $\cos\theta_E$ goes to zero it shrinks to nothing. Both limits occur on every lit object with a rounded outline: the lamp’s rays are tangent to it along the edge of its lit side, and the eye’s rays are tangent to it along the outline the eye draws.

Why the two ratios need two different remedies #

The two ratios are not only different quantities; they are different kinds of quantity, and that is what decides what a renderer can do about them.

The distance ratio is a property of a point’s position. Two surfaces through the same point, whatever their orientation, have the same $d_L/d_E$. So it can be corrected by deciding how the map’s texels are spread over the lamp’s picture: spend more of them in the part of the picture that lands near the eye and fewer in the part that lands far away. That redistribution is a choice about the lamp’s projection, and it is the same kind of choice a tile is an off-centre frustum made when it cut one frustum into many: split the eye’s view into slabs by distance, and give each slab a map of its own, so that the near floor is photographed by a lamp frustum that sees nothing else.

The cosine ratio is a property of a surface’s orientation at a point, and the lamp’s picture does not know orientations. A texel is one direction from the lamp; two surfaces that meet along that direction at different angles — a floor and a wall at their join, or the curved flank of an object near its silhouette — receive the same texel with wildly different stretches. No rearrangement of texels in the lamp’s picture can give one direction a different density for two surfaces at once. What a renderer does about the cosine ratio it does to the surface afterwards, by softening or biasing the comparison where the lamp grazes, not by spending texels.

Graphics texts call the first perspective aliasing and the second projective aliasing. The closed form says where the names come from: one factor is the ratio of the two projections’ distances, and the other is the ratio of the two foreshortenings.

The map each lamp needs #

The last figure turns the footprint into a budget.

Sample the floor the eye sees, out to 60 m, every 6 px of its picture, and find the largest texel each lamp lands on it. Beside the eye the largest is 0.79 px, 5.3 m out. Overhead it is 6.13 px, 5.4 m out. Facing back it is 9.31 px, 5.1 m out. In every case the worst is within a few metres of the eye, where the eye’s distance is smallest.

Since a texel’s footprint halves exactly when the map’s texels double — the figure checks it to six decimals — the map each lamp needs for texels no larger than a pixel follows directly. The lamp beside the eye needs 811 texels across, a little less than the 1024 it has. The overhead lamp needs 6275. The lamp facing back needs 9531 texels across, which is 91 million texels, to draw a shadow on the near floor as sharply as the eye’s picture can show it; and nearly all of those texels would be spent on floor far from the eye, where the same map already lands texels smaller than a pixel — half a pixel by 20 m out. That is the budget the redistribution of the previous section exists to cut.

The three maps do not even cover the same floor. The lamp beside the eye covers all of the floor the eye sees, and the overhead lamp’s 100° field covers 99 per cent of it, but the lamp facing back, with a 60° field, covers 90 per cent: the rest lies outside its picture, under and behind the lamp’s axis. Widening that lamp’s field to take in the last tenth would lower its focal length and enlarge every texel it already lands, by the same factor that cost the overhead lamp 1.72 — so the facing lamp’s 9531 texels are a lower bound for a map that covers everything the eye can see.

The depth stored in each texel is itself a depth-buffer code, with all of the precision a depth buffer has left and none of the eye’s, because it is measured from the lamp. How close to a surface the comparison can tell lit from shadowed is a separate budget from the one measured here, and it runs out in a different place.

What this does not settle #

The comparison. A texel stores one depth for a patch of surface, and a point is judged against that one number. How far a sloping surface departs from its texel’s stored depth across the texel — the reason renderers bias the comparison — follows from the same stretch but was not measured.

Filtering. Renderers soften a shadow’s edge by comparing against several neighbouring texels. That is sampling, and what it does to the staircase is outside this geometry.

Spotlights. Every lamp here is a point with a frustum. A distant sun is a lamp at infinity, the eye taken to infinity applied to the lamp, and its map is a parallel projection whose distance factor is constant across its picture; the overhead lamp, 30 m up, is only an approximation of it.

Still open: one map warped for the eye #

Splitting the eye’s view into slabs spends several maps. The alternative is one map whose projection is itself chosen so that the distance factor comes out nearly constant over the floor the eye sees — a lamp’s picture drawn through a perspective warp that gives the near floor more of its texels.

The question that leaves is how far one warped map can go: how much of the 9531 texels the facing lamp needed a warp of its picture can remove, what the best warp is for a given eye and lamp, and whether what remains after it is exactly the cosine ratio, which no warp of the lamp’s picture can reach.