A frame is an interval

The previous essay took a frame apart along one axis: the rows are read at different times, so the frame is a stack of projections indexed by height. This one takes the other axis. Even a single row is not read at an instant — it is integrated over an interval, and everything that moves during the interval contributes to more than one place.

A photograph is therefore not a projection of a scene. It is a projection of a scene’s history, over a few milliseconds, added up.

The reassuring half: a streak is straight #

Start with what does not go wrong, because it is a consequence of the site’s oldest fact and it is worth having explicitly.

A point moving in a straight line at a constant speed traverses a straight world segment during the exposure. The image of a straight segment is a straight segment. So the streak is straight, and it is straight whatever the depth, whatever the direction of travel, and whatever the field of view.

The gate measures it rather than asserting it: the worst departure from the chord, over twenty-four samples along each streak, is 0.0 px exactly at the noise floor. Not approximately straight — the image of a segment is a segment, and a departure would mean the projection was wrong rather than the model.

That matters because it says what the blur is. It is not a diffusion, not a smear, not a low-pass filter. It is a line integral along a drawn segment, which is a very specific and very tractable thing.

What is not constant along the streak is its speed. The point moves uniformly in the world and non-uniformly on the page — the same change of parameter the interpolation essay derives, for the same reason. So a streak from a receding object is a straight line with more exposure at one end than the other, which is a blur kernel with a shape.

The measurement: length goes as one over depth #

A point at depth $z$ moving at speed $v$ perpendicular to the axis draws a streak of length

$$L = \frac{f\,v\,T}{z}$$

so two points at different depths moving at the same speed draw streaks in the ratio of their depths reversed. The gate asserts the ratio against the depth ratio at 2% and finds 2.000 against 2.000 for a 3 m and a 6 m point.

That is the whole argument against treating motion blur as a filter. A convolution applies one kernel to a whole image. A frame of a scene with depth in it needs a different kernel at every depth, and the depths are not recorded anywhere in the frame. So deblurring a photograph of a real scene is not a deconvolution — it is a deconvolution with an unknown spatially-varying kernel, which is a different and much harder problem wearing the same word.

The control makes the claim discriminating rather than merely true: two points at the same depth blur by lengths within 5% of each other, so a fronto-parallel scene does have one kernel and a single deconvolution is exactly right for it. Which is why the technique appears to work on test images, and why test images are usually flat.

The same structure the light field already found #

This site has met a spatially-varying kernel before, and naming the earlier case makes this one easier to place.

The penumbra is the lamp’s image establishes that a shadow’s soft edge is not a blur applied to a sharp shadow — it is the image of the source, projected by the occluder’s edge acting as a pinhole, and its width depends on the distances involved. Two objects at different heights above the ground have penumbras of different widths in one photograph, for exactly the same reason two objects at different depths have streaks of different lengths.

Both are the same statement: an extended aperture in one variable produces a kernel whose size is a projection. For the shadow the extended thing is the lamp and the variable is space; for the blur it is the exposure and the variable is time. In both cases the naive picture — a sharp thing that has been softened — is wrong in the same way, and gets the dependence on distance backwards.

The two intervals compose #

A real camera has both departures at once, and they are independent.

The rolling shutter says each row was exposed at a different time — the frame’s rows are indexed along the interval. The exposure says each row was exposed over an interval rather than at a point in it.

So a frame is a double integral: over the readout, and over the exposure at each row. The two are usually of very different sizes — a readout of 30 ms and an exposure of 1 ms, say — which is why the artefacts look nothing alike. A short exposure with a long readout gives sharp, sheared geometry: the case the previous essay measured. A long exposure with a fast readout gives blurred, unsheared geometry.

The interesting case is when they are comparable, and it is worth stating what happens because it is not the sum of the two. The eye is moving during each row’s exposure as well as between rows, so the streak a point draws is a projection from a moving centre — and the streak is then no longer guaranteed straight. The straightness result above assumes the camera is stationary during the exposure; drop that and the streak bends by the same mechanism a scroll bends a straight line.

That is a satisfying place for the two essays to meet, and it is worth being honest that this site measures the two separately and does not draw the composite. The composite is a real object and drawing it well would take a phase of its own.

The streak is a projection, so the whole battery applies #

The cultures phase’s contribution to this site was a battery: five questions asked of any drawing system, each answered by a computation on a map rather than by a judgement. Does it have a centre? Does it keep true measure? Does size fall with distance? Is its depth range bounded? Are its straight lines straight?

A blurred frame can be run through it, and the answers are worth setting out because four of the five are unaffected and the fifth is the whole subject.

It has a centre, exactly — provided the camera itself is still. Every ray in the frame passes through the eye; what varies is when, and the eye has not moved. This is the sharp difference from a rolling shutter, where the eye does move and the centre goes.

It keeps whatever measure a pinhole keeps, since each instant’s projection is an ordinary pinhole projection and the frame is their sum.

Size falls with distance, in every one of them.

Its depth range is bounded as a pinhole’s is.

And its straight lines are straight, which is the result asserted above at the noise floor.

So a blurred frame passes the battery. It is a projection through a point of a scene — of a scene that existed over an interval rather than at an instant, which is a statement about the subject of the picture rather than about the picture’s geometry.

That is a genuinely different kind of departure from the others this phase measures, and it is worth naming as such. A rolling shutter breaks the geometry. A depth buffer quantises it. A pixel aspect misreports it. An exposure leaves the geometry entirely intact and changes what the geometry is a projection of.

The exposure is a choice with no free side #

Every photographic account of shutter speed presents it as a trade against noise, and that is true and is not the geometric statement.

The geometric statement is that the exposure decides how much of the scene’s history the frame contains, and every value has a cost:

• a short exposure records a nearly-instantaneous projection and records almost no light;
• a long one records more light and integrates over more history, so it is a projection of a scene that no longer exists as photographed.

There is no setting at which a frame is a projection of an instant, because an instant carries no light. That is not a limitation of any particular camera; it is a consequence of a photograph being made of something.

It is worth setting that beside the other departures this phase has measured, because it is the only one with no fix at all. A depth buffer can have more bits. A pixel grid can be sampled at its centres. A rolling shutter can be replaced with a global one. An exposure cannot be replaced with an instant, and the whole of photography is a negotiation with that.

The direction the streak does not have #

One asymmetry is worth extracting because it is the reason a single frame cannot be run backwards.

A streak records a segment of a path, and a segment has two ends and no arrow. Nothing in the drawn streak says which end was first. The exposure was an integral and an integral forgets the order of its integrand.

So a frame containing a streak determines the path, up to reversal, and determines nothing about the direction of travel — which means it determines the speed’s magnitude and not its sign. That is a small loss and it has a familiar shape: it is the same kind of gap as the four camera poses a fundamental matrix admits, where the geometry leaves a discrete ambiguity that no amount of precision resolves and one extra fact settles instantly.

There the extra fact is cheirality — the reconstructed points must be in front of both cameras, which rules out three of four. Here there is no equivalent, because both directions of travel are perfectly possible and the frame contains no constraint that distinguishes them. A second frame does, immediately.

Which is a small illustration of a claim this site makes constantly. A picture is a complete record of one pencil of rays, and every question it cannot answer is a question about something outside that pencil. The direction of travel is outside it, so the picture is silent, and no reading of it will ever be otherwise.

The exposure and the aperture are the same trade twice #

Setting this beside the site’s earlier measurement of a pinhole makes the shape of the trade clearer than either does alone.

The pinhole crossover is the point at which a pinhole camera’s image stops sharpening as the hole shrinks — geometric blur falls with the hole and diffraction rises, and the two cross at a computable size. There is no aperture at which the picture is both bright and perfectly sharp.

An exposure is that trade with time in place of space. Shorten it and the streak shortens; shorten it far enough and there is not enough light to record anything. The crossover is set by the sensor’s noise rather than by diffraction, which is why this site does not compute it — noise is not geometry.

But the shape is identical, and the shape is what is worth carrying away. A picture is made by integrating over an extent, and every extent that can be reduced costs light. Aperture, exposure, and the sensor’s own photosite size are three instances of one constraint, and a picture with none of the three would be a picture of nothing.

What a streak actually records #

Ending on the thing that makes this more than a catalogue of an artefact.

A streak is the drawn image of a world path. So a photograph with motion blur in it contains more information than a sharp one, not less: it records where something went, over a known interval, projected exactly. A sharp photograph records one instant and says nothing at all about motion.

That is the reason the artefact is worth measuring rather than merely suppressing. The streak’s direction is the direction of travel projected; its length is $fvT/z$, so with two of $v$, $T$ and $z$ known the third follows; and its varying density along its length carries the change of parameter that says whether the object was approaching or receding.

None of which is a claim about being able to recover any of it robustly from a real photograph — that is an image-analysis problem and this site does not own it. The claim is about what is there, which is a geometric question with an exact answer. A blurred frame is a projection of a segment of history, and a segment of history contains a segment’s worth more than a point does.

Which is the same argument this site makes about every departure it measures. A refracted picture has no viewpoint and it still records the scene; a scroll has no centre and it still measures distance along its roll exactly. A picture that is not a projection is not a picture of nothing. It is a picture of something else, and the work is saying what.