Straight lines that are not

This site has an essay called wide-angle is not distortion, and it is right: the marginal stretch of a wide frame is a correct projection, the price of casting a wide world onto a flat surface, and it is exactly undone by standing where the picture is correct from.

This field is about the thing that is distortion. It is a departure from the projection rather than a property of it, and no viewing position undoes it, because it is not a projection of anything.

The polynomial #

The model everybody fits — every calibration library, every raw converter’s lens profile, every photogrammetry package — is a polynomial in the radius from the principal point. In normalised image coordinates $x = (u - c_x)/f$, $y = (v - c_y)/f$, with $r^2 = x^2 + y^2$:

$$x_d = x\left(1 + k_1 r^2 + k_2 r^4\right) + 2p_1xy + p_2(r^2 + 2x^2)$$

$$y_d = y\left(1 + k_1 r^2 + k_2 r^4\right) + p_1(r^2 + 2y^2) + 2p_2xy$$

The $k$ terms are the radial part; the $p$ terms are tangential, and they model a lens whose elements are not quite coaxial. In almost every real lens the radial part dominates by an order of magnitude, so the essays here mostly set $p_1 = p_2 = 0$ and say so.

Two conventions are worth stating because getting them wrong is the usual way this arithmetic goes astray. The radius is measured from the principal point, not from the centre of the image — and those are not the same place on a shifted or cropped frame. And the coordinates are normalised by the focal length, which is what makes the coefficients dimensionless and comparable between lenses of different focal lengths on different sensors.

A negative $k_1$ is barrel distortion: the image contracts toward the centre as $r$ grows, so a square bows outward. A positive $k_1$ is pincushion, and a square bows inward. Wide-angle lenses barrel; long lenses tend to pincushion; a lot of zooms do both at different focal lengths, which is why a single profile per lens is never enough.

The line that is left alone #

The visual signature of distortion is usually described as the edges of the frame bow. That gets the mechanism wrong, and the wrongness matters as soon as anything is measured.

The map above is radial: it moves every point along the line joining it to the principal point, and changes nothing else. So consider a line that is radial — one passing through the principal point. Every one of its points moves along the line itself. The line cannot leave itself. It is left exactly straight, at any coefficient, at any radius.

Every other line has points whose radial directions differ, and they move by different amounts in different directions, and the line bends.

So the quantity that decides how much a line bends is not where it is in the frame. It is its distance from the principal point. Two lines equally far from that point bend identically whether one is near an edge and the other is not; a line running diagonally through the centre of the frame from corner to corner does not bend at all.

That correction sounds pedantic and it is not. It is the difference between “correct the edges” and “know where the centre of the radial pattern is”, and the second is what a calibration actually determines. It is also the reason a crop of a distorted image looks strange: cropping moves the frame’s centre and leaves the principal point where it was, so the pattern is no longer symmetric about the picture and stops looking like distortion at all. It looks like a badly assembled panorama.

Why this is not a projection #

The site’s premise is that a picture is a projection through a centre, and the departure here is small enough to be easy to wave through. It is worth being exact about why it cannot be.

A projective map of the plane is a ratio of linear forms:

$$u' = \frac{a u + b v + c}{g u + h v + 1}, \qquad v' = \frac{d u + e v + f}{g u + h v + 1}$$

The radial factor $1 + k_1r^2 + k_2r^4$ is even in $r$ and quartic in the coordinates. There is no choice of the eight parameters above that produces it, at any coefficient other than zero. Not approximately-not — structurally not: a projective map sends lines to lines, and this one does not, and a single bent line is a proof.

That is the whole content of the next essay, measured: the cross-ratio, the one thing a projection preserves, does not survive a lens, and the departure is a number rather than an argument.

The two distortions, side by side #

It is worth separating the two things this site now calls by names that sound alike, because they are opposite in almost every respect.

Marginal stretch — the subject of wide-angle is not distortion — is what a correct perspective projection does at the edge of a wide frame. A sphere images as an ellipse stretched by $1/\cos\theta$; a face at the corner of a group photograph looks broadened. It is a property of the projection, it is exactly right, and it disappears entirely when the picture is viewed from the point it is correct from.

Radial distortion is a departure from the projection. It bends straight lines, which marginal stretch never does; it is not undone by any viewing position, because there is no viewpoint from which the drawn line is straight; and it can be measured from the picture alone, which marginal stretch cannot, since a correctly projected wide frame is indistinguishable from what it should be.

The test that separates them in one question: is a straight world line drawn straight? If yes, whatever else is going on is projection, and any complaint about it is a complaint about where the reader is standing. If no, the picture has been through something that is not a projection.

That test also settles the common confusion in the other direction. A photograph of a building whose sides converge is not distorted; it is a correct three-point perspective from a tilted camera, and the sides are straight. A photograph of a building whose sides bow has both things happening at once, and only the second is the lens’s fault.

What a real lens’s coefficients look like #

Some scale, so the figures are not floating free.

A modern kit zoom at its wide end runs $k_1$ around $-0.10$ to $-0.15$. A 20 mm ultra-wide might reach $-0.30$. A portrait telephoto pincushions gently at $+0.05$ to $+0.15$. A cheap phone camera before software correction can be past $-0.40$, which is why phone pictures of buildings looked so bad before the software started fixing them silently.

Translated into pixels on the figure’s 690 px frame, $k_1 = -0.32$ moves a corner by about 17 px and bows the outermost horizontal by 17.8 px. That is plainly visible; it is why the picture on the page looks obviously wrong. A more typical $-0.11$ bows the same line by about 6 px, which is not obvious at a glance and is enough to cost a measurement two per cent.

A fisheye is not in this table, and the omission is deliberate. A fisheye is not a distorted rectilinear lens; it is a different picture surface, following $r = f\theta$ or $r = 2f\sin(\theta/2)$ rather than $r = f\tan\theta$, and the curved field owns it. Fitting a Brown–Conrady polynomial to a fisheye is the mistake of modelling a deliberate design decision as an error.

The inverse, and why it is an iteration #

Undoing the polynomial is needed constantly — every correction, every calibration fit, every measurement off an uncorrected frame — and it has no closed form worth writing. Inverting $r_d = r(1 + k_1r^2 + k_2r^4)$ means solving a quintic.

What every implementation does instead is a fixed-point iteration: guess that the undistorted radius is the distorted one, evaluate the radial factor there, divide, and repeat. It converges quickly wherever the radial factor stays comfortably positive, which is the whole region a lens is usable in.

There is a failure mode worth naming because it is silent. If the radial factor approaches zero — which happens at large $r$ with a strongly negative $k_1$ — the iteration stops contracting and starts wandering. It does not diverge to infinity and crash; it settles somewhere, and returns a point that is merely wrong. The machinery here refuses instead: a non-positive radial factor means the candidate lens has folded the image over itself, and there is no point to return.

That refusal has one caller which needs a different answer, and the distinction is worth recording. The plumb-line fit walks a bracket of candidate coefficients and will step into inadmissible ones on the way; for it, “this lens folds the image” is a perfectly good objective value — infinitely bad — rather than an error. Swallowing the refusal as zero would be the bug, because the search would then prefer exactly the candidates the model cannot represent.

Two routes, as usual #

The check that the inverse is right is a round trip, and it is the cheapest thing in the field: distort a point, undistort it, compare. Over the points in the figure the worst discrepancy is about 10⁻¹² px.

That matters more than it looks, because everything downstream is built on the inverse. The plumb-line fit works by undistorting an observed line with a candidate coefficient and measuring how straight the result is. An inverse that was merely close would make the fit report the error in the inverse rather than the error in the coefficient — the fit would converge, to a number that describes the iteration.

This is the same discipline that runs the whole site. The camera recovered from the picture it drew agrees with itself to one part in 10¹⁵, and the value of that number is not that it is small but that the two halves are separated by an interface narrow enough to inspect. Here the interface is narrower still: the forward polynomial and the inverse iteration share no arithmetic beyond the coefficients themselves.

Where the polynomial comes from #

A fair question about a model that is fitted everywhere and derived nowhere: why a polynomial in $r^2$?

The answer is a symmetry argument rather than an optical one, which is why it applies to lenses of wildly different designs. A lens made of surfaces of revolution about a common axis has a map from direction to image that commutes with rotation about that axis. A map on the plane that commutes with every rotation about a point must send each point along its own radius, and the amount it moves it can depend only on the radius. So the map is $r \mapsto r\,g(r)$ for some function $g$.

Then: the map is smooth, and it is odd in the sense that reversing the direction reverses the image, so $g$ must be even. An even smooth function is a series in $r^2$. Truncated after two terms, it is Brown–Conrady’s radial part.

That is why the model works so well across designs it was never derived for. It is not a model of glass; it is the general form of any rotationally symmetric smooth departure, and the coefficients are whatever the particular glass happens to produce.

It is also why the tangential terms exist: they are the correction for the symmetry assumption failing, which is what a decentred element does.

The tangential terms, and why they are usually zero here #

The $p_1$ and $p_2$ terms model decentring: elements in the lens that are not quite on a common axis, so the pattern of displacement is not quite symmetric about a point.

They are included in the model here and set to zero in almost every figure, for a reason worth stating. On a well-made lens they contribute an order of magnitude less than the radial terms, and — more to the point for this site — they do not change any of the structural claims. A tangential term is still not projective; it still breaks the cross-ratio; it still leaves the picture unable to be corrected by a homography. What it does change is the shape of the pattern, and drawing that shape would make every figure here about a second-order effect rather than about the first-order one.

The one place they matter is a calibration report. A fit that frees $p_1$ and $p_2$ on data that does not constrain them will return values, and the values will be noise dressed as measurements. That is the same disease the correlation between $k_1$ and $k_2$ has, and the same cure: report the conditioning beside the parameter, or do not report the parameter.

Correcting it, and what correction costs #

Every raw converter now applies a lens profile automatically, and it is worth knowing what is being done.

The correction resamples the image: for each pixel of the output, compute where it came from in the distorted input — which is the forward polynomial, not the inverse, so it needs no iteration — and interpolate. Geometrically it is exact, to the accuracy of the fitted coefficients.

The costs are not geometric. Correcting barrel distortion pushes the corners outward, so the corrected frame is larger than the sensor at the corners and has to be cropped back, which loses field of view — often several per cent, and always the part a wide lens was bought for. And the resampling redistributes detail: the parts of the frame that were compressed are stretched, so their effective resolution falls.

That is why lens designers do not treat distortion as free even now that it is corrected in software. It is corrected at the price of the field and the corner sharpness, and both of those are what a wide lens is for.

The geometric point stands regardless: after correction, the picture is a projection again, and every theorem on this site applies to it exactly. That is the actual reason correction matters for anything measured. It is not about the picture looking right; it is about restoring the property that makes cross-ratios, vanishing points and camera recoveries mean anything.

Where this field goes #

Four questions, and each is an essay.

If distortion is not projective, what happens to the invariant everything here rests on? The cross-ratio comes back one and a third per cent out where the pinhole is exact to fifteen digits, and a height measured off an uncorrected wide frame is two per cent wrong — with the error tracking a quantity nobody would guess.

Can the coefficient be recovered without a calibration target? It can, from nothing but the knowledge that some edges were straight — and the two coefficients everyone fits turn out to be correlated at −0.997, so the fit determines their combination and neither of them.

Is the principal point the middle of the frame? No, and assuming it is costs one and a half per cent of the focal length at a fifth of a frame’s shift — while the site’s own three-point recovery computes it instead and has no such error.

And is the centre of projection a point? It is a place, and rotating a camera about the wrong one leaves a parallax that falls exactly as one over the distance.