One more habit takes two hands, and only part of a walk
Worth reading first: A tiring hand draws a different habit · The picture whose lines spread.
A stepped hand’s walk is in the creeps it was given found a way to convict a divergent picture of not having been painted by one steady hand. Fit each strip alone for its habit and its creep, fit the whole picture once with a single habit and a single creep, and take the ratio an analysis of variance takes: how much the strips disagree with the shared fit, per degree of freedom given up, over how badly each strip disagrees with its own marks. A tiring hand’s pictures sit below one. A hand that steps its dividers, so that every row inherits the slips of the rows before it, puts a walk into each strip that the strip’s own fit absorbs as a creep of its own, and those creeps disagree; on eight strips the stepping pictures sit far above.
The essay closed on what the verdict cannot say. Two painters who share a picture strip by strip, with habits one apart, were called a single stepping hand seventy times in a hundred, because a second habit is also a disagreement between strips that no single habit and creep can take. The test says “not one steady rule”. It does not say which of the two common alternatives broke it, and the measurement it proposed to separate them was the obvious one: give the picture one more number — a second habit — and see how much of the disagreement it takes.
It takes most of two hands’ disagreement and a shrinking share of a walk’s, and the share, rather than what is left over, is the reading that separates them.
One more number takes a gap, and a slice of a walk
The paintings are the ones the last measurement used: a divergent interior of strips at the splays of a floor (1.12), a footstool (1.20), a table (1.32) and a book (1.38), cycled, every strip divided into six bays with a creep of 2 per cent a bay and its starting edge alternating. A stepping hand slips 0.4 pixels a step. Two tiring hands slip as the single tiring hand did, tripling along each strip, with habits of 1 and 2 — the first the even spacing down the page an untrained hand draws, the second a step further from a camera’s spacing, crowding the rows a little towards the near edge — and they share the picture so that each takes every splay once and both edges. That last condition matters. A pattern that simply alternated strips would hand one painter all the floors and tables from the near edge, and any difference in what the two hands’ strips report could then be a difference in splay or edge rather than in hand.
The refit keeps one creep for the whole picture and allows two habits, the strips divided into the two groups that fit best. Finding that division is a search, and the obvious shortcut fails. Sorting the strips by the habit each reports when fitted alone does not work, because on one strip a habit and a creep counterfeit each other almost exactly — the observation a tiring hand draws a different habit began with — so a strip’s free habit is set mostly by its slips. Sorting by the habit each strip would choose under a shared creep is better but still misleads when a strip has been read from the wrong edge. On eight strips the search simply tries all 127 ways to divide them into two non-empty groups, each refitted with two habits and one creep and every strip’s edge re-chosen; on sixteen, where there are 32,767, it starts from the best division at each of thirty-one trial creeps and moves single strips while that lowers the cost. Both search over the creep, the two habits and every strip’s edge together.
The pale bars are the disagreement the last measurement read: how far, root mean square on the page, the rows of one shared habit and creep sit from the rows each strip would have chosen for itself. The solid bars are the same with two habits. For the two hands in this painting the solid bars collapse: the shared fit missed the strips by up to 2.16 pixels, and with a second habit the largest miss is 0.58. The picture’s ratio of disagreement to slip falls from 9.88 to 0.68, below where a single tiring hand’s pictures sit. For the stepping hand the solid bars shrink too, but less, and the ratio falls only from 7.54 to 2.35.
The slider runs through the other paintings. On most of them the two hands’ bars all but vanish and the stepping hand’s are cut by somewhere between a half and four-fifths. A second habit always takes something, because it is a free number fitted to the data, and the question is never whether it helps but whether it helps as much as it would if it were true.
Why a walk gives up a slice and a gap gives up nearly all
The share is measured in the currency the ratio already uses. The one-habit fit leaves a disagreement over the strips’ separate fits; the two-habit fit leaves a smaller one; the share is the fraction of the first that the second removed. A picture whose share is 1 disagreed only by something one more habit could absorb. A picture whose share is 0 gained nothing from it.
For two hands with distinct habits, everything the strips disagree by beyond their slips is the gap between the hands, and the gap is exactly one number. What the second habit leaves is the tiring hands’ own noise in the strips’ fitted habits and creeps, which is small. The share is therefore set by the size of the gap against that noise, and it does not change much with the number of strips: more strips bring more gap and more noise in the same proportion.
A walk is different. Each strip’s walk is independent of every other’s, so the strips’ disagreement is spread across every direction the separate fits used, two per strip. A second habit is one direction, and the best division of the strips into two groups chooses that direction to take as much of the disagreement as it can. With four strips there are few directions to spread over and the best of seven divisions catches a great deal. With sixteen strips the disagreement is spread over thirty directions and the best division, however well chosen, catches a smaller share.
That is what the clouds do. On four strips the stepping hand’s median share is 0.74 and the two hands’ 0.96; on eight, 0.72 and 0.93; on sixteen, 0.59 and 0.91. The two hands’ cloud stays near the top whatever the count. The walk’s cloud sinks as strips are added, because a single division has to account for more independent disagreements and the best of them can explain less of the whole.
The line drawn at 0.85 was not chosen from these pictures. It was set on a hundred other paintings of eight strips by each kind, where it is the value at which the two mistakes balance — 93 of 100 two-hand pictures above it and 90 of the 96 stepping pictures the creeps convicted below — and it is then held fixed. Hollow rings in the figure are pictures the first test never convicted, so they are never asked the second question; on four strips nine of the forty stepping pictures fall there, the same weakness the creeps essay found on short pictures. On eight strips the line names 34 of 40 two-hand pictures and 37 of 40 stepping pictures. On sixteen it names every stepping picture and 31 of 40 two-hand ones: the walk has fallen well clear of the line, and the two hands’ cloud, with its median at 0.91, now has a lower tail that crosses it. A line set on sixteen-strip paintings would sit lower, but it would have to be set on such paintings rather than guessed from these.
The share, not what is left over
There is a second reading available from the same refit, and it is the one the obvious proposal implied: ask whether what is left after the second habit falls back below the line the creeps were judged by. A picture by two hands should drop back to where one tiring hand’s pictures sit, near or under one. A walk should stay high.
The remainder reading is right about two hands as often as the share is, and badly wrong about walks: on four and on eight strips it calls a stepping hand two painters about half the time. The reason is that a walk’s size varies enormously from one painting to the next. Across fifty eight-strip stepping pictures the creeps essay measured, the ratio ran from 1.37 to 18.18, because the walks a slip of 0.4 pixels a step produces are sometimes nearly straight and sometimes bend hard. Take a walk that happened to be small, give it a second habit, and what remains is small too, and falls under any line set against the slip. The remainder is a measurement of how much walk is left in absolute terms, and a small walk has little to leave.
The share does not care how large the walk was. It asks what fraction of the disagreement one number could take, and a small walk spread across eight strips is as unlike one gap as a large one is. That is why the share holds the stepping pictures on eight strips at 93 per cent while the remainder lets half of them through. The two readings are drawn from the same two fits; the difference is entirely in which ratio of them is compared with a line.
On sixteen strips the remainder recovers, 85 per cent for both, because thirty degrees of freedom are enough for even a modest walk’s remainder to clear the line. By then the share reading is at 89 per cent and limited by the two hands’ tail, not by walks. Neither reading is useful on four strips, where only seven divisions are possible and the best of them takes most of anything: the share names walks 57 times in a hundred, little better than a coin.
A map of who painted what, blurred where the edge is unknown
The split the second habit chooses is not only a verdict. It assigns every strip to one of two groups, and if the picture really was painted by two hands the groups should be the hands. The rows count hands, not cameras argued that a divergent picture’s strips reporting different habits are the signature of more than one painter. This is the first reading that says which strips belong to which.
On eight strips the reading puts 94 per cent of strips with their own painter. It is not equally good everywhere, and the place it errs was not the one the obvious guess predicts. A weakly splayed strip — the floor, at 1.12 — might be expected to say least about its painter, since its rows are nearly even whatever the habit. Instead the floor’s strips are assigned best, every one on eight strips, and the table’s worst, 85 per cent.
The cause is the edge the reader does not know. A strip keeps its ratio, not the end it began showed that the rows of a crept strip do not say which edge the painter divided from, so the refit chooses each strip’s edge for itself. On a steep strip that freedom lets one habit counterfeit the other. With the hands’ shared creep, a table strip of habit 2 divided from its near edge is drawn by habit 1 divided from its far edge to within 0.39 pixels root mean square, a book strip to within 0.42 — less than the slip on any single row. On the floor no such twin exists: the nearest is 1.85 pixels away, four times the slip. The table’s strips are misplaced because they can be read either way, and the floor’s are placed correctly because they cannot.
The whole-picture figures are sobering. With eight strips the map is perfect 57 times in a hundred; with sixteen, 15. More strips make the verdict surer and the map’s mistakes more numerous, because each strip is another chance for one twin to be taken for the other. A reader who wants a map rather than a verdict should treat each steep strip’s assignment as a suggestion and each weakly splayed strip’s as evidence, which is the reverse of the instinct that a weak splay says less.
Half a habit apart is the hardest gap
The two hands here differ by a whole habit, which is large: it is half the distance between a camera’s spacing, habit −1, and a hand’s even spacing, habit 1. Two painters trained in one workshop may differ by much less.
The verdict runs in two stages — first whether the strips disagree past the creeps’ line at all, then whether one more habit takes enough of the disagreement — and the gap moves each picture through all three answers. At a quarter of a habit, sixty per cent of pictures are not convicted; their strips agree as well as one tiring hand’s would, and the honest reading is that the picture is consistent with one steady painter. At one habit, 83 per cent are named as two hands. At two, all of them.
Between, at half a habit, something worse happens than failure to detect. Seventy per cent of the pictures are convicted and then called a stepping hand. The gap is large enough to make the strips disagree and not large enough for one number to take nearly all of their disagreement, because at that size the gap and the tiring hands’ own noise are comparable, so the share lands in the walk’s range. A reading that stops at the first stage says, correctly, “not one steady hand”. The second stage, applied to a picture at this gap, says confidently and wrongly “one hand that stepped”.
This is the reading’s real limit. The line at 0.85 was set on hands one habit apart, and against a smaller gap it is the wrong line: a share of 0.7 means something different when the alternative is a narrow gap rather than a wide one. A reader who suspects two hands of a common training would need a line set on paintings at the gap they suspect, and would find it overlaps the walk’s cloud much more.
When both are true, the larger one is named
Every picture so far has had one cause of disagreement. A workshop of two painters who both stepped their dividers has two.
The reading asks which single explanation takes the disagreement, and a picture with both is given whichever is larger. With habits half a habit apart, the walks dominate and the picture is called one stepping hand 63 times in a hundred; with habits two apart, the gap dominates and it is called two hands 97 times. Neither answer is wrong. Each is half the truth, and the half it gives is the half that left the larger mark.
The map, though, survives the walk. With habits two apart, 93 per cent of strips are put with their own painter even though both painters walked, because a gap of two habits is much larger than either hand’s walk on any one strip. At one habit apart the map is still right 77 per cent of the time while the verdict is split. A reader can therefore be told a useful thing even when the verdict is unsure: which strips go together, and how confidently.
What one extra habit settles
Put together, the measurements answer the question the creeps essay left with a qualified yes. One more habit takes a median 93 per cent of two hands’ disagreement on eight strips and 72 per cent of a stepping hand’s, and the share against a line set on separate paintings tells the two apart 89 times in a hundred — 85 per cent of two-hand pictures and 93 of stepping ones. The remainder after the extra habit, the reading the proposal first suggested, manages 69, because a small walk leaves a small remainder.
The general point is about what an extra parameter is evidence of. Every extra parameter improves a fit, so improvement alone proves nothing; the reading the same series has used since a camera count needs a tolerance is that a model earns a number only by what the number takes relative to what it could take by chance. Here the chance is measured directly: the best division of a walk’s strips takes a share that falls from 0.74 on four strips to 0.59 on sixteen, so a number that takes 0.93 is doing more than any division of noise does. One camera means one horizon, not one point made the same argument for cameras — fit what is shared once, and a part’s disagreement with it becomes evidence — and this is the next step: give the shared fit exactly one more degree of freedom, and the way the disagreement responds says what kind of disagreement it was.
What it cannot do is tell two causes apart when they are present in comparable amounts. Half a habit apart, two tiring hands look more like one stepping hand than like two hands; two stepping hands are named by whichever mark is larger. The reading is a choice between explanations, and a picture made by several causes gets the largest.
The premises the split rests on
The two hands share a creep. The refit gives each group its own habit and the picture one creep. Two painters who tire at different rates would differ in creep as well, and a second habit would take only part of that difference, pushing their pictures towards the walk’s range. A refit with two habits and two creeps is the natural extension, and it spends a second extra number that a walk could also use.
The two hands each paint both edges and every splay. The pattern was chosen so that neither hand is confounded with a kind of strip. A workshop that gave one painter the floors and the other the furniture would make hand and splay the same division, and the map could not say which it had found. Whether such a division is evidence of two hands or of two kinds of strip would then rest on knowing the habits are not splay-dependent.
The line is set on hands one habit apart. At other gaps it is the wrong line, as the half-habit pictures show. It also assumes the stepping slip of 0.4 pixels; a hand that steps much more carefully leaves walks too small to convict at all, and one that steps carelessly leaves walks whose division can take more.
The rows are measured far better than the slip. As for the creeps’ test, a measuring error independent from row to row inflates every strip’s residuals and pulls every share towards what noise alone produces. How far it moves the line was not measured here.
Still open: whether two groups are two painters or one painter twice
A second habit splits a picture into two groups that each agree within themselves, and a picture painted by one hand who changed habit halfway — after a break, after a lesson, after moving from the floor to the ceiling — splits the same way. The verdict cannot tell two people from two periods of one person, because both are two habits.
The order of painting can. A tiring panel keeps its order, not its direction found that a creep growing as a panel is worked leaves the strips’ painting order readable, up to reversal. A hand that changed habit once divides the panel into strips painted before the change and strips painted after, so its two groups are contiguous in that order; two hands working in turn interleave. The measurement that settles it paints a panel with a growing creep twice — once by one hand whose habit changes at a stated strip, once by two hands alternating — recovers each picture’s order and its two groups, and asks how often the groups come out contiguous in the recovered order for the one hand and interleaved for the two. The question with a number in it is whether the order, which six strips recovered nineteen times in twenty, survives the second habit’s own freedom, and how many strips a panel needs before “contiguous” can be told from “interleaved by chance”.
Shares its objects with
Essays that name at least two of the same things, and that neither author linked.
- A stepped hand passes for a tiring one on a short strip — both name degrees of freedom, drawing convention, least squares, model error, residual
- A strip's scatter points to the end drawn last — both name degrees of freedom, drawing convention, least squares, model error, residual
- The wedge recovered with the camera — both name degrees of freedom, identifiability, least squares, model error, residual
- A panel's creeps keep the order it was painted in, while the hand is steady — both name drawing convention, identifiability, least squares, residual
- A wedge moves the centre, not the lens — both name identifiability, least squares, model error, residual
- How many lamps a drawing has — both name identifiability, model error, overfitting, residual
Named objects
A flat tag is an object no other essay names yet.
degrees of freedomDrawing conventionIdentifiabilityleast squaresModel errorOverfittingResidual