Which measurement is worth making

A ranking is only useful if somebody could act on it, and three different rankings of the same four numbers are available here.

Two other constants of the same machinery have been priced the same way, and they say what the ranking looks like when the quantity has no declaration behind it.

The claim #

Asking which of four measurements is worth improving has three plausible readings, they give three different answers, and the one that is usually computed is the one with nothing attached to it.

• By contribution, the lens wins outright: eighty-one per cent of the population’s disagreement on a narrowband match, against the macular pigment’s forty-six. This is the ranking three essays here have quoted since the population was built.
• By doubt — how much each width’s own uncertainty moves what is published — the macular pigment comes first at 0.65 against the lens’s 0.60. The margin is eight per cent, so the honest reading is that the two are interchangeable rather than that one beat the other.
• By which one breaks something, the answer is neither. The pigment peaks, third on both other lists, are the width that takes two of this collection’s three published statements about confusion points nearest to failing.
• The third question is the one an experiment can be designed around, because it names a claim as well as a measurement.
• And a fourth reading is available and empty: the cone optical density is last on every list, at a doubt of 0.07 against the leader’s 0.65, so a better measurement of it would change nothing here whatever it found.

Why contribution is not the same as doubt #

The distinction is easy to state and easy to skip past. A term can be large and precisely known, and a term can be small and a guess, and improving a measurement is worth something only in the second case.

The attribution this collection already has answers the first question. It holds each variate live in turn with the other three at their medians and reports what share of the disagreement each accounts for. That is a decomposition of the answer, and the lens dominates it because fifty years of yellowing is a large optical change and the other three are small ones.

The second question multiplies each variate’s elasticity — how strongly a published number responds to that width — by how badly that width is itself known. The lens’s own uncertainty is not really an uncertainty about eyes at all: it is a fact about who a study recruited, and a span of ages is a design decision rather than a measurement error. The macular pigment’s is a genuine disagreement between methods, at about a factor of two.

There is a second reason the two questions come apart, and it is arithmetic rather than epistemology. The attribution measures each variate’s contribution to one quantity — the disagreement over a narrowband match — because that is the quantity the population model was built to produce. The doubt ranking sums over six quantities, so a variate with a moderate response to all of them can outrank one with a large response to a single one. The macular pigment is exactly that: never the largest term anywhere, and never absent.

So the product reorders them, and by very little. 0.649 against 0.601 is a lead of eight per cent, which is the wrong size to call a reversal and the right size to say the two leading terms are not distinguishable on this evidence. Below the top the two lists are identical, which is worth reporting because it is what a reader would not predict: the reordering does not cascade.

The two leaders lead everywhere #

Eight per cent between the top two invites the question of whether the ordering is stable claim by claim, and it turns out to be far more stable than the margin suggests.

Ranked by doubt on each of the seven claims separately, the lens and the macular pigment take first and second place on every one of them, in one order or the other. Neither is ever third. The lens leads on four, the pigment on three, and the cone density and the pigment peaks lead on nothing.

So the summary and the detail agree, which is not guaranteed and is the useful part. The two totals are eight per cent apart because the two terms alternate — each wins outright somewhere — rather than because one of them is spread thinly across claims it never wins. A ranking whose top two swap places from row to row and never admit a third is a ranking with two leaders and no first place, which is a stronger reading of the eight per cent than a tie.

One row deserves its own sentence. The standard observer’s own position in the population reads 0.000 against all four widths: the one published claim of the seven that none of these measurements moves at all. It contributes nothing to either ranking, so the totals are a sum over six claims wearing the label of seven — which is worth knowing before designing a campaign against them.

The reading with an action attached #

Neither ranking says what to do, because neither names a consequence. A width that carries forty per cent of a number’s uncertainty is only worth measuring better if that number is doing something, and here several of them are: they are the numbers in statements, with thresholds written into the checks this site runs on every build.

The claim that a fitted adaptation transform is not a set of cone responses rests on three distances measured in the population’s own standard deviations: the protanope’s and the deuteranope’s implied confusion points at least two σ outside the population’s clouds, and the tritanope’s inside four. Those are not quantities, they are lines that can be crossed.

Their margins — how far each measured value sits from its own line — are 1.12, 1.87 and 1.54. Their headroom — the factor by which one declared width would have to be wrong for each to cross — is 1.34, 1.55 and 1.79. The two orderings are not the same ordering: the middle two change places, because the deuteranope’s cloud is the one that grows fastest when the pigment peaks are allowed to spread.

And every one of the three is inside a factor of two. That is the number worth carrying away from this essay. The margins say two of the three are comfortable; the headroom says none of them is.

The two orderings differ because a margin and a headroom divide by different things. A margin divides a measured value by a threshold and is a fact about where the number sits. A headroom divides a width by its declared value and is a fact about how steeply the number responds — so a claim with plenty of room and a steep response can be less safe than one with little room and a shallow one. The deuteranope’s statement has two thirds more room than the protanope’s and reaches its line at a factor only fifteen per cent further out.

That is not a subtlety about this model. It is the reason a table of margins, which is what a specification usually contains, cannot be read as a table of risks.

Which width does the breaking #

The third ranking falls out of the same table read down its columns rather than across its rows.

• The pigment peaks take the protanope’s statement under its threshold at a factor of 1.34 and the deuteranope’s at 1.55 — both of them, and soonest.
• The lens range takes the tritanope’s statement over its threshold at 1.79.
• The macular pigment is close behind on two of the three and first on none.
• The cone density breaks nothing inside a factor of eight.

The mechanism is visible in the picture above and is not a coincidence. The clouds are not the same size, and they are not the same shape. The protanope’s is the tightest — a root-mean-square radius of 0.025 in chromaticity — so a σ is a small unit there and a small widening of the cloud moves a σ-distance a long way. The deuteranope’s cloud is the one the pigment peaks move most, because a deuteranope’s confusions are about the two long-wavelength pigments and those are exactly the two whose peaks the width is scaling.

So the answer to which measurement is worth making is: the peak wavelengths of the L and M pigments, because they are the width that decides whether two of this collection’s published sentences remain true. That is not the answer either of the other two rankings gives, and it is the only one of the three that names an experiment.

The one that breaks nothing #

The cone outer segment’s optical density is last on every ranking here, and it is worth a paragraph because a null result of this shape is actionable in the other direction.

Its elasticities are 0.035, 0.055, 0.037, 0.008, 0.063 and 0.042 — an order of magnitude below the others on every quantity. Its declared span is a factor of two, the largest in the set, and even multiplied through it contributes a doubt of 0.074 against the leader’s 0.649. No width of it inside a factor of eight breaks any of the three statements.

So a better measurement of the cone optical density would change nothing this collection publishes. That is a fact about this model rather than about the eye: the density enters as an axial self-screening term that broadens a pigment’s absorption without moving its peak, and broadening a pigment without moving it turns out to be almost invisible to every quantity here — a spread over a population, a distance between two members, or the position of a confusion point.

An experiment planned from the attribution alone would have put the density third, at twenty-eight per cent of the answer. It is third by that measure and useless by every other.

What was computed, and how #

Each headroom is a bisection. One width is multiplied by a factor, the claim is recomputed, and the search narrows on the factor at which the claim crosses its threshold — twenty-two halvings on the logarithm of the multiplier, over a range of a factor of eight either way.

A claim that does not cross inside that range is reported as not reachable rather than as a large number. An extrapolated multiplier of fourteen is not a statement about anything, and the temptation to quote one is the same temptation this collection has already caught itself out on twice.

The search does not assume monotonicity beyond checking that the far end has crossed. A claim that crossed and came back would be reported at its first crossing, which is the conservative reading and is what a threshold means: the statement stops being true the first time it stops being true.

One number in the table is a control and is exactly zero. The median member’s distance from the standard observer responds to no width at all, because scaling a width leaves every median where it was. Its margin against its own threshold is 2.07 and its headroom is infinite — the only genuinely safe claim in the set, and safe for a structural reason rather than by luck.

Where the model stops #

A doubt ranking depends on the spans, and the spans are declared. They are round numbers chosen to be defensible: a factor of two on the two psychophysical densities, 1.6 on the age range, 1.4 on the peaks. A reader who disagrees can multiply differently, and the elasticities — which are the part computed from this collection’s own machinery — do not move when they do. That separation is deliberate: the measured half and the asserted half are kept in different columns.

A headroom ranking depends on the thresholds, and those are not arbitrary but they are choices. Two standard deviations is a convention. Four is a convention. If the claim had been written at three σ instead of two, the protanope’s statement would already be false and this essay would be about something else.

And a threshold crossed is not a claim refuted. If the pigment peaks turn out to vary by a third more than this model gives them, the protanope’s σ-distance falls under two and the sentence as written stops holding — but the underlying observation, that the published transforms are nowhere near anybody’s receptors on two of the three points, does not evaporate at 1.99σ. What fails is a particular way of saying it. The right response to a small headroom is to find a form of the claim that does not depend on the width, not to stop believing the claim.

And none of this says the claim is wrong. It says the claim is a claim about a population whose width was chosen, and that the width would have to be about a third larger for the sentence to stop holding. Whether a third larger is plausible is a question about the individual-observer literature that this collection cannot answer by computation — which is exactly why the number is reported as a factor rather than as a probability.

The generalisation #

The pattern generalises past colour and is worth stating in the form that makes it checkable.

Before improving a measurement, find the claim it is nearest to breaking. A sensitivity analysis that ranks inputs by how much they move an output is answering a question about the output; a decision about what to measure next is a question about a threshold, and the two rankings agree only when every conclusion is equally close to its own edge.

The reason the first is computed so much more often is that it needs less. An elasticity needs a model and a perturbation. A headroom needs a model, a perturbation, and somebody to have written down what the model is being used to claim — and that last requirement is the one that fails, because most models are used to produce numbers rather than statements.

This collection is in an unusual position to run the check, because its claims are already written as assertions with thresholds in them — that is what makes a caption a fact about a drawing rather than a promise. The thresholds were written to catch a regression in the machinery. They turn out to double as the anchors a value-of-information calculation needs, which was not why they were put there.

Who found it, and when #

The distinction between a variance decomposition and a value-of-information calculation is standard in decision analysis and in the risk literature, where the second is usually called the expected value of perfect information and is computed against a decision with a stated loss. This essay computes something coarser — a distance to a threshold rather than an expectation over a loss — because there is no loss function here and inventing one would be inventing the answer.

Colour science does propagate observer variation, and has done since the 1980s when individual-observer models began to be fitted; the CIE’s own set of individual observer functions exists for that purpose. What is rare is the second derivative of the same question: not how much does an observer’s variation move this number, but how much does an error in the model of that variation move it, and which of the model’s own parameters is worth pinning down first.

Where the ladder goes next #

Three of the six numbers audited here are statements with thresholds and three are quantities without them, and the three with thresholds turn out to be the exposed ones. That is a suggestive division, and it points at the question the next rung asks of the whole collection rather than of one model: for every published claim on this site, what is the smallest thing that would have to be wrong for it to stop being true?

The answer is a table with one alarming row in it and a great many reassuring ones, and the alarming row is not the one this essay would have predicted.

The same three-way split — a contribution, a doubt and a distance to a line — has an exact counterpart one level up, where the object being ranked is not a variate but a colour space in a table of eight. There the widths belong to the ruler rather than to the observer, and the question of which adjacent pair in the ranking is really ordered has two answers that disagree.