The three numbers a gain cannot see

Nine numbers of the observer are loose, and the arithmetic that fixes them has been counted twice on this site without either count noticing the other. Put the two together and something falls out that neither predicted: an adaptation model cannot see three of the nine, and the three it cannot see are exactly the three the dichromat experiment does not supply.

Two more views say that the invisible directions are invisible to the other objective as well, which is what makes them a property of the arithmetic rather than of one criterion.

The claim #

A von Kries gain is exactly invariant to the scale of each row of its basis. Three of the nine free numbers therefore do nothing at all to an adaptation model — and they are the same three the confusion-point construction has to invent. The six that remain are supplied outright by dichromat data, so the receptor basis is not a starting point for a fit. It is the answer to a fit with no free parameters.

• The invariance is exact. Scaling the rows by 3.7, 0.21 and 11 moves the residual on every change of light in the census by at most 7 × 10⁻¹⁵ ΔE00.
• The arithmetic is one line. A gain in basis B is B⁻¹ diag(d) B, and a row’s scale appears once in B and once in B⁻¹. It cancels.
• So the counts line up. Six numbers come from the three copunctal points and three from a choice of units; the model sees the six and is blind to the three.
• And the determined basis is not the best one. It leaves 1.65 ΔE00 averaged over the census, against 1.14 for Bradford, 1.31 for CAT16 and 0.97 for a basis free to be any nine numbers at all.
• Which is a result rather than a disappointment, because it is the first time either quantity has been put beside the other.

What was already counted, twice #

The freedom itself is a theorem and it is settled. A colour match is an equality between two triples of integrals, so applying any nonsingular 3×3 to the colour-matching functions leaves every match exactly where it was: the matrix is applied to both sides of an equation and is therefore applied to nothing. Matching data of any quantity and any precision leave the observer’s three curves undetermined up to nine numbers, and the residual over sixty-four random bases is machine zero.

What closes the freedom is a different experiment. A dichromat is missing one cone class, so the stimuli they confuse lie on lines that all meet at one point, and that point is the chromaticity of the missing cone’s own response direction. Three classes give three points, a point is two numbers, and three times two is six. The remaining three are the scale of each row, fixed here by requiring the adopting white to come out as unit signals in all three channels.

The other count is older and belongs to adaptation. A change of illumination is exactly a 3×3 matrix on a three-dimensional family of surfaces, and an adapted observer answers it with a diagonal in some basis — which basis being the whole question. The literature’s answers are fitted: Bradford, CAT02 and CAT16 were all obtained by minimising error over corresponding-colour data, and all three have entries no cone response could have.

Nobody appears to have asked what happens if the two counts are put beside each other. They are counts of the same nine numbers.

The row scale does nothing, and the proof is shorter than the measurement #

Write the adapted observer’s operation out. Tristimulus values go into the basis, each channel is multiplied by the ratio of the two whites read in that basis, and the result comes back:

The gain is d_i = (B w_a)_i / (B w_b)_i. Replace row i of B by k·row i. Then (B w)_i becomes k (B w)_i for both whites, so d_i is unchanged. And in B⁻¹, column i is divided by k, which cancels the k that row i of B introduced. The whole operator B⁻¹ diag(d) B is identical.

The same statement can be made without any algebra at all, and it is worth having both. A basis is three directions in tristimulus space; what the model does is decompose a signal along those directions, multiply each component by a number, and put it back. How long the three measuring sticks are cannot matter, because whatever a longer stick does to the decomposition, the reassembly undoes. The scale of a row is a unit, and the operator is a statement about directions.

There are three rows, so there are three such numbers, and they are dead. That is not a small technical point about parametrisation. It means the space of distinguishable adaptation bases is six-dimensional, not nine — and six is exactly what three confusion points supply.

Which makes the confusion points a determination, not a constraint #

The usual shape of an argument like this is that data constrain a model and a fit finds the best remaining parameters. Here there are none. The three copunctal directions fix the three rows up to scale, and the scale is invisible, so the basis is decided.

That is worth stating plainly because it removes an escape route. If the receptor basis performed badly at adapting, one might reasonably suspect the normalisation, or the choice of adopting white, or some other convention downstream of the measurement. There is nothing to suspect. Every normalisation gives the identical operator, and the construction returns the published Smith–Pokorny fundamentals to four decimals when run backwards, so the algebra is the discipline’s own.

What it costs #

Averaged over every change of illumination this collection models — daylights, thermal radiators, discharge lamps, coloured walls and two filters inside the eye — the receptor basis leaves 1.65 ΔE00 after the gain an adapted observer actually applies.

The comparisons that matter are these. Hunt–Pointer–Estévez leaves 1.58 — barely better, which is a small surprise on its own, because that matrix exists precisely to make a von Kries gain behave and it beats the actual receptors by four per cent. Bradford leaves 1.14 and is the transform most colour management uses. CAT16 leaves 1.31. And a search over all nine numbers reaches 0.97, which is the floor.

So imposing the confusion points costs seventy per cent against the floor. It is not a rounding error and it is not a catastrophe; it is the size of the gap between a physiological account and an engineering one, quantified for the first time on this site.

Whether any eye closes the gap #

The determined basis leaves 1.65 against a floor of 0.97, and the natural question is whether that deficit belongs to one particular construction or to eyes.

Scoring four hundred members of this collection’s own population — each basis built from that member’s own three confusion points, so each is a determination rather than the determination — the adaptation cost runs from 1.187 at best to 2.443 at worst, with a median of 1.758 and a fifth-to-ninety-fifth range of 1.431 to 2.138.

The best of the four hundred is 21.8 per cent above the floor. Not one member reaches it, and the gap between the luckiest eye in the set and a basis free to be any nine numbers at all is a quarter of a unit. So the deficit is not an artefact of the reference construction. It is what six numbers supplied by a retina cost against six chosen by a search.

Setting the published transforms against the same population makes the ordering uncomfortable. Not one of the four hundred beats Bradford at 1.140 — zero out of four hundred. Eight of them beat CAT16 at 1.312, which is two per cent. The fitted transforms are not merely better than the reference receptor basis; they are better than almost every eye in a population of eyes.

The reference construction itself sits at the 31st percentile, with 123 of the 400 doing better. So it is a good member rather than an exceptional one, which is what it ought to be, being built from published median confusion points rather than from any optimum.

Those three facts together say what the six numbers are worth. They are supplied outright, which is the finding above and is worth a great deal. They are not chosen well in the sense a fit means: the whole population occupies a band beginning 22 per cent above the floor and ending 151 per cent above it, and every published transform sits below the band’s lower edge.

None of which is a criticism of anything. A retina is not solving this problem; a matrix fitted to corresponding-colour data is solving exactly this problem and nothing else. What the comparison establishes is that the two are genuinely different objects, and that the fitted ones win at the job they were fitted for by a margin no eye in four hundred reaches.

The wrong lesson, and the right one #

The wrong lesson is that the eye is badly designed for its own adaptation. Nothing here supports that, and the sentence does not even parse: the eye is not applying a matrix to anything, and a von Kries diagonal is a model of adaptation rather than a mechanism inside a retina. What has been measured is how well this particular model, in these particular axes, reproduces the change a change of light makes.

The right lesson is about where the residual lives. A diagonal in the receptor basis fails by 1.65 units on average; the best diagonal in any basis fails by 0.97; and the difference between those two is not something a better transform could recover, because the matrix that would fix each change is a different matrix for every change and the observer has one mechanism. What the extra 0.68 buys is a basis chosen to be wrong in a more useful direction, on average, across a census somebody chose.

What this does to the site’s own simulations #

Every colour-vision figure in this collection still runs on Hunt–Pointer–Estévez, and what that choice costs the simulation is measured elsewhere here: 1.77 ΔE00 on average across the sRGB cube and 14.8 at worst. This essay adds a second column to the same decision. The matrix in use is 1.58 at adapting and 0.13 away in chromaticity from the measured protanope point; the construction from the confusion points is 1.65 and 0.00.

Neither is better at both. That is a shape these essays keep meeting and it is taken up properly elsewhere; what belongs here is only the observation that the choice cannot be made by measuring one thing.

What was computed, and how #

The residual is the one the essays on adaptation bases already used, and it predates this essay by a long way — which matters, because it was not written to make this comparison and could not have been tuned to it.

For each change of illumination, the change itself is computed exactly. A surface’s tristimulus values under light a and under light b are related by a 3×3 matrix T, and T is not fitted — it is R(b) R(a)⁻¹ where R is the response of the standard observer to the three-dimensional family of reflectances this collection constructs. The build asserts that this is exact, at a tolerance of 10⁻⁹, on every row.

What an adapted observer does about it is the diagonal d, read in the candidate basis as the ratio of the two whites. The residual is then the mean CIEDE2000 between where each surface actually is under the second light and where the gain puts it. Two hundred and forty constructed surfaces, fourteen changes, eight bases.

The invariance check multiplies row two of the receptor basis by 0.21, 1, 3.7 and 11 in turn and requires the residual on all fourteen rows to be unchanged to 10⁻¹². It is asserted at an identity’s tolerance rather than a fit’s, deliberately: if it were merely approximate then determined would have to be softened to nearly determined, and the whole argument above would be a fit with three badly conditioned parameters rather than no parameters at all.

The comparison against the floor uses a Nelder–Mead search over all nine entries with restarts, which finds 0.9741 at two hundred and fifty steps and four restarts and the same four decimals at six hundred and eight. The number quoted is therefore a floor that has stopped moving rather than the best a particular budget happened to find.

Who kept the two counts apart, and why #

Von Kries proposed the diagonal in 1902. König’s construction of cone fundamentals from dichromat confusion data is of the same era, and Smith and Pokorny’s fundamentals — the ones this collection checks its algebra against — are from 1975. So both halves of the arithmetic here are old, and one of them is very old.

The reason they were never put together is visible in who published them. The confusion-point construction belongs to colour-vision physiology, where the question is what are the three receptor sensitivities, and the answer is judged against dichromat data and against microspectrophotometry. Adaptation transforms belong to colour appearance modelling and colour management, where the question is what diagonal best reproduces a set of corresponding-colour judgements, and the answer is judged against a fit. The first field’s answer is a candidate for the second field’s parameter, and the second field has, for a century, preferred to fit it.

That preference is well founded on its own terms — the fitted transforms do better, and this essay measures by how much. What has been missing is the observation that fitting has nothing to fit. Six numbers determine the model completely; the other three are decoration; and the choice being made is not which values but which experiment.

Bradford’s entries were published in 1993 and CAT02’s in 2002, both from minimisations over corresponding-colour data sets, and both are routinely described in print as cone responses. They are not, and the distance is measurable: Bradford implies a deuteranope confusion point 2.42 away in chromaticity from the measured one and CAT02 implies one 4.09 away.

The generalisation #

The pattern here is not about colour. A model can be invariant to some of the parameters a measurement leaves free, and when it is, the useful question stops being how many numbers are loose and becomes which loose numbers the model can see.

Written that way it is nearly obvious, and it is nearly always skipped. Nine free numbers and six constraints reads as an underdetermined problem with three parameters left to fit. It is not: the three left over are in the model’s null space, so the problem is exactly determined and the fit that appears to remain is a fit over nothing. The same shape appears in a different guise elsewhere here, where a least-squares residual was flat in precisely the directions the data could not constrain — which is the mirror image of this, a model that cannot see what the data do not say.

The practical consequence is a habit rather than a formula. Before reporting that a model has k free parameters after a set of constraints, apply the model to two parametrisations that differ only in those parameters and see whether anything moves. Here nothing moved, at 7 × 10⁻¹⁵, and three of the nine numbers turned out to have been quietly irrelevant to every adaptation transform ever published.

Where the model stops #

Three of the nine numbers being invisible is a fact about a von Kries model and not about vision. A model with an off-diagonal term, or a nonlinearity, or a second stage would see them. The invariance is a property of B⁻¹ diag(d) B specifically, and it disappears the moment anything is done between the two matrices that is not a per-channel multiply.

The census is a construction. Fourteen changes of illumination, weighted equally, over this collection’s own family of smooth reflectances. A different census would move every number in the table by some amount, and what a census does to a ranking is a question with its own essay. What it would not move is the invariance, which is exact and has no census in it.

A residual is not an appearance error. What is being measured is the distance, in CIEDE2000, between the tristimulus values an adapted observer’s model produces and the ones the second light actually delivers. That is a colorimetric quantity computed downstream of an adaptation stage, and it is not the same thing as a judgement by a person that two patches in two rooms look alike. The gap between matching and appearance is the oldest distinction on this site and it applies to every number here.

And the confusion points are quoted measurements. They are among the small number of things on this site that are not computed from something else, and they carry the between-observer variation of the people who were measured, which is not small.

Where the ladder goes next #

The determined basis is worse than three fitted ones and the fitted ones are not cone-like. That raises the question this essay has carefully not answered: is the optimum near the receptors and merely missed, or does the diagonal model actively want axes the eye does not have? The answer is the second, and it is unambiguous.