Best on the average, undefined at the edge

A default is chosen on an average and is met at every input a system receives, and the two are not the same test.

The claim #

The transform with the lowest mean residual over this collection’s census of illumination changes is one of two whose gain changes sign inside the family that census was drawn from. Where that happens the von Kries model is not inaccurate; it is undefined.

• On the mean, Bradford wins: 1.140 ΔE00, against CAT16’s 1.312, CAT02’s 1.323, Hunt–Pointer–Estévez’s 1.584 and XYZ scaling’s 2.373.
• Under a deep narrow notch, Bradford’s middle gain is −1.0 × 10¹⁹. Its middle row’s reading of the second white passes through zero, and the gain is that reading divided into the first.
• CAT02 does the same thing at 5.3 × 10¹⁸, and Hunt–Pointer–Estévez stays positive and reaches 5,800, which is not a gain any adapted observer applies.
• CAT16 and XYZ scaling stay finite, at 21.3 and 18.2 ΔE00 — and XYZ scaling, the oldest mistake still shipping and the worst of the five on the mean, is the better of the two there.
• So the two rankings disagree, and the disagreement is not a matter of degree: one list is of residuals and the other is of whether the model has a value.

What a gain is, and where it can fail #

An adapted observer, in the von Kries model, resolves a tristimulus triple along three directions, multiplies each component by a number, and reassembles: B⁻¹ diag(d) B. The three numbers are the ratios of the two whites read in that basis — dᵢ = (B wₐ)ᵢ / (B w_b)ᵢ — and are not fitted to anything. The whole model is that one line.

There is a division in it. If (B w_b)ᵢ is small, dᵢ is large; if it passes through zero, dᵢ passes through infinity and changes sign.

Whether that can happen depends on the basis. A basis whose rows are all non-negative — every entry positive — reads every physically realisable spectrum as a positive number, because a tristimulus triple of a real light has all three components positive and a positive combination of positives is positive. Such a basis cannot divide by zero.

Three of the five transforms in this collection’s table have negative entries: Bradford has three, CAT16 has three, Hunt–Pointer–Estévez has two, CAT02 has two, and XYZ scaling has none. Negative entries are not a defect on their own — none of these matrices is a set of cone responses and none needs to be — but they are what makes a sign change possible.

Where it happens #

Not at any illuminant on anybody’s list. The lights in the census — daylights, a tungsten lamp, three discharge sources — all read positive in every one of the five bases, and their gains are ordinary numbers between about a half and about three.

It happens in a painted room. A wall is a reflectance with a notch in it; a corner applies the same reflectance twice; and light that has bounced three times off a wall with a deep narrow notch at the short-wavelength end has a spectrum with almost nothing left in the band the third row of these matrices weights most. Read through a row with a large negative entry, the resulting number is a difference of two nearly equal quantities, and it crosses zero.

Under the worst such paint the search finds — a 25 nm notch at 420 nm, 81 per cent deep on an 11 per cent base, three bounces — the five bases read:

The residual quoted for a divergent row is not worth reporting. A mean CIEDE2000 saturates: two colours far enough apart are about a hundred units apart however much further they go, so a model that has produced nonsense returns 140 and reads as though it were merely bad. It is not bad there. It has no value.

What that says about a default #

Bradford is not an academic curiosity. It is the chromatic adaptation transform the ICC specifies for converting between profiles with different white points, which makes it the transform that runs when a document made under D50 is shown on a display calibrated to D65 — which is nearly every document.

The failure mode described here is not reachable through that path. An ICC transform adapts between two standard white points, both of which are ordinary daylights, and the gains are all near one. Nothing about a photograph of a green room passes a green room’s illuminant through the adaptation stage; what passes through is D50 and D65.

So the practical exposure is narrow, and saying so is more useful than an alarm. Where it does bite is anywhere the scene illuminant is used as the adapting white: appearance modelling under a measured light, a camera pipeline that estimates the illuminant from the image and adapts to it, a viewing-condition calculation in a room whose walls are painted. An estimated illuminant is an estimate, and an estimate that lands in the wrong part of the spectrum lands in a part of the model that has no value there.

Why CAT16 survives and CAT02 does not #

CAT02 was withdrawn from CIECAM’s recommended path because it produced negative or otherwise implausible cone-like responses for saturated colours in practice, and CAT16 was constructed to fix exactly that. The fix worked, and this is the first measurement in this collection that shows it working on something other than the cases it was designed against.

Both matrices have negative entries; that is not the difference. The difference is where the negative entries are and how large they are relative to the positive ones in the same row. CAT16’s third row reads the shortwave-starved spectrum at a value that is small and positive; CAT02’s reads it at a value that crosses zero.

That is a narrow escape rather than a design guarantee, and it should be read that way. Nothing in CAT16’s derivation bounds its rows away from zero on arbitrary spectra, and a search over a wider family than the one here would be an obvious next thing to run. What can be said is that on the family this collection models, the transform that exists because its predecessor went negative does not go negative.

The order the two rankings put things in #

Setting the two columns beside each other is the whole result, and the individual reversals are worth naming because each says something different.

XYZ scaling moves from last to first among the finite entries. It is the transform with no basis in it at all — adapt by scaling the tristimulus values themselves — and this collection has called it the oldest mistake still shipping, correctly, on the mean. Its rows are the coordinate axes and have no negative entries, so it cannot divide by zero, and at the edge it is the best-behaved of the five at 18.2. Being crude and being robust are compatible, and the pairing is common enough to deserve suspicion whenever a sophisticated method wins narrowly.

CAT16 moves from third to second, at 21.3, which is close enough to XYZ scaling’s 18.2 that the two are not meaningfully separated. What separates them from the rest is not accuracy; it is having a value.

Hunt–Pointer–Estévez moves from fourth to nowhere in particular. Its gains stay positive and reach 5,800, which is not a sign change and is not a usable number either — a gain of 5,800 multiplies a channel by nearly four orders of magnitude, and whatever comes out the other side is not a prediction about an observer.

And Bradford moves from first to undefined, which is the sentence this essay exists for.

The four reversals share a shape. The mean rewards a transform for being fitted, and every transform here except XYZ scaling was fitted — to corresponding-colour judgements, under ordinary illuminants, using stimuli within a gamut. A fit is a statement about the region it was fitted in, and nothing in a residual says how far that region extends.

What was computed, and how #

The gains are computed exactly as the model specifies: the two contexts’ whites, read in the candidate basis, divided channel by channel. Nothing here is a search over gains; the gain is determined by the two lights and the basis.

The paint is found by maximising the residual over a four-parameter wall family, and it is that search — rather than the divergence — that the assertion in the build is careful about. A divergent gain is easy to produce by allowing an arbitrary spectrum; the claim being made is that it happens inside a family of paints, so the box is restricted to what an ordinary architectural pigment can be and the search is required to end inside that box in at least one parameter.

The assertion requires three things: that the transform with the lowest mean residual is one whose gain changes sign; that CAT16 does not; and that at least one basis which is worse on the mean is finite at the edge, so that the two rankings genuinely disagree rather than one merely being incomplete. XYZ scaling supplies the third, at 18.2 against CAT16’s 21.3.

The box is a declaration rather than a measurement, and two other declarations are available.

The tightest of the three bounds is the one a specifier would actually work inside, and it is the one that makes the worst case smallest.

The cheapest defence #

If the exposure is narrow and the failure is catastrophic where it happens, the useful question is what a system should do about it, and the answer is a check rather than a different transform.

Look at the gains. Three numbers, computed anyway, on every adaptation. A gain that is negative is a model outside its domain; a gain outside a range of roughly a tenth to ten is a model being asked something implausible. Both are one comparison, both are free, and neither requires knowing which transform is in use.

That is worth stating because the alternative defences are worse. Clamping the adapting white to a nearby daylight throws away the thing the calculation was for. Choosing a non-negative basis — XYZ scaling, or a cone-like construction with positive rows — costs a great deal on the mean, which is where the system spends its life. Switching to CAT16 helps on this family and is not a guarantee on any other.

A domain check is the only defence that is honest about what is known, which is that the model has an edge, that the edge has been located in one family, and that nobody has looked for it in the others.

All five bases nearly annihilate the white; two land on the wrong side #

The table of gains is presented as separating two bases that fail from three that do not, and read as a single quantity it separates them much less than that.

The largest gain in each row is the reciprocal of how much of the reference the starved white still reads as. Across the five:

Every one of the five reads the wall’s light as between one per cent and nothing in its most starved direction. The wall has removed almost all of the spectrum in that band, and no choice of basis recovers it — what a basis decides is only whether the sliver that remains lands above zero or below.

That reframes the finding usefully. Bradford and CAT02 are not doing something qualitatively different from Hunt–Pointer–Estévez; all three are reading a number that is within a hundredth of a per cent of zero, and two of them happen to be on the wrong side. A basis a fraction of a degree away from either would swap them, and nothing in the matrices says which side any of them will land on for a spectrum nobody has tried.

So the useful quantity is the margin, not the sign. XYZ scaling’s 1.06 per cent is five times CAT16’s 0.20 and sixty times Hunt–Pointer–Estévez’s 0.017, and that ordering is the same as the robustness ordering with the two divergent rows added at the bottom. A system wanting to be safe here should be choosing the basis with the largest margin rather than the one whose sign happens to hold.

The bounces cube the notch #

The wall’s parameters and the number of bounces do the whole of the work, and the arithmetic is a cube.

A base reflectance of 0.11 with an 81 per cent notch has a floor of 0.0209, so the notch is 5.26 times darker than the rest of the band after one bounce. After three it is 146 times darker, because the contrast is cubed.

In absolute terms three bounces leave 9.1 × 10⁻⁶ of the incident light inside the notch and 1.3 × 10⁻³ outside it. The band is gone, and the row that weights it is reading a residue five orders of magnitude below what it reads elsewhere.

That is why the number of bounces matters more than the paint. One bounce off the same wall leaves a 5.3-fold notch, which no basis in the table reads anywhere near zero; two leave 28-fold; three leave 146. The failure is a property of the corner rather than of the pigment, and it arrives at the third bounce because a cube of five is a hundred and forty-six.

It also says what a bound would look like. A search restricted to one bounce would not have found this at all, and a search allowing four would find it on a shallower notch. The reachable edge is a statement about the enclosure as much as about the paint, which is a variable the census’s wall rows carry and its illuminant rows do not.

What the two finite bases are worth against each other #

The two survivors are quoted at 18.2 and 21.3 and the gap between them is smaller than it looks.

XYZ scaling is 17 per cent better than CAT16 at the edge, having been 108 per cent worse on the mean — so the reversal is enormous in one direction and modest in the other. The essay’s close enough that the two are not meaningfully separated is right, and worth putting beside the fact that the mean-ranking gap it reverses is six times larger.

Their gain spreads say the same thing more directly. XYZ’s three gains span 4.0 to 94.4, a factor of 24; CAT16’s span 7.9 to 500, a factor of 63. So CAT16 is asking for a more extreme rebalancing and delivers a slightly larger residual, which is what one would expect and is the whole of the difference between them.

Neither is good there. Eighteen and twenty-one ΔE₀₀ are both catastrophic in any ordinary sense; the distinction being drawn is between a bad answer and no answer, and once that line is crossed the two survivors are on the same side of it and nothing else about them matters.

Where the model stops #

A sign change is a property of the model, not of vision — and no adaptation removes everything even where it is defined. No observer’s response passes through zero and comes back negative. What has failed is a linear model of an observer, applied to a light it was never fitted against, and the honest description is that the model’s domain of validity has an edge and the edge has been found.

The wall family is a caricature. A Gaussian notch on a grey base is four numbers, and real pigments are not that. What the caricature establishes is that the edge is reachable by a smooth, four-parameter, physically-signed reflectance applied a small number of times — not that any particular room does it.

And the census is a sample. Every mean in the left-hand column of the table above is a mean over fourteen changes somebody chose, and that list is not the family. A different census would move the means and might move their order; it would not move the sign change, which is a property of a matrix and a spectrum.

The generalisation #

A default is met at every input, and an average is a statement about typical inputs, so choosing a default on an average is choosing on the wrong statistic.

The useful form is not never use averages. It is a two-column habit: report the mean and the domain of validity, and rank on both. A method that is five per cent better on average and undefined on a reachable input is not five per cent better; it is a different kind of object, and the comparison that put them in the same column was the mistake.

The tell that this is happening is a saturating error metric. Where a bad answer and a meaningless one produce similar numbers — a mean CIEDE2000, a clipped percentage, an accuracy that bottoms out at chance — the summary statistic cannot distinguish wrong from undefined, and the distinction has to be made by checking the model’s own internals instead. Here that check is one line: look at the gains, not at the residual.

The second half is about what negative entries mean. A matrix with negative entries is a matrix that can read a physically realisable input as zero, and whether it does depends on which inputs exist. That is a question about the input family, so it cannot be answered by looking at the matrix — which is why a property this consequential has stayed invisible in a table of matrices for thirty years.

Who found it, and when #

That CAT02 could produce negative responses in practice is well documented — it is the stated reason for CAT16’s existence in the 2016 revision of CIECAM, and the failures reported there are for saturated stimuli under ordinary illuminants rather than for ordinary stimuli under extreme ones. The mechanism is the same: a row with a negative entry reading a spectrum concentrated where that entry sits.

Bradford’s exposure to the same mechanism does not appear to have been reported, and the reason is visible in how it is used. It is the ICC’s adaptation and the ICC’s whites are standard illuminants, so the input family it meets in practice is small and safe. It has never been asked what it does under an arbitrary adapting light, because nothing in its normal use supplies one.

Its normal use is not its only use. Any appearance calculation that adopts a measured scene white — which is what a viewing-condition specification asks for — hands it exactly the input nobody tested.

Where the ladder goes next #

Everything so far has held the observer fixed and varied the light, the basis or the constraint. The remaining question the census cannot answer is what happens when the observer varies — because the receptor construction that all of this is priced against rests on three points measured on one population, and the price it charges turns out to move more than the table it is being compared in.