No basis is good at both

Two of this collection’s arguments have been running in parallel for a long time without meeting. One is about adaptation: a change of light is a matrix, an adapted observer answers it with a diagonal, and which basis the diagonal is taken in decides how much is left over. The other is about difference: a lightness–chroma space applies a nonlinearity after a basis, and which basis decides how badly the discrimination ellipses fail to be circles. Both are decisions about the same nine numbers, and until they are put on one pair of axes there is no way to see that they disagree.

The claim #

The two objectives are nearly opposed. The basis that minimises the adaptation residual leaves an ellipse anisotropy of 7.70 — worse than every published cone basis and well past a display’s own primaries. The basis that minimises the anisotropy leaves an adaptation residual of 1.79 — worse than the receptors and worse than three of the four transforms in ordinary use.

• Best at adapting: 0.974 ΔE00, anisotropy 7.70.
• Best at discriminating: anisotropy 1.61, residual 1.79 ΔE00.
• The receptors: 1.65 and 2.60 — sixth of eight on the first, second of eight on the second, and within a factor of two of both floors, which three of the eight manage.
• CAT16 is the least badly placed of the published transforms, at 1.31 and 2.71; Bradford, which most colour management actually uses, buys 0.17 units of adaptation for a 36 per cent worse discrimination geometry.
• Nothing here says the eye is a compromise. What is shown is that two engineering objectives over the same numbers point apart, and that the direction they point away from is the receptors.

Two arguments that were never on one axis #

The first argument is an old one here. A change of illumination acts on a three-dimensional family of surfaces as an exact 3×3, an adapted observer applies a diagonal in some basis, and the mean CIEDE2000 left over is a score for that basis. Published transforms differ by nearly a factor of two on it.

The second is recent. CIELAB divides by a white point and takes a cube root, and neither operation commutes with a change of basis — so a lightness–chroma space is a property of the observer and of a basis chosen before the nonlinearity. Applying the same arithmetic after six different bases gives mean ellipse axis ratios from 2.60 to 12.45.

Each argument was made about its own set of candidates and each ended in a table. What neither did was score a candidate on the other’s measure, and there is no reason it would have occurred to either: adaptation transforms and colour-difference spaces are different chapters of different books, produced by different committees, and a basis is a parameter in both without anybody saying so.

Putting them on one pair of axes #

The two scores are never combined into one number here, and the refusal is deliberate. An adaptation residual is a ΔE00 and an anisotropy is a ratio of two lengths; a weighted sum of them would be an exchange rate between how wrong a white balance is and how oval a discrimination contour is, and inventing one would be exactly the undeclared decision these essays are about.

So the picture is a scatter, and it is read the way a scatter is read.

The rank correlation is negative. XYZ scaling is bad at both, which is unsurprising. Every other entry that is good at one is poor at the other, and the two winners are each other’s worst entries but one.

The walk, again #

The clearest form of the result is not the table but the path.

Thirteen stops, each a legitimate basis with its rows renormalised on the white. Adaptation improves from 1.65 to 0.97 monotonically. Anisotropy rises from 2.60 to 7.70. If the two objectives were merely uncorrelated there would be a stop somewhere on the path where one had improved and the other had not yet suffered. There is not.

Why they point apart #

The mechanism is the same one that makes sharpening work, read in both directions.

A diagonal reproduces a matrix exactly when the matrix is diagonal in that basis. Broad overlapping channels turn a spectral change into three correlated signals — an off-diagonal — so adaptation wants channels as narrow and separate as possible. That is what the winner is: three narrow, nearly independent bands.

A difference formula wants something else entirely. The discrimination ellipses are small and their shapes are set by how a small change in the stimulus divides among the channels. Narrow channels make that division lopsided: a stimulus that lies mostly in one band produces a large signal there and almost nothing elsewhere, so a step in one direction is enormously more visible than a step in another, and the ellipse is long and thin. Overlapping channels spread every change across all three, which is what makes the contour rounder.

Overlap is bad for the first job and good for the second. The cones overlap heavily, and the two objectives are two ways of asking whether that overlap helps.

Where the receptors sit #

Not first at either. Sixth of eight at adapting and second of eight at discriminating, and — the observation this essay exists for — the only basis in the table within a factor of two of both floors.

That sentence needs guarding, because it invites a story the measurement does not support. It is tempting to read it as the eye striking a balance between two demands, and nothing here shows that. The two objectives are artefacts of two models the discipline invented; the retina is not minimising either; and a basis that is mediocre at two arbitrary things is what a point not chosen for either usually looks like.

What is supportable is the negative version, and it is worth having: no single objective explains where the cone fundamentals are. They are not where adaptation wants them, they are not quite where discrimination wants them, and any account that derives them from one desideratum has to explain the other.

The generalisation #

Two objectives written over the same parameters are a decision, and a discipline that reports only one of them has made the decision without recording it.

That is a general shape and it is easy to state and hard to notice, because the two objectives usually live in different documents. Nothing about the CIE’s recommendation of CAT16 is wrong; it is a good transform on the criterion it was chosen for. What is missing is any statement anywhere that choosing it is also choosing a position on a second criterion that a different committee cares about.

The diagnostic that reveals it is cheap: find each objective’s own optimum and score it on the other. If the two winners are each other’s ordinary entries, the objectives are compatible and there is nothing to decide. If each winner is the other’s worst or nearly worst — which is what happens here — then every candidate in use is a compromise, and the compromises have been arrived at by people who were optimising one thing.

The habit generalises past colour without effort. It is the same question as asking what a regularisation parameter costs on a criterion the regulariser was not chosen for, and the answer is usually available for the price of one extra evaluation.

Who found each half, and when #

The two literatures are almost exactly contemporaneous and have almost no overlap in their citations.

Von Kries’s diagonal is 1902. The idea of choosing its basis to make the diagonal work better — rather than to model a receptor — belongs to the computational colour constancy work of the early 1990s, and Bradford’s entries date from 1993. CAT02 followed in 2002 and CAT16 in 2016, each fitted to a corresponding-colour data set and each reported with a residual.

CIELAB is 1976 and was built on XYZ because that is what the 1931 committee had standardised, for reasons about non-negativity that have nothing to do with discrimination. Every attempt since to improve perceptual uniformity has either changed the nonlinearity, changed the difference formula, or added parametric factors. Changing the basis before the nonlinearity has been done — CIECAM’s space uses CAT16’s axes — but as a by-product of using an appearance model rather than as a choice made on uniformity grounds.

So neither field has been careless. Each has been optimising its own quantity with its own data, and the parameter they share has been invisible from both sides because it is called something different in each: the adaptation basis in one, the space it is built on in the other. Putting the two scores on one pair of axes is the entire contribution here, and it took nothing more than running two existing pieces of this collection’s machinery over one list of eight matrices.

Row by row, and ellipse by ellipse #

An average can hide a reversal that only happens on a subset, so both objectives are worth looking at unaveraged.

The adaptation ranking is not uniform, and how it fails to be is the useful part. Against these two rivals the optimum has the lowest residual on eight of the fourteen rows. The six it loses are the blackbody comparison, three discharge lamps, the three-emitter source and the macular one — all of them either narrowband or already easy, and none of them large. What it wins are the hard rows: two bounces off a green wall at 2.21 against 3.75 and 3.37, tungsten at 0.60 against 2.28 and 1.64.

So a fitted basis does not improve everything a little. It gives ground on the changes that were already cheap and takes it on the ones that were expensive, which is what minimising a mean over a census does and is worth seeing rather than assuming. The receptor basis, by contrast, is the best of the three on the blackbody row and the three-emitter row and nothing else — for the reason narrowband sources reverse every ranking here.

The anisotropy is a mean over twenty-five shapes and the shapes are not all alike: at the floor of 1.61 the roundest of them is nearly circular and the worst is still more than twice as long as it is wide. So the number being minimised is an average over a set whose members disagree, and a space that reached 1.0 would be one in which a step of a given size means the same thing in every direction everywhere, which no space does.

Both objectives therefore survive being taken apart, and neither survives in the way a reader would guess from its summary. One wins its average by trading small rows for large; the other averages over shapes that stay stubbornly various.

What was computed, and how #

Both objectives are older than this essay and neither was written for it.

The adaptation residual computes the exact change matrix for each of fourteen illumination changes, applies the diagonal an adapted observer would use, and takes the mean CIEDE2000 over a hundred and twenty-five constructed surfaces. The anisotropy carries the boundary points of MacAdam’s twenty-five measured ellipses through the candidate basis and CIELAB’s arithmetic and takes the mean of the longest radius over the shortest.

Both are minimised by the same Nelder–Mead routine over the same nine parameters from the same starting point. Both objectives are invariant to the scale of a row, so both searches have three exactly flat directions and both need restarts for the same reason.

The one thing worth checking is that each winner really is a winner. The adaptation optimum is the lowest point on the horizontal axis of the scatter and the discrimination optimum is the lowest on the vertical, and the figure asserts it: a search that had failed would show up as a marked point that something else beats.

What a working pipeline actually contains #

The eight bases in the table are not equally consequential. Four of them are in daily use in software that renders images, and it is worth saying which decision each represents.

Bradford is what the ICC’s own colour management applies when a profile’s white point differs from the connection space’s, so it is running, unnoticed, in every operating system’s display pipeline. It is the best of the published transforms at adapting, at 1.14, and the worst but one at discriminating, at 3.68.

CAT16 is CIECAM16’s own step, so it is what runs inside any appearance model, any gamut-mapping algorithm built on one, and the newer difference formulae. It is 1.31 and 2.71 — second-best of the published set on one and best on the other.

CAT02 is CIECAM02’s, withdrawn in practice for going negative on saturated colours, and it is 1.32 and 4.29: no better than CAT16 at what it was fitted for and considerably worse at what it was not.

XYZ scaling is what happens when a piece of software adapts a white point without a transform at all, which is more common than it should be. It is 2.37 and 3.44, and it is bad at both for the same reason: it is not a cone basis of any kind.

The pattern is that the two transforms in widest use sit at opposite ends of a disagreement neither was chosen against, and that the difference between them on the unmeasured objective is larger than the difference on the measured one.

The diagonal is empty because nobody has drawn the front #

A scatter with a winner at each end and nothing in the middle invites the reading that the middle is unreachable. It is not. The middle is unoccupied because none of the eight candidates was built to sit there.

Minimising a weighted sum of the two objectives, each divided by its own floor, and sweeping the weight, produces a curve of bases the picture does not contain. At a weight of 0.4 on discrimination, from five independent starts, the best basis found sits at an adaptation residual of 1.073 and an anisotropy of 1.997 — 10.2 per cent above the first floor and 23.9 per cent above the second.

That single basis beats every published entry in the table on both objectives at once. Against CAT16’s 1.312 and 2.706 it is 18 per cent better on the first and 26 on the second; against Bradford’s 1.140 and 3.680, 6 and 46; against the receptors’ 1.651 and 2.596, 35 and 23; against XYZ’s 2.373 and 3.443, 55 and 42.

So the empty diagonal is not a front. Sampling the actual front at ten weights and asking which of the eight it dominates: all six published bases are dominated, several of them heavily, and only the two constructed optima survive — as they must, each being an endpoint of the curve.

What the front’s shape says instead #

The two objectives are nearly opposed at their optima and not in the region between them.

At either end the other objective does collapse — 7.70 at the adaptation optimum, 1.79 at the discrimination one — and that much is not in doubt. But the curve joining those two points is not the straight line they suggest. It has a knee, the knee is deep, and a basis sitting on it pays a tenth of one floor and a quarter of the other at the same time, which is a place where both objectives are nearly satisfied.

That changes what the picture is evidence for. It is not evidence that the two objectives cannot both be met. It is evidence that nobody has ever tried to meet both — each published transform was fitted against one criterion by people who did not have the other in front of them, and a fit against one criterion has no reason to land near a two-criterion knee.

The dominance check among the eight makes the same point with no optimisation at all. Three of them are beaten on both objectives by another entry in their own table: XYZ by the receptors, the Hunt–Pointer–Estévez axes by CAT16, and CAT02 by CAT16. Three of eight candidates in a comparison being beaten on every axis by another candidate in the same comparison is a statement about how the candidates were assembled rather than about the trade between them.

What this does not overturn #

The structural argument survives intact and gets sharper.

Two objectives written over the same nine numbers do point apart. Nothing reaches both floors, and 7.70 against 1.79 at the two optima is not a small disagreement. What the front adds is the size of the genuine trade: between the two floors there is a basis costing a tenth of one and a quarter of the other, so the real price of serving both is about a third of a floor’s worth of performance rather than the factor of five the endpoints imply.

And the receptors are still not that basis. They sit at 1.651 and 2.596 and are dominated by the knee on both counts, which is a more precise version of best at neither than the table on its own can give.

Where the model stops #

Two objectives is not all the objectives. Noise amplification, gamut behaviour, hue linearity and computational cost are all decided by the same nine numbers, and none of them is here. A third axis could change the picture — it could contain a point good at everything, though nothing suggests it does.

The anisotropy is measured at one luminance. The ellipses are placed at a fixed Y, so the axis ratio being minimised is a shape in a plane through the space rather than in the space. Whether the ellipsoids agree is a separate question with its own data.

And a residual is not a judgement. Neither number is a report of what anybody saw. The first is a colorimetric distance after a modelled adaptation; the second is a geometric property of measured thresholds after a transformation. Matching is not appearance, and both of these live on the matching side of that line.

Where the ladder goes next #

If two objectives disagree, the transforms in daily use are each an unstated position on the disagreement — and one of them is doing both jobs at once. The matrix inside an appearance model adapts in the same axes it then applies its nonlinearity in, so a single 3×3 decides both scores and was selected against one of them.