The appearance model takes XYZ

An appearance model is the most elaborate machinery in this collection, and everything it does happens downstream of three numbers. This round is about how those three numbers are made.

The claim #

An appearance model’s input is a projection through a standard observer, so every departure this round measures is applied before the model is called and none of them is visible to it.

• CIECAM16 takes XYZ, a white point, an adapting luminance, a background and a surround. The first two carry the observer; the last three are the viewing condition.
• Six departures totalling about three colour differences are already in the input by the time the model sees it.
• The model then applies its own cone transform, CAT16’s, which is not the physiological basis and is a further fifteen units away from it.
• And the model cannot compensate, because a model whose input is three numbers cannot recover the spectrum those numbers came from.

What the model’s inputs are #

CIECAM16 is a function of a stimulus’s tristimulus values, the tristimulus values of the adopted white, an adapting luminance, a relative background luminance and a categorical surround. It returns lightness, brightness, chroma, colourfulness, saturation and hue.

A viewing condition is an argument established the structural point about the last three: an appearance model differs from a matching model precisely in taking them, and that is why it can say a patch looks different in two rooms while a colorimetric calculation says it is the same colour.

What has never been said here is what the first two arguments are. They are integrals of spectra against three tabulated functions — the observer, applied before the model exists — and the model has no argument for which observer produced them.

So the model is exact about everything it is given and inherits everything about how the giving was done. That is the ordinary situation for any model taking pre-processed input and it is worth naming because the pre-processing here is a contract rather than a measurement.

Two observers inside one calculation #

There is a subtlety that makes this worse than a simple inheritance, and this collection has already found half of it.

The cones an appearance model uses established that CIECAM16’s internal cone space is CAT16’s, which is a sharpened basis fitted to corresponding-colour data rather than a set of physiological fundamentals. The model transforms its input XYZ into that space, adapts, applies its nonlinearities and comes back out.

So an appearance prediction passes through two observer-like objects: the standard observer that made the XYZ, and CAT16’s cones that the model adapts in. Neither is a receptor set, and this round measures that the second is fifteen colour differences from the physiological basis at the median.

The two are not composable into one. The standard observer decides what the stimulus is; CAT16’s cones decide how the adaptation is done to it; and a change in either moves the answer in a different way.

A model with two internal observers has two places for the audit’s terms to enter, and the literature discusses the second and not the first.

What the model would need to see them #

A model that could account for observer variation would need something a colour cannot supply.

The departures are spectral: they depend on where the stimulus’s structure sits relative to where an observer’s parameters live. Three tristimulus values have thrown that away — that is what a projection onto three numbers means — so no function of XYZ can recover how much observer spread the stimulus carries.

The model would have to take the spectrum as its argument. That is not a small change: CIECAM16’s whole structure is a chain of transforms on three numbers, and every published test vector, every implementation and every ICC workflow is built on that interface.

A spectral appearance model is not an unheard-of idea and nothing in this collection is one. What is worth noting is that the barrier is the interface rather than the arithmetic: a model taking a spectrum could apply any observer, compute the spread, and return an appearance with an uncertainty on it, and everything it needs beyond the current model is already available.

What an observer argument would give #

A cheaper version is available and it is what the CIE’s own 2006 fundamental observer suggests.

An appearance model could take an observer as an argument beside its viewing condition — an age and a field size, which is what the 2006 observer is parameterised by. It would compute the stimulus’s tristimulus values through that observer rather than being handed them, which means the interface moves upstream but only by one step.

That would let an appearance prediction be made for a specific person, which is the thing every appearance model’s evaluation data is an average over. It would also let the spread be computed by running the model twice.

Nothing prevents it. The obstacles are that the input to most workflows is already XYZ by the time an appearance model is reached, and that the corresponding-colour data the models were fitted to are themselves averages over unreported panels, so a per-observer model would be fitted on population-average data.

A model parameterised by an observer and fitted on an average is a model that is precise about the wrong thing, which is a real objection and is not a reason to leave the parameter out.

The one thing the model does absorb #

There is a term the model handles well and it is worth crediting, because it is the one most likely to be confused with the ones it does not.

An appearance model’s adaptation step divides by a white point, and a great deal of what a change of observer does to a stimulus is shared with what it does to the white. So the relative effect is smaller than the absolute one, and an appearance prediction is less exposed than a raw colorimetric one.

That is the pairing structure again, and it is why a neutral is everyone’s colour — a stimulus that is a scalar multiple of the white is unaffected by any observer, and an appearance model computing on the ratio inherits that identity exactly.

So the model absorbs the part of the observer’s departure that is common to the stimulus and the white, and is exposed to the part that is not. That is the same division the whole round has been making, and it means an appearance prediction’s exposure scales with the stimulus’s departure from the white in the same linear way everything else does.

Putting the ladder beside the model’s argument list is the clearest statement of the essay’s point. CIECAM16 takes five arguments and is exact about all of them; six more exist, they are of comparable size to the effects the model predicts, and there is nowhere to put them.

That comparison is worth making carefully. The model’s own predictions — the Hunt effect, the surround’s effect on lightness contrast, the shift in colourfulness with adapting luminance — are effects of between a few per cent and a factor of two on their respective correlates. The observer departures are two to three colour differences. Those are not directly comparable quantities and they are the same order of consequence.

So an appearance prediction for a specific person carries an uncertainty comparable with the phenomena the model exists to predict. That is not an argument against the model; it is an argument for the parameter it does not take.

What this does to the model’s evaluation #

The audit has something to say about how appearance models are judged against each other, and it is a caution rather than a finding.

Models are compared by how well they predict measured corresponding colours: pairs of stimuli reported to look alike under two different lights. The measurements come from panels of observers and are averaged, and the averaging is over people whose lens densities, macular pigments and cone peaks differ by the amounts this round measures.

So the data a model is fitted to carry an observer spread of a few colour differences, and models are being distinguished by residuals of a similar size. A fit can be exact and empty, and a comparison between two models whose residuals differ by less than the data’s own observer spread is a comparison the data cannot support.

This collection cannot check that, because it has no corresponding-colour data — an absence recorded for six rounds and now recorded for a seventh, with one more reason attached.

What was computed, and how #

Nothing is recomputed. The six departures and the fifteen-unit route gap are this round’s, and the description of CIECAM16’s interface is from the model as implemented in this collection, which is checked against the published test vector to four significant figures in all six correlates.

The claim that no function of XYZ can recover the spectral information is a statement about the projection rather than a measurement: three numbers are three numbers, and a metameric black is invisible to them by construction.

The claim that the model absorbs the common part follows from its adaptation step being a division by the white, which is the same argument the round’s identities rest on.

One further consequence is worth drawing out for anybody building a soft-proofing or colour-appraisal application on top of an appearance model. Such an application predicts what a viewer will see, in a stated room, from a stated stimulus, and presents the result as a picture or a number.

The prediction is exact for the standard observer and carries the round’s spread for the actual viewer. So a soft proof shown to a fifty-year-old is a prediction for a young eye, presented without qualification, and the gap is largest exactly for the saturated colours a proof is usually being consulted about.

The repair is the same one the round has proposed everywhere: compute the allowance, which needs the spectrum and is available in a spectral workflow, and present it. A prediction with an uncertainty is a different object from a prediction, and every part of the machinery needed to produce the first already exists.

Where the model stops #

Everything here is about the inputs. Nothing in this round examines CIECAM16’s internal parameters — its nonlinearity exponent, its surround constants, its hue quadrature — which have their own uncertainties and their own fitted origins.

The audit’s observers are constructed rather than tabulated, so the departures fed into an appearance calculation would carry the construction’s residual as well.

And the whole essay is about a matching uncertainty entering an appearance calculation. Whether a person with a different observer also has a different appearance response is a question about neural processing downstream of the receptors, and nothing here bears on it. It is quite possible that the visual system’s own normalisation absorbs a large part of what the arithmetic reports, which is the argument the lens essay makes and cannot settle.

The two internal observers could be reconciled. There is a version of the model that would remove one of the two, and it is worth sketching because it is closer than it sounds.

If the model took cone responses rather than tristimulus values, its first internal transform would disappear: the stimulus would arrive in a receptor basis and the adaptation would be applied there. The question of which cone space to adapt in would become the question of which cone fundamentals to use, and those are measurable objects.

What stops it is that the model’s adaptation transform was fitted in a sharpened basis, and it was fitted there because a receptor-basis gain does not predict corresponding colours as well. So moving the interface to the receptors would require refitting the adaptation on data the field has and this collection does not, and the refit might well be worse.

That is the same standing difficulty the basis work has recorded from several directions: the basis that is physiologically motivated and the basis that fits the behaviour are not the same, and every model in use has chosen the second. This round adds the size of the gap and does not resolve it.

The field size is the sharpest illustration of where the interface sits, because it is a viewing-condition parameter that is also an observer parameter and the model takes viewing conditions.

CIECAM16 takes a surround, a background luminance and an adapting luminance — three properties of the room — and no field size, because a field size acts on the observer rather than on the room and the observer arrives already applied. So a model that carefully distinguishes a dim surround from an average one is silent about whether the sample subtends two degrees or ten, which is worth 1.33 ΔE₀₀ and is a property of the same viewing situation.

That is the interface’s boundary drawn in one example: everything about the room is an argument and everything about the eye is upstream, and the field size sits on the line and went upstream.

The generalisation #

The habit is about the interface a model is written against.

A model’s interface fixes what it can be right or wrong about. A model taking three numbers can be a very good model of everything downstream of those numbers and can say nothing whatever about how they were made — and if the making carries a term larger than the model’s own accuracy, the model’s accuracy is not the limiting factor.

The move is to look upstream of an interface before improving what is downstream of it. That is unfashionable, because downstream is where the model is and upstream is somebody else’s code.

The failure mode is a chain of well-validated components each accurate to a tenth of a unit, connected by an interface that discards the information a term of three units depends on. The accuracy of a chain is not the accuracy of its parts, and the losses are at the joins.

A last note about the size of what is at stake, because the essay could be read as claiming an appearance model is unreliable and it does not. CIECAM16 is checked here against the published test vector to four significant figures in all six correlates and round-trips across the whole sRGB cube to 4 × 10⁻¹³ under three surrounds and five decades of adapting luminance. It is an exact and well-behaved piece of arithmetic.

What the round says is that its inputs are a projection through a contract and its internal adaptation is in a fitted basis, and that both carry terms of a few colour differences that the model neither creates nor can see. Those are statements about the surrounding machinery rather than about the model, and they would be true of any model with the same interface.

Who found it, and when #

CIECAM97s and its successors have taken tristimulus values since the beginning, because that is what a colour management pipeline has and because the models were built to slot into one.

The CIE’s 2006 fundamental observer, parameterised by age and field size, is the standardised object that would let an appearance model take an observer argument, and it has been available since 2006. No appearance model in general use takes one, and the reason is the interface rather than the science.

One thing this round does leave the appearance work in better shape about. Its two internal observers are now measured against each other and against the physiology, so a reader of any appearance result here knows that the adaptation basis contributes and by how much. That was previously a known choice with no number attached, and a choice with a number is a different object from a choice.

Where the ladder goes next #

The audit’s terms have now been traced through every part of this collection’s machinery. What remains is the case they were always about: two observers and one metamer, which is where an observer difference stops being an abstraction and becomes a disagreement about whether two things match.