Only one of these devices adapts

This site models four devices that colour passes through, and each of them meets the same problem. The light changes; the numbers change; something has to be done about it.

The eye applies a gain in a fixed basis it did not choose. A camera applies a gain in a basis that turns out to move with the light it is balancing. A display can shift its white point, which is a gain in its own primaries. And a press applies nothing whatever, because a sheet of paper does not adapt to the room it is read in.

Putting all four through the same census gives a table, and the table’s ordering is not the ordering anybody would predict from how good each device is at colour.

The claim #

Each device has at most one gain, in axes it did not choose, and the quality of those axes is unrelated to the quality of the device.

• The eye is best, at ΔE00 1.41 averaged over the census — CAT16’s axes, applied without fitting anything.
• A silicon camera is second, at 1.70. Its axes are three dye transmittances arrived at for cost and sensitivity, and they are better adaptation axes than a colorimetrically correct sensor’s.
• A display is third, at 2.43. Its axes are its primaries, which were chosen by a standards committee for gamut coverage.
• A colorimetric camera is fourth, at 2.52 — the worst of the three that have a mechanism, because its channels are the matching functions and a gain on those is the transform this site calls the oldest mistake still shipping.
• And a press is last at 15.82, which is not a mechanism failing but the absence of one: eleven times the eye’s residual, and equal to the change itself by construction.

The press column is not a residual #

The row that makes this a comparison rather than a ranking is the one with no mechanism in it.

Four of the five numbers are what is left after something was done. The press’s is what happens when nothing is. A printed sheet has a fixed reflectance; the light in the room multiplies it; and the sheet has no gain, no white balance, no adaptation of any kind. Whatever the change of light does, arrives.

The site’s gate asserts that the press column equals the census’s unadapted change exactly, to floating point, because a table in which four numbers mean one thing and one means another is a table that has to say so.

That is not a criticism of print, and reading it as one would miss the point. A press does not need a mechanism because the reader supplies it. What is worth noticing is where the compensation happens: for a camera it is in the device, for a print it is in the observer, and colour management’s substrate rule is the one place where a machine tries to do the reader’s adaptation on their behalf — in the worst available basis.

The camera that is right about colour and wrong about light #

The result that reorders the table is the two camera rows.

A silicon sensor leaves ΔE00 1.70 averaged over the census. A sensor satisfying the Luther condition exactly — sensitivities that are a linear combination of the matching functions, the theoretical ideal that no manufacturer has achieved — leaves 2.52. It is half again as bad.

Not at colour. At adapting. This sensor’s channels are the matching functions in nearly the proportions they come in, so applying a per-channel gain to them is nearly scaling X, Y and Z, which is last in every basis comparison this site has run.

The natural reading of that is that the two properties cannot both hold — a perfect colorimeter is a poor von Kries observer, and a sharpened cone-like sensor is a good one that has metamers of its own no matrix repairs. The natural reading is wrong, and it took measuring the family rather than the specimen to say so: the condition constrains the shapes of the three curves and leaves the mixture free, and the mixture is exactly what a colorimetric camera’s adaptation basis turns out to be. Mixed differently, the same ideal sensor reaches the best adaptation residual any basis achieves.

What is true of the comparison in this table is that these two particular cameras trade off that way, and that neither of them was designed with the other property in mind.

The display, and the axes a committee chose #

A display’s white point control multiplies R, G and B, so its adaptation basis is the inverse of its primary matrix. It averages ΔE00 2.43.

Those axes were not chosen for adaptation and were not chosen for colorimetry either. sRGB’s primaries came from the phosphors of a 1990s cathode-ray tube; P3’s came from a cinema projector’s filters; Rec. 2020’s are monochromatic wavelengths selected to enclose as much of the chromaticity diagram as possible. Every one of those is a decision about gamut, and each of them silently fixes the axes in which every white point adjustment on that display is performed.

The row where the display does worst is the one that says most: two bounces off a green wall leaves it ΔE00 10.50, against the eye’s 3.37 and the silicon camera’s 4.62. A display in a coloured room, adjusting its white point to compensate, is doing so along axes that are badly placed for that particular change — and there is no setting anywhere that would improve it. The room is already taking a great deal from the display before any of this.

Row by row, and the reversals #

The means hide two reversals worth stating.

On the daylight rows the camera beats the eye: 0.42 against 0.46 on D65 to D50, 0.91 against 0.98 on D65 to daylight at 4000 K. A camera’s dye channels are more sharpened than CAT16 and happen to sit closer to the axes daylight picks out — which is a coincidence of chemistry rather than a design achievement, and it does not survive a change to any other kind of light.

On the triphosphor row the colorimetric camera beats everything, at 1.54 against the eye’s 2.32. That is the same reversal the basis essay found: a broad, overlapping set of channels is hard to knock out of alignment by three narrow phosphor bands, and the matching functions are the broadest, most overlapping channels here.

So no device wins everywhere, and the device that wins on the hardest row is the one that loses on average. That is what a table of compromises looks like when none of the compromises was made deliberately.

The one row where every device fails together #

Sorting by device hides something the census makes obvious, which is that the rows disagree more than the columns do.

The spread across devices on any one change of light runs from 1.29 to 6.24, and is under three on nine of the twelve rows. The spread across changes of light for any one device runs from 12.8 to 34.0. Whatever a device is, the light it is asked about decides more about the outcome than the device does — and the row every one of them does badly on is the same row.

Two bounces off a green wall costs the eye 3.37, the silicon camera 4.62, the colorimetric one 5.10 and the display 10.50. It is the worst row for all four. A triphosphor tube is the second-worst for the eye and the silicon camera, and the best row of all for the colorimetric one.

The generalisation is the census’s, restated: what a gain can do about a change of light is mostly a property of the change. A device brings axes, and axes are worth a factor of two or three; the change brings its spectral shape, and that is worth a factor of ten.

Which means the useful engineering lever is not the device at all. It is the lamp, and it is under the control of whoever specifies the lighting rather than whoever designs the camera. A photograph taken under a broad source and one taken under three narrow emitters are two different problems, and no sensor design closes the gap between them.

How much of this table is the device at all #

The two spreads in the last section are order-of-magnitude statements and they are the wrong instrument for the claim they support, because each is a ratio of two extremes and throws away the other ten rows. The claim deserves the whole table.

Taking the logarithm of every residual and splitting the variance three ways — between rows, between devices, and what is left — gives 77.6 per cent to the change of light, 7.3 per cent to the device, and 15.2 per cent to the interaction between them. So the conclusion above survives its own arithmetic and then some: the light decides ten times as much as the device does, and even the interaction term — the part that is genuinely about which device meets which light — is twice the size of the device’s own main effect.

That reframes what the four columns are. They are not four qualities of adaptation. They are four small perturbations on a quantity that is mostly set before any device is chosen, and the ordering the essay opens with is a real ordering of a small effect.

Two of the extremes are worth naming, because both are larger than the earlier statement allowed. The widest disagreement between devices is on D65 to daylight at 10,000 K, where the display sits at 0.37 and the colorimetric camera at 2.30 — a factor of 6.2, on one of the easiest rows in the census. The widest range within a device is the display’s, from 0.31 to 10.50, a factor of 34. The display is both the most volatile column and the winner of the widest row, which is the same fact seen twice.

And there is a third reversal the row-by-row section does not mention: the display wins all three daylight rows outright, at 0.31, 0.77 and 0.37 against the eye’s 0.46, 0.98 and 0.51. Its primaries were chosen for gamut coverage by a committee that was not thinking about adaptation, and on the changes of light that actually happen outdoors they are the best axes on the page. Counting first places rather than means, the eye takes eight of the twelve rows, the display three and the colorimetric camera one — and the silicon camera, which is second on the mean, wins nothing at all.

What the ordering does not measure #

It is worth being explicit about what this table is not, because it is easy to read as a ranking of devices and it is not one.

It does not say a camera is nearly as good as an eye. It says that on the single operation of removing a change of illumination by a per-channel gain, the two are within twenty per cent. A camera also has to demosaic, has a fixed exposure, quantises, and cannot see what is behind it. It is a fourth observer with a different set of problems, of which this is one.

It does not say a press is bad. It says a press has no mechanism, which is a design fact rather than a failing, and that everything print does about changing light happens in the reader.

And it does not say the display is poorly designed. It says a display’s adaptation axes are a side effect of a gamut decision, which is true and is not something anybody has traded off, because the connection between the two has not been drawn.

What each device would have to record #

The table’s four columns are four decisions, and each was made without the information that would have let it be made deliberately. It is worth saying, for each, what would have to be written down.

A camera would have to publish its spectral sensitivities. They are measured during profiling and are almost never released; without them the basis its white balance is diagonal in cannot be computed by anybody outside the manufacturer, and neither can the drift. Two cameras with identical profile matrices can have differently placed bases, and no published quantity distinguishes them.

A display would have to publish its primary spectra rather than its primary chromaticities. Chromaticity is the quantity that has already been averaged over the observer, so a coverage figure cannot be turned back into anything about a population or about adaptation. The spectra are measured; the chromaticities are what is printed.

A press would have to record the substrate’s spectrum, which it does, and the basis its profile normalises in, which nothing does — because until somebody notices that dividing by the substrate is an adaptation transform there is no reason to think a basis was chosen.

And an eye needs nothing recorded, which is the asymmetry worth ending on. Its axes are three pigments and a fixed neural stage; they are the same axes in every room, for every stimulus, at every level. Every other device on this list has axes that are an accident of its construction, and two of the four have axes that move.

That is not an argument that the eye is well designed. It is an argument that a fixed basis is a strong property, that three of these four devices lack it, and that none of the three knows.

Who found it, and when #

Von Kries’s diagonal is 1902 and is the mechanism in three of these four columns. Luther’s condition is 1927. The observation that a sensor cannot satisfy the condition is as old as colour photography, and the observation that sharpening a basis improves adaptation is from the 1990s.

Putting the two together does not appear to have been done. The reason is probably that the two literatures belong to different trades: the Luther condition belongs to sensor design and colorimetry, and adaptation basis selection belongs to colour appearance modelling. A device is designed by the first group and adapted by the second, and nothing in either process asks what the first group’s decision did to the second’s.

What was computed, and how #

Each device’s basis is computed rather than assumed. The eye’s is CAT16. The camera’s is [S diag(E) R][A diag(E) R]⁻¹, exact on this site’s three-dimensional reflectance family and evaluated under the reference light of each census row. The display’s is the inverse of the sRGB primary matrix. The press has none, so its column is the unadapted change.

Every residual is the mean CIEDE2000 over a hundred and twenty-five surfaces after the ratio-of-whites gain in the named basis, with no fitting anywhere. The census’s twelve non-trivial rows are used; the level row and the two ocular rows are excluded, the first because it is the control and the second two because a filter inside an eye is not something a camera or a press has.

The gate requires the colorimetric camera to leave more than the silicon one by at least twenty per cent, requires the eye to beat both, and requires the press column to equal the census’s own change exactly.

Where it stops #

The colorimetric sensor is an idealisation with no noise, no infrared response and no manufacturing tolerance, and it exists here to isolate one property. It turned out to be carrying a second, undeclared one — the mixture of the matching functions its channels are made from, which nobody chose for this purpose and which decides its whole column. A real attempt to build one would fail the condition slightly and would sit between the two camera rows.

The display’s white point control is modelled as a diagonal on linear RGB, which is what the control does on most panels and is not what all of them do — some apply a full matrix, which is outside the arithmetic here and would do better.

And every device is assumed to know the white exactly. Estimating it is a separate problem with its own error, larger than most of the numbers in this table, and it is not modelled.

That last exclusion deserves its own sentence, because it is the one most likely to make these numbers read as more useful than they are. A grey-world estimator reaches forty-three degrees of angular error on an all-green scene, and an error of that size in the estimate of the white swamps every difference between the columns here. What this table compares is the mechanisms, with the input to each of them handed over correct. It is a comparison of ceilings, not of performances, and the ceilings turn out to be close enough together that in practice which device is in front of a scene matters far less than whether it worked out what the light was.

Where the ladder goes next #

The table has four columns and each of them is one decision made by somebody who was not thinking about adaptation. That suggests a question these essays can pose and not answer: what would a device look like if the adaptation axes were the thing being designed?

For a camera the answer is partly known — sharpened sensitivities are better adaptation axes and worse colorimetry, and the trade could be quantified. For a display it is unexplored: choosing primaries for the adaptation properties of their inverse, subject to a gamut floor, is exactly the shape of problem the multi-primary work already solves, and nobody has posed it.

And for the press the answer is that there is nothing to design, which is why the substrate rule is the interesting case: it is the one place where a machine performs the reader’s adaptation for them, and it is the one place where the axes were written into a specification and could be changed by amending it.