Colour Science & Perception
what the eye settles, and what it cannot
What a scene does

Three numbers the scene supplies

An adaptation model's parameters are not all the same kind of thing. Some are numbers an observer must estimate from the room it is standing in; others could have been settled once by evolution. Counting them separately turns the diagonal gain from a crude approximation into the only model of the set that gets a large answer from information the observer can actually have.

Assumes A gain needs a basis, A theorem about a family and What no adaptation can remove.

15 min read 7 figures Computed, not quotedThree numbersThe instrument is the reader

A change of light acting on this collection’s family of surfaces is exactly a 3×3 matrix, with no residual at all. The exact answer is available, it is nine numbers, and the visual system does not use it.

What an observer is left with, by how much it is allowed to know about the room. Six ways of discounting a change of light, averaged over the fourteen changes in the adaptation census and 125 test surfaces each. The bar is what each leaves behind, on a logarithmic axis because the models span two orders of magnitude. The second line under each name is the count that matters: how many numbers about this room the model has to be given. Doing nothing leaves 15.7 ΔE₀₀. A single gain read off the two whites' luminances leaves 15.3. A matrix fitted across half the census and then applied everywhere, knowing nothing about the room at all, leaves 12.5. The published von Kries gain, which is told the white and nothing else, leaves 1.312 — and bolting a fixed correction onto it, at no cost in scene information, leaves 1.368, which is very slightly worse. The exact matrix leaves nothing and is not on the chart: its nine numbers are the change of light, which is the quantity being discounted.
Fig. 1 Six ways of discounting a change of light, averaged over the adaptation census. The bar is what each leaves behind, on a logarithmic axis. The second line under each name is the count that matters: how many numbers about this room the model has to be told.

The claim

A model’s parameter count is a claim about what an observer can find out, and separating the parameters it must measure from the ones it could have been born with is what makes the diagonal gain look like a good model rather than a crude one.

  • The exact matrix needs nine numbers about the room, and those nine numbers are the change of light. An observer that could measure them would not need to discount them.
  • A von Kries gain needs three, and they are the white — a quantity a visual system has several plausible ways of estimating.
  • Those three remove 91.6 per cent of what a change of light costs an unadapted observer.
  • One of them alone removes 2.6 per cent. Scaling by the ratio of the two whites’ luminances, which is the easiest of the three to estimate and the only one a photographic meter measures, buys almost nothing.
  • And nine numbers with no access to the room remove 15.4 per cent, held out — more than the luminance, and not a model of anything.

Two kinds of parameter

Model complexity is usually counted as a single number, and for the usual purpose — how much can this fit before it starts fitting noise — that is right. It is the wrong count here.

A scene number is something the observer has to estimate from the room it is in, now, from the light arriving at its retina. It costs the observer a mechanism, it is subject to error, and it has to be re-estimated whenever the room changes.

A fixed number is something that could have been settled once and never estimated again — by evolution, by development, or in an engineered system by a calibration. It costs nothing at run time and cannot be wrong about a particular room, because it does not know about particular rooms.

The distinction decides the question this essay is about, and it collapses the moment the two are added together. The exact 3×3 has nine parameters and every one of them is a scene number, because the matrix is a property of the change of light. A model that requires the observer to know the change of light in order to discount it is not a model of adaptation; it is a restatement of the problem.

The ladder

Six models, ordered by how many numbers about the room each is told, averaged over the fourteen changes of light in the adaptation census and 125 test surfaces each.

model scene numbers fixed leaves
no adaptation at all 0 0 15.67
one matrix, the same in every room 0 9 12.48
a single gain on everything 1 0 15.25
a diagonal gain in a fixed basis 3 9 1.31
the diagonal gain, then a fixed correction 3 18 1.37
the whole change of light, inverted 9 0 0

The shape is a cliff. Nothing below three scene numbers gets within a factor of eight of the diagonal gain, and the diagonal gain gets within a factor of — well, of nothing, because the exact model leaves zero. All of the useful work is done by the step from one scene number to three.

That step is the white. The three numbers a von Kries gain needs are the tristimulus values of the light the observer is adapting to, expressed in a cone-like basis, and the mechanism applies the ratio of the new white to the old in each channel independently.

What the luminance alone buys, and why so little

The one-scene-number model is worth dwelling on because it is the one an engineer would try first.

Scale everything by the ratio of the two whites’ luminances. That is a single number, it is what a photographic light meter measures, and a visual system certainly has access to it. It removes 2.6 per cent of a change of light.

The reason is arithmetic and is the census’s own control row. A change of level — the same light, dimmer — is removed exactly by a scalar gain, and the census carries such a row precisely to establish that: its residual is zero to numerical noise. Every other row in the census is a change of colour, and a scalar cannot touch a change of colour at all.

So the 2.6 per cent is not the luminance gain doing a little of the job. It is the luminance gain doing all of its job on the small part of each change that happens to be a level shift, and nothing on the rest. A model that addresses the wrong dimension of a problem does not partially solve it.

Every test surface under each model — daylight to tungsten. The 125 surfaces of the test set, sorted for each model from the one it costs least to the one it costs most. Doing nothing is a nearly flat line high on the chart: a change of light of this kind moves almost every surface by about the same amount, which is why it looks like a change of light rather than like a change of scene. A luminance gain barely moves that line. The von Kries gain drops it to the floor and changes its shape: 5 of the 125 surfaces cost exactly nothing, because they are flat greys on which an adapted observer's gain is exactly right, and what is left rises steeply over the most saturated members. The published residual for this row is a mean over that curve.
Fig. 2 Daylight to tungsten, surface by surface, under each model. The unadapted line and the luminance-gain line sit on top of each other; the von Kries line drops to the floor and changes shape, with a block of surfaces at exactly zero.

Two more rows say whether what the white’s three numbers buy is the same on every change of light, and it is not.

Every test surface under each model — a green wall, two bounces. The 125 surfaces of the test set, sorted for each model from the one it costs least to the one it costs most. Doing nothing is a nearly flat line high on the chart: a change of light of this kind moves almost every surface by about the same amount, which is why it looks like a change of light rather than like a change of scene. A luminance gain barely moves that line. The von Kries gain drops it to the floor and changes its shape: 5 of the 125 surfaces cost exactly nothing, because they are flat greys on which an adapted observer's gain is exactly right, and what is left rises steeply over the most saturated members. The published residual for this row is a mean over that curve.
Fig. 3 The harshest row in the census, surface by surface. The white’s three numbers remove a smaller share here than anywhere else, because a corner is not a change of white in any useful sense.

A filter inside the observer is the row where the question stops making sense at all, because there is no white in it for a scene to supply.

Every test surface under each model — the macular pigment. The 125 surfaces of the test set, sorted for each model from the one it costs least to the one it costs most. Doing nothing is a nearly flat line high on the chart: a change of light of this kind moves almost every surface by about the same amount, which is why it looks like a change of light rather than like a change of scene. A luminance gain barely moves that line. The von Kries gain drops it to the floor and changes its shape: 5 of the 125 surfaces cost exactly nothing, because they are flat greys on which an adapted observer's gain is exactly right, and what is left rises steeply over the most saturated members. The published residual for this row is a mean over that curve.
Fig. 4 And a filter inside the observer rather than a change of lamp. There is no white to estimate here at all, and the three numbers a scene supplies are three numbers about the wrong object.
How much of each change of light three numbers about the room remove. The ladder's headline figure, per row rather than averaged. The bar is the share of the unadapted difference that a von Kries gain removes — the gain being three numbers, the white, read off the room. It runs from 67.3 per cent on a triphosphor tube to 97.6 per cent on the macular pigment. The two ends are the two kinds of change: a filter inside the observer or a wall in the room changes the light in a way a gain handles almost perfectly, and a discharge lamp with emission lines in it does not, because a gain cannot see structure narrower than a cone.
Fig. 5 How much of each change of light the white’s three numbers remove, row by row. From 67.3 per cent on a triphosphor tube to 97.6 on the macular pigment — and every row above two thirds, which is what makes the model plausible rather than merely conventional.

Nine numbers that know nothing about the room

The most interesting row in the table is the one that is not a model anybody proposes: a single 3×3, fitted across half the census, applied to every change of light indiscriminately.

It has no scene numbers at all. It cannot tell one room from another, it applies the same compensation in daylight and under a sodium lamp, and it is fitted to whatever the census happens to contain. Held out — fitted on seven rows and scored on the other seven — it leaves 12.48 against no adaptation’s 14.75.

Fifteen per cent, from nine numbers and no perception whatever. That is six times what the luminance gain buys with one number it has to measure.

What it is doing is not mysterious. The census’s fourteen changes are not distributed symmetrically about the identity: most of them make the light warmer, because most artificial light is warmer than daylight and most of the census’s rows go from daylight to something. A fixed matrix that leans slightly cool removes the average of that, and the average is worth fifteen per cent.

Which is a warning about fitted transforms in general. A matrix fitted across a set of changes that share a direction will always show a gain, and the gain is a property of the set rather than of the physics. The census’s set of changes was chosen editorially, and this row is the price of that choice made visible.

The table and the text use two different baselines

Three of the essay’s percentages come from the ladder and one does not, and the discrepancy is large enough that a reader recomputing from the table will not reproduce the claim.

Against the table’s own no-adaptation figure of 15.67, the six models remove 0, 20.4, 2.7, 91.6, 91.3 and 100 per cent. Two of those match the claim section exactly — 91.6 for the diagonal gain and 2.7 for the luminance one. The third does not: the fixed matrix removes 20.4 per cent by the table and the claim says 15.4.

The 15.4 is recoverable and the text supplies the ingredient: it leaves 12.48 against no adaptation’s 14.75. Held out on seven rows, no adaptation costs 14.75 rather than the 15.67 the full census costs, and (14.75 − 12.48) / 14.75 is 15.4 per cent exactly.

So the figure is right and it is computed against a baseline that appears nowhere in the table. That is not a small bookkeeping point in an essay whose argument is a comparison down a column: the fitted row is scored on half the census and every other row on all of it, and the two are printed in the same column with no mark distinguishing them.

It also moves the comparison the section is built on. Six times what the luminance gain buys with one number mixes the two baselines — 15.4 against 2.6. On a single baseline the fixed matrix removes 20.4 per cent and the luminance gain 2.7, which is 7.6 times, not six. The conclusion is unchanged and the multiplier is a quarter larger than stated.

Which figure the eighteen-parameter row is

The same mixing runs through the two rows the essay’s most careful argument depends on, and here it inflates the effect fourfold.

The table gives the diagonal gain at 1.31 and the diagonal-plus-correction at 1.37 — a gap of 4.6 per cent. The essay then itemises the four underlying numbers: the diagonal at 1.2724 in sample and 1.3511 out, the correction at 1.2592 in and 1.3679 out. Neither pair is 4.6 per cent apart. Out of sample the two models differ by 1.2 per cent and in sample by 1.0 the other way.

The two table entries reconcile as different quantities. The diagonal’s 1.31 is the mean of its two halves, 1.3117 — which is legitimate for an unfitted model, since both halves are equally valid scores. The correction’s 1.37 is its out-of-sample figure alone, 1.3679; the mean of its two halves would be 1.3135, indistinguishable from the diagonal’s.

That is the right pair of choices — a fitted model must be scored where it was not fitted — and it means the column is comparing a mean over fourteen rows against a mean over seven. The neighbouring essay measures what that costs: the two halves of this census differ by 6.19 per cent for a model with nothing fitted in it. A 4.6 per cent gap between two entries drawn from differently sized halves of a census whose halves differ by 6.19 per cent is not a measurement of the correction.

The finding survives it, because the essay’s own itemised comparison is clean: 1.3679 against 1.3511, both out of sample, both on the same seven rows, is the correction losing 1.2 per cent. The table overstates the loss by a factor of about four and the argument only needs the itemised pair, which is already in the text one section later.

Where the observer’s three numbers come from

The plausibility of the whole model rests on the white being estimable, and it is worth saying what that costs, because the ladder above assumes the observer gets it exactly right.

It does not. Estimating the illuminant from an image is an underdetermined problem — a reddish surface under white light and a white surface under reddish light give the same signal — and every method for doing it is a prior about scenes rather than a measurement. The grey-world assumption, the brightest-patch assumption, the gamut-mapping methods, and whatever the visual system actually does are all guesses whose error this collection has measured.

So the honest form of the ladder’s third row is: three numbers, if they can be got right, remove 91.6 per cent. An observer whose white estimate is off by some amount removes less, and the shortfall is a separate quantity with its own essay.

That does not weaken the argument, because the alternative models suffer the same discount and worse. A nine-number model needs nine estimates of a quantity even harder to get at, and its error would compound accordingly.

A correction an observer could have been born with, fitted on half the census and tested on the other. The same six models, each scored twice: on the seven census rows the fixed matrices were fitted to, and on the seven they were not. The split alternates by position so both halves contain daylight changes and discharge lamps. The upper bar is in sample and the lower is out, on a logarithmic axis. For the four models with nothing fitted the two bars differ only because the halves are different questions. For the two fitted ones the gap is the finding, and it is largest where it matters least: bolting a fixed correction onto the von Kries gain takes it from 1.2724 to 1.2592 on the rows it was fitted to, and from 1.3511 to 1.3679 — worse — on the rows it was not. There is no correction to the diagonal that an observer could arrive with.
Fig. 6 The same six models scored twice: on the census rows the fitted matrices were fitted to, and on the ones they were not. The four unfitted models differ between the two halves only because the halves are different questions; the two fitted ones show the gap that fitting buys.

The eighteen-parameter model is the worst kind of worse

One row in the table has more parameters than any other and does not sit where more parameters should put it.

The diagonal gain followed by a fixed correction has three scene numbers and eighteen fixed ones — the basis, plus a 3×3 correction fitted across half the census. In sample it is very slightly better than the plain diagonal, 1.2592 against 1.2724. Out of sample it is worse, 1.3679 against 1.3511.

That is the textbook shape of a model that has fitted its training set, and it is worth having in the table for exactly that reason. Nine extra fixed parameters, no extra demand on the observer, and the result is a model that is 1 per cent better on the rows it was shown and 1.2 per cent worse on the rows it was not.

The conclusion is stronger than “the correction does not help”. It is that there is nothing left for a fixed correction to remove. If the diagonal’s 1.31 contained any component that was the same across changes of light, one matrix would capture it and would survive being held out. It does not, so the residual is specific to each change — which is another way of saying it is information about the particular room, and the observer does not have it.

This collection has an essay about the fits that are exact and empty, and this is the small, honest version of the same thing: a fit that is barely better in sample and worse out of it, reported rather than dropped.

Why the count and not the residual

There is an obvious objection: the diagonal gain leaves 1.31 and the exact matrix leaves zero, so the diagonal gain is 1.31 worse and the parameter count is a distraction.

The objection has the causality backwards. The exact matrix is not an alternative model that happens to be better. It is the answer written down. Its nine numbers are the entries of the change-of-light matrix, so producing it requires knowing the change of light, and knowing the change of light is the thing an observer cannot do — it is why the problem exists.

Put the other way: there is no experiment in which an observer applies the exact matrix, because there is no mechanism by which it could be obtained. The row is in the table as a bound, and a bound’s job is to say how much is left rather than to be competed with.

What the parameter count adds is the ability to compare the models that are available on the axis that separates them. Two models with the same residual and different scene-number counts are not equally good, and two with the same count and different residuals are directly comparable. The count is what makes the table a ladder rather than a list.

What the ladder would look like for a camera

The same count applied to a device rather than an observer gives a different shape, and the contrast is what makes the count worth having.

A camera’s white balance is a diagonal gain too, in a basis of its own, and it needs the same three numbers about the room. But a camera can be told them: a grey card in the frame, a manual colour temperature, or a reading from a separate sensor. Where an observer has to infer the white from the image alone, a camera has channels an observer does not.

That changes which rungs of the ladder are reachable rather than which are useful. A camera could in principle apply the exact matrix, given a spectral measurement of the illuminant and a model of the surfaces, and some studio workflows come close by profiling under the actual lighting. What stops it is not information in principle but cost and the fact that a matrix fitted under one light is wrong under another, which is the same failure as a fixed correction and for the same reason.

So the count is relative to the knower, and stating the knower is part of stating the model. The same six rows scored for a spectroradiometer on a tripod would put the exact matrix at the top as the obvious choice, and the diagonal gain would be a curiosity.

Where the model stops

Everything here is on one family of surfaces and one census of changes, both constructed, both audited elsewhere and neither measured. The 91.6 per cent in particular is a mean over fourteen editorially chosen changes, and a mean over a chosen set is a statement about the set.

The basis is held at CAT16 throughout. Which basis a diagonal gain should be taken in is a question with its own round of work, and the nine numbers of the basis are counted here as fixed — which is right, since a visual system’s cone-like axes are not re-estimated per room, but it does mean the diagonal model’s 9 fixed parameters are carrying a fit that was done somewhere.

And “scene number” is a modelling word, not a physiological one. Nothing here says the visual system estimates three numbers and multiplies; the mechanisms are receptor gain control, post-receptoral normalisation and cortical processes, and the diagonal is a description of their net effect rather than an account of them.

How much of the residual a partial correction removes. Between the diagonal gain and the exact matrix there is a line: apply the correction that would make a row exact, but only a fraction of it. The horizontal axis is that fraction and the vertical is the share of the row's residual it removes, for all fourteen census rows. The straight diagonal is where a correction worth exactly its fraction would fall, and in the published unit every curve lies on it to within 2.2 percentage points. The lower band of curves is the same interpolation measured in CAM16-UCS, which departs by up to 17 points — because its distance is a power of the Euclidean one and a power is not homogeneous along a ray, where every ordinary norm is. The straight line is therefore a property of the ruler rather than of the correction, and the exception is what says so.
Fig. 7 The line between the diagonal and the exact answer, walked. Where the remaining 1.31 could come from, and how much of it a partial correction buys, is the next question.

Who found it, and when

Von Kries’s hypothesis is 1902 and is a statement about mechanism: adaptation acts as an independent gain on each receptor class. Its persistence for a century and a quarter is usually explained by its accuracy, and the accuracy is real — 91.6 per cent here, and the standard adaptation transforms in every colour-management system are refinements of it.

The argument from information is not usually the one made. The nearest thing in the literature is the observation, common in computational vision, that colour constancy is ill-posed and that a solution must therefore be a prior rather than a computation — which is the same point about the estimate rather than about the model. Applying it to the model’s own parameter count is the move here, and it does not require anything the field does not already know; it requires counting two kinds of parameter separately, which nobody has a reason to do until the two kinds appear in the same table.

Where the ladder goes next

If the remaining 1.31 is information the observer does not have, there might still be a cheap approximation to it — a small, rough correction applied after the gain, worth a disproportionate share of what a perfect one would give. Whether such a thing exists is a question about the shape of the residual along the line from the diagonal to the exact answer, and the shape turns out to be the least interesting one available.

Where this goes next

Every published adaptation transform, and one computed from daylight, on every change. What each basis leaves an adapted observer with, row by row. Darker is worse. The last column is not a published transform: it is the basis in which a change from D65 to D50 is exactly diagonal, computed in closed form from the two spectra with nothing fitted. It is far the best on the daylight rows and it is beaten on the discharge lamps, which is the trade the published transforms are sitting in — they were fitted to data containing both kinds of light and are therefore optimal for neither. Over the census as a whole the winner is Bradford at ΔE00 1.14. A gain needs a basis Adaptation scales three signals, and which three is a choice. The basis in which a change from D65 to D50 is exactly diagonal can be computed in closed form from the two spectra, it beats every published transform on daylight by a factor of five, and it loses to all of them on a fluorescent tube. What a fourth reflectance dimension costs the theorem that a change of light is a matrix. Four rising curves on axes of the fourth dimension's amplitude, left to right, against what is left of daylight to tungsten after the exact 3×3 change-of-light matrix has been applied, in ΔE₀₀. All four begin at exactly zero: on the three-dimensional family the matrix is solved rather than fitted and there is no remainder at all, which is the theorem this collection's adaptation argument is built on. Adding a fourth reflectance dimension breaks it, and how badly depends far more on the fourth function's shape than on its size — at five per cent amplitude the four shapes cost 0.329, 0.572, 0.063, 0.124 ΔE₀₀ respectively, a factor of 9.1 between the dearest and the cheapest. For scale, the smallest von Kries residual anywhere in the census is 0.26 ΔE₀₀, so the cheapest of the four is a quarter of it and the dearest is twice it. A theorem about a family A change of light acts on the test surfaces used here as an exact 3×3 matrix with no residual whatsoever, and the whole adaptation argument is built on that being exact. It is exact because the surfaces span exactly three dimensions, and they span exactly three dimensions because three basis functions were written down. five greens, no neutral, under D65 — four guesses at the illuminant. The scene's surfaces as they reach the eye, and each estimator's angular error in degrees against the illuminant that actually lit them. Grey-world 43.1°, Max-RGB 27.7°, Shades-of-grey (p = 6) 31.4°, Grey-edge 34.4°. The best here is Max-RGB, which is best because this scene happens to satisfy its assumption — something reflects fully in every band — and not because it is the better algorithm. Any hatched patch is a surface this display cannot show under this light. The algorithms that guess the light What reaches a sensor is an illuminant multiplied by a reflectance, and no arithmetic separates a product into its factors. Every white-balance algorithm therefore works by assuming something about the world — and the interesting content of each is not its formula but the assumption, because a scene can violate it.

What this makes readable

Essays that name this one as a prerequisite.

Named alongside this one

Essays reaching for the same objects. Nobody chose these; they are what the index of named objects makes visible.

What links here

Every essay whose body links to this one.

The objects this essay names

Each one links to every other essay that touches it.

BasisChromatic adaptationHeld-outIlluminant estimationMatrixModel complexityResidualTest setThe von Kries transformWhite point