Colour Science & Perception
what the eye settles, and what it cannot
Concept

Illuminant estimation — where it appears

Working out what the light was from an image, which every camera does before it can white balance. It is underdetermined in principle and solved by assumption in practice, so its errors are systematic rather than random.

Named by 6 essays across 2 fields — each of them below, with the objects they name alongside it.

One reflectance, two illuminants, two colours. A reflectance peaking near 580 nm, and the colours it produces under D65 and A. The object has not changed. The light has, and colour is a property of the pair.

Constancy is the default

A sheet of paper looks white in daylight and white under a tungsten lamp, although the light reaching the eye differs enormously. The visual system is solving one equation with two unknowns, and it solves it by assumption.

brain · Appearance five greens, no neutral, under D65 — four guesses at the illuminant. The scene's surfaces as they reach the eye, and each estimator's angular error in degrees against the illuminant that actually lit them. Grey-world 43.1°, Max-RGB 27.7°, Shades-of-grey (p = 6) 31.4°, Grey-edge 34.4°. The best here is Max-RGB, which is best because this scene happens to satisfy its assumption — something reflects fully in every band — and not because it is the better algorithm. Any hatched patch is a surface this display cannot show under this light.

The algorithms that guess the light

What reaches a sensor is an illuminant multiplied by a reflectance, and no arithmetic separates a product into its factors. Every white-balance algorithm therefore works by assuming something about the world — and the interesting content of each is not its formula but the assumption, because a scene can violate it.

brain · Appearance How short of determining the light a photograph is, as the scene grows. Each cell is the number of unknowns left over after every equation the image supplies: three sensors, a three-dimensional illuminant, and reflectances confined to a linear model of the dimension on the left. At one and two dimensions more surfaces close the gap. At three the gap never closes, because each further surface adds three equations and three unknowns; at four it widens. The count is arithmetic and has no algorithm in it.

An image does not determine the light

A photograph of a scene under one illuminant gives three numbers per surface and asks for the illuminant plus three numbers per surface. The count closes only if reflectances lie in a two-dimensional model, and no number of surfaces helps — at three dimensions the alternative scenes can be written down, and they reproduce every sensor response exactly.

scene · Scene What an observer is left with, by how much it is allowed to know about the room. Six ways of discounting a change of light, averaged over the fourteen changes in the adaptation census and 125 test surfaces each. The bar is what each leaves behind, on a logarithmic axis because the models span two orders of magnitude. The second line under each name is the count that matters: how many numbers about this room the model has to be given. Doing nothing leaves 15.7 ΔE₀₀. A single gain read off the two whites' luminances leaves 15.3. A matrix fitted across half the census and then applied everywhere, knowing nothing about the room at all, leaves 12.5. The published von Kries gain, which is told the white and nothing else, leaves 1.312 — and bolting a fixed correction onto it, at no cost in scene information, leaves 1.368, which is very slightly worse. The exact matrix leaves nothing and is not on the chart: its nine numbers are the change of light, which is the quantity being discounted.

Three numbers the scene supplies

An adaptation model's parameters are not all the same kind of thing. Some are numbers an observer must estimate from the room it is standing in; others could have been settled once by evolution. Counting them separately turns the diagonal gain from a crude approximation into the only model of the set that gets a large answer from information the observer can actually have.

scene · Scene A correction an observer could have been born with, fitted on half the census and tested on the other. The same six models, each scored twice: on the seven census rows the fixed matrices were fitted to, and on the seven they were not. The split alternates by position so both halves contain daylight changes and discharge lamps. The upper bar is in sample and the lower is out, on a logarithmic axis. For the four models with nothing fitted the two bars differ only because the halves are different questions. For the two fitted ones the gap is the finding, and it is largest where it matters least: bolting a fixed correction onto the von Kries gain takes it from 1.2724 to 1.2592 on the rows it was fitted to, and from 1.3511 to 1.3679 — worse — on the rows it was not. There is no correction to the diagonal that an observer could arrive with.

A model is a claim about what can be known

The exact answer to chromatic adaptation is nine numbers, and the nine numbers are the change of light itself. A model whose parameters are quantities the observer cannot obtain is not a worse model of the same thing — it is a model of something else, and counting parameters without asking where they come from hides the difference.

scene · Scene How far the answer for the average is from the average answer, over eight spreads. For each spread of inputs, the distance between the mean of the model's answers and its answer for the mean input, as a share of the spread of the answers. Over a population of observers it is 1.4 per cent. Over surfaces it is 21 on smooth natural reflectances, 27 on a banded family with lightness in it, and 9 on a set of pale surfaces. Over the light one room sees in a day it is 59.

The average surface does not look average

Over a population of observers the appearance model is so nearly linear that the mean of its answers is its answer for the mean, to 1.4 per cent of the spread. Over the surfaces in a scene it is not. On 240 smooth reflectances the gap is 21 per cent of the spread, and the mean surface looks 4.2 units lighter than the surfaces look on average. The grey that matches the average light is a 47 per cent reflectance; the grey that matches the average look is a 43 per cent one.

brain · Appearance

Named alongside it

The objects these essays reach for when they reach for this one.

Colour constancyReflectanceChromatic adaptationThe grey-world assumptionThe von Kries transformWhite balanceAdaptationBasisHeld-outIlluminantMatrixModel complexity

All concepts