Colour Science & Perception
what the eye settles, and what it cannot
The collection

Every essay — page 32

Page 32 of 40, continuing through the fields in the same order.

What light is What the eye does Matching and measuring Difference and uniformity What the brain does What a scene does What a camera does Where the model breaks What it takes to deliver it

SeriesObserversNamed objectsRefutationsSearch

What a camera does

A camera is three functions of wavelength collapsing a spectrum onto three numbers, which is what an eye is — and they are different three functions. Everything a photograph can and cannot establish about colour follows from that, and from the fact that silicon sees three hundred nanometres past where anybody is looking.

A camera profile is a fit, and the sample set is a hidden argument to it. Per-surface ΔE00 after the best 3 × 3 from raw to XYZ, fitted on 12 surfaces at chroma 0.2 and tested twice: on those same surfaces (mean 0.65) and on 12 at chroma 0.9 (mean 1.75). Both bars come from the same matrix; only the surfaces differ.

A camera profile is a fit

Since no matrix is exact, the one a manufacturer ships is a least-squares compromise over a set of surfaces somebody chose. Fitted on desaturated patches it is excellent on desaturated patches — and the sample set is a hidden argument to every camera profile in existence.

9 figures The best a camera profile can do on the surfaces it was fitted to. Per-surface ΔE00 after the best 3 × 3 from raw to XYZ, fitted on 12 surfaces at chroma 0.7 and tested on those same 12 surfaces — mean 1.45, worst 2.58. This is the most favourable measurement it is possible to make of a camera and it is the one usually published.

No matrix is right everywhere

The best three-by-three this sensor admits, fitted and tested on the same twenty-four surfaces, leaves a worst case of ΔE00 2.85. A control sensor built to satisfy Luther's condition reaches ten to the minus seven under the identical computation, which is what makes the first number a measurement.

7 figures A grey edge, reconstructed from a Bayer row, arrives coloured. Above: an achromatic step through 24 sensor sites, with green sampled on the even ones and red on the odd. Interpolating each channel separately reconstructs them from data taken on either side of the edge, so their ratio moves. Below: the resulting chroma, peaking at 144 per cent of the local mean, and 112 per cent once colour differences are interpolated instead.

A grey edge arrives coloured

A sensor site measures one channel and the other two are interpolated from neighbours that sat somewhere else. Across a black-and-white step that reconstruction gives an achromatic scene a chroma of 144 per cent of its own local mean, and nothing in the scene or the sensor was coloured.

7 figures Recovering the lamp from a highlight rather than from an assumption. A green scene under illuminant A. The faint curve is the true lamp; the solid one is what the interface component of the glossy surfaces recovers, which is 0.30° from it. Grey-world on the same scene is 42.1° and max-RGB 17.5°, because both are assumptions about the surfaces and a Fresnel reflection is not.

The highlight is the white balance

Every estimator a camera uses is an assumption about the scene wearing the costume of a measurement. The specular component of a glossy surface is not — it carries the illuminant to within a third of a degree on a scene where averaging is fifty times worse, and it does so without knowing what colour the paint underneath is.

10 figures Correcting colour costs noise, and the two cannot both be least. Sweeping the colour matrix from its own diagonal — a white balance with no cross terms — to the full least-squares fit. Mean ΔE00 falls from 18.61 to 1.29; photon noise rises by a factor of 1.03. At 10000 photons per pixel.

Correcting colour costs noise

The matrix that turns a sensor's raw into XYZ has large off-diagonal terms of both signs, because that is what correcting a sensor which fails Luther's condition requires. Differences of large numbers are where noise grows, and the full correction amplifies photon noise by 1.66 times.

7 figures A clipped channel turns the hue of what is left. CIELAB hue shift against exposure for one saturated stimulus, measured against the same stimulus rendered without clipping. Nothing moves until the first channel reaches the ceiling at 0.25 stops; after that the recorded hue rotates by as much as 67 degrees, with nothing in the scene having changed colour.

A blown highlight turns

An exposure that clips nothing changes no hue at all. Once one channel reaches the ceiling the recorded hue rotates by as much as sixty-seven degrees, with nothing in the scene having changed colour — and every response a converter can make to that is an invention.

8 figures A camera's spectral sensitivities, with the filter removed. Silicon quantum efficiency times the colour-filter dye times nothing else, per channel, on a grid running to 1100 nm rather than to 780. With the filter removed, 68 per cent of the area under the three curves lies beyond the visible band, and all three curves are the same curve out there.

The grid outside every figure

Every figure here is computed on 380 to 780 nanometres, which is exactly right for an eye and insufficient for a sensor. This field met the first subject the grid cannot hold, and the decision was not to widen it — because widening it honestly is impossible.

5 figures Everything between the photons and the picture, and what each stage decides. The 8 stages of a camera pipeline. Only the second is physics; every one after it is a decision somebody made, and the reason two cameras pointed at the same scene disagree is that they made different ones.

A photograph is not a measurement

A photograph is a measurement made by an instrument whose kernel nobody published, under an illuminant nobody recorded, corrected by a matrix fitted to somebody else's surfaces, with two thirds of every pixel invented. It supports relative claims well and absolute ones badly, and it is used for the second.

7 figures How wrong a camera profile is, and who it is wrong for. A colour matrix fitted against the 1931 observer, evaluated four ways. Its residual against that observer is ΔE00 0.92 — the Luther failure, which is a property of the sensor and is the honest measurement of the camera. Against a person drawn from a population of 160, the worst-off twentieth report 3.35. And a camera with no spectral error whatever, reporting the standard observer's own tristimulus values exactly, would leave 3.47. The camera is not the problem. It was fitted to somebody who does not exist, and so is the standard it was fitted against.

Fitted to an eye nobody has

A camera profile's residual against the observer it was fitted to is under a unit, and that is the number everybody quotes. Against a person drawn from a population it is 3.35 at the ninety-fifth percentile — and a camera with no spectral error whatever, reporting the standard observer's tristimulus values exactly, would carry 3.47. The camera is not the problem.

7 figures The same flicker, written across the frame. A rolling shutter exposes each row of the sensor at a different moment, so a lamp that flickers above fusion is recorded as bands. There are 1.7 of them here — the readout time times the lamp's frequency, which is a camera setting and not a property of the light — spanning 2.37 stops. Held at matched luminance the lightest band and the darkest are still ΔE00 5.40 apart in colour, because the two drive currents are two spectra and the shutter caught one of each.

The shutter samples the lamp

A lamp switched between two drive currents at a hundred hertz is one steady colour to a person and two spectra to a camera. A thousandth-of-a-second exposure catches whichever phase the shutter opened at — two and a third stops of exposure and five units of colour, decided by nothing but timing — and a rolling shutter writes the difference across the frame as bands.

6 figures The basis a camera balances in is a different basis for every light. A camera's white balance is a per-channel gain on raw values, which is a von Kries adaptation in whatever basis the filter dyes give it. That basis is not a property of the dyes alone: it is the dyes and the light in the room, and it moves when the light does. Each bar is how far the basis has turned, in degrees, from where it sits under D65. A sensor satisfying the Luther condition would have a bar of exactly zero on every row, because for such a sensor the light cancels — which is the one property nobody buys a sensor for.

A camera balances in another basis

White balance is a per-channel gain on raw values, which makes it a von Kries adaptation in whatever axes the filter dyes happen to give. Those axes are not a property of the dyes alone — they move with the light, by up to seventeen degrees across the adaptation census — and the sensor for which they would not move is the one that adapts worst of all.

6 figures Two lamps of the same colour, and one sheet that is two colours under them. The same brightened sheet under a xenon flash, the same flash behind its cover glass, and a phosphor-converted white LED of nearly the same chromaticity. Each patch is computed relative to its own lamp's white, which is what a perfect white balance does — so everything a camera can see and correct has already been removed. What is left is ΔE00 11.8 between the first and the last, against 3.2 between the two lamps themselves. The LED has no emission below 380 nanometres at all, because its pump die is at 450, so the sheet simply does not fluoresce under it and nothing in the photograph records why.

A camera cannot record the excitation

Two lamps of nearly the same chromaticity, one with ultraviolet and one with none, put a brightened sheet twelve units apart after a perfect white balance. Nothing in the camera measured the difference — the filter stack removed the band before the sensor saw it — so the correction that would fix the picture needs a quantity the file does not contain.

7 figures