Colour Science & Perception
what the eye settles, and what it cannot
Concept

Spectral sensitivity — where it appears

How strongly a detector responds at each wavelength, whether the detector is a cone, a camera channel or an instrument. Three of them together decide what matches what, so two devices with different sensitivities disagree about which pairs are the same colour.

Named by 32 essays across 5 fields — each of them below, with the objects they name alongside it.

A camera's spectral sensitivities, after the infrared-cut filter. Silicon quantum efficiency times the colour-filter dye times the infrared-cut filter, per channel, on a grid running to 1100 nm rather than to 780. With the filter removed, 68 per cent of the area under the three curves lies beyond the visible band, and all three curves are the same curve out there.

A camera is a fourth observer

The 1931 functions, the 1964 functions and a person's own cones are three sets of three curves that collapse a spectrum onto three numbers. A camera is a fourth, built from silicon and dye rather than from pigment and neural wiring, and it agrees with none of them.

imaging · Capture Everything between the photons and the picture, and what each stage decides. The 8 stages of a camera pipeline. Only the second is physics; every one after it is a decision somebody made, and the reason two cameras pointed at the same scene disagree is that they made different ones.

Raw is not a picture

A raw file is three integrals per site and a list of decisions nobody has taken yet. Every one of those decisions has a defensible answer and none of them has a correct one, which is why two converters open the same file and disagree.

imaging · Capture The blue channel, as the three things multiplied to make it. Silicon's quantum efficiency, the colour-filter dye's transmittance, and their product. The dye is the only stage carrying any colour information and it is clear above about 800 nm — transmittance 0.92 at 900 nm against 0.06 at 550.

Silicon sees past the visible

A sensor's response ends at 1107 nanometres because that is the band gap, and the colour-filter dyes have stopped absorbing three hundred nanometres before it. So all three channels measure the same thing over the last third of the range, and every published sensitivity plot is drawn after the component that hides it.

imaging · Capture How much of what an unfiltered sensor records is invisible. The share of a camera's raw signal coming from beyond 780 nm, against the colour temperature of the lamp, for one surface with a near-infrared reflectance of 0.62. Without the filter it runs from 98 per cent at 2000 K to 51 at 9000; with it, under five per cent everywhere.

The filter that makes colour possible

An infrared-cut filter is not a refinement on a colour camera. Removing it collapses the separation between the three channels by a factor of nearly nine under tungsten, and the component that keeps colour photography working is the one nobody photographs.

imaging · Capture One reflectance, four different infrared tails, and what the camera makes of each. The same visible reflectance continued past 780 nm to four different near-infrared values. A spectrophotometer reports only the left-hand part; an unfiltered sensor integrates all of it. The channel spread falls from 0.14 at a tail of 0.05 to 0.02 at 0.85, with nothing about the visible half changed.

Most things are pale in the infrared

A spectrophotometer stops at 780 nanometres and a black cotton shirt reflecting five per cent of visible light reflects more than half the near infrared. The measurement everybody has and the quantity a camera integrates are different quantities, and nothing in the first says so.

imaging · Capture A silicon sensor's best possible impersonation of the standard observer. The 1931 matching functions in outline, and the closest linear combination of the sensor's three sensitivities laid over them; underneath, what is left over at each wavelength. The residual is 31.7 per cent of the matching functions' own magnitude, worst at 440 nm. Colour reproduction is exact if and only if this is zero.

Luther said when it would work

There is an exact condition under which a fixed three-by-three matrix converts camera raw to XYZ correctly for every spectrum in existence. It was stated in 1927, it is a theorem rather than a guideline, and no camera ever built satisfies it.

imaging · Capture Two reflectances the camera records as identical. Constructed by projecting onto the null space of the sensor's own sensitivities, so the two raw triples agree to 0.0000 per cent. To the eye they are ΔE00 15.33 apart, which the swatches show.

The camera has its own metamers

Two surfaces a camera records as identical can be plainly different to a person, and two a person cannot tell apart can be recorded as different. Both pairs are constructed rather than found, from one projection, used for both.

imaging · Capture The best a camera profile can do on the surfaces it was fitted to. Per-surface ΔE00 after the best 3 × 3 from raw to XYZ, fitted on 12 surfaces at chroma 0.7 and tested on those same 12 surfaces — mean 1.45, worst 2.58. This is the most favourable measurement it is possible to make of a camera and it is the one usually published.

No matrix is right everywhere

The best three-by-three this sensor admits, fitted and tested on the same twenty-four surfaces, leaves a worst case of ΔE00 2.85. A control sensor built to satisfy Luther's condition reaches ten to the minus seven under the identical computation, which is what makes the first number a measurement.

imaging · Capture Correcting colour costs noise, and the two cannot both be least. Sweeping the colour matrix from its own diagonal — a white balance with no cross terms — to the full least-squares fit. Mean ΔE00 falls from 18.61 to 1.29; photon noise rises by a factor of 1.03. At 10000 photons per pixel.

Correcting colour costs noise

The matrix that turns a sensor's raw into XYZ has large off-diagonal terms of both signs, because that is what correcting a sensor which fails Luther's condition requires. Differences of large numbers are where noise grows, and the full correction amplifies photon noise by 1.66 times.

imaging · Capture A camera's spectral sensitivities, with the filter removed. Silicon quantum efficiency times the colour-filter dye times nothing else, per channel, on a grid running to 1100 nm rather than to 780. With the filter removed, 68 per cent of the area under the three curves lies beyond the visible band, and all three curves are the same curve out there.

The grid outside every figure

Every figure here is computed on 380 to 780 nanometres, which is exactly right for an eye and insufficient for a sensor. This field met the first subject the grid cannot hold, and the decision was not to widen it — because widening it honestly is impossible.

imaging · Capture Everything between the photons and the picture, and what each stage decides. The 8 stages of a camera pipeline. Only the second is physics; every one after it is a decision somebody made, and the reason two cameras pointed at the same scene disagree is that they made different ones.

A photograph is not a measurement

A photograph is a measurement made by an instrument whose kernel nobody published, under an illuminant nobody recorded, corrected by a matrix fitted to somebody else's surfaces, with two thirds of every pixel invented. It supports relative claims well and absolute ones badly, and it is used for the second.

imaging · Capture Grassmann's four laws, exact — and the two things that break them. For a linear observer every one of the four is exact and the residual is floating point, which is the control that makes the two failures below measurements rather than artefacts. Rods break a cone-metameric match by 23 per cent of a rod excitation at dusk; bleaching breaks it by 0.59 per cent of a cone excitation in the sun. Both are stated as fractions of a receptor's own response, so they can be put on one scale.

The laws that make colour add up

Colorimetry is an integral, and an integral assumes matching is linear. Grassmann's four laws are exact for a linear observer, to floating point — and they fail at both ends of the light range, by two different mechanisms, leaving colorimetry an operating band of three and a bit decades that no standard states.

matching · Gamut One match, and what it says about the observer making it. The anomaloscope: a monochromatic 589 nm yellow set against a mixture of 545 and 670 nm. Only two cone classes respond at those wavelengths, so the match is two equations in two unknowns and has one solution for any observer whose two pigments differ. The bar is the fraction of the accepted band; the mark is the solution. A normal observer accepts 0.7 per cent of the scale; an observer whose two pigments are the same accepts all of it, because their two equations are one equation twice. Nothing here is fitted to clinical data: the pigments are the same template used for every observer here, at stated peaks, and the match is the solution of the linear system.

One match names the observer

A yellow at 589 nanometres set against a mixture of 545 and 670 is two equations in two unknowns. It has one solution for a normal observer, a solution somewhere else for an anomalous one, and no unique solution at all for a dichromat — whose two equations are one equation twice.

matching · Gamut One light, two eyes. The same stimulus through two sets of ocular media differing only in macular pigment (0.35 and 0.41) and lens age (55 and 55 years). Compared under one white the two differ by ΔE00 1.00; compared with each eye adapted to its own long-run white — which is what the visual system does — by 0.00. The second number is why nobody notices, and the first is why a person who has had one lens replaced reports that the other eye has turned yellow.

Nobody here has two eyes

One person's two eyes differ in macular pigment and lens density, so the same light produces two colours — a whole ΔE00 apart for an ordinary pair, seven for one replaced lens. Adaptation hides it exactly, which is why nobody notices and why nothing in colorimetry has a term for it.

limits · Limits What the same eye reports about one field, in the middle and at the edge. Each row is a uniform field, drawn at the most saturated version of itself this page can show — the percentage is how much of the full stimulus survived, the rest being the adapting light added to bring it inside the gamut. The left patch is what the centre of gaze reports and the right one what 10 degrees out reports, each adapted to the same light as that position sees it. The adapting white comes out identical to 5e-13, because an adapted eye cancels its own filter exactly. Nothing else does, and the largest difference is in the blue. tungsten light is not drawn: it cannot be shown at any useful saturation, and at full strength it differs by ΔE00 1.87.

One person is two observers

The macular pigment is a yellow screen over the fovea and nowhere else, so a cone at the centre of gaze and a cone ten degrees away have different colour-matching functions in the same eye. A match made in the middle comes apart at the edge by six units — and fitting one filter to the gap between the CIE's two standard observers gives a density of 0.40 against a measured 0.35.

eye · Cones How wrong a camera profile is, and who it is wrong for. A colour matrix fitted against the 1931 observer, evaluated four ways. Its residual against that observer is ΔE00 0.92 — the Luther failure, which is a property of the sensor and is the honest measurement of the camera. Against a person drawn from a population of 160, the worst-off twentieth report 3.35. And a camera with no spectral error whatever, reporting the standard observer's own tristimulus values exactly, would leave 3.47. The camera is not the problem. It was fitted to somebody who does not exist, and so is the standard it was fitted against.

Fitted to an eye nobody has

A camera profile's residual against the observer it was fitted to is under a unit, and that is the number everybody quotes. Against a person drawn from a population it is 3.35 at the ninety-fifth percentile — and a camera with no spectral error whatever, reporting the standard observer's tristimulus values exactly, would carry 3.47. The camera is not the problem.

imaging · Capture The share of itself each change leaves behind, and the smallest is inside the eye. The residual as a fraction of the change rather than as a colour difference, which sorts the census differently. At the top is the macular pigment — the filter in front of the central few degrees of one's own retina — leaving 2.4 per cent of itself. It is a fixed transmittance multiplying the light and the white together, which is as close to a pure gain as anything here gets, and it is why nobody notices they have one.

The filters inside the eye

The macular pigment leaves 2.4 per cent of itself after adaptation — the smallest share of anything in this site's census of light changes, and less than half the next smallest. Fifty years of lens yellowing leaves 7.9 per cent, and the difference between the two says what a gain is actually good at.

eye · Cones The basis a camera balances in is a different basis for every light. A camera's white balance is a per-channel gain on raw values, which is a von Kries adaptation in whatever basis the filter dyes give it. That basis is not a property of the dyes alone: it is the dyes and the light in the room, and it moves when the light does. Each bar is how far the basis has turned, in degrees, from where it sits under D65. A sensor satisfying the Luther condition would have a bar of exactly zero on every row, because for such a sensor the light cancels — which is the one property nobody buys a sensor for.

A camera balances in another basis

White balance is a per-channel gain on raw values, which makes it a von Kries adaptation in whatever axes the filter dyes happen to give. Those axes are not a property of the dyes alone — they move with the light, by up to seventeen degrees across the adaptation census — and the sensor for which they would not move is the one that adapts worst of all.

imaging · Capture Six functions of wavelength, and the six different places they stop. Every table this collection integrates against, drawn over the range the body that published it defined it on. The scale is logarithmic so that the ultraviolet and the near infrared both fit. The bottom row is the range used here before the infrared band was added, and it is the intersection of the two rows that matter for an eye looking at a reflector — which is the right answer only while everything in the integral is being multiplied together. The daylight basis runs 80 nanometres further down than that intersection, and it was published that way because the ultraviolet in daylight is what makes a brightened sheet of paper glow. The analytic row is drawn to the edge of the plot because it has no edge: Planck's law is a formula and is exact at every wavelength, which is why illuminant A needs no table at all.

The tables do not stop together

This collection integrates from 380 to 780 nanometres, and decided once, in writing, that the range could not honestly be widened. The argument was correct at the long end and wrong at the short one — the CIE publishes the daylight basis from 300 nanometres, and publishes it from there for exactly the reason it matters.

light · Light What reaches the retina, and why the observer's table stops at 360 nanometres. The transmittance of the eye's own optics across the short-wave band, at three ages, with the brightener's absorption shaded underneath. The upper curve is an eye whose lens has been removed — the cornea alone, opaque below about 295 nanometres and transparent above it. The photopigments absorb perfectly well in this band; what stops the light is a piece of optics in front of them, which is why the short-wave limit of colour vision moves with age and can be removed surgically. A twenty-year-old receives 21 times as much of the band a brightener works in as a seventy-year-old does.

The eye stops at the lens

Neither standard observer is tabulated below 360 nanometres, and the reason is not that the photopigments stop absorbing there. It is that the light never arrives — the cornea and the crystalline lens take it — so the short-wave limit of human colour vision is a piece of optics, it moves by a factor of twenty across a lifetime, and it can be surgically removed.

eye · Cones Two lamps of the same colour, and one sheet that is two colours under them. The same brightened sheet under a xenon flash, the same flash behind its cover glass, and a phosphor-converted white LED of nearly the same chromaticity. Each patch is computed relative to its own lamp's white, which is what a perfect white balance does — so everything a camera can see and correct has already been removed. What is left is ΔE00 11.8 between the first and the last, against 3.2 between the two lamps themselves. The LED has no emission below 380 nanometres at all, because its pump die is at 450, so the sheet simply does not fluoresce under it and nothing in the photograph records why.

A camera cannot record the excitation

Two lamps of nearly the same chromaticity, one with ultraviolet and one with none, put a brightened sheet twelve units apart after a perfect white balance. Nothing in the camera measured the difference — the filter stack removed the band before the sensor saw it — so the correction that would fix the picture needs a quantity the file does not contain.

imaging · Capture What a camera matrix reports on its own chart, and what it delivers off it. The same camera fitted on charts of increasing chromatic range. The left bar of each pair is the mean error on the chart the matrix was fitted to, which is the number a profile comes with; the right bar is the error on a saturated set it never saw. At the thinnest chart the fit reports 0.19 ΔE00 and delivers 1.65, a factor of 8.6. The gap closes as the chart widens, and it closes because the chart improves rather than because the camera does.

The chart decides the profile

A camera's colour matrix is nine numbers fitted to a set of patches somebody chose, and the number that comes with it is the error on those patches. On a chart with no chromatic range that number is 0.19 ΔE00 and the matrix delivers 1.65 — and a second matrix, indistinguishable on the chart, delivers 2.03.

imaging · Capture The dyes a camera has, and the dyes an adaptation basis would want. Three sensor sensitivities drawn twice: faintly, the silicon-and-filter-array set this collection models, and boldly, three Gaussian dyes chosen to make the inverse of their own response matrix a good basis for a white-balance gain. The designed dyes sit at 610, 542, 449 nm with widths of 35, 26, 30 nm — narrower and further apart than the real ones, which is what sharpening looks like when a search rather than a committee does it. They leave 0.97 ΔE00 against the real sensor's 1.62, and they are held within 0.28 of the Luther condition so that the result is still a camera.

A sensor designed for its inverse

A camera's white balance is a gain in a basis made from its own dyes and the light in the room. Choose the dyes for that basis instead of for cost and quantum efficiency, hold the sensor within a stated distance of the Luther condition, and the design reaches the best adaptation figure any basis achieves — and then the room moves it.

imaging · Capture Four cameras that all satisfy the Luther condition exactly. Four sensors whose sensitivities are linear combinations of the colour-matching functions — the theoretical ideal, satisfying the condition to machine precision, each with an adaptation basis that does not move when the light does. They differ only in which linear combination, which the condition does not constrain, and they leave 2.46, 0.97, 1.65, 2.37 ΔE00 after a white balance. The best of them reaches 0.974, which is the best any basis at all achieves. Being a perfect colorimeter costs nothing in adaptation; what costs is the mixing matrix, and the control measured here carries one nobody chose.

The condition chooses no axes

It has long been said here that a sensor satisfying the Luther condition exactly adapts worse than a silicon one, and offered a reason — that its channels are the matching functions, and a gain on those is the oldest mistake in the subject. The measurement was of one sensor. The condition leaves the axes entirely free.

imaging · Capture Which of a camera's three dyes each direction moves. A grid with one column per direction — stiffest on the left, flattest on the right — and one row per parameter of a camera's three dyes. Each cell's bar length is that parameter's share of that direction, so a column with one long bar is a direction that moves one thing. The stiffest column is dominated by blue centre, at a weight of 0.96. The flattest column is spread across blue width, red width, green width — a combination rather than any single number, which is why a specification listing one tolerance per parameter cannot express it.

Where a camera is blind to itself

A colour filter array is six numbers — three dye centres and three bandwidths — and how well the resulting sensor adapts is far more sensitive to some combinations than to others. The stiffest direction is almost entirely where the blue dye sits. The flattest is all three bandwidths at once, and the design can move twenty-five times further along it for the same cost.

imaging · Capture The three cone absorptances at two settings of the rods. Solid and dashed are the same construction at the two ends of a tenth of the cone response, which is a dim room. The curves are built from one pigment template through its ocular media, which is the same model its population of two hundred eyes is drawn from. The largest difference between the two sets is 9.1 per cent of the peak, and where it sits along the wavelength axis is what decides which stimuli the two observers disagree about — a departure concentrated in the blue is invisible on a sample with no blue in it.

The rods are a fourth curve

Colorimetry has three numbers and a rod signal is a fourth. It cannot be absorbed by a gain, it cannot be removed by a white point, and adding a tenth of one to a three-curve observer costs 1.60 ΔE₀₀ — the narrowest distribution of the six departures, because an addition behaves quite differently from a filter.

eye · Cones The three cone absorptances at two settings of the pigment peaks. Solid and dashed are the same construction at the two ends of two standard deviations, and the L/M polymorphism on top. The curves are built from one pigment template through its ocular media, which is the same model its population of two hundred eyes is drawn from. The largest difference between the two sets is 8.9 per cent of the peak, and where it sits along the wavelength axis is what decides which stimuli the two observers disagree about — a departure concentrated in the blue is invisible on a sample with no blue in it.

The peaks move the flanks

Shifting a cone's peak wavelength by three nanometres changes its sensitivity at its own maximum by almost nothing and on its flanks by several per cent, because a maximum is flat and a flank is steep. Every display primary sits on a flank, which is why a pigment polymorphism is worth 3.60 ΔE₀₀ under a laser projector.

eye · Cones The arguments a standard observer does not have. Seven choices inside a set of colour-matching functions, each with the shape it takes and what it is worth in ΔE₀₀ on a red pigment under a 6500 K radiator. Six are measurements: a field size, an age, a macular density, a cone optical density, three peak wavelengths and a rod contribution. The seventh is not — a change of basis is a change of curves and not a change of observer, and its entry is exactly zero because the space an experiment measures is what an observer is. Printing that zero beside the others is the clearest statement of what the other six are measurements of.

A sensor has no lens

A camera is a fourth observer and it is the only one with none of the six arguments this round measured. It does not age, it has no macular pigment, its response does not broaden with density and its peaks do not vary — so it is perfectly reproducible and is not any of the observers it is fitted to reproduce.

imaging · Capture One infrared-cut filter, crossed at four angles. The transmittance of the same interference filter for light crossing it at the centre of a frame and at three steeper angles, from 560 to 760 nm. Its half-transmission edge is at 665 nm straight on and moves to 658, 646 and 630 nm, following the edge wavelength times the square root of one minus the squared sine of the angle over the square of the stack's effective index, 1.8. The deepest reds are what it takes away, and a lamp decides how much light there was in them.

The corner of the frame has another filter

A camera's infrared-cut filter is an interference stack, and light crossing it at an angle sees its edge moved towards the blue — from 665 nm at the centre of a frame to 646 at twenty-five degrees. The corner's red channel loses a tenth of its response under daylight and a sixth under tungsten. A grey-card gain map makes the corner exactly right under the lamp it was made under; under a tungsten lamp the same grey is 2.6 colour differences wrong, under a white LED 4.2, and even under its own lamp the coloured patches are not corrected, because a moved edge is not a gain.

imaging · Capture A black level slightly wrong, through the balance, under daylight. A grey ramp from half a per cent to seventy-two per cent reflectance, with a pedestal error of a tenth, three tenths and one per cent of white left in all three raw channels before the white balance, against the same ramp with none. The horizontal axis is the grey's reflectance, logarithmic. At a three-tenths error a two per cent grey is 2.4 colour differences off, most of it chroma, and a seventy-two per cent grey 0.21. An equal offset in the raw channels is not equal after three different gains.

A black level is multiplied by the balance

Every raw value carries a pedestal that is subtracted before anything else, and a white balance is then a different gain in each channel. Subtracting a constant and multiplying by one commute only when the constant is zero or the gains are equal. So a pedestal left three thousandths of white too high becomes 2.4 colour differences in a two per cent grey, most of it chroma, in the colour the lamp starves — and pushing that shadow four stops in editing makes it 8.9. An offset in proportion to the lamp's own white is the exception, and it is exactly grey.

imaging · Capture Three corrections for the corner of a frame, each made under daylight. The mean colour difference over twenty-four coloured patches between the centre of a frame and its corner, against the angle light arrives at, after three corrections each fitted under daylight, D65 and used under it: a grey-card gain map, a correction confined to the red channel's row, and a full three-by-three matrix. All three leave the grey exact. At 25° the gain map leaves 1.86, the red row 0.93 and the matrix 0.90; at 35°, 3.81, 1.95 and 1.81. Six more numbers buy almost nothing, because the moved edge is in one channel.

A corner is corrected by one row

A grey-card gain map makes the corner of a frame exactly right on grey and leaves coloured patches 1.86 colour differences wrong at twenty-five degrees. A three-by-three matrix fitted at that position halves it — and six of its nine numbers do nothing, because the moved filter edge is in one channel. The three that matter rebuild the lost red from green and blue, they carry to another lamp better than a gain map in eleven cases of twelve, and in the twelfth, a row fitted under tungsten and used in daylight, they leave the grey 9.2 off.

imaging · Capture Four ways to correct the corner, under each of four lamps, at 25°. The mean colour error left on the chart's coloured patches at the corner of the frame, under each lamp, after: the red row fitted under that lamp; one red row fitted on the chart under all four lamps with the grey held under daylight; a grey-card gain map calibrated under daylight; and the worst of the other lamps' rows used by mistake. daylight: 0.93, 1.33, 1.86, 4.91; tungsten: 0.99, 1.45, 1.87, 3.26; white LED: 0.36, 1.39, 3.28, 1.77; fluorescent: 0.37, 1.56, 3.09, 2.45. The pooled row holds every lamp between 1.33 and 1.56.

One row for every lamp costs the lamps that lose least

A corner correction fitted under one lamp is right under that lamp and can be badly wrong under another, so a converter unsure of its lamp was offered a single row fitted under several at once. Pooled over four lamps, one row holds every lamp's corner between 1.33 and 1.56 colour differences — no lamp worse than a grey-card map, and none near the 4.91 a wrong row can leave. The price falls on the white LED and the fluorescent tube, which it leaves four times worse than their own rows, because they lose the least red at the corner and the pooled row is built to repair the lamps that lose the most. Two rows, one per class of lamp, keep almost all of both.

imaging · Capture

Named alongside it

The objects these essays reach for when they reach for this one.

Camera rawStandard observerColour matrixLuther conditionWhite balanceIlluminantIndividual variationColour filter arraySiliconChromatic adaptationInfraredLeast-squares

All concepts