Colour Science & Perception
what the eye settles, and what it cannot
Theme

The thread: The instrument is the reader

This is the one subject where the page is displayed on the apparatus under discussion, and the reader's own eye is the measuring device. Several figures here are experiments rather than illustrations.
Two identical grey patches on different surrounds. Both inner squares are #818181. The one on the dark field looks lighter. The values are checked to be equal before the figure is drawn, so the claim is a fact about the drawing rather than a promise. What the brain does

These two patches are identical

It is the most repeated and least checkable sentence in visual perception, because the whole point is that it does not look true. Every instance here is computed, and checked before the figure is drawn.

 The Cornsweet edge, with its luminance profile. The two plateaux are both #898989 and are flat to exactly — the profile below shows that everything which differs lies within a narrow band at the boundary. Cover the centre line and the two halves become obviously identical. What the brain does

Brightness is inferred from edges

The Cornsweet effect makes two identical regions look different by altering nothing except a narrow band at the boundary between them. Cover the boundary and the difference vanishes, which says the visual system is reconstructing surfaces rather than reading off intensities.

 One reflectance, two illuminants, two colours. A reflectance peaking near 580 nm, and the colours it produces under D65 and A. The object has not changed. The light has, and colour is a property of the pair. What the brain does

Constancy is the default

A sheet of paper looks white in daylight and white under a tungsten lamp, although the light reaching the eye differs enormously. The visual system is solving one equation with two unknowns, and it solves it by assumption.

 A gamma probe: which grey matches a half-white dither?. The striped block on the left is half white and half black, so it carries half the luminance of white. Stand back until the stripes blur and find the patch that matches it. On an sRGB display the answer is code 188, not 128 — code 128 has only 22 per cent of white's luminance. Where the model breaks

The display is an unknown

This site is displayed on the very apparatus it is about, and it knows almost nothing about that apparatus. Two figures here stop assuming and ask instead — a probe for the transfer function and a probe for the gamut.

 The sRGB transfer function, and the gamma 2.2 curve it is not. Code value against relative luminance. The sRGB function is piecewise — a short linear segment near black, then a 2.4 power law with an offset — and it is close to but not the same as a plain 2.2 power law. Half-way along the axis of stored values sits at 21 per cent luminance, and half the luminance of white is at code 188. Difference and uniformity

The midpoint is not half

Code 128 sits halfway along the sRGB scale and carries about a fifth of white's luminance. Half the luminance is code 188. Almost every gradient, blur and resize on the web gets this wrong, and the errors are visible once known.

 One stimulus, three rooms, three appearances. The same XYZ in a dark, a dim and an average surround. The stimulus does not change and is drawn identically in all three panels; what changes is what CIECAM16 says it looks like. Predicted lightness runs from 65.6 to 57.5 — a spread of 8.1 — with chroma and colourfulness moving too. Colorimetry returns one answer here because it has nowhere to put the room. What the brain does

A viewing condition is an argument

An appearance model takes a stimulus and a situation. The second argument is not a refinement of the first — it is the content of the claim, and a model that returns an appearance from a colour alone has assumed a room without saying which.

 The same two surfaces, weighed by each system. A long-wavelength and a short-wavelength surface under D65, with their relative luminance under the photopic curve and under the scotopic one. Under daylight vision the red surface is 1.33 times the blue; under rod vision it is 0.15 times, a reversal by a factor of 9.0. The swatches are the photopic appearance, which is the only one a display can produce: rod vision has no colour, and drawing a guess at it would be an invention. What the eye does

The eye that has no colour

Rods outnumber cones twenty to one, work alone below a hundredth of a candela, and are absent from the centre of gaze. Between dusk and a lit room both systems run at once, and neither standard curve describes what is happening.

 How far a match comes apart when the observer changes. A broad source and a three-primary source, solved at each primary width so the pair is an exact tristimulus match for the CIE 1931 observer. The pair is then handed to the 1964 observer, and the gap between them is plotted. For the observer they were built for the gap is arithmetic noise at every width. For the other it grows as the primaries narrow, reaching 0.012 at 10 nm — and displays have been getting narrower for twenty years. Where the model breaks

Whose eyes

The standard observer is an average over seventeen people, and no reader is it. What that costs was small when displays were broad and grows every time the primaries get narrower.

 Three ways to spend a thousand code values. Normalised code value against luminance, for PQ, HLG and a conventional gamma curve, all covering 0.001 to 10000 cd/m². PQ spends 51% of its range below 100 cd/m² — roughly where a picture lives — and the gamma curve spends 15%, leaving the rest for highlights. A PQ code is the only one of the three that names a luminance rather than a fraction of whatever the display can manage. Where the model breaks

How bright is white

An sRGB value says a pixel is some fraction of whatever the display can manage. A PQ value says it is two hundred candelas. The change is the largest in display encoding since gamma, and what it removed was the honest admission that nobody knew.

 A 20 nm bandpass, and what it does to the sample. The true reflectance, the same reflectance as a 20 nm instrument reporting every 20 nm returns it, and the slit function responsible, drawn at its own scale around 550 nm. The reconstruction is what every calculation downstream will use, and it differs from the truth by ΔE 2.97 under D65. Nothing in the file the instrument writes marks which values were measured and which were interpolated. Matching and measuring

What the instrument reports

A spectrophotometer sees a sample through a slit of finite width, at a finite number of wavelengths. What it writes down is the truth convolved with its own bandpass, and nothing in the file says which values were measured and which were invented.

 Longitudinal chromatic aberration of the eye, focused at 555 nm. Thibos's chromatic eye, plotted as defocus in dioptres relative to 555 nm. The curve crosses zero exactly once, at the wavelength the eye is accommodating on, and everything else is out of focus — by -1.22 D at 420 nm and 0.50 D at 680 nm. Through a 3 mm pupil that first figure is a blur circle of 12.6 minutes of arc, against a foveal acuity limit of about one. The eye is never in focus across the spectrum and never can be. What the eye does

Nothing is in focus at both ends

The eye carries about two dioptres of chromatic aberration across the visible band, which is a strong reading prescription. Whatever it is focused on, most of the spectrum is landing somewhere other than the retina — and the cone class that gets the worst of it is the one the retina bothered least to sample.

 five greens, no neutral, under D65 — four guesses at the illuminant. The scene's surfaces as they reach the eye, and each estimator's angular error in degrees against the illuminant that actually lit them. Grey-world 43.1°, Max-RGB 27.7°, Shades-of-grey (p = 6) 31.4°, Grey-edge 34.4°. The best here is Max-RGB, which is best because this scene happens to satisfy its assumption — something reflects fully in every band — and not because it is the better algorithm. Any hatched patch is a surface this display cannot show under this light. What the brain does

The algorithms that guess the light

What reaches a sensor is an illuminant multiplied by a reflectance, and no arithmetic separates a product into its factors. Every white-balance algorithm therefore works by assuming something about the world — and the interesting content of each is not its formula but the assumption, because a scene can violate it.

 How far a match comes apart when the observer changes. A broad source and a three-primary source, solved at each primary width so the pair is an exact tristimulus match for the CIE 1931 observer. The pair is then handed to the 1964 observer, and the gap between them is plotted. For the observer they were built for the gap is arithmetic noise at every width. For the other it grows as the primaries narrow, reaching 0.012 at 10 nm — and displays have been getting narrower for twenty years. Where the model breaks

Two degrees or ten

This site names the observer on every figure it draws, which was supposed to be the point. Measuring what it had actually done found two hundred and four figures naming the same observer, three naming the other, and sixty-two of sixty-four generators printing a string that had never once varied.

 The ΔE = 1 contour at points across the L* = 55 plane, magnified 14×. Each closed curve is the set of colours exactly one unit of difference from the dot at its centre, traced by bisection along 30 directions and drawn 14 times life size. Under ΔE2000 the contours run from 0.68 to 5.07 CIELAB units across this slice — a ratio of 7.44 — and they are not circles and not aligned with each other. A formula whose contours were circles of one radius everywhere would be claiming that CIELAB is uniform, which is what the 1976 formula claims and what the measurements refuse. Difference and uniformity

Where the formula is not smooth

A colour difference formula is a distance, and a distance ought to vary gently. CIEDE2000's does not everywhere — it carries a hue-rotation term with a hard edge in it, and the discontinuity sits where a great many industrial samples live.

 A glossy surface returns two spectra, and only one of them is the paint. The dichromatic reflection model, computed rather than assumed. Light that enters a dielectric binder, scatters off pigment and comes back carries the reflectance — the body component, ΔE00 = 33.7 from the lamp. Light reflected at the interface never entered, so it carries the lamp's spectrum with only Fresnel's slight dispersion on it: ΔE00 = 1.06. The interface term is computed from Fresnel's equations on a Cauchy index at 55°, so its near-neutrality is a result here rather than an assumption. This is why a highlight is the one region of a photograph that tells a white balancer what the light was, and why removing highlights removes the evidence. What a scene does

The highlight is the lamp

Light reflected at the surface of a glossy material never entered it, so it never met a pigment. Fresnel's reflectance varies by 5.5% across the visible band and the paint underneath varies by a factor of twenty — which makes a highlight the one region of a photograph that reports what the light was rather than what the object is.

 One white balance across a scene lit by two lamps. A neutral surface of albedo 0.6 under 7 mixtures of A and D65, corrected by one diagonal transform chosen for the middle of the run — which is what a camera does when it estimates a single illuminant. The middle patch comes out neutral to ΔE00 = 0.00 and both ends do not: 19.8 at the A end and 16.9 at the D65 one. The failure is structural rather than a matter of a better estimator: white balance is one transform for the whole image, and a scene with two lamps in it has no single answer for that transform to be. Every patch here is the same surface. What a scene does

A scene has no white point

White balance is one transform applied to a whole image, and a scene lit by two lamps has no single answer for that transform to be. The failure is structural rather than a matter of a better estimator — and every colour-managed workflow in existence takes exactly one white point as an input, with no field in which to say there were two.

 A glossy surface returns two spectra, and only one of them is the paint. The dichromatic reflection model, computed rather than assumed. Light that enters a dielectric binder, scatters off pigment and comes back carries the reflectance — the body component, ΔE00 = 33.7 from the lamp. Light reflected at the interface never entered, so it carries the lamp's spectrum with only Fresnel's slight dispersion on it: ΔE00 = 1.14. The interface term is computed from Fresnel's equations on a Cauchy index at 45°, so its near-neutrality is a result here rather than an assumption. This is why a highlight is the one region of a photograph that tells a white balancer what the light was, and why removing highlights removes the evidence. What a scene does

Gloss changes the measurement

The same sample measured with the specular component included and excluded returns two different numbers, and both are correct answers to different questions. Colour is one of four appearance attributes and the only one most instruments report — so a specification that names a colour has silently named a geometry too, and usually does not say which.

 A camera's spectral sensitivities, after the infrared-cut filter. Silicon quantum efficiency times the colour-filter dye times the infrared-cut filter, per channel, on a grid running to 1100 nm rather than to 780. With the filter removed, 68 per cent of the area under the three curves lies beyond the visible band, and all three curves are the same curve out there. What a camera does

A camera is a fourth observer

The 1931 functions, the 1964 functions and a person's own cones are three sets of three curves that collapse a spectrum onto three numbers. A camera is a fourth, built from silicon and dye rather than from pigment and neural wiring, and it agrees with none of them.

 Everything between the photons and the picture, and what each stage decides. The 8 stages of a camera pipeline. Only the second is physics; every one after it is a decision somebody made, and the reason two cameras pointed at the same scene disagree is that they made different ones. What a camera does

Raw is not a picture

A raw file is three integrals per site and a list of decisions nobody has taken yet. Every one of those decisions has a defensible answer and none of them has a correct one, which is why two converters open the same file and disagree.

 The blue channel, as the three things multiplied to make it. Silicon's quantum efficiency, the colour-filter dye's transmittance, and their product. The dye is the only stage carrying any colour information and it is clear above about 800 nm — transmittance 0.92 at 900 nm against 0.06 at 550. What a camera does

Silicon sees past the visible

A sensor's response ends at 1107 nanometres because that is the band gap, and the colour-filter dyes have stopped absorbing three hundred nanometres before it. So all three channels measure the same thing over the last third of the range, and every published sensitivity plot is drawn after the component that hides it.

 How much of what an unfiltered sensor records is invisible. The share of a camera's raw signal coming from beyond 780 nm, against the colour temperature of the lamp, for one surface with a near-infrared reflectance of 0.62. Without the filter it runs from 98 per cent at 2000 K to 51 at 9000; with it, under five per cent everywhere. What a camera does

The filter that makes colour possible

An infrared-cut filter is not a refinement on a colour camera. Removing it collapses the separation between the three channels by a factor of nearly nine under tungsten, and the component that keeps colour photography working is the one nobody photographs.

 One reflectance, four different infrared tails, and what the camera makes of each. The same visible reflectance continued past 780 nm to four different near-infrared values. A spectrophotometer reports only the left-hand part; an unfiltered sensor integrates all of it. The channel spread falls from 0.14 at a tail of 0.05 to 0.02 at 0.85, with nothing about the visible half changed. What a camera does

Most things are pale in the infrared

A spectrophotometer stops at 780 nanometres and a black cotton shirt reflecting five per cent of visible light reflects more than half the near infrared. The measurement everybody has and the quantity a camera integrates are different quantities, and nothing in the first says so.

 A silicon sensor's best possible impersonation of the standard observer. The 1931 matching functions in outline, and the closest linear combination of the sensor's three sensitivities laid over them; underneath, what is left over at each wavelength. The residual is 31.7 per cent of the matching functions' own magnitude, worst at 440 nm. Colour reproduction is exact if and only if this is zero. What a camera does

Luther said when it would work

There is an exact condition under which a fixed three-by-three matrix converts camera raw to XYZ correctly for every spectrum in existence. It was stated in 1927, it is a theorem rather than a guideline, and no camera ever built satisfies it.

 A camera profile is a fit, and the sample set is a hidden argument to it. Per-surface ΔE00 after the best 3 × 3 from raw to XYZ, fitted on 12 surfaces at chroma 0.2 and tested twice: on those same surfaces (mean 0.65) and on 12 at chroma 0.9 (mean 1.75). Both bars come from the same matrix; only the surfaces differ. What a camera does

A camera profile is a fit

Since no matrix is exact, the one a manufacturer ships is a least-squares compromise over a set of surfaces somebody chose. Fitted on desaturated patches it is excellent on desaturated patches — and the sample set is a hidden argument to every camera profile in existence.

258 essays on this thread, page 1 of 11 · all threads · all essays