The highlight is the lamp

The central difficulty of colour constancy is that what arrives at a sensor is a product. The illuminant multiplied by the reflectance, integrated — and no arithmetic separates a product into its factors. Every algorithm that guesses the light works by assuming something about the world, because there is nothing in the signal itself to work from.

Except in one place. On a glossy surface, some of the light never entered the material at all, and what never entered never met a pigment.

The claim #

What leaves a dielectric surface is the sum of two spectra with different provenance: one carrying the material’s reflectance, and one carrying the illuminant almost unaltered.

That is the dichromatic reflection model, and its usefulness is entirely in the word almost. If the interface term carried the illuminant exactly, it would be a perfect illuminant probe. If it were as strongly coloured as the body term, it would be useless. It is neither, and how nearly it carries the lamp is a number rather than an intuition.

Why the interface term is nearly neutral #

At a boundary between air and a dielectric, the fraction of light reflected without entering is given by Fresnel’s equations. For unpolarised light at normal incidence they reduce to

$$R = \left(\frac{n-1}{n+1}\right)^{2}$$

with $n$ the refractive index of the material. The only wavelength dependence is whatever $n$ has, and for the transparent binders that pigments are suspended in — oils, acrylics, the resin in a car’s clearcoat — $n$ is close to 1.5 and barely disperses.

Computed here from Cauchy’s two-term dispersion, $n = A + B/\lambda^2$, the index runs from 1.5309 at 400 nm to 1.5132 at 700 nm. Push that through the Fresnel expression and the normal-incidence reflectance runs from 0.04400 to 0.04169 — a variation of 5.52% across the whole visible band.

Compare that with the body term. The paint in the figure above has a reflectance running from 0.03 to 0.75, a factor of twenty-five. So the two components differ in spectral flatness by well over two orders of magnitude, and the resulting chromaticities differ accordingly: 1.06 units of $\Delta E_{00}$ from the lamp for the interface term against 33.7 for the body.

The interface term is not exactly the illuminant. It is the illuminant tilted very slightly towards the blue, because $n$ is slightly higher there. A perfect white-balance cue would be a $\Delta E$ of zero and this is 1.06, which is around the threshold of visibility — small enough to be an excellent estimator and large enough that calling it exact would be wrong.

What the angle does #

Fresnel reflectance rises steeply towards grazing incidence, and that is why a highlight exists as a distinct bright region rather than a uniform sheen.

Averaged across the band, the reflectance is 0.0424 at normal incidence, 0.0726 at 55°, and 0.3908 at 80° — nearly ten times the head-on value. A glancing view of any dielectric returns a great deal of light without it ever entering the material, which is why a matte sheet of paper viewed almost edge-on looks glossy, and why wet asphalt at night is a mirror.

That near-separability is the property the whole technique rests on. If the highlight changed colour with viewing angle, its chromaticity would report the geometry rather than the lamp, and it would be no more useful as a probe than the paint is.

The exception that proves it #

Everything above is a statement about dielectrics, and it fails completely for metals.

A metal has a complex refractive index, its reflectance is strongly wavelength-dependent, and there is no body component at all — light either reflects at the surface or is absorbed within a fraction of a wavelength. So gold’s yellow and copper’s red are properties of their specular reflection, which is exactly the component that is neutral for a dielectric.

This is the standard way of separating the two classes, and it is a much cleaner division than the everyday categories suggest. Plastic, paint, wood, skin, paper, ceramic glaze and water all behave one way; gold, copper, brass and aluminium behave the other. A “metallic” car paint is a dielectric clearcoat over aluminium flake and does both at once, which is why it is difficult to photograph and harder to specify.

The practical consequence for an illuminant estimator is worth stating plainly: a highlight on a metal reports the metal, not the lamp, and an algorithm that assumes otherwise will confidently white-balance a photograph of brass to the wrong answer.

What was computed, and how #

The refractive index is computed, not assumed constant. That is deliberate, because the claim is about how nearly flat the interface term is, and fixing $n$ would have made the answer exactly flat by construction. The 5.52% and the residual 1.06 $\Delta E$ are the outputs of letting it disperse.

The Fresnel expression is the unpolarised average of the two polarisation components, $\tfrac{1}{2}(r_s^2 + r_p^2)$, with the normal-incidence case handled by its own closed form rather than by taking a limit numerically. Both branches are in the code because the general expressions are indeterminate at $\theta = 0$.

The assertion is comparative. The generator refuses to draw unless the interface term is within 2 $\Delta E_{00}$ of the lamp and the body term is more than 15 away. Asserting only the first would pass on a surface with no colour of its own, where both components are neutral and the figure demonstrates nothing.

The illuminant is A rather than D65 in the hero. Illuminant A is a Planckian radiator at 2856 K and is strongly yellow, which makes the “the highlight is the lamp” claim visible as a colour rather than as a number. Under D65 both the lamp and the highlight are near-neutral and the figure proves the same thing while looking like less.

Why constancy algorithms want this #

The estimators on this site all work by assuming something about the scene: that the average reflectance is neutral, that something in the scene reflects fully in every band, that the average edge is neutral. Every one of those assumptions is something a scene can violate, and the measured failures are large — grey-world reaches 43.1° of angular error on an all-green scene.

A highlight is a different kind of evidence. It is not an assumption about the distribution of surfaces; it is a region of the image where the illuminant is present nearly undisturbed. Methods that find highlights and use them do not need the scene to be statistically well-behaved.

What they need instead is for the highlight to be findable, and that is where the approach gets difficult:

• Highlights are usually clipped. The interface term at a glancing angle is many times brighter than the body term, so in an ordinary exposure the highlight is at the top of the sensor’s range and its chromaticity has been destroyed by saturation. The most informative pixels in the image are the ones most likely to be unrecoverable.
• They are small. A specular region may be a few dozen pixels, and any noise in them is not averaged away by area.
• They are mixtures. A pixel in the penumbra of a highlight is a sum of both components in unknown proportion, so the illuminant has to be recovered from the direction the pixel values move as the specular fraction rises rather than from any single pixel’s colour.

That last point is the actual algorithm, and it is elegant: the set of pixel values across a single glossy object of uniform colour forms a plane in colour space, spanned by the body colour and the illuminant. Fitting the plane and finding the illuminant direction recovers the lamp without ever needing a pure specular pixel.

Which lamp, though #

There is a complication this field is well placed to raise and the machine-vision literature mostly does not.

A highlight carries the spectrum that arrived at that point on that surface. In a room, that is not the lamp. It is the lamp multiplied by every reflectance along every path that reached the surface — which is the whole content of the first rung of this ladder. A highlight on a glossy object standing next to a red wall reports a reddened illuminant, faithfully, because a reddened illuminant is what is actually falling on it.

So the highlight is an excellent estimator of a quantity that is not quite the one the estimator wanted. This is not a defect — arguably it is reporting the more useful number, since the local illuminant is what actually determines every colour nearby — but it does mean two highlights in one photograph can legitimately disagree, and that neither is wrong.

It also sets a limit on the accuracy worth chasing. The interface term’s own bias is 1.06 $\Delta E_{00}$; the variation in local illuminant across an ordinary room is several times that. Refining the Fresnel model past the point where interreflection dominates is polishing the smaller of two errors.

The two components on the diagram #

Plotting the split rather than tabulating it makes the geometry of the estimator visible.

Two more readings say that the interface term is the same spectrum however the sample and the geometry are chosen, which is the property the whole method rests on.

Every pixel on a uniformly coloured glossy object is a non-negative combination of two fixed spectra. In chromaticity terms that puts every pixel on the line segment between the body colour and the illuminant, with position along the segment set by how much specular the geometry happened to deliver.

That is why the method works from pixels that are nowhere near pure specular. The direction of the segment is what carries the illuminant, and a scatter of pixels spread along even a short stretch of it determines that direction. What kills the method is not the absence of bright highlights but the absence of variation — a perfectly matte object gives a single point, and a point has no direction.

The failure worth naming: an object whose body colour happens to lie close to the illuminant’s chromaticity gives a very short segment, and the direction is then poorly determined. A white object is the worst possible illuminant probe for this method, which is the exact opposite of the intuition that a white object tells anybody what the light is. That intuition belongs to the max-RGB estimator, which wants a white object and fails when there is none; the two methods want opposite scenes.

The highlight gets less coloured towards grazing #

The highlight gets brighter without getting more coloured is the property the plane-fitting method is said to rest on, and the arithmetic says something better and slightly different: it gets brighter and less coloured, by a large factor.

Recomputing the interface term’s distance from illuminant A at three angles, each normalised to one luminance so only colour is compared:

A factor of 7.6 in the bias, running the helpful way. The mechanism is in the Fresnel expression itself: towards grazing incidence the reflectance climbs towards one at every wavelength, so the dispersion is squeezed out — a spectral difference of five and a half per cent between two reflectances near 0.04 becomes a much smaller one between two reflectances near 0.4, because both are being pushed against the same ceiling.

So the two things a highlight estimator wants are not in tension: a highlight is brightest exactly where it is most nearly the lamp. At 80° it carries nine times the light and a fifth of the bias.

That also amends the separability claim rather than confirming it. Fresnel’s angular and spectral dependences are not separable — if they were, the bias would be 1.51 at every angle — and the non-separability is what makes the glancing end accurate. The plane-fitting method’s segment is therefore slightly curved rather than straight, since the illuminant endpoint moves by 1.3 ΔE₀₀ between the normal and grazing ends of one highlight. That is a real caution for the method and a small one: it is a bend of about a unit across a segment tens of units long.

Fifty-five degrees is Brewster’s angle #

The hero figure’s viewing angle is worth naming, because it is not an arbitrary choice and it is the one angle at which the essay’s closing caveat about polarisation is at its most extreme.

For this dielectric at 555 nanometres the index is 1.5183, so Brewster’s angle is 56.6°. The figure is drawn at 55, which is 1.6 degrees away — and at 55° the two polarisation components are r_s² = 0.1448 and r_p² = 0.00028, so the reflected light is 99.6 per cent polarised.

Two things follow, and they pull in opposite directions.

A polarising filter would remove essentially the whole highlight there. Removing them destroys exactly the illuminant evidence this essay is about is true at every angle and it is true to four significant figures at this one. A photographer using a polariser to kill reflections on glass or water is working near Brewster by instinct, and is deleting the best illuminant cue in the frame.

And Brewster is the worst angle for the estimator’s other requirement. The unpolarised reflectance at 55° is 0.0726, only 1.7 times the head-on value, because the p component has been driven to zero — whereas at 80° it is 9.2 times. The angle at which the highlight is most nearly pure polarisation is the angle at which there is least of it.

So the hero is drawn at the angle that shows the split most clearly and probes the lamp least well, which is the right choice for a figure and the wrong one for a measurement. A highlight estimator wants 75 to 85 degrees: nine times the light, a fifth of the bias, and a degree of polarisation back down to forty per cent, so that a polariser in the path removes less than half of it rather than all.

One smaller dependence, for completeness. The bias is a property of the lamp as well as the interface: the same 55° highlight sits 1.06 ΔE₀₀ from illuminant A and 0.92 from D65, because a warm lamp puts more of its power where Fresnel’s tilt has further to act. The estimator is therefore slightly better outdoors than under tungsten, on top of everything else that is.

Past Brewster’s angle the interface component is stronger again, and a red pigment under daylight is where the split is largest.

Where the model stops #

The transport model here is Lambertian and this is bolted on afterwards. The radiosity solve underlying the rest of this field has no directions in it at all, so it cannot produce a highlight. The dichromatic split is computed separately and added, which means nothing here can show a highlight moving as a reader moves, or a specular interreflection between two glossy surfaces.

The interface term is a single reflection. Real gloss has microstructure, so a real highlight is spread over a range of angles rather than mirror-sharp, and light that reflects off one microfacet onto another has bounced twice at the interface. The second bounce squares the Fresnel term, which is small but is not zero.

Subsurface scattering is folded into the body term. For paint that is a good approximation. For skin, marble and milk it is not — the light enters, travels laterally, and emerges somewhere else, which is a matter of path length rather than of reflectance.

No polarisation is tracked beyond the unpolarised average. Specular reflection from a dielectric is strongly polarised near Brewster’s angle, which is why a polarising filter removes highlights and why removing them destroys exactly the illuminant evidence this essay is about.

And the display cannot show most of what is computed here. The body component under illuminant A is a saturated orange at low luminance and falls outside the sRGB triangle; where that happens the swatches are hatched rather than clipped, which is the marking this site uses everywhere in place of the nearest available lie. A reader seeing hatching is seeing a correct report that the display is an unknown and has run out of gamut, not a broken figure.

A near-normal view of a green pigment under a tungsten lamp is the case where the interface component is weakest and still decisive.

The generalisation #

The transferable idea is about provenance rather than magnitude.

Two contributions to one measurement may be similar in size and completely different in what they have been multiplied by. Separating them is not a matter of filtering out the smaller one; it is a matter of noticing that one of them took a path that left it uncontaminated by the quantity being solved for. The highlight is not useful because it is bright. It is useful because of where it has not been.

The same shape appears wherever a measurement contains a direct and an indirect path — a reference beam in an interferometer, an internal standard in a chemical assay, a control channel in an experiment. In each case the useful component is the one that skipped the step whose effect is unknown.

A grazing view is the other extreme, and it is where nearly all of what reaches the eye is the lamp.

Who found it, and when #

Fresnel derived the reflection coefficients in 1823, from the wave theory, and they are among the oldest results in this field still used unmodified.

The dichromatic reflection model as a colour statement is Shafer’s, from 1985, and its framing was explicitly computational: he was arguing that the two components give a machine-vision system a way to separate illumination from surface colour, which nobody had a principled route to before. Klinker, Shafer and Kanade built the plane-fitting method on it shortly afterwards, and the observation that the pixel cloud of a glossy object is planar in colour space is theirs.

The physics of why dielectric specular reflection is achromatic was of course understood long before 1985 — it is one line of Fresnel. What was new was noticing that it made a measurement possible, which is a recurring pattern: the useful consequence of an old result is often not the result but the thing it lets somebody stop assuming.

Where the ladder goes next #

The rung above asks what a renderer loses by carrying three numbers instead of a spectrum, and finds an error that is exactly zero at the first product and grows at every one after it.

Further up the ladder, gloss stops being a source of information and becomes a source of disagreement: the same sample measured with the specular component included and excluded gives two different answers, and both are correct reports of different questions.