Where the model breaks

Either factor being zero

Four different departures from the colour integral turn out to have one algebraic form — each is an inner product of something the sample does that the model has no slot for with something the light does that the model assumed away. Either factor being zero makes the departure exactly zero, and the sizes of the two factors decide neither how large it is nor which way it goes.

Assumes The model has six arguments, Three choices reached and A model is a claim about what can be known.

Four departures from one equation, in four different physics, measured with four different instruments. The useful thing about having all four at once is not the ranking; it is that they turn out to be the same sentence.

Eight conditions under which the model equation is exact, and how exact each one is. Each of the four departures vanishes if either of its two factors is empty, which is eight conditions. The axis is logarithmic in the residual that is left when the condition is imposed. Three of the eight are identities: the fluorophore's loading is zero so the emitted term is an empty sum, and a Lambertian surface or a uniform field makes the pairing's second argument identically zero. The other five are limits — a Gaussian excitation band has no edge, an opaque sample still has a kernel a few microns wide, a four-metre aperture is still finite, and the observer is small rather than absent at 380 nanometres. Each limit is drawn with the sequence its residual falls along as the condition is pushed, because a small number is not evidence of a limit and a falling sequence is.
Fig. 1 Each departure vanishes if either of its two factors is empty, which is eight conditions. Three of them are identities in floating point and five are limits, drawn as the sequence each residual falls along when the condition is pushed.

Those eight conditions belong to four departures, and the four are one decision seen four ways: which of a surface’s six arguments the model is allowed to keep.

The six arguments a surface's response has, and the one this model keeps. A surface's response to light is a function of six arguments: the wavelength, direction and place the light arrives with, and the wavelength, direction and place it leaves with. The model every colour here is computed from keeps one number per wavelength, which means it takes the diagonal of the first pair, integrates the second away, and assumes the third pair equal. Each departure drawn here restores one of them. The fourth departure is not on the diagram: the wavelength grid is the range of the index that was kept rather than an index that was dropped, which is why it is the cheapest of the four to fix and was still not fixed.
Fig. 2 The four departures as one decision. A surface’s response has six arguments and the model keeps a diagonal of one pair, so each departure is one of the arguments it dropped, coming back with a magnitude attached.

The claim

Every departure here is a pairing of two deviations — one belonging to the sample, one to the light — and the pairing is exact rather than approximate.

  • Written as an inner product, each departure is ⟨ what the sample does that the model cannot hold , what the light does that the model assumed away ⟩.
  • Either factor being zero makes it exactly zero, which gives eight conditions from four departures.
  • It is bilinear, not linearised. Doubling either factor doubles the departure, at any size, with no small-parameter expansion anywhere.
  • The prediction that the two factors’ sizes decide the answer was refused. Two deviations of a given size pair to anything between plus and minus their product, and a viewing booth and a window move the same gloss sample in opposite directions.
  • And one of the four is rank one, which is the special case where the sizes really do decide it — and the reason is a rule about molecules rather than about arithmetic.

The shape, in one departure

Take the directional index, because it is the one where the algebra can be written in three lines.

A detector looking at a surface in a field of radiance L reads a reflectance factor of π ⟨f, p⟩, where f is the bidirectional reflectance towards the detector, p is the arriving light’s own distribution over the hemisphere, and the pairing is the cosine-weighted average. Under a uniform field the same detector reads π ⟨f, u⟩, which is the sample’s own reflectance, by reciprocity. Subtract:

error = π ⟨f, p − u⟩

and then notice that p and u both integrate to one, so ⟨c, p − u⟩ = 0 for any constant c. Subtracting the surface’s Lambertian part changes nothing:

error = π ⟨f − ρ/π , p − u⟩

Both arguments now have mean zero. The first is everything about the surface that a single reflectance cannot hold; the second is everything about the room that a uniform field would not have had. Neither is an approximation and no term was dropped.

The same shape appears three more times. For the aperture, the sample’s factor is its kernel and the light’s is the aperture’s shortfall against an infinite one, and the reading minus the reflectance is their pairing over the plane. For the grid, the sample’s factor is whatever it does outside 380 to 780 nanometres and the light’s is whatever power it has there — the two do not stop at the same wavelength, which is the whole of that departure. For fluorescence, the sample’s factor is the fluorophore’s excitation band and the light’s is the lamp’s power inside it.

Four departures, four physics, one line of algebra. It is worth being suspicious of that, because a form general enough to hold anything holds nothing — so it is worth saying what the form excludes. It excludes any departure that is not linear in the light, and there is one on this site: a brightener is used up while it is being measured, so a sample’s response depends on how long the lamp has been on it. That one is not a pairing and does not appear here.

Eight conditions, and they are not all the same kind of statement

Four departures with two factors each gives eight ways of making the error vanish. Computing all eight was meant to be a formality and produced the round’s first surprise.

Three of them are identities. A Lambertian surface reads the same in five different fields to four parts in a hundred million million; every surface reads its own reflectance under a uniform field to the same precision; and a sample with no fluorophore emits exactly nothing, because a loading of zero makes the absorbed-photon sum empty. These are not small numbers, they are floating-point noise, and they are that way because the second argument of the pairing is identically zero rather than merely tiny.

Five are limits. A Gaussian excitation band has no edge, so no cut filter is entirely outside it: a filter at 400 nanometres leaves four parts in a thousand, at 450 six in a million, at 500 four in a hundred thousand million. An opaque sample still has a kernel a few microns wide. A four-metre aperture is still finite. And the observer is small rather than absent at 380 nanometres, so a substrate that reflects there is seen no matter how little of it there is.

The distinction matters because a limit and an identity look the same when the numbers are small, and one of them is a statement about the world while the other is a statement about a construction. So each limit is reported with the sequence its residual falls along as the condition is pushed, and the assertion behind the figure requires that sequence to be monotone. A small number is not evidence of a limit; a falling sequence is.

The pairing against the direct computation, for each departure that admits both. Each departure can be computed twice: directly, by taking the difference between the fuller model and the integral one, and as a pairing — an inner product of the sample's deviation with the light's. The bar is how far apart the two answers are, relative to the answer, on a logarithmic axis. The three directional rows agree to a part in a thousand billion, which is the arithmetic of one shared quadrature. The lateral row agrees to three parts in a hundred thousand, and the gap there is the radial quadrature rather than the identity: the two integrals are taken over different grids. The pairing is not an approximation to the departure. It is the departure, written so that its two factors are separate.
Fig. 3 The pairing against the direct computation. Three rows agree to a part in a thousand billion because they share one quadrature; the fourth’s gap is two different grids rather than two different answers.

The prediction, and what the arithmetic did with it

Before any of the cells were computed, the round wrote down what it expected: that the error would be the product of the two factors’ magnitudes, so that a surface twice as glossy in a room twice as directional would be four times as wrong.

Half of that is right and it is the uninteresting half. Bilinearity does hold exactly, so the error is proportional to each factor separately.

The other half is wrong, and it is wrong in the way an inner product is always wrong when it is mistaken for a product. A pairing depends on the angle between its arguments. Two deviations of fixed size can give anything from plus their product to minus it, and the size of neither says which.

Measured over twenty-four cells — six roughnesses in four non-uniform fields — the fraction of the Cauchy–Schwarz bound actually used runs from −0.101 to 0.365. Six of the twenty-four are positive and eighteen are negative. The positive ones are all the viewing booth, which lights a surface mostly from near its normal, where a detector eight degrees off the normal is looking at the specular lobe. The negative ones are the window, the lamp and the sun, which light it from thirty-five or forty-five degrees, where the lobe points somewhere else entirely.

So a viewing booth reads a gloss sample high and a window reads the same sample low, and no comparison of anisotropies could have said so.

The one that really is a product

The fluorescent departure is the exception, and its exception has a physical cause worth stating.

What a fluorophore returns does not depend on where in its excitation band the photon was absorbed. That is Kasha’s rule — the molecule relaxes to the lowest excited state before it emits, so the emission spectrum is the same shape whatever excited it — and it makes the off-diagonal part of the response matrix rank one: one spectrum belonging to the sample, multiplied by one number belonging to the light.

A rank one pairing has no alignment left to vary. There is only one direction, so the inner product really is a product, and the departure really is proportional to the two magnitudes. Scaling the part of a lamp below 400 nanometres by a factor and watching the emitted photon count follow it is exact to twelve figures over a range including zero and twice the real lamp.

The sweep is affine rather than proportional, and the intercept is instructive: 17.7 per cent of what a full D50 lamp excites is still excited when everything below 400 nanometres is removed, because the excitation band’s long tail reaches into the visible. That intercept is why a measurement condition that cuts the ultraviolet is a condition rather than a control — it does not turn the fluorescence off, it turns five sixths of it off, and the sixth that remains is why the four standard conditions are four quantities rather than four degrees of care.

The sign is a property of the field alone

The sign split is reported as a count and a description — six positive of twenty-four, and the positive ones are all the viewing booth — and the two together say something stronger than either.

The cells are six roughnesses in four fields, so six is exactly one field’s worth. The viewing booth’s six cells are the six positives and the other three fields’ eighteen are the eighteen negatives, with no cell of any field on the wrong side.

So the roughness never changes the sign, across whatever range six roughnesses span, and the alignment’s sign is decided entirely by the light. That is a much cleaner statement than the essay’s and it is the practically useful half: a laboratory does not need to know its sample’s gloss to know which way its instrument is wrong, only its own geometry.

It also sharpens the refusal. The prediction the round wrote down was that the two magnitudes decide the answer; the measurement says the sample’s factor decides neither the sign nor, on this evidence, much of the magnitude — the whole of the sign and most of the variation belongs to the field.

How loose the bound is, at both ends

The essay quotes the bound’s looseness once, as a factor of two hundred, which is its worst case. The utilisations run from −0.101 to 0.365, so the bound is loose by 2.7 at best and by about 200 at worst — a range of a factor of seventy-five across twenty-four cells of one experiment.

That range is worth having because it changes what the bound is for. A bound loose by two hundred everywhere would be useless; a bound loose by 2.7 in the worst-behaved geometry is a genuinely informative upper limit. The viewing booth — the geometry a laboratory actually uses — is the one where the bound is nearly tight, at 0.365 of it, and the fields where it collapses are the ones a specification does not describe anyway.

So the pairing form gives a usable bound exactly where measurements are made and a useless one where they are not, which is an oddly convenient distribution and is worth stating rather than averaging away.

What the residual sequence is really measuring

The three filter residuals — 4 × 10⁻³ at a cut of 400 nanometres, 6 × 10⁻⁶ at 450, 4 × 10⁻¹¹ at 500 — are offered as evidence that the condition is a limit rather than an identity. The rank-one result two sections later makes them a good deal more than that.

If the departure is exactly proportional to the excitation the lamp supplies — which is what rank one means, and which the sweep confirms to twelve figures — then the sequence is a direct measurement of the excitation profile’s own tail. Dividing through by the unfiltered case, which the 17.7 per cent intercept fixes at 2.26 × 10⁻², gives the share of the excitation surviving each cut:

cut residual share of the excitation above it
400 nm 4 × 10⁻³ 17.7%
450 nm 6 × 10⁻⁶ 0.027%
500 nm 4 × 10⁻¹¹ 1.8 × 10⁻⁷ %

Two numbers the essay reports separately turn out to be the same measurement, and the intercept it singles out is simply the first entry of the sequence it uses for something else.

The three shares are also checkable against the fluorophore this collection models — a Gaussian excitation centred at 358 nanometres with a standard deviation of 22. Its unweighted tails above the three cuts are 2.8 per cent, 1.4 × 10⁻⁵ and 5.4 × 10⁻¹¹, so the measured shares exceed them by factors of 6.3, 18.4 and 32.6.

Those factors rise, and they have to. The share is weighted by the lamp’s own power, and a daylight spectrum climbs steeply across 350 to 500 nanometres — so the further out the cut, the more the surviving tail is being weighted by a brighter part of the lamp relative to the band’s centre. A rising sequence of correction factors is what the physics requires, and finding one is a check that the residual sequence, the intercept and the fluorophore’s stated profile are all describing one object.

What was computed, and how

The eight conditions and the four pairings are computed in one place, which imports the three departures rather than reimplementing them, and the identity check is done twice for each.

Each pairing is computed both ways: as an inner product of two explicitly-constructed deviations, and directly, as the difference between two whole model evaluations. The two have to agree. For the three directional cells they agree to about a part in a thousand billion, which is the arithmetic of a shared quadrature — the same hemisphere, summed in a different order. For the lateral cell they agree to three parts in a hundred thousand, and the gap there is honest: the two integrals are taken over different radial grids, so this is two numerical quadratures agreeing rather than one sum rearranged.

That difference in precision is worth keeping. A pairing checked on a shared grid demonstrates an algebraic identity; a pairing checked on two grids demonstrates the identity and the quadrature. Reporting them at one tolerance would hide which of the two was tested.

The share of a sample's reflectance an aperture recovers, by how wide it is. Six materials, and the fraction of each one's true reflectance that a measurement recovers through an aperture of the stated radius. The horizontal axis is logarithmic in millimetres; the vertical is a share, so 1.0 is the whole of it. The dashed line is a 4-millimetre radius, which is about what a hand-held spectrophotometer has. At that aperture coated paper reads 98 per cent of its own reflectance and candle wax reads 48. Every curve approaches one from below and none of them reaches it: the kernel's tail is what is being cut, and it falls as one over the aperture rather than exponentially.
Fig. 4 The second factor of the lateral pairing, drawn from the other end: the share of a sample’s own reflectance an aperture recovers. The shortfall is what pairs with the kernel, and the curves approach one without reaching it.

Where the model stops

The pairing form says what makes a departure zero. It says almost nothing about what makes one large, because the alignment term is the thing that decides that, and the alignment is a property of the pair rather than of either member.

That is a practical limit and not a technicality. A practitioner with a glossy sample and a directional room has both factors, and would like to know how much trouble that is. The collection has met this shape of answer before: a constraint is a direction and a distance, and a bound that knows the distance and not the direction is exactly half of one. The pairing gives an upper bound — the two norms multiplied — and the measured cells use between half a per cent and thirty-six per cent of it. A bound loose by a factor of two hundred is not a useful answer to how much.

What the form does give is a decision procedure for whether: measure either factor, find it near zero, and stop. That is worth a great deal more than it sounds, because it is the question a specification actually asks. A tolerance cannot cross a measurement condition, and the reason is that the condition is one of the two factors.

The generalisation

The shape has a name outside colour: it is a first variation. A model is a functional of two inputs; the fuller model differs from it by a term that is bilinear in the two departures; and the bilinear term is the pairing.

What is worth carrying is not the algebra but the reporting discipline that comes with it. Where a departure can be written as a pairing, three separate statements become available and are usually confused:

  • The conditions, which are what makes it exactly zero, and are the only part that is a theorem.
  • The bound, which is what a product of magnitudes can promise, and is usually enormously loose.
  • The value, which needs both factors in full and cannot be estimated from summaries of either.

A published claim that a model is safe because the sample is nearly Lambertian is a claim about one factor. A claim that it is safe because the room is nearly uniform is a claim about the other. Both are legitimate and each on its own is complete, which is the useful consequence: either half of the argument suffices, and half an argument is often all that is available.

Four departures from the model equation, each at an ordinary strength. What each of the four assumptions inside a colour integral costs, in ΔE₀₀, on a stated sample under a stated light. The wavelength index is a coated printing paper measured with and without the ultraviolet of D50; the range is the same paper integrated from 300 nanometres and from 380; the place index is a pigmented plastic through a four-millimetre radius; the direction index is an eggshell paint beside a window. The spread is a factor of 7.0. This is a ranking of four examples rather than of four departures — each of them can be made larger by choosing a more extreme sample, and the marble in the same collection of materials reaches 12.7 on the index that comes third here.
Fig. 5 The four departures at ordinary strengths, in one unit. Each is a pairing whose two factors are named in the row’s own description, and each is above the tolerance a specification is written in.
An aperture and a gloss lobe, apart and together. Six materials, each measured through a four-millimetre radius and each given a gloss lobe, alone and at the same time. The pale bar is what the two cost added together as if they were independent; the dark one is what they cost when both are present. Every material comes out below the sum, by between 0.8 and 3.6 ΔE₀₀. The two departures partly cancel: the aperture removes light that went into the material and came back out too far away, and the interface returns light that never went in at all. Measuring either one alone therefore overstates what both together do, which is the opposite of the way interacting errors are usually assumed to behave.
Fig. 6 And what happens when two departures are present at once, which the pairing form does not predict: they partly cancel, on every material tested.

Who found it, and when

None of the four pairings is new as physics. Each is the leading term of a perturbation somebody wrote down decades ago in its own field: the bidirectional one in radiative transfer, the fluorescent one in photometry of brightened materials, the lateral one in the diffusion theory of turbid media.

What is new here is that they are the same leading term, and that saying so is only possible once all four are computed in one unit on one set of samples. That is an argument for the shape of this collection rather than for any result in it: four departures in four literatures, each with its own instrument and its own vocabulary, are a list of unrelated cautions until somebody puts them on one axis.

The refused prediction has a longer pedigree still. Mistaking an inner product for a product of magnitudes is the error that Cauchy and Schwarz’s inequality exists to bound, and the bound is famous precisely because the gap between it and the truth is where all the information lives.

Where the ladder goes next

Two things about the pairing are unfinished, and one of them is cheap.

The alignment has no model. Twenty-four cells give twenty-four cosines, and nothing predicts them from the field and the surface separately, because nothing could — but a family of fields might have a low-dimensional description that does. The obvious candidate is the projection of the field onto the first few spherical harmonics, which is how graphics has handled this since the 1990s.

The alignment is also what a figure can show and a number cannot, which is why the refusal above is drawn as a scatter with both signs on it rather than reported as a correlation coefficient. A single number summarising twenty-four cells that fall on both sides of zero would have been about 0.1 and would have said nothing at all — the same failure a site average hides in a link count.

And the interaction between departures is not a pairing. Two departures at once are not the sum of two pairings; they partly cancel, by more than the smaller of the two on every material tested. That is a second-order term the form above has no room for, and the round measured it rather than modelling it.

What this makes readable

Essays that name this one as a prerequisite.

Named alongside this one

Essays reaching for the same objects. Nobody chose these; they are what the index of named objects makes visible.

What links here

The 8 essays that link to this one and share the most of its objects, of 15 that link here.

The objects this essay names

Each one links to every other essay that touches it.

ApertureBidirectional reflectanceBilinearityThe Donaldson matrixFalsificationIllumination uniformityMarginalisationPredictionStructural choiceSubsurface scattering