A theorem about a family

The most load-bearing sentence in this collection’s account of adaptation is that a change of light is exactly a 3×3 matrix. It is exact, it is a theorem rather than an approximation, and it is a theorem about a family of surfaces somebody constructed.

The claim #

The exactness is a property of the test surfaces spanning three dimensions, and the amount it is worth is not small compared with the effect it was built to isolate.

• On the three-dimensional family the residual after the matrix is 7 × 10⁻¹⁴ ΔE*₀₀ — machine zero. The matrix is solved, not fitted.
• A fourth dimension at five per cent amplitude puts 0.33 ΔE*₀₀ into it for the most natural fourth basis function.
• The whole von Kries residual — the entire subject of the census — runs from 0.26 to 3.37 across fourteen changes of light. So the idealisation is worth more than the mildest row of the thing it makes measurable.
• The shape matters more than the size. At the same amplitude a third cosine costs 0.33 and a narrow absorption band costs 0.063 — five times apart — and the widest pair of the four shapes spans a factor of nine.
• Nothing about this is hidden. The file that builds the family says the basis is a caricature and says why. What was never measured is what the caricature is worth.

What the theorem is #

Take a set of surfaces spanned by three fixed reflectance functions, so that every member is ρ = a₁b₁ + a₂b₂ + a₃b₃ for some coefficients. Illuminate it with light E, integrate against the observer, and the tristimulus of a member is linear in the coefficients: XYZ = R(E) · a, where R(E) is a 3×3 whose columns are the tristimulus values of the three basis functions under E. Swap the light for E′ and the same surface gives R(E′) · a. Eliminate a and what is left is XYZ′ = R(E′) R(E)⁻¹ · XYZ.

One matrix, the same for every surface in the family, exact. It is the fact the whole adaptation census is built on. Not a least-squares fit with a small residual: an identity, holding to whatever precision the arithmetic carries, which here is fourteen decimal places.

Everything downstream depends on it. The residual the census reports is defined as what an adapted observer’s diagonal gain fails to undo of this matrix, and the gap between the two is meaningful only because the matrix half has no error in it. If the matrix were itself approximate, the census would be reporting the sum of two errors of unknown proportion, and the claim that what is left is not something a better transform could fix would lose its footing — because part of what was left would be the family’s own failure to be a family.

What it rests on #

Exactly one thing: that the surfaces span three dimensions and no more.

That is not a fact about surfaces. It is how the set is built. Three basis functions were written down — a constant, a half-cosine and a full cosine across the visible band — and every member of the set is a combination of them. The construction cannot produce anything else, so the theorem cannot fail on it.

The file is explicit that this is a caricature. Its docstring says the basis is not a measured principal-component basis, that this site has no measured reflectance collection, and that the family is exactly three-dimensional so that the matrix result is a theorem about this family rather than a good approximation on it. Every word of that is true and it is stated in the right place.

What is not stated anywhere — and not on the previous audit’s list of what it could not reach — is the size of the difference between a theorem about this family and a good approximation on real surfaces. That is what the rest of this essay measures.

Why three is nearly right, and why nearly is the problem #

There is a real fact behind the caricature and it is a remarkable one. Measured collections of natural reflectance spectra are very nearly low-dimensional: three basis functions reconstruct most such collections to within measurement error, which is why linear models of surface colour work at all and why an image does not determine the light is a solvable problem rather than a hopeless one.

“To within measurement error” is a statement about variance, and the residual after a matrix is not a variance.

The reconstruction studies report that three components capture ninety-something per cent of the variance in a collection of spectra. That is a statement about how much of the spectrum is left over. What matters here is how much of the left-over spectrum survives integration against three cone fundamentals and then a change of light — and those are two different projections, so a small spectral residual can produce a large or a small colorimetric one depending entirely on its shape.

Which is precisely what the measurement shows.

What a fourth dimension costs #

Each surface in the lattice is given a fourth coefficient at amplitude ε, scaled by the surface’s own brightness so that a dark surface does not acquire a modulation deeper than it is bright. Anything that leaves the physical range is dropped rather than clipped, because a clamp is the non-linearity the construction exists to keep out. At ε = 0 the result is the original lattice, member for member, band for band, checked as an identity.

All in ΔE*₀₀, all on daylight-to-tungsten, all against a quantity that is zero in the model.

For scale, the census’s own numbers: the mildest change of light in it costs 0.263 after a von Kries gain, and the harshest costs 3.375. So a fourth dimension at five per cent — well inside what a reconstruction study would call negligible — puts more into the matrix’s error than the entire measured residual on the mildest row.

The idealisation that makes the theorem a theorem is, in the realistic regime, the same size as the effect the theorem is used to isolate.

The shape matters more than the size #

The table’s four rows span a factor of nine at the same amplitude — 0.572 for a fourth cosine against 0.063 for a narrow band, with the third cosine and the dye edge between them — and that is the finding a variance figure cannot express.

A fourth cosine term oscillates across the whole visible band with a period comparable to the separation between the cone fundamentals’ peaks. That is the worst possible shape: it puts energy exactly where three broad, overlapping sensitivities differ from one another, so the three channels see it differently under one light and differently again under another, and no fixed matrix can carry both.

A narrow absorption band at 550 nm sits under the peak of the medium-wave fundamental and inside the shoulder of the long-wave one, and its effect on the three integrals is very nearly a fixed reweighting — which a matrix absorbs almost perfectly. It costs a fifth of what the cosine does.

A dye edge sits between the two, for the obvious reason: a step has broad spectral content but no oscillation, so part of it looks like a tilt the matrix can carry.

So “how much fourth dimension do real reflectances have” is not answerable by a number. It needs the shape, and the shape is exactly what a variance figure integrates away. A collection whose fourth component is a narrow pigment band is nearly three-dimensional for this purpose; one whose fourth component is a broad ripple is not, at the same variance.

Real pigments produce absorption bands, which is the reassuring direction. Real mixtures of pigments, and real interference and structural colours, produce broader structure, which is not.

The cost is linear where it matters, and the shape sets the exchange rate #

The four columns of the amplitude table are not four independent readings. Taken as growth per doubling — where a strictly linear response would give exactly 2.00 — they read:

All four are linear to within three per cent over the first doubling and saturate thereafter, which is what one expects of a residual that grows with a perturbation and is measured in a metric with a chroma weighting in it. The saturation is a property of ΔE00 rather than of the surfaces.

That linearity is what makes the headline transferable, and it lets the essay’s comparison be stated as a threshold instead of a coincidence. Extrapolating each shape back to the census’s mildest row at 0.263:

So the question is the idealisation worth worrying about has an answer that runs from two per cent to twenty-one, depending entirely on what shape the fourth component has — the same factor of nine, now in the currency somebody with a measured collection could check against.

That is a more usable form of the essay’s finding than the table of costs. A collection whose fourth principal component is a pigment band is safe up to a fifth of the surfaces’ own modulation depth; one whose fourth component is a broad ripple is compromised at a fortieth of it. Nobody needs the variance figure to decide which case they are in — they need the component drawn.

The shape ordering is stable and the row ordering is not #

Two robustness readings, and they point opposite ways.

The four shapes keep their order across a factor of eight in amplitude. The second cosine is worst, the first cosine next, then the dye edge, then the narrow band, at 5 per cent and at 40, with the spread barely moving — 9.08 against 8.52. So the shape ranking is a property of the shapes and not of how much of them there is, which is what makes it worth quoting at all.

The census rows do reorder, and the essay’s assurance that only unordered rows move is checkable on the five it prints. At ε = 0 the order runs triphosphor, tungsten, halogen, display, blackbody; at ε = 20 per cent it runs triphosphor, halogen, tungsten, display, blackbody. Exactly one exchange, between tungsten and halogen — separated by 0.052 at ε = 0 and by 0.010 the other way at twenty per cent.

That is the smallest gap among the five and it is well inside what the paired standard errors on the census’s steps can resolve. So the assurance holds on this subset, with the margin quoted: the one pair that swaps is the one pair the test set was never able to separate.

The response is not proportional to the residual #

One relationship the row table invites and does not have. The five rows’ percentage responses run +90, +34, +29, +3 and −6, and their starting residuals run 0.980, 1.583, 1.635, 0.263 and 2.323 — the largest response belongs to the second smallest starting residual, and the only negative one to the largest.

Read as absolute increases instead, they are +0.885, +0.539, +0.477, +0.009 and −0.146, which puts the three-primary display ahead by nearly two to one and leaves the ordering otherwise similar.

Neither reading makes the response a function of the starting residual. What predicts it is the mechanism the essay names — how narrow the light’s own spectrum is — and the display’s +90 per cent against the blackbody’s +3 is a thirtyfold difference between two rows whose unperturbed residuals differ by less than four. A fourth dimension’s cost is a property of the light, not of how badly the light was already handled, which is a sharper claim than the essay’s and is what its own five rows show.

Which changes of light care #

Adding the fourth dimension does not raise every row by the same factor, and the row that responds most is not one anybody would nominate.

The three-primary display nearly doubles, and the reason is the whole argument in one row: a source made of three narrow lines is exactly the instrument that cannot see a fourth reflectance dimension. Two surfaces that differ only in the fourth component arrive at the eye as the same three numbers under three narrow primaries and as different numbers under daylight, so the mapping between the two lights stops being a function of tristimulus at all — which is the failure of the matrix, in its purest form.

The triphosphor tube falls, and it is the only row that does, for a reason worth a rung of its own.

Why nothing caught it #

Because the theorem is checked and the theorem is true.

colourcheck asserts that a change of light is exactly a 3×3 on this family and reports a residual of 3 × 10⁻¹⁴. That assertion has never failed, cannot fail, and is worth having — it is what catches an arithmetic error in the matrix construction. It is also a statement about a set the same file built, which means the check and the thing being checked share their only assumption.

An assertion that a property holds on the object constructed to have it is a consistency check, not a test. That is not an argument against writing it; it is an argument for writing something else alongside it, and the something else is the table above. The distinction is the same one this site draws between an assertion that has rejected something and one that never has: the exactness check has rejected arithmetic errors and cannot reject the assumption it shares.

What it changes about what has been published #

Not the ordering, and not the arguments. The census’s conclusions are comparative — and its middle was never ordered — and the comparisons survive — the extremes of the table are unmoved at every amplitude tested, and the rows that reorder are the rows that were never ordered.

What it changes is one sentence’s scope. A change of light is exactly a matrix, so what is left after adaptation is not something a better transform could fix is true of this family. Of real surfaces the honest form is: a change of light is a matrix to within the fourth dimension’s contribution, that contribution depends on the shape of the fourth component rather than on its size, and for a smooth fourth component at a few per cent it is comparable to the smaller adaptation residuals themselves.

That is a weaker sentence and it is the one the evidence supports.

Where the model stops #

Nothing here says how much fourth dimension real surfaces carry, and the file is as loud about that as it can be. The four candidate fourth functions are constructions, chosen to span a range of shapes rather than to be right, and none of them is a measured principal component.

What the exercise establishes is a sensitivity rather than a value: the answer moves by this much per unit of fourth dimension, and by this much more depending on its shape. Somebody with a measured collection could turn that into a number, which is the absence the round keeps arriving at, and the number would be theirs and their collection’s. Anybody without one — which is this site — has to say which regime a claim holds in, and now can.

Who found it, and when #

The three-dimensionality of surface reflectance is Cohen’s, from 1964, and the linear-model machinery built on it is Maloney and Wandell’s from the mid-1980s; the recovery arguments in this collection’s scene essays are theirs. Every one of those papers states the dimensionality as an empirical approximation and states its residual.

The slippage happens downstream, and it happens here as it happens everywhere: an approximation adopted for computational convenience becomes a construction, the construction makes something exact, and the exactness is then quoted as though it were a result rather than a definition. The tell is that the residual is machine zero. A number that comes back at 10⁻¹⁴ is not measuring the world; it is measuring the arithmetic, and the question of what it would be if it were measuring the world had never been asked.

What the two zeros prove #

The frequency sweep’s control deserves a section of its own, because it is the only thing separating this measurement from an artefact.

Every number in the dimension tables is a residual against a matrix computed on the unperturbed family. If the perturbation were being applied wrongly — added at the wrong point, scaled by the wrong factor, or applied to a set the matrix was silently refitted to — the curves would still rise, still be smooth, and still look like a measurement.

They would not touch zero. At one and two half-cycles the fourth basis function is exactly the family’s own second and third basis function, so the perturbed set spans the same three dimensions and the matrix stays exact. A construction that had drifted in any of the three ways above would put a small non-zero value there, and the drift would be visible as a curve with no zeros in it.

The zeros come out at 10⁻¹⁶ and are asserted at 10⁻⁹. That is the check this collection’s habit demands: an assertion that has never rejected anything proves nothing, and this one rejects the class of error the measurement is most exposed to.

It also settles a smaller question. The perturbed set has 375 members against the unperturbed 125, so the two means are over sets of different size — and the exact zeros show that the size is not doing anything, since a mean over 375 members of a three-dimensional family is exactly a mean over 125 with three copies of each.

Where the ladder goes next #

The fourth dimension’s shape turns out to span a factor of nine at one amplitude, which makes the shape the thing to characterise — and makes one row of the census, the one that gets better, worth understanding on its own.