The census is a construction too

Five of the fourteen rows this collection scores adaptation models against are not measurements of anything.

The claim #

The census is half a list of standard illuminants and half a set of inventions, and perturbing the inventions moves the numbers a great deal and the ranking not at all — except in one place, where the table’s own gap was never large enough to be a result.

• Nine of the fourteen rows are what they are. D50 is D50; illuminant A is a Planckian radiator at 2856 K; a blackbody at 6504 K is defined by its temperature.
• Five are constructed: a green wall as a Gaussian band at 540 nm of width 60 and depth 0.6 on a base of 0.25, the same wall applied twice, a red wall a hundred nanometres along, the macular pigment at a density of 0.35, and a lens at twenty against a lens at seventy.
• Moving those constants by defensible amounts changes the mean residual by up to 39 per cent and never changes which transform is best on the average.
• Two of fourteen perturbations reorder the table below the top, and both swap the same pair — two transforms the census separates by six parts in a thousand.
• The perturbations are in the units the constants live in, and getting that wrong is what made the first version of this measurement produce a dramatic and worthless result.

What the census is for #

The census exists to score adaptation models against something more varied than the two daylights the standards are written in. It sorts its rows by where the change of light came from — the daylight the eye was built under, something hot enough to glow, a gas or a phosphor, the room rather than the lamp, and a filter inside the observer.

That taxonomy is the census’s own finding: the residual a gain leaves behind depends much more on the kind of change than on its size, and the world’s own light commutes while a room’s does not.

The constructed rows are constructed because there is nothing to quote. There is no standard green wall. The illuminant standards define lamps and daylight phases and say nothing about what a room does to them, so a row about a painted room has to be invented — and inventing one is the right thing to do, since the surface rows are where the interesting failures live.

There is a second reason a constructed row belongs in a census, and it is not merely that nothing can be quoted. A constructed spectrum can be varied, which is what makes the whole family behind it searchable — and searching that family is where the worst change of light a room can produce came from. A census of measured spectra is a list; a census with a construction in it is a list with a handle on it, and this essay is that handle turned in the other direction.

What the perturbation has to be #

The measurement is simple and its one difficulty is a trap this essay fell into first.

The obvious way to perturb a set of constants is by a uniform percentage. Twenty per cent of a centre wavelength is 108 nanometres, which does not move a green wall — it repaints it violet, and at thirty per cent it puts the band off the end of the visible spectrum altogether. The first version did exactly that, and produced a headline result: the winner changes.

It was a result about a wall nobody could paint. Corrected to moves that are plausible for each quantity in its own units — twenty nanometres on a centre wavelength, fifteen on a band width, 0.15 on a depth, 0.1 on a base, 0.1 of optical density on the macular pigment, eight years on each end of the lens range — the winner does not change under any of them.

That is worth recording as a methodological finding rather than a mistake corrected quietly. A uniform relative perturbation across parameters with different units is not a sensitivity analysis; it is a sensitivity analysis of the units. It produces the largest effect wherever the units happen to make a percentage large, which here was the one parameter measured in hundreds.

The corrected perturbations are worth listing with their reasons attached, because each is a small judgement and a reader may disagree with any of them. Twenty nanometres on the wall’s centre is a different green rather than a different colour. Fifteen on its width is a broader or narrower pigment band, well inside the range organic and inorganic colourants span. A depth of 0.15 is a stronger or weaker tint of the same paint; a base of 0.1 is a lighter or darker wall. A macular density of 0.1 is roughly the spread between two foveas. And eight years on each end of the lens range is a study that recruited a little wider.

None of those is a measurement and all of them are arguable, which is the point: they are declared, they are in the units of the thing they move, and a reader who prefers different ones can see immediately which conclusions would change.

What moves and what does not #

Under the corrected perturbations the best transform’s own mean residual spans a factor of 1.39, so the census’s headline number is soft by two fifths in either direction depending on what somebody paints their wall.

The ranking is not soft. Bradford holds first place in all fourteen perturbations, at a gap of 1.26 to the second — a quarter clear, which is a margin no plausible repainting closes.

The asymmetry between those two facts has a mechanism worth naming. A perturbation moves the constructed rows and five of fourteen is a minority, so the mean moves by a fraction of what those rows move by. But every transform is scored on the same fourteen rows, so a change to one row moves every transform’s mean in the same direction — and a common movement cancels from a comparison. The census’s redundancy protects its ranking and not its numbers.

Two perturbations reorder the table below the top, and both do the same thing: they swap CAT16 and CAT02, which the census as built separates by six parts in a thousand. Moving the wall’s band twenty nanometres to the red does it, and lightening the wall by 0.1 of base does it.

So the honest summary is three-layered. The number is soft, the winner is not, and the pair whose gap was under a per cent was never ordered. A table that prints all five to four significant figures says none of that.

The audit measures a floor and a ceiling and not the threshold between them #

The three-layer summary — the number is soft, the winner is not, the pair under a per cent was never ordered — leaves a gap in the middle that is worth sizing, because it is the gap a reader will want to put their own adjacent pair into.

Bradford’s lead is 26 per cent and it survives every perturbation. The CAT16-against-CAT02 gap is 0.6 per cent and two perturbations close it. Between those two figures is a factor of 43, and nothing in the fourteen runs says where in it the boundary falls.

So the audit establishes that a gap of a quarter is safe and a gap of six parts in a thousand is not, and a reader with a pair separated by five per cent — which is an ordinary size for two published transforms — has been told nothing about it. The essay’s rule, any adjacent pair separated by less than that is not ordered, needs a that, and the measurement as run does not supply one.

The repair is in the same fourteen runs and costs no further computation. What decides whether a pair reorders is not how much the mean moves but how much the two transforms move differently — a common movement cancels, which is the essay’s own mechanism for why the ranking is protected. Reporting the largest differential movement of each pair across the fourteen perturbations would give the threshold directly, and it is one subtraction per pair per run against numbers already computed.

Without it the audit’s headline figure is the wrong one to compare a gap against. The mean moves by 16.3 per cent about its centre, and that number does not bound anything: it is a common movement, and common movements are precisely what a ranking is immune to. Quoting 16 per cent beside a gap invites exactly the comparison the essay’s own argument forbids.

What the mean’s softness implies about the constructed rows #

The 1.39 span is a fact about the census’s mean and it can be read backwards to say something about the rows underneath it, which is more alarming than the headline.

A span of 1.39 is a movement of ±16.3 per cent about the centre. Only five of the fourteen rows move at all. If those five contribute roughly their share of the mean, then to shift the whole by a sixth they must themselves be shifting by ±46 per cent — nearly half, under perturbations each of which is a small and defensible change in one constant.

Individual constructed rows are therefore about three times as soft as the headline suggests, and the headline’s softness is the average of five very soft rows with nine rigid ones. That matters for anybody quoting a single row rather than the mean — and the collection does quote single rows constantly, since the wall rows and the ocular rows are the subject of essays of their own.

The estimate rests on the constructed rows contributing their numerical share of the mean, which is not checked here and which the census’s own figures could settle in one line. If those five rows are larger than average — and the wall rows are among the largest in the census — the amplification is smaller than 46 per cent; if smaller, larger.

Five of fourteen is not half #

A small correction to the claim section, which describes the census as half a list of standard illuminants and half a set of inventions. It is five constructed rows against nine defined ones — 36 per cent against 64.

The difference matters slightly more than a rounding, because the redundancy argument the essay depends on runs off exactly this ratio: nine unmoving rows against five moving ones is what damps the mean by a factor of 2.8, and seven against seven would damp it by only 2.0. The census is better protected than “half and half” implies, and the protection is the reason the ranking survives.

The pair that was never a result #

The CAT16-against-CAT02 row deserves its own section, because it is the useful half.

Their mean residuals differ by six parts in a thousand. That is smaller than the effect of a plausible change in a wall’s colour, smaller than the effect of lightening the wall, and — as the other audit this round makes clear — smaller than most things that could reasonably be different about the model.

So no essay here should say that one of them is better than the other on this census, and none does. That was luck rather than judgement: the sentence would have been easy to write, it would have been true of the numbers as printed, and it would have been a statement about a wall.

The comparison that is available between those two transforms is a different one entirely, and it is structural. CAT02 was withdrawn from practice because its gains go negative under some illuminants; CAT16 exists because of that. Under the worst wall a search can find, Bradford’s middle row passes through zero and CAT16’s does not — which is a fact about the matrices rather than about a census, and does not move when a wall is repainted.

What was computed, and how #

Each perturbation rebuilds the five constructed rows with one constant moved and leaves the nine defined rows exactly as they are. The five transforms are then scored over all fourteen and re-sorted.

Holding the defined rows is what makes the measurement mean what it says. If a perturbation also moved D50 or illuminant A, the result would be a sensitivity of the ranking to the census rather than to the part of the census somebody made up, and those are different questions with different answers.

The perturbations are applied one at a time and then all at once in both directions, which is fourteen cases and not a random sample. With six knobs the interesting question is which sign pattern is worst, and a sample of random draws would report the middle of that distribution rather than its edge — the same reason a sampled maximum understates an extremum, arriving in a smaller room.

Where the model stops #

The five constructed rows are still five rows. Perturbing their constants asks how much the answer depends on the numbers inside a chosen construction, not on the construction itself. A census with a different kind of surface row — a metallic paint, a translucent curtain, a wall with two pigments in it — is a different census and this says nothing about it.

Nothing here perturbs the reflectance family the residual is averaged over. Every row of the census is scored by how far apart a set of surfaces looks before and after the change, and that set is this collection’s own linear family. It is held fixed throughout, which makes the comparison clean and leaves an obvious second audit undone.

And the ocular rows are the strangest members of the set. A change from the fovea to ten degrees out, or from a lens at twenty to one at seventy, is not a change of illumination at all: it is a filter inside the observer, and whether an observer adapts to their own macular pigment is a question the census does not ask and cannot answer. Their constants are perturbed here along with the rest, and their inclusion in an adaptation census is a modelling decision with more in it than a number.

The generalisation #

The rule is short and it is the phase’s, applied to a table rather than to a model.

A ranking’s meaning is bounded by its gaps, and the gaps have to be compared with something. Here the something is the census’s own construction: how much does the mean move when the invented parts of the test set move plausibly? Any adjacent pair separated by less than that is not ordered, whatever the decimals say.

The second half is about units, and it is the part that cost this essay a rewrite. A perturbation has to be plausible for the quantity being perturbed. A percentage is not a unit; it is a unit divided by a value, and applying one uniformly across a centre wavelength, a width, a fraction and an age is applying four different-sized changes chosen by an accident of scale.

And the third half is a small piece of practical advice. When a perturbation produces a dramatic result, check what it did to the object being perturbed before writing it down. Drawing the perturbed wall — which is what the figure of three searched reflectances in the neighbouring essay does — would have caught the first version in ten seconds, because a violet wall is not a green wall moved a little.

What a robust census would look like #

The audit suggests, without quite establishing, what a census designed to be robust would have in it — and it is not more rows.

Redundancy in the number of rows protects the numbers and not the ordering, because a common movement cancels. What protects an ordering is diversity in kind: rows that stress the transforms differently, so that a perturbation which flatters one transform on one row penalises it on another.

The census already has that structure by accident of its taxonomy. Its five kinds ask genuinely different questions — a smooth change of daylight, a spiky discharge spectrum, a multiplication by a surface, a filter inside the eye — and a transform good at one is not automatically good at another. That is why Bradford’s quarter-length lead survives a two-fifths movement in the mean: the perturbations move the surface rows, and Bradford’s lead is built on the illuminant rows as well.

A census of fourteen rows all of one kind would have a soft ranking and a stable mean, which is exactly the wrong way round, and is what a set of corresponding-colour data collected in one laboratory under two lamps is at risk of being.

Who found it, and when #

Testing an adaptation transform against a set of illuminant changes is the standard practice and predates every model in the table; the sets used are usually corresponding-colour datasets — real judgements by real observers under two lights — rather than constructed spectra.

That difference is the interesting one and it cuts both ways. A corresponding-colour dataset is a measurement and cannot be perturbed like this: its constants are people. A constructed census can be perturbed and audited exactly as it is here, and cannot be checked against anybody’s actual perception. This collection has the second kind and says so, and the absence of the first kind is a shortfall it has recorded for four rounds.

Where the ladder goes next #

This audit holds the reflectance family fixed and moves the illuminants. The obvious complement is to hold the illuminants and move the surfaces, which is a larger job because the family is not five constants but a basis with a hundred and twenty members in it.

The more immediate question is what happens when the thing being scored is not a transform at all but a device — where the constants are not invented, the tolerances are manufactured, and the requirement that binds turns out not to be the one anybody nominates.