The tables do not stop together

A grid is the least interesting decision in a spectral calculation and the hardest one to revisit. Every array in a library is that long; every integral assumes it; every figure plots it. This collection has used 380 to 780 nanometres at five-nanometre steps since its first commit, and the range has been defended twice — once for its step and once for its long edge.

The second of those defences contains a sentence that is false, and finding out took the work below.

The claim #

A shared wavelength grid is the intersection of several published ranges, and an intersection is only the right choice when every function in it is being multiplied by every other. The moment one of them is an operator rather than a multiplier, the intersection throws away the part that matters.

• The six tables stop in six different places, and until now this collection had replaced all six with one number.
• The CIE daylight basis functions are published from 300 nanometres, not from 380. The essay that declined to widen the grid said no defensible extension of D65 past the visible band existed; that is true going up and false going down.
• The extension is checkable rather than assumable. Reconstructing D65 from the basis below 380 and comparing against the CIE’s own tabulated D65 gives a worst disagreement of 0.021 relative units over seventeen bands, which is the rounding in the published coefficients.
• For an eye it buys nothing measurable: extending the integral moves the chromaticity of daylight by 2.4 × 10⁻⁵, two orders of magnitude below anything a figure here could show.
• For a sheet of paper with a brightener in it, it changes the answer by a factor of three. Every fluorescence number this collection has published was a floor, as it said, and the floor sits between 2.1 and 3.2 times below the value.

Six ranges, not one #

The habit that produced the single grid is worth naming, because it is not carelessness. A spectral calculation of an object colour is

XYZ = ∫ ρ(λ) E(λ) x̄ȳz̄(λ) dλ

and the three factors are multiplied. A product is zero wherever any factor is zero, so integrating over the intersection of their supports is not an approximation at all — it is exact, and computing the terms outside it is arithmetic on zeros. That is why nobody has to think about the range: for the calculation the collection does most often, the intersection is the answer.

The six ranges are these. The 1931 and 1964 colour-matching functions are tabulated from 360 to 830 nanometres. The daylight basis functions S₀, S₁ and S₂ run from 300 to 830. Planck’s law has no range, being a formula. Silicon responds from about 190 nanometres to 1107, where its band gap stops it. And this collection chose 380 to 780, which is the intersection of the first and — near enough — the last thing an eye looking at a reflector needs.

The interesting number in that list is 360. Not because the twenty nanometres matter — they do not — but because of what is true of the band below it. The CIE does not tabulate the observer under 360 nanometres. That is not a tabulated zero; it is an absence, and the difference is the whole subject of the essays around this one. A tabulated zero says the eye does not respond. An absence says nobody measured, and the reason nobody measured is that the light does not get to the retina — the lens absorbs it, which is a fact about the eye’s optics rather than about its photochemistry.

Meanwhile the light in that band is not absent at all.

What the earlier essay got right, and the sentence it got wrong #

The essay that declined to widen the grid gave three reasons and was right about two of them.

It touches everything — true, and the reason this extension lives in a library of its own rather than in the spectral core, exactly as the infrared extension does. It buys nothing for most of the collection — true, and now measured rather than asserted: 2.4 × 10⁻⁵ of chromaticity.

The third reason was that it could not be done honestly:

The CIE D-series daylight illuminants are not measurements of daylight; they are reconstructions from three basis functions, and those basis functions are tabulated to 780 nanometres and stop. There is no defensible extension of D65 past the visible band.

Every clause of that is true of the long end. Past 780 the basis functions genuinely stop at 830 and there is nothing beyond; extending D65 into the infrared would mean inventing a tail and printing it with the authority of a measurement, which is why the sensor’s grid is a second grid rather than a longer one.

Going the other way there is nothing to invent. The tables run to 300. They have run to 300 since the D-series was standardised, and they run there for exactly the reason these essays exist: a standard that could not state the ultraviolet content of daylight could not specify a measurement of paper, and paper is the most-measured surface in the world — the substrate every printed colour is reported against.

The fact was already in this collection’s own machinery. The daylight reconstruction carries a warning next to it saying that the CIE publishes these tables from 300 nanometres and that indexing them from 380 reads the wrong part of the array — a note left by whoever got the indexing wrong the first time, and every word of it correct. It has sat underneath an essay asserting the opposite ever since.

The extension, and how it was checked #

The three basis functions are published at ten-nanometre intervals and this collection’s visible copy of them is that table linearly interpolated to five. So the extension is the same table interpolated the same way, sixteen bands further down: nothing new in kind, and nothing chosen.

That leaves the question of whether it is right, and the answer is available from an entirely separate document. The CIE publishes the reconstruction machinery — the basis functions and the coefficient formulae — and it separately publishes the resulting D65 spectral power distribution as a table. Running the machinery below 380 and comparing against that table is a check with no free parameters in it.

The worst disagreement over the seventeen bands from 300 to 380 nanometres is 0.021 relative units, at 370, on values of order 50. It is nearly constant across the band, which is the signature of rounding in the published M₁ and M₂ coefficients rather than of an error in the tables — and the assertion that carries it reports the number rather than a pass.

That is the strongest form this collection’s habit takes. A quantity nobody had computed here was computed two ways and required to agree.

What it buys, and for whom #

For the eye: nothing. Extending the integral to 300 nanometres moves the chromaticity of D65 by 2.4 × 10⁻⁵, which is about a thousandth of a MacAdam ellipse’s smallest semi-axis. It is not zero, and the difference between “not zero” and “zero” is the same difference as before: the twenty nanometres between 360 and 380 are tabulated and small, and everything below 360 is untabulated and therefore contributes exactly nothing to an integral that has no values to put there.

For anything fluorescent: everything. A brightener absorbs where the observer is zero and re-emits where it is not, so the excitation band lies almost entirely outside the range the eye needs. Of the light a brightener takes out of D65, 37.8 per cent arrives below 360 nanometres, in the band where neither standard observer exists at all.

What the floor was a floor by #

This collection has carried a caveat about exactly this since its first year. The fluorescence machinery says that the excitation band runs off the short end of the grid, that only its tail is visible to the calculation, that the numbers are therefore floors rather than estimates, and that widening the range would raise them.

A caveat of that shape has two properties worth having: the sign has to be right, and the size has to be small enough that a reader who takes the floor for the value is not badly wrong. The first is what the caveat claimed and nobody had checked. The second was not addressed at all.

The sign is right. Every stock’s emission on the wide grid exceeds its emission on the truncated one, which had to be true — a truncated excitation cannot add light — and is now asserted rather than reasoned.

The size is not small.

The lightly brightened sheet’s emission is understated by 3.20×, the heavily brightened one’s by 2.29, and the laundered shirt’s by 2.10. As colour differences the two grids are between 4.5 and 6.8 ΔE00 apart, which is several times any tolerance a printer would accept. As CIE whiteness the gap runs to 36.9 points on a scale where an unbrightened sheet of the same base measures 82.

The ordering is worth a sentence because it is the opposite of the obvious one. The least brightened sheet is understated by the largest factor. A brightener’s absorption saturates — Beer’s law puts the second molecule behind the first — so a lightly loaded sheet is taking a thin bite out of the whole excitation band and loses proportionally more when the band is cut, while a heavily loaded one has already absorbed nearly everything in the part of the band that survived the truncation.

Why a caveat was the wrong instrument here #

The earlier essay set out the test explicitly, and it is a good test: a caveat works when the omission has a known sign and a bounded magnitude, and fails when it does not. It ruled that the fluorescence case passed and the infrared case failed.

On the evidence the fluorescence case fails too, and in a way the test as stated does not catch. The sign is known. The magnitude is bounded — by the total excitation available, which is finite. What is missing is the third condition nobody wrote down: the bound has to be tight enough to be useful. “Between one and three times the true value” is a bound. It is not a number anybody can put in a specification.

So the general form of the rule wants a third clause. A caveat replaces a measurement when the omission has a known sign, a bounded magnitude, and a bound within the tolerance of whatever the number is for. A floor a factor of three below the value is a bound that is technically honest and practically a placeholder, and the honest thing to do with one is to go and compute it.

What was computed, and how #

The wide grid is 300 to 780 nanometres at the same five-nanometre step: ninety-seven bands, of which the last eighty-one are this collection’s own grid sample for sample. The two share a step and an end rather than a step and an origin, which is the opposite of the arrangement the infrared grid uses and is forced by which direction the extension runs. What matters is the same either way — moving between them is a slice, so a quantity computed on one is comparable with the same quantity computed on the other without resampling.

Three rules govern what may live there, and they are the earlier essay’s rules with one sign changed.

Nothing tabulated is extrapolated. The daylight basis is used where it is published and nowhere else. Illuminant A is Planck’s law, which extends exactly because it is a formula. The observer carries its four published short-wave rows and zeros below 360, and the zeros are marked in the code as untabulated rather than as measured.

The projections between the grids refuse the wrong argument. The narrowing function takes a ninety-seven-band spectrum and returns eighty-one; handed an eighty-one-band one it throws rather than truncating something already truncated. The widening function has no default at all for what a spectrum does below 380 — that is the quantity this whole library exists to stop assuming, so a caller has to say.

And the wide grid does not replace the narrow one. Every figure elsewhere in this collection is still computed on 380 to 780, still correctly, and every number in every earlier essay stands. What has changed is that the fluorescence numbers now have a companion computed on the wider range and the gap between them is published rather than gestured at.

A fourth filter sits inside the observer, and its own table stops at 360 nanometres for a reason the other tables do not share.

Where the model stops #

The two-band fluorophore is a caricature and always was. A real brightener has vibrational structure in both bands and its emission depends slightly on where in the excitation band a photon was absorbed. The model here obeys Kasha’s rule exactly — everything absorbed comes back in one emission shape — which makes the operator rank one and makes the arithmetic tractable. Real Donaldson matrices are measured, and measuring one is a specialist instrument’s whole purpose.

The stocks are constructed, not measured. A paper base is a level with an exponential rise in absorption towards the blue; the brightener loadings are chosen to span the commercial range. The shapes of every conclusion here survive that; the exact factor of 3.2 does not, and is a property of a stated excitation band centred at 358 nanometres.

And the extension does not reach the whole excitation band either. A commercial brightener absorbs meaningfully to about 320 nanometres and the daylight basis begins at 300, so this grid contains the band and the previous one did not — but the same argument that condemned the old floor applies to any future one, and the honest statement is now a number with a stated support rather than a bound with none.

Who found it, and when #

The D-series was standardised in 1964 and its basis functions were published from 300 nanometres in the same document. Judd, MacAdam and Wyszecki derived them from 622 measured daylight spectra, and the reason the ultraviolet is in the reconstruction is that the measurements had it.

Donaldson published bispectral measurements of fluorescent materials in 1954, five years before the CIE had a way to specify an illuminant’s ultraviolet at all. Optical brightening agents were commercialised through the 1940s and 1950s, and the reason ISO 13655 has four measurement conditions rather than one is that by the 1990s essentially every sheet of white paper sold had one in it.

What is new here is only the arithmetic in one direction: the size of the error a truncated grid introduces, computed on a stated sample set, against a floor this collection had declared and never quantified.

The generalisation #

The failure has a shape that has nothing to do with colour.

A body of work adopts an assumption because it is exactly right for its central case. The assumption is examined, defended in writing, and the defence is good. Then the work meets a case where the assumption is wrong, notices, writes a caveat, and the caveat is also good — it states the direction of the error and it is honest about it.

And then nothing happens for years, because a caveat is a terminal state. There is no mechanism by which “this number is a lower bound” becomes “this number is 2.6 times too small”, since the second requires doing the work the caveat exists to avoid. A caveat is not a note to self; it reads as a completed action.

The thing that broke it here was not diligence. It was reading the earlier essay’s third reason carefully enough to notice that it is a claim about a table, and that the table was three metres away.

Where the ladder goes next #

If the range was hiding something, the next question is what it was hiding, and the answer is not a number but a category. A fluorescent sample is not a reflectance — it is an operator on the spectrum, of which a reflectance is the diagonal — and everything downstream of that, from what an instrument reports to what a proof can promise, follows from the difference.

The other direction is the one the glazing figure opens. If the ultraviolet a sheet receives depends on the window it is behind, then where the sheet is is part of its colour in a way no viewing condition on this site has had to model, and the standard measurement conditions are a laboratory’s attempt to name one place and stand by it.