The grid is a range, not an index

An audit that finds four things wrong is tidier if all four are the same kind of thing. Three of these are; the fourth is a different animal wearing the same costume, and the difference is what makes it the only one still outstanding.

The claim #

The wavelength grid is not a dropped argument but a truncated one, which makes it cheaper to fix and easier to forget.

• Three departures restore an index: a matrix instead of a diagonal, two directions instead of an average, a kernel instead of a point.
• This one restores nothing. It widens the range of the argument the model already keeps, from 380 nanometres down to where the tables actually start.
• It costs 6.70 ΔE₀₀ on a coated printing paper under D65, and between 4.47 and 6.82 across the brightened stocks in this collection’s table.
• It costs exactly nothing on anything unbrightened, which is why nineteen rounds of essays never met it.
• And it is a table lookup. The CIE’s daylight basis functions are published from 300 nanometres; the colour-matching functions from 360. The grid stops before either.

Where the number 380 came from #

It came from the visible band, which is where a colour-matching function is worth having, and it was fixed in the first weeks of this collection alongside the 5-nanometre step and the 780-nanometre top.

Two of those three have been examined. The step was measured and found to cost a fraction of a unit on smooth spectra and much more on spiky ones. The upper bound was measured for a camera sensor, which sees into the infrared, and the decision there was to build a second grid inside the imaging module rather than widen the shared one.

That decision was made with three arguments, and the third one — that the D-series illuminants are reconstructions from basis functions which stop at 780 and cannot be extended past it — is true upwards and false downwards. The basis functions are published from 300 nanometres, and they are published from 300 for exactly the reason this essay exists: the ultraviolet of daylight is what makes a brightened sheet of paper glow, and a standard that could not state it could not specify a measurement of paper.

So the collection’s own arithmetic has known this from the start. There is a comment beside the daylight reconstruction warning that the tables start at 300 and that indexing them from 380 reads the wrong part of the array. The fact was sitting in the source for eleven rounds underneath an essay saying the opposite.

What the truncation costs #

The band below four hundred nanometres was built as a separate wide grid in a previous round, precisely so the question could be asked without touching the shared one. That makes the measurement direct: compute a sample’s colour on the wide grid under the full lamp, then under the same lamp with everything below 380 removed, and compare.

For the five brightened stocks the answers are 4.47, 6.19, 6.70, 6.82 and 6.47 ΔE₀₀, rising with the loading and then falling slightly at the heaviest, where the emission band has started to cost more than it buys.

For the unbrightened sheet the answer is 0.01, and the residual is not the truncation at all — it is that the observer is small rather than absent at 380 nanometres, so a substrate reflecting there is seen faintly.

That contrast is the whole reason the departure hid. Every reflectance in this collection outside the shortwave module is a construction with no fluorophore in it, so the sample-side factor is zero and the truncation is exactly free. Either factor being zero makes a departure zero, and the collection’s own habits guaranteed one of them was.

Why a range is a different kind of departure #

The three other departures each say the model is the wrong shape: it has one number where the object has a matrix, a function, or a kernel. Fixing one means adding an index, which means adding an instrument.

A range says the model is the right shape and stops too early. Fixing it means a longer array.

That difference propagates through everything about the two kinds. A dropped index cannot be recovered from existing measurements, because the measurement summed over it; a truncated range can, because the data outside it exist and were simply not read. A dropped index changes what a specification has to say; a truncated range changes only what a computation does. And a dropped index has a literature, an instrument and a name, while a truncated range has a comment in a source file.

The last of those is why the range is the departure most likely to survive an audit. There is nobody whose job it is to notice. A laboratory measuring paper is on M1 and its instrument reports from 360 nanometres; a computation reading that data on a 380-nanometre grid silently discards the first four bands, and every number downstream is self-consistent.

What was computed, and how #

The comparison uses the wide grid this collection already has: 300 to 780 nanometres at the same 5-nanometre step, ninety-seven bands, of which the last eighty-one are the site’s own grid sample for sample.

The two grids share a step and an end rather than a step and an origin, which is forced by the direction the extension goes in and has one useful property: moving between them is a slice, so a quantity computed on one is directly comparable with the same quantity on the other, with no resampling anywhere.

The wide illuminants are the CIE’s own reconstruction — the same S₀ S₁ S₂ basis and the same chromaticity-driven coefficients, run over sixteen extra bands rather than extrapolated. That is checked against the CIE’s published D65 in a band this collection had never computed, and agrees to 0.03 of a relative unit, which is the rounding in the published coefficients.

The condition is imposed two ways, and both are limits rather than identities. Scaling the substrate’s reflectance below 380 towards zero drives the residual down by a decade per decade; scaling the lamp’s ultraviolet does the same. Neither reaches zero, because the observer is not identically zero at 380 nanometres — which is a fact about a tabulated function rather than about the physics.

The three ranges this collection has now measured #

A grid has three numbers in it — a bottom, a top and a step — and each has been examined in a different round, by a different method, for a different reason.

The step, at five nanometres, was measured against a lamp with emission lines in it, because a step is a quadrature and a quadrature’s error is decided by how spiky the integrand is. A smooth daylight costs a fraction of a unit; a triphosphor tube costs several.

The top, at 780, was examined for a camera, which sees past it, and the answer there was to build a second grid inside the imaging module rather than widen the shared one. That was the right call: silicon’s response extends far into the infrared and the observer’s does not, so the two need different ranges and nothing is gained by giving them one.

The bottom, at 380, is this essay, and it is the one where the same reasoning gives the opposite answer. Below 380 the observer does not extend and the lamp and the sample do, and the sample can move energy from where the observer cannot see it to where it can. That is what makes the bottom different from the top: the top has no mechanism carrying energy inwards across it, and the bottom has one.

Three numbers, three methods, three answers, and the pattern is that the right range depends on what crosses the boundary. A boundary nothing crosses can be put anywhere convenient. A boundary something crosses is a modelling decision.

The cost belongs to the lamp, and the lamps differ tenfold #

The 6.70 ΔE₀₀ is quoted under D65, and the fraction of a lamp’s power lying below 380 nanometres is not a constant. Computed on the wide grid for the illuminants this collection carries:

Equal energy’s 16.49 per cent is exactly sixteen bands out of ninety-seven, which is the check that the measure is a plain band count with nothing weighted into it, and it gives the flat baseline the others are read against: D65 carries 41 per cent of that, D50 20 and illuminant A 4.

Two things follow that the essay’s single number does not carry.

The number is quoted under the wrong lamp for the industry it is about. The measurement condition the paper trade uses is built on D50, which has half the ultraviolet of D65 — so the truncation on a brightened stock costs less under the light the stock is actually specified under than under the light this collection’s figures default to. The 6.70 is the departure at its convenient maximum among the daylights, not at the value a laboratory would meet.

And under tungsten the departure is nearly gone. Illuminant A puts 0.65 per cent of its power below 380, a tenth of D65’s, which is the arithmetic behind the oldest piece of trade knowledge about brighteners: a brightened sheet under an incandescent lamp is not brightened. The collection’s either-factor-being-zero structure covers it exactly — the lamp-side factor is the one that goes to zero — and it is worth having the number, because it says the grid’s truncation is free for every figure on this site drawn under illuminant A whatever the sample is.

None of that changes the ranking of the four departures, since the same lamp is used throughout the comparison. It changes what the range is worth outside the comparison, which is where a number gets quoted from.

The extension is worth about one band to the eye #

The extension is not for the eye is asserted and can be measured, using the same wide observer the comparison runs on.

Summed over the sixteen new bands from 300 to 375 nanometres, the 1931 functions come to 0.00152 in x̄, 0.0000453 in ȳ and 0.00712 in z̄. The single band at 380 — the bottom of the existing grid, which the collection has always integrated — carries 0.00137, 0.000039 and 0.00645.

So the whole extension adds between 10 and 16 per cent more observer weight than the grid’s own last band already holds. Sixteen new bands are worth about one-tenth of one old one apiece, and the statement that the eye gains nothing from the extension is not a rounding claim but a ratio anybody can check.

That also explains the unbrightened sheet’s residual precisely. Its 0.01 ΔE₀₀ is described as the observer being small rather than absent at 380 nanometres, and the arithmetic says which side of the boundary that weight sits on: it is the 380 band itself, inside the existing grid, not the new ones. A sheet reflecting below 380 is seen faintly through a weight the collection already integrates, and extending the grid adds a tenth again of it.

Which sharpens the essay’s own distinction between a range and an index. A dropped index removes a mechanism; this truncation removes almost no direct sensitivity at all. What it removes is the energy that a fluorophore can move across the boundary into a region the observer weights heavily — and that is why the cost is 0.01 on a sheet without one and 6.70 on a sheet with one, on the same grid, under the same lamp, with the observer’s contribution below 380 unchanged between them.

Where the model stops #

The extension goes to 300 nanometres and stops there, and the stop is a real boundary rather than an arbitrary one.

Below about 360 nanometres the observer is not small, it is undefined: the CIE does not tabulate colour-matching functions there, because the light does not reach the retina in useful amount. So a wide grid below 360 can carry a lamp and a sample and cannot carry an observer, and any colour computed on it is using zeros for a function that has no values.

That asymmetry is worth stating because it is the reason the extension is bounded. The extension is not for the eye — the twenty nanometres between 360 and 380 are worth about a part in ten thousand of a chromaticity — it is for the samples that return light at wavelengths they did not receive. The eye never sees the ultraviolet; it sees what the ultraviolet made.

What the extension would have to decide #

Widening the grid is a table lookup and it is not only a table lookup, because two questions have to be answered before the first extra band can be integrated.

What is the observer below 360 nanometres? Not small — undefined. The CIE does not tabulate colour-matching functions there, and the reason is that the light does not reach the retina in useful amount, which is a statement about the lens and the macular pigment rather than about the receptors. Filling those bands with zeros is a decision, and it is the decision that makes a wide grid usable, but it should be recorded as one.

And what does every existing reflectance do in the new bands? This collection has several hundred constructed reflectances, none of which says anything about 340 nanometres. The wide-grid module already refuses to guess: its conversion from a visible spectrum to a wide one has no default for what happens below 380, on the grounds that this is exactly the quantity worth not assuming.

Those two are the real work, and neither is arithmetic. They are the reason a one-line change is a body of work rather than an afternoon.

The generalisation #

The transferable claim is about audits rather than about colour: a list of departures should be sorted by what kind of thing each one is, not only by size.

Three of these four are modelling choices with alternatives in the literature, and the previous round’s method reaches them: find the published alternative, recompute, report the spread. The fourth is a range, and a range has no alternatives to enumerate — it has an interval, and the question is where its endpoints should be.

Ranges are systematically under-audited for that reason. There is no menu to sweep, no instrument to buy, and nobody to disagree with, so a range gets fixed once by whoever wrote the first version and inherited thereafter. This collection has a step, an upper bound, a lower bound, and a set of surfaces its census is taken over — four ranges, all of them decided early, and three of them now measured.

The practical form of the rule: when auditing a computation, list every interval in it as well as every constant. An interval is a pair of constants that does not look like one.

How much that discarded stretch is worth depends on where the sheet is, and between ordinary places it varies by more than an order of magnitude.

The building is the other supplier of that band, and unlike the sky it is something a specifier chose without ever being asked about ultraviolet at all.

An instrument makes the same choice on a sample’s behalf, and it makes it four different ways under the four standard conditions.

Those four conditions are one way the range shows up: an instrument that lights a sample differently below 400 nanometres reports a different sample. The observer’s own optics are the other way, and they are why the matching functions stop where they do.

The one number that does not move #

Widening the grid changes almost everything by nothing, and it is worth saying which quantities are in that category, because they are most of the collection.

A reflectance with no response below 380 nanometres, under a light with power there, integrated against an observer that is nearly zero: the product of three terms of which one is zero and one is nearly zero. Every constructed surface in this collection is such a reflectance. Every blackbody and every daylight has power below 380 and it does not matter, because there is nothing to reflect it and almost nothing to see it with.

So the extension is free for the collection’s ordinary work and decisive for one corner of it, and that ratio — nothing for most, seven ΔE₀₀ for a few — is what makes it easy to defer. A change that mattered a little everywhere would have been made years ago; a change that matters enormously in one place and nowhere else waits for somebody to be standing in that place.

Who found it, and when #

The CIE published the daylight basis functions from 300 nanometres in 1964, and the reason is recorded in the documents: the paper and textile industries had brighteners and needed a defensible way to specify a measurement.

The colour-matching functions are tabulated from 360, and the twenty-nanometre gap between the two tables is a real and deliberate difference — the light is specified where the sample can respond, and the observer is specified where the eye can. The tables do not stop together, and that is not an oversight.

What this collection did was replace both ranges with one, at the intersection, which is the correct operation only when everything in the integral is being multiplied together. The moment a sample can move energy from one part of the range to another, an intersection is the wrong object.

Where the ladder goes next #

The fix is stated and not done, and the honest reason is that it is a collection-wide change rather than a local one.

Widening the shared grid moves every spectral integral on this site by a small amount — the observer functions, every illuminant, every constructed reflectance, the metamer constructions, and every figure downstream. Most of those move by nothing, because most of the samples do not respond below 380 and the observer is nearly zero there. But nothing has to be measured rather than assumed, and the measurement is a full rebuild with every assertion re-run.

That is a large piece of work and it should be done deliberately. Two things would have to come with it: a decision about what the observer is below 360, where it is undefined rather than small, and a rule about what every existing reflectance does in the new bands, which is a quantity this collection’s own conversion refuses to default precisely because it is the thing worth not assuming.