Two ends and one is empty

The collection’s ledger has named its wavelength range as an outstanding item for two rounds, on the strength of one number measured on one brightened paper. This is the same question asked of both ends at once, over a family of surfaces, under six lights.

The claim #

The two ends of a spectral range are not two instances of one decision. One of them is worth thousands of times the other, and which one is decided by the observer’s shape rather than by where the light’s power is.

• Below 380 nanometres a red pigment under a 6500 K radiator moves by 0.502 ΔE₀₀. Above 780 it moves by 0.00015 — a factor of three thousand three hundred.
• A fifth of that light’s power lies outside the range, and about three per cent of its visual product does. The two shares differ by an order of magnitude, and for a tungsten lamp by two.
• A light with no ultraviolet pays nothing at either end, exactly. The range’s cost is a product of what the range excludes and what the light puts there.
• And the asymmetry is not about the infrared being unimportant. It is about the observer being identically zero on one side of the range and merely small on the other.

The two ends are different questions #

At 780 nanometres the CIE’s colour-matching functions are of order 10⁻⁵ of their peaks and falling exponentially, and every thermal source is still radiating strongly. At 380 the functions are of order 10⁻³ and falling much more slowly, and daylight is still substantial. The products are therefore four orders of magnitude apart before any sample is chosen.

That is the whole of the asymmetry and it is a property of the eye rather than of the spectrum. The visible band is bounded above by the pigments’ absorption edge and below by the ocular media, and an absorption edge and a filter have very different shapes: the edge is abrupt, the filter is a tail. So the observer’s short-wavelength side has a long shoulder on it and its long-wavelength side does not.

This collection stops its integrals at 380 and 780 because the CIE tables did, and the CIE tables did because that was where their contributors’ data ran out. The tables have since been extended to 830, and the extensions do not all stop in the same place, which is a separate and equally awkward fact.

What each end costs, measured #

Six lights, a red pigment, and a comparison between the site’s range and each extension, all at a tenth of a nanometre so that no step error is mixed in.

Every entry in the right-hand column is below any tolerance anybody writes. Four of the six entries in the left-hand column are too. The whole of the range’s cost, for this sample, is two rows — and both of those rows are thermal radiators, which are the only sources in the set with substantial ultraviolet.

Two conclusions follow and they point in opposite directions for different readers. For a collection computing under daylight and blackbodies, the range is worth about half a colour difference and is the largest single tabulation defect it has. For anybody working under LED or fluorescent light, it is worth nothing at all and refining the step matters instead.

There is a third reading of the table that is easy to miss and is the most useful one for anybody choosing what to measure. The two thermal radiators differ from each other by a factor of six at the ultraviolet end — 0.502 against 0.079 — and by a factor of two and a half the other way at the infrared end. A tungsten lamp is hotter in the infrared and colder in the ultraviolet than a daylight radiator, and the range’s cost follows the ultraviolet rather than the total. Colour temperature predicts the range’s cost and radiant power does not, which is the same point the next section makes in a different currency.

Power outside, and colour outside #

The commonest way this argument goes wrong is to reach for the share of the light’s power that lies outside the band, which is a number a radiometer reports and is easy to obtain.

A 6500 K radiator puts 21.4 per cent of its power outside the range and 3.4 per cent of its visual product. Tungsten puts 21.5 per cent of its power out and 0.28 per cent of its product. A white LED puts out 0.011 per cent of its power and 0.0027 per cent of its product.

The gap between the two columns is the observer’s tails doing their job. Most of the power outside the band is in the infrared, where the observer is zero, so it contributes nothing whatever to a colour. Reading the power share as though it were the colour share overstates the range’s importance by between four times and eighty times depending on the source, and it overstates it most for exactly the sources where it matters least.

There is a subtler version of the same mistake, and it is the one a careful person makes. The visual product’s share is still not the colour error, because a truncation removes light from the sample’s integral and from the white’s integral together, and the normaliser cancels most of what both lose. Three and a half per cent of the product goes missing and half a colour difference results, which is a great deal less than three and a half per cent of anything.

The sample matters as much as the light.

The 0.502 above is a red pigment. The same measurement on a white paper is 0.165, and on an interference filter with a notch at 545 nanometres it is 0.0193.

Over the family of forty-two analytic surfaces under the same 6500 K radiator, the range costs a median of 0.542 and a maximum of 2.024, with a minimum of 0.098 — a factor of twenty across surfaces nobody would call exotic. The step, over the same family and under the same light, costs a median of 0.060.

So the ranking between the two decisions is stable across the family under a smooth light — the range wins everywhere — and it reverses completely under a fluorescent tube, where the step’s median is 0.834 and the range’s is 0.00013. There is no answer to “which end of the tabulation should be repaired” that does not name a lamp.

Two samples, ten pairs of bars, and one invariant: the ultraviolet end costs more than the infrared end in every case, by between two and four orders of magnitude. A sample can change how much a range costs by a factor of twenty and cannot change which end it is spent at.

The pairing, for the third time #

The structure here is the one the previous round established for the sample and this round keeps finding for the index. A range costs the product of two things: what lies outside it, and what the light puts there. Either factor being zero empties it.

The three-emitter LED’s row is exactly zero at both ends, and it is exactly zero rather than very small. Its emitters are Gaussians centred at 455, 528 and 625 nanometres with widths of tens of nanometres, so their value at 380 is below the floating-point floor. That is the light’s factor emptied, and it makes the range decision irrelevant no matter how badly the range was chosen.

The other route to zero is the sample’s, and it is the identity of the previous essay: a flat reflectance reads the same on any range, because the truncation removes the same fraction from the sample’s integral and from the white’s. Two independent conditions, each exact, and this collection’s ordinary computations sit near both of them.

The notch filter is the useful third case because it is spectrally the most structured sample in the set and the cheapest to truncate. Structure and range sensitivity are unrelated: what decides a truncation is how much of the sample’s contrast sits near the edges, and a notch at 545 nanometres has none there at all. A sample can be difficult for one tabulation decision and trivial for another, which is the clearest argument for measuring the three separately rather than as one resolution.

What extending it would actually take #

Widening the range is often described as a table lookup and it is not, and the reason is that four different tables would have to be extended together.

The colour-matching functions exist to 830 and stop there, which fixes the upper end. The CIE daylight basis functions run from 300 to 830, which is generous. The blackbody spectra are computed from Planck’s law and are available at any wavelength. And the reflectances in this collection are formulae, so they too are available anywhere — but the formulae were fitted or chosen against the visible band and have no claim to be correct outside it.

The last of those is the real obstacle and it is a modelling obstacle rather than a data one. A reflectance extrapolated below 380 nanometres is a guess, and the guess matters: the difference between assuming a paper’s reflectance holds constant below the band and assuming it falls to zero is worth more than the whole extension. So widening the range trades a known truncation for an unknown extrapolation, which is a trade rather than a repair.

For samples that genuinely respond below 380 the situation is worse still, because the response is not a reflectance at all: an optical brightener absorbs there and emits in the blue, and no widening of a diagonal can represent that.

What this collection has already spent on the same question #

Some of this ground is held, and it is worth being precise about which parts, because the ledger’s outstanding item is narrower than it reads.

The essay that named the grid as a range rather than an index measured one number: 6.70 ΔE₀₀ on a coated brightened paper under D65, integrated from 300 and from 380. That number is much larger than anything here, and the reason is the brightener rather than the range — it is a fluorescence measurement wearing a truncation’s clothes, and it belongs to the wavelength index rather than to the range.

The wide grid itself already exists in this collection’s machinery, running from 300 nanometres with the daylight basis extended to match, and it is used by the fluorescence family. So the extension is not unbuilt. What is unbuilt is the decision to make it the default, and the measurement above says what that decision would buy on ordinary samples: about half a colour difference under daylight, nothing under LED lighting, and an extrapolation problem on every reflectance in the collection.

One further consequence follows for anybody building a spectral pipeline rather than auditing one. A range is the cheapest of the three tabulation decisions to change and the most expensive to change correctly, and those two facts are usually confused. Adding sixteen rows to the bottom of a table is an afternoon; deciding what belongs in them is the work, and it is work on every sample rather than once. The step and the origin have no equivalent — refining a step needs no new physical claim about anything, and moving an origin needs none either.

That asymmetry is why the range has stayed outstanding here for two rounds while the other two were never even named. It is not that nobody noticed; it is that the repair is a modelling commitment and the other two are arithmetic.

What was computed, and how #

Both ends are measured at a tenth of a nanometre so that no quadrature error is folded in. The first version of this measurement used the site’s own five-nanometre step for both sides and reported numbers about a tenth larger, because the step error over 300–780 is not the step error over 380–780 and the difference showed up in the range column.

The share of the power and the share of the visual product are integrated over 300 to 830 at the same tenth of a nanometre, with the visual product taken as the sum of the three observer functions rather than as luminance alone — luminance would have understated the ultraviolet end by a further factor of ten, since ȳ falls faster there than x̄ and z̄ do.

The assertion this family carries is that the ultraviolet end costs at least as much as the infrared end for every light in the set, which is a claim about the observer’s asymmetry rather than about any particular source, and which a source with strong infrared structure and no ultraviolet could falsify.

Where the model stops #

The observer here is this collection’s analytic construction, and its tails are the pigment template’s tails through the ocular media. The template’s long-wavelength tail is the part an earlier round found to be most of what a template is, and the ultraviolet end is where the ocular media are doing nearly all the work. Both tails are model rather than measurement, and the factor of three thousand is a property of the model’s shape.

The reflectances are extended below 380 by their own formulae, which is the optimistic assumption. A real pigment’s behaviour in the near ultraviolet is often quite different from the continuation of its visible curve, and every number in the left-hand column would move if it were.

And no sample here fluoresces. The one case where the ultraviolet is genuinely load-bearing is excluded from the measurement by construction, and it is excluded because it is not a range problem.

That pair of figures is worth reading side by side, because between them they show which of the two shares is a measurement of anything. The power share is identical on both samples, to every digit printed, since a reflectance cancels out of a ratio of integrals of the light alone. The product share changes with the sample by a factor of twelve. Only one of those two numbers is about the calculation anybody is doing.

The generalisation #

The habit is about how to compare two shares of the same thing.

A quantity that is an integral of a product has as many shares as it has factors, and they can differ by orders of magnitude. Asking what fraction of the power lies outside a band is a well-posed question with an easily obtained answer, and it is not the question anybody wanted. The question they wanted is what fraction of the answer lies outside, and the difference between those two is the shape of the other factors.

The failure mode is to use the available share because it is available. It is nearly always the wrong one, and it is nearly always wrong in the direction that makes the neglected region look more important than it is — because the neglected region is neglected precisely where the other factors are small.

Who found it, and when #

The CIE’s original 1931 tables ran from 380 to 780 at five nanometres, and the extension to 360–830 came with the 1986 revision, which is when the shoulder below 380 became available to anybody who wanted it. Very little practice took it up, and standards for industrial colour measurement generally specify 360 to 780 or 380 to 730 for reasons of instrument range rather than of the observer.

The asymmetry between the two ends is not, so far as this collection can tell, stated anywhere as a number. It is implicit in every table — anybody can see that z̄ has a shoulder and that all three functions die abruptly at the top — and the consequence for a truncation appears to have been left as an exercise.

Where the ladder goes next #

Two decisions are measured and their ranking is known to depend on the lamp. The last thing the grid section owes is a way of choosing between them that does not require reading four tables, and that turns out to be a single length compared against a single other length.