The index is a choice too

The previous round of this collection opened the first factor of its own arithmetic and found a projection where it had assumed an object. This round opens the index the arithmetic is summed over, which turns out to hold three decisions rather than one.

The claim #

Eighty-one numbers from 380 to 780 in steps of five is a tabulation rather than a measurement, it holds three separable choices, and only one of the three is the sampling error everybody calls it.

• The step is a quadrature rule. It decides how well a sum approximates an integral, and on a smooth light it is worth hundredths of a colour difference.
• The range is a truncation. It decides what is left out entirely, and on the same smooth light it is worth ten times as much as the step.
• The rule for the values between the points is an interpolation, and it exists only because the step is finite. Two of the three rules in common use leave the answer further from the truth than not interpolating at all.
• The three are not versions of one another. They scale differently with the step, they respond to different properties of the light, and a repair to one does nothing for the others.

What is being summed, and over what #

A tristimulus value in this collection is one line of arithmetic:

X = k Σ R(λ) S(λ) x̄(λ) Δλ

with the sum running over λ = 380, 385, … 780. A spectrum is not a colour until it meets an observer, and this is the meeting written down. The audit two rounds ago took the first term apart and found six arguments where the model keeps one. The audit of this round takes the index apart.

The index looks like bookkeeping and is not. It carries a lower limit, an upper limit and a spacing, and every one of the three was decided by a committee that had to publish tables on paper. Eighty-one rows fit on a page; eight hundred and one do not. Nothing about an eye, a lamp or a pigment prefers five nanometres to four or to six.

What makes the three decisions worth separating is that they are not the same kind of decision. Coarsening the step leaves the same integral badly approximated. Narrowing the range leaves a different integral, exactly computed. Interpolating between points invents values that were never measured. Those are three failures with three mechanisms, and any account that calls all of them “the resolution” has already lost the ability to say which one to fix.

Measuring a grid needs something finer than a grid, and that obstacle stands in front of all three measurements.

A five-nanometre error cannot be measured with five-nanometre data. Handing a tabulated observer to an interpolator and integrating the result more finely measures what the interpolator does; the true curve never enters the calculation at all. The answer that comes back is a property of the fill-in rule, and it will happily report that five nanometres was perfect, because the fill-in rule was built from the five-nanometre table and agrees with it at every one of its points.

So everything summed in this round is analytic in the wavelength. The observer is built from this collection’s own pigment template through its own ocular media, both of which are closed-form functions of λ and can be asked about 412.7 nanometres as readily as about 415. Every test light is a formula: a Planck radiator, a sum of Gaussian emitters, a phosphor bed with mercury lines on it. Every sample is a formula too.

The price is stated rather than hidden. This is an observer of the right shape and not the CIE’s tabulated one, so the third decimal place of every number here belongs to the construction. What does not belong to it is the structure — which lights are safe, how the errors scale, what cancels against what — and structure is what the round is about.

The step, and the tell in its scaling #

Coarsening the step from one nanometre to twenty moves a red pigment under a 6500 K radiator by 0.012 to 0.221 ΔE₀₀ — a factor of eighteen for a factor of twenty in the step, which is very nearly linear. At this collection’s own five nanometres it is 0.064.

Under a tungsten lamp it is smaller still, 0.013, because a tungsten spectrum is smoother than daylight in the region the observer cares about. Under a white LED with one phosphor it is 0.0001, a part in ten thousand of a just-noticeable difference. Three of the six lights in the set are, at five nanometres, exactly as well tabulated as anybody could want.

The other three are not, and they fail in a way that has nothing to do with how many points are used. A fluorescent tube costs 0.98 ΔE₀₀ at five nanometres, 0.66 at two, and 1.45 at ten and at twenty — a sequence that is not monotone and does not converge. A three-laser projector costs 37 units at five nanometres, 43 at two and 46 at twenty. These are not approximations that improve with effort. They are answers to a different question, and the collection has an essay saying which sources make a grid expensive without saying why the sequence refuses to settle. It is the same asymmetry a camera meets at the other end of the band, where the grid stops before the sensor does.

The reason is in the next section, and it is not the step.

Return to the smooth column and read the sequence again: 0.0025, 0.0052, 0.0130, 0.0247, 0.0417 for steps of 1, 2, 5, 10 and 20 nanometres. Doubling the step doubles the error.

That is not what a sampling error does. A sum of a smooth function sampled at spacing h has an error that falls as h² when the endpoints are handled properly, and the deviation from h² is a signal that something else is in the number. What is in the number is the end cells: the integrand is not zero at 380 or at 780, and a rectangle sum carries a whole cell where the integral wants half of one, at each end.

That term is first order in the step and proportional to the value at the endpoint, which is exactly the linear behaviour observed. So the quantity universally described as the cost of five-nanometre sampling is, on a smooth light, mostly the cost of stopping at 380 and 780 — the range wearing the step’s clothes.

The separation is made by a rule change rather than by an argument. Halving the two end cells is the trapezoid rule, it is a one-line edit, and it takes the five-nanometre error on a smooth light from 0.064 to 0.0045: a factor of fourteen, for free. On the fluorescent tube the same edit changes the answer in the fifth decimal place, because there the error really is the sampling.

Two mechanisms living inside one number, told apart by a change that costs nothing and that no colorimetric standard makes. It is worth being clear about what that does and does not mean: the summation everybody performs is the one the standards specify, and the standards specify it because it is what the tables were built for. The finding is not that the practice is wrong. It is that the number the practice produces has two terms in it, and only one of them is about how finely the spectrum was sampled.

Where the grid begins #

The three lights whose error would not fall have a property in common: each has a feature narrower than the step. That suggests a test which no amount of refining can substitute for. Hold the step fixed and slide the grid’s origin through one whole cell.

An error belonging to the quadrature rule does not care where the grid starts. An error belonging to which features happen to be sampled cares about nothing else.

Across five origins the smooth lights move by 0.010 and 0.050 ΔE₀₀, and even that movement is the end cells rather than the sampling — sliding the grid slides its endpoints. The fluorescent tube moves by 3.18 and the laser projector by 35.0. That is a factor of sixty between the two mechanisms, on the same step, with the same number of points.

An average over origins would report a single figure for the discharge lamp of about two units and conceal all of it. This is the same shape of mistake a mean over a set makes about its members, arriving from an unexpected direction: the set here is not a set of surfaces but a set of grid alignments, and nobody has ever thought of a tabulation as having members.

Two ends that are not alike #

The range is the second decision and it is asymmetric to a degree that is easy to state and easy to get backwards.

Extending the lower limit from 380 down to 300 nanometres moves a red pigment under a 6500 K radiator by 0.502 ΔE₀₀. Extending the upper limit from 780 to 830 moves it by 0.00015 — a factor of three thousand three hundred. Under a tungsten lamp the two are 0.079 and 0.00038.

The asymmetry is not about how much power is out there. A thermal radiator has about a fifth of its total power outside 380 to 780, and almost all of that fifth is in the infrared, where the observer is identically zero. Below 380 the observer is small rather than absent, and daylight is still strong, so the product of the two is small rather than zero. What decides a truncation is the product, and the product is what a colour is made of — which is why a brightened sheet under a lamp with ultraviolet in it pays far more for this range than an unbrightened one.

For a light with no ultraviolet at all — an LED lamp, a laser — both ends cost exactly nothing. That is the pairing structure the previous round found, arriving in the tabulation: a range only costs what the light puts in it, so the range’s factor and the light’s factor multiply, and either being zero empties the product.

Which repair is worth buying #

Two decisions, both measurable, and a collection with limited attention has to choose. Measuring each over a family of forty-two analytic surfaces rather than on one example makes the choice legible.

Under a 6500 K radiator the step costs a median of 0.060 ΔE₀₀ over the family and the range costs 0.542 — a factor of nine. Under a tungsten lamp, 0.011 against 0.071. So for the lights this collection draws most often, widening the range is worth roughly an order of magnitude more than refining the step, and the site’s own ledger has named the range as its outstanding item for two rounds without ever comparing it against the alternative.

Under a fluorescent tube the ranking reverses outright: the step costs a median of 0.834 and the range 0.00013, a factor of six thousand the other way. There is no answer to “which end of the tabulation should be fixed” that does not name a light. That is the third appearance of the same structure in one essay, and the third time the answer has been a pairing rather than a property.

Why five nanometres has survived a century #

One number in the step column deserves a closer look before the round moves on, because it is smaller than it has any right to be.

At five nanometres the raw sum for a red pigment under daylight is wrong by 0.024 per cent in Y. The finished colour is wrong by 0.064 ΔE₀₀. Those two are not the same size, and the reason is that the grid appears twice: a tristimulus value is divided by a normaliser computed on the same grid, and the two errors are largely the same error.

Dividing removes between a third and three quarters of what the sum got wrong. That is not a large factor, but it is a systematic one, and it is available for nothing because nobody would think to compute a white point on a different grid from the sample. Five nanometres has been good enough for a century partly because it is fine enough and partly because the arithmetic everybody uses cancels its own error without being asked to.

The cancellation has a condition, and the condition is the one the whole essay keeps returning to. It works when the error is a smooth function of wavelength that both integrals share. On a spectrum with lines narrower than the step there is no shared smooth error to cancel — the sample’s sum and the white’s sum miss different lines — and the factor collapses.

What was computed, and how #

One module was written for this section of the round and it is small, because most of what it needed already existed.

The observer is the collection’s own, made grid-generic. This collection has held an analytic pigment template and an analytic ocular-media filter since the foundation phase, and a population of two hundred eyes is drawn from them. Both were evaluating onto the site’s eighty-one-point grid because nothing had ever asked them for anything else. Adding an optional list of wavelengths to three functions made the whole apparatus answerable at any resolution, and the normalising peak is still read off the site’s own grid so that a coarse request samples the pigment rather than rescaling it.

The reference is a tenth of a nanometre from 300 to 830, and it is checked rather than assumed: halving it again moves the sharpest case in the file by 3 × 10⁻¹³ ΔE₀₀. That is the difference between a reference and another entry in the table.

Every comparison is within one range. A step is measured against a tenth-nanometre sum over the same two endpoints, and a range against the full one, and the two never appear in the same subtraction. The first version of this file did not do that, and its step column rose as the step got finer — the truncation, constant in the step, drowning the thing the column was about.

Where the model stops #

Three limits, all of them about the observer rather than the arithmetic.

The curves here are a model of the standard observer and not a copy of it. Their residual against the tabulated 1931 functions is 2.3 per cent root-mean-square on ȳ and 16.4 on z̄, which is a median of 1.42 ΔE₀₀ over the surface family. Every number above should be read beside that rather than against zero.

The samples are all smooth, because reflectances are smooth for physical reasons — a pigment absorbs over a band of a hundred nanometres or more. The sharpest sample in the set is an interference filter with a notch forty times wider than a mercury line. So every result here about narrow structure is a result about lights, and whether a sample can have structure fine enough to matter is a question this file cannot ask.

And nothing here is measured. Every light is a construction from stated coefficients and every sample is a formula. The structural results do not depend on the numbers; the numbers do.

The generalisation #

There is a habit worth taking away from the arithmetic.

A quantity computed by summing a table has an index, and an index has a lower limit, an upper limit and a spacing. Those are three decisions, they fail by three mechanisms, and they are almost always discussed as one. The move that separates them is available in most such calculations and costs nothing: change the rule rather than the resolution, and slide the grid rather than refining it. The first isolates the endpoints; the second isolates the aliasing. What is left after both is the sampling error, which is usually the smallest of the three and is the only one anybody names.

The failure mode is to treat an index as bookkeeping. A table’s rows look like a fact about how the data were stored rather than a choice about what is being computed, and a choice that looks like storage never gets audited. Nineteen rounds of this collection have said name the observer and name the illuminant on every figure. Not one has said name the grid.

Who found it, and when #

The arithmetic here is old and unglamorous. Newton–Cotes rules and their error terms are eighteenth-century, and the fact that a rectangle sum on a finite interval carries an endpoint term linear in the spacing is in every numerical-analysis text.

What is specific to colorimetry is the tabulation itself. The CIE published its 1931 functions at five-nanometre intervals from 380 to 780 because that is what fitted the tables of the day, extended them to one-nanometre steps much later, and issued a technical report in 2005 on how to interpolate them when the finer table is not to hand. The ASTM’s practice for computing tristimulus values carries weighting tables at both intervals, and its own text is careful to say that abridging a spectrum and interpolating one are different operations. That care has not survived into general use, which is where this essay’s third choice comes in.

Where the ladder goes next #

The step and the range are measured and the third decision has only been named. A rule for the values between the tabulated points is the next essay, and its result runs the other way from expectation: two of the three fill-in rules in common use leave the answer further from the truth than summing the table as it stands.

After that comes the thing that makes the whole tabulation defensible, and it is not in the arithmetic at all. A spectrometer does not sample a spectrum; it integrates one through a slit, and the blur everybody would remove if they could is what keeps a five-nanometre table honest about a mercury line.