The endpoint term has a name

A number that is called a sampling error should behave like one, and this one does not. Its scaling gives it away, the diagnosis takes one line of arithmetic, and the repair is available to anybody and taken by nobody.

The claim #

On a light with no feature narrower than the step, a colorimetric summation’s error is the end cells of the truncated range rather than the sampling, and the two can be told apart by a change that costs nothing.

• The scaling is the tell. Doubling the step doubles the error, and a properly-ended sum of a smooth function has an error that falls as the square of the spacing.
• Halving the two end weights removes it. The five-nanometre error on a red pigment under a 6500 K radiator goes from 0.064 ΔE₀₀ to 0.0045, a factor of fourteen, on the same eighty-one numbers.
• On a spiky light the change does nothing, to five decimal places, because there the error genuinely is the sampling.
• And no colorimetric standard makes the change, for a reason that turns out to be about what a tabulated value means rather than about arithmetic.

The sequence that does not fit #

Coarsening the step from one nanometre to twenty gives, for a red pigment under a 6500 K radiator: 0.0122, 0.0255, 0.0640, 0.1232, 0.2209. For a tungsten lamp: 0.0025, 0.0052, 0.0130, 0.0247, 0.0417.

Both sequences are very close to proportional to the step. Doubling from one to two multiplies the error by 2.09 and 2.09; going from five to ten multiplies it by 1.93 and 1.90; from ten to twenty by 1.79 and 1.69.

That is first-order behaviour, and a quadrature error on a smooth integrand should not be first order. The midpoint and trapezoid rules are both second order — halving the spacing quarters the error — and the rectangle rule is second order too in the interior, because its errors on the rising and falling sides of a smooth hump cancel to leading order. First-order behaviour means something is not cancelling, and on a finite interval there is exactly one candidate.

A rectangle sum differs from an integral in one identifiable place. The sum a colorimetric calculation performs is Σᵢ f(λᵢ) Δλ with λᵢ running from 380 to 780. Geometrically it is a run of rectangles, each of width Δλ, each centred on nothing in particular: the first is anchored at 380 and extends to 385, and the last is anchored at 780 and extends to 785 — which is outside the range entirely.

The integral over [380, 780] wants half a cell at each end. The sum supplies a whole one at the bottom and, depending on how the loop is written, a whole one or none at the top. The difference is

(Δλ/2) · [ f(380) + f(780) ]

which is proportional to the step and to the integrand’s value at the ends. That is the whole of the discrepancy, and it is the Euler–Maclaurin formula’s first term: the sum and the integral differ by half the endpoint values plus terms in the derivatives.

Everything else about the sum is second order. So on a smooth integrand the entire first-order error is a statement about f(380) and f(780), which are not properties of the sampling at all. They are properties of where the range stops.

The rule change, measured #

Halving the first and last weights turns the rectangle sum into the trapezoid sum and removes exactly that term.

At the collection’s own five nanometres the improvement is a factor of fourteen. At twenty nanometres it is only four, and the reason is that by then the genuine sampling error — the second-order term — has grown large enough to be a real share of the total. The ratio peaking in the middle of the range is what a first-order term being removed from a sum of a first- and a second-order term looks like.

At one nanometre the ratio is also lower, at 8.2, and there the limit is different: the trapezoid residual has stopped being a truncation and is approaching the floating-point noise of the comparison.

The two smooth lights make the proportionality visible without any algebra. Daylight’s value at the lower endpoint is about five times tungsten’s, and the gap between the rules on daylight is about five times the gap on tungsten at every step. A term that scales with the integrand at one point rather than with anything about the sampling is not a sampling term, and no amount of refining will make it behave like one.

There is a second reading of the same pair that is worth having. The trapezoid line on daylight is not flat — it falls from 0.054 at twenty nanometres to 0.0015 at one — and that fall is the genuine sampling error, second order, behaving exactly as it should. It has been there the whole time, underneath a first-order term fourteen times its size, and nobody could have seen it without removing the larger one.

The falsification #

A claim this specific should have a case where it fails, and it has one that is easy to arrange.

On a light whose structure is narrower than the step, the first-order endpoint term is a negligible share of a large sampling error, and the rule change should therefore do nothing. It does nothing:

Five decimal places of agreement, on a light where the error is a hundred times larger. The rule change is not a general improvement to the arithmetic; it is a surgical removal of one identified term, and it is silent where that term is absent.

That pair of tables is the strongest form of the argument. A repair that improved everything would be evidence of nothing in particular. A repair that improves exactly the cases the diagnosis predicts and leaves the others untouched is a test the diagnosis could have failed.

Why the repair is not made #

Nothing about the trapezoid rule is difficult, and the CIE and ASTM both specify the summation without it. That is worth understanding rather than deploring, because the reason is a real one.

A tabulated value is not a sample of a spectrum. It is the spectrum integrated over a band, and if the band is one interval wide, the tabulated value at 380 nanometres already represents the light from 377.5 to 382.5. Summing all of them with equal weights is then the correct thing to do — the rectangles are not an approximation to an integral, they are a partition of one — and halving the end weights would introduce an error by discarding half of a band that was genuinely measured.

So the standard’s rule is right for measured data and wrong for sampled data, and the two are indistinguishable in a file. This collection computes from formulae, which is to say it produces sampled data, which is to say the trapezoid rule is correct here and the standard’s rule is not.

That is an uncomfortable conclusion and it is the honest one. A collection that computes its spectra should not use the summation a collection that measures its spectra uses, and this collection has been using it for nineteen rounds.

That figure closes a loop opened two essays ago. The origin sweep reported a spread of 0.050 ΔE₀₀ on daylight and attributed it to the endpoints rather than to aliasing, on the strength of a rule change. This is that rule change, measured: applying the trapezoid weights takes the same spread below 0.004, and leaves the fluorescent tube’s 3.18 exactly where it was.

Two diagnostics, one term, and each confirming the other’s account of it. That is worth more than either alone, because a single diagnostic that agrees with a hypothesis is weak evidence and two independent ones that agree with each other and with the hypothesis are not.

What it would cost to change #

The change is one line in one function and it would move every number this collection has ever published.

That is the argument against making it casually, and it is the same argument that governs any shared-kit change: a repair that touches everything has to be measured against everything before it is believed. The size is known — a factor of fourteen on the smooth-light term, which is between 0.01 and 0.06 ΔE₀₀ on ordinary samples — so nothing published here would change by more than a twentieth of a tolerance.

There is also a reason to prefer not making it, and it is not laziness. A collection that uses the standard summation is computing what a laboratory would compute, and a great many of its arguments are comparisons against laboratory practice. Switching to a rule that is more accurate but not the one in use would make those comparisons slightly incomparable, in a direction nobody could see.

The resolution is to state which is being done and why, rather than to choose once and forget. That is recorded rather than implemented, and it is recorded here because a defect this collection knows about and has not repaired belongs in an essay rather than in a comment. That is the same discipline the shortfall queue exists for, applied to a line of arithmetic rather than to a missing measurement.

The same term, in a place nobody looks #

The endpoint term is proportional to the integrand at the ends of the range, and there is a second consequence of that which has nothing to do with quadrature.

f(380) for a tristimulus integral is R(380) S(380) x̄(380). The observer’s value there is small — of order a thousandth of its peak — but daylight’s power is substantial and a reflectance is of order one, so the product is not negligible. That is precisely why the ultraviolet end of the range costs three thousand times what the infrared end does: both facts are about the same number, f(380), and both would go away if the range were extended to where the observer is genuinely zero.

So the endpoint term and the range cost are not two defects. They are one defect measured two ways, and widening the range repairs both at once. A range extended to 300 and 830 nanometres has f at its ends smaller by three orders of magnitude, and the first-order quadrature term goes with it — after which the summation really is second order and the step really is a sampling parameter.

That is the cleanest argument for the range repair that this section has produced, and it arrived from the direction of the arithmetic rather than from the physics.

What was computed, and how #

The two rules differ only in the first and last weight, and both are applied to the same eighty-one function evaluations, so the comparison cannot be contaminated by anything else — not by the range, not by the observer, not by the sample.

The white point is recomputed under each rule as well. Applying the trapezoid rule to the sample and the rectangle rule to the white would mix two fidelities in a ratio, which is the mistake the normaliser essay is about, and it would have reported a much larger and entirely spurious improvement.

The assertion the figure family carries is written conditionally: on a light with no narrow feature, the trapezoid rule is never worse. It is written that way because the unconditional version is false — on a spiky light the two rules differ in the seventh decimal place and the sign of the difference is arbitrary.

One more consequence follows for anybody reading a published guidance number about spectral resolution, and it is uncomfortable. Nearly all such guidance is derived by exactly the procedure this collection used: coarsen a fine calculation, watch the error, and pick the interval where it falls below a tolerance. If the calculation was a rectangle sum over a truncated range — and it was, because that is the specified summation — then the interval chosen is the interval at which a truncation term falls below the tolerance, and it has been reported as a resolution requirement.

The number that comes out is not wrong for the practice it describes, since that practice carries the same term. It is wrong as a statement about how finely a spectrum needs to be sampled, and it is wrong by the factor of fourteen measured above. A recommendation is only as portable as the arithmetic it was measured under, and this one has never travelled with its arithmetic attached.

Where the model stops #

The Euler–Maclaurin argument is exact for a smooth integrand and the integrand here is smooth only because both the light and the observer are. A real tabulated observer has a small amount of measurement noise in it, and noise has no derivatives, so the second-order term’s cancellation is not available to a table in the way it is available to a formula.

The endpoint term also assumes the sum’s first and last points sit exactly at the range’s ends. A grid that overshoots — running to 785 because the loop condition was written with the wrong inequality — has a different and larger error, and that is a class of bug no accuracy argument will find.

And nothing here has been checked against the ASTM weighting tables, which fold the illuminant and the observer into a single set of factors per interval. Those tables are constructed to reproduce a fine-interval result, so they may already contain a correction of this kind; establishing whether they do is a table comparison nobody here has run.

That last observation deserves to be stated as a correction rather than as a curiosity. This section has been presenting the step and the range as two separable decisions, and on a smooth light they are not fully separable: a good part of what has been charged to the step is the range appearing in the quadrature. The separation survives because the two repairs are still different — one is a weight and the other is sixteen new rows of data — but the accounting is not as clean as two columns suggest, and the ranking between them is if anything more lopsided than it appeared.

The generalisation #

The habit is to read the scaling before reading the magnitude.

An error’s size says how much trouble there is. Its order says where the trouble is. A first-order term where second order was expected means a boundary is involved; a term that does not fall at all means information is missing rather than approximated; a term that falls faster than expected usually means two errors are cancelling and will stop.

The diagnostic move that follows is to change the rule rather than the resolution. Refining tests the same mechanism at a different size. Changing the rule tests a different mechanism at the same size, and it is the only one of the two that can isolate a term.

The failure mode is to see an error fall as the discretisation refines, conclude that the method is converging, and stop. It was converging, at first order, towards an answer that a one-line change would have reached fourteen times faster — and the first-order behaviour was visible in the third and fourth entries of the same table that established the convergence.

Who found it, and when #

Euler and Maclaurin published the formula independently in the 1730s, and its first term — the half-endpoint correction — is the reason the trapezoid rule exists at all as a distinct thing from a Riemann sum.

Its application to colorimetry is not in the standards, and the reason given above is why: the standards are written for band-integrated data, where equal weights are correct. The distinction is stated clearly in the CIE’s recommended practice for tabulating spectral data, and is then absent from most implementations, which apply the standard summation to whatever array is handed to them.

Where the ladder goes next #

Every number in this section has been measured against a reference, and a reference for a tabulation cannot be another tabulation. What that forced, what it cost, and what it means for a collection to check its arithmetic against a model of its own subject matter.