The slit is what makes it legal

A tabulated spectrum arrives with a number beside each wavelength, and it is natural to read the number as the spectrum’s value there. It is not. It is the spectrum integrated against a slit, and the difference between those two readings decides whether a coarse table is safe.

The claim #

A finite instrument bandpass makes a tabulated colour more accurate, not less, and for a line spectrum the improvement is a factor of a hundred.

• Point sampling is the worst case, and it is the case every calculation performs when its spectrum comes from a formula rather than from an instrument.
• The slit is a low-pass filter, and the tabulation that follows it is a sampling of an already-smoothed function, which is exactly the arrangement sampling theory asks for.
• The repair is published and cheap. A three-term deconvolution recovers most of what the slit itself cost, and leaves the anti-aliasing.
• And this collection computes the wrong way round. Every spectrum here is evaluated at eighty-one wavelengths from a closed form, which is a zero-width slit, so its line-spectrum figures carry an error no instrument would have made.

What a spectrometer actually returns #

A monochromator disperses light across an exit plane and a slit selects part of it. The slit has width, so what reaches the detector is not the radiance at one wavelength but the radiance integrated over a band, weighted by how much of the slit each wavelength illuminates. When the entrance and exit slits are the same width — the ordinary design — that weight is a triangle, of full width equal to twice the nominal bandwidth and of half-width equal to the sampling interval.

The tabulated number at 545 nanometres is therefore ∫ S(λ) T(λ − 545) dλ with T a triangle, and not S(545). For a spectrum that is smooth over the triangle’s width, the two are the same to a part in many thousands, which is why the distinction is usually left out. For a spectrum that is not, they are different quantities.

Everything in this collection sits on the second side of that boundary at least some of the time. The lamps that light most rooms are not smooth: a fluorescent tube is a phosphor bed with mercury lines standing on it, and a mercury line is about a nanometre wide.

The measurement, in one table #

Six lights, a notch-filter sample, a five-nanometre tabulation, and a triangular slit of full width zero, five, ten and twenty nanometres. The comparison is against the same colour computed at a tenth of a nanometre with no slit at all.

The columns to the right of the fifth-nanometre one are the ordinary bandpass story: a slit wider than the reporting interval oversmooths, and every row grows. What is not ordinary is the direction between the first two columns, and it is the whole essay.

The first two rows are flat: a slit narrower than any feature the light has changes nothing, which is the expected behaviour and the reason nobody thinks about slits.

The last two rows are the finding. Going from no slit to a five-nanometre slit improves the fluorescent tube by a factor of ninety and the laser projector by a factor of a hundred and ninety-three. The instrument’s own blur is not an error being added to the measurement; it is the anti-aliasing filter that makes the measurement samplable.

What was already known here, and what is new #

This collection has met half of this before and it is worth saying exactly which half, because the earlier result is right and incomplete rather than superseded.

The essay that first coarsened this site’s grid compared two ways of dropping points — taking every fourth value, or averaging the values in each new cell — and found the second better by about a factor of six on a fluorescent tube. That is the same mechanism as this one, seen from inside the arithmetic: averaging over a cell is a rectangular slit of one cell’s width, and it works for the reason a triangular slit works.

Three things are new. The comparison there was between two coarsenings of a five-nanometre table, so the true answer never entered it and the improvement could only be reported as a ratio; here every column is measured against an analytic reference and the absolute error is available. The kernel there was implicit and rectangular; here it is the triangle a real monochromator has, and its width is a variable rather than a consequence of the coarsening. And the earlier essay could not ask the question this one ends on, which is what happens when the spectrum on the disk was never measured at all.

Why the direction is surprising and should not be #

Every account of instrument bandpass in the literature treats it as a distortion. It is: the slit smears a real spectral feature, a narrow peak comes back shorter and wider than it is, and a bandpass correction exists to undo that. All of that is true of the spectrum.

It is not true of the colour computed from the spectrum, and the reason is that a colour is itself an integral. A tristimulus value is the spectrum integrated against a smooth observer function, and smearing the spectrum against a symmetric kernel of width small compared with the observer’s own variation barely changes that integral at all. What it changes enormously is whether the intermediate tabulation can represent the spectrum, and that is where all the error was.

So the slit costs the spectrum something and buys the colour a great deal, and the two are not in competition because they are answers to different questions. An instrument’s reading is a construction with a geometry and a condition in it, and this is one more entry on that list — one that happens to be working in the user’s favour.

A point sample of a line spectrum computes a different object, and the laser row is worth taking apart, because its failure is not a large error but a different quantity.

The three lines are at 465, 532 and 638 nanometres and each is a fifth of a nanometre wide. On a five-nanometre grid beginning at 380, the sample points are 380, 385, … so 465 is a grid point and 532 and 638 are not. Evaluated at the grid points, the projector is a monochromatic source at 465 nanometres. Not a poor approximation to three lines: one line, and the other two are simply absent from the arithmetic.

Everything downstream follows from that and none of it is subtle. The computed white is blue. The computed colour of any sample is that sample’s reflectance at 465 nanometres times a fixed vector, so every sample in the file comes out the same hue. And because a single wavelength scales in and out of the calculation, the sample’s chromaticity does not depend on the sample at all.

Sliding the grid by two nanometres catches 532 instead and produces a completely different set of answers, which is why the origin of the grid is a measurement in its own right and why the laser’s spread across five origins is thirty-five units.

The published repair leaves the anti-aliasing behind. If a slit rescues the tabulation, the obvious next question is whether the slit’s own cost can be removed afterwards, and it can. Stearns and Stearns published a three-term deconvolution for a triangular slit whose full width equals the sampling interval: replace each tabulated value by −0.083 of its left neighbour, 1.166 of itself and −0.083 of its right neighbour. It is exact for a quadratic and it is one line of arithmetic.

Applied to the five-nanometre column above, it takes the fluorescent tube from 0.0140 to 0.0039 and the laser projector from 0.1916 to 0.0567. So the slit’s own smearing was costing about a factor of three, and the correction recovers it.

What the correction cannot do is undo the anti-aliasing, and that is the point rather than a shortcoming. It operates on the tabulated values, which is to say on a signal that has already been low-pass filtered and sampled. It restores the shape the filter distorted. It has no access to what the filter removed, and what the filter removed is exactly the structure that would have aliased.

A correction that recovered everything would put the aliasing back. That is worth stating plainly because the arithmetic makes it look like a lossless operation on a table, and a table is where the loss has already happened.

Which of this collection’s figures are on the wrong side #

The site’s own machinery evaluates a spectrum by calling a function at each of eighty-one wavelengths. That is a zero-width slit — the first column of the table above, the worst one.

For most of what the collection draws this costs nothing, because most of what it draws is smooth. Planck’s law is smooth. The CIE daylight reconstructions are built from three tabulated basis functions and are smooth. Reflectances are smooth for physical reasons. Cone absorptances are smooth by construction. On all of those the difference between the first two columns is in the fifth decimal place.

The exceptions are the lamps built with narrow emitters, and this collection has several. Anywhere a figure here draws a discharge lamp, a mercury line or a laser primary and quotes a colour from it, the quoted colour is a point sample and an instrument would have returned something else. On the constructions in this file that gap reaches 1.26 ΔE₀₀ for a tube and thirty-seven for a projector.

This is a real defect and it is recorded as one. It does not invalidate the arguments those figures make, because every one of them compares two computations done the same way and the pairing structure means the shared term largely cancels. It does mean that an absolute colour quoted for a narrowband source anywhere in this collection is a point-sampled colour, and the honest thing is to say so rather than to quietly widen a slit.

Widening one would not be a small change either, and the reason is instructive. A slit is a property of an instrument, and a figure that draws a constructed lamp has no instrument in it. Adding a five-nanometre triangle to the constructed lamps would be adding an imaginary spectrometer to a calculation whose whole claim is that it computes from stated physics — which is a different kind of dishonesty from the one it repairs. The defensible options are to compute those figures at a resolution fine enough that the point sampling is not a point sampling, which costs a factor of twenty in build time on every affected family, or to say in the caption that the source is being read at the grid’s own points. The second is what the collection can afford and the first is what it owes.

There is a third option that looks attractive and is not. Moving the grid so that its points land on the lines would make each of these figures exact, and would make the collection’s answers depend on a choice nobody could defend for any other spectrum. A tabulation tuned to one light is not a tabulation.

The condition, and both halves of it #

The structure here is the one the previous round found for the sample and this round keeps finding for the index: a cost that is a pairing, vanishing when either factor is empty.

A slit is worth nothing when the light has no feature narrower than it. A step is worth nothing when the light has no feature narrower than it. Both conditions are about the same property of the light, and the two are related in the way that matters: the slit’s width and the step’s width are usually the same number, because instruments are built with the bandpass matched to the reporting interval. That is not a coincidence and it is not an accident of manufacture. It is the design that makes the reported table mean something.

So a spectrum reported at five nanometres from a five-nanometre instrument is a well-formed object, and a spectrum evaluated at five nanometres from a formula is not. They look identical in a file. Nothing in a data format records which one is on the disk, and no gate in this fleet could tell them apart.

The consequence for anybody combining data is sharper than it first looks. Two spectra tabulated at the same interval may have been produced by instruments of different bandpass, or by an instrument and a model, and multiplying them together — which is what a reflectance under an illuminant is — multiplies two differently-filtered signals. The product is not the filtered product, and the gap is largest exactly where one of the two has structure. A reflectance is smooth, so the ordinary case is safe; a fluorescent lamp multiplied by a tabulated observer is the case where it is not, and it is the commonest calculation in the field.

The cancellation drawn there is the other reason a coarse grid usually works, and the slit is why it works here too. Two sums that both miss the same lines have errors that divide out; two sums that miss different lines do not. A slit removes the lines from both, which is what puts a spiky light back into the regime where the division helps.

What was computed, and how #

The slit is integrated numerically at a tenth of a nanometre across the triangle, which is fifty times finer than the narrowest line in the set, and the reference colour is the same tenth-nanometre grid with no slit at all. Both halves therefore share a quadrature and the comparison is of slits rather than of integrators.

The Stearns correction is applied to the smeared, tabulated values and then read back through a linear interpolation, because that is the operation a user of corrected data actually performs. Applying it and then integrating analytically would measure the correction’s algebra rather than its use.

The assertion this figure family ends on is that the zero-width column is worse than the five-nanometre column by more than a factor of ten for the laser projector, and it is written that way round on purpose: the interesting claim is a comparison between two instrument designs, not a tolerance on a number.

Where the model stops #

The slit here is an isosceles triangle, which is what matched entrance and exit slits give. Real instruments depart from that — array spectrometers have a pixel-and-optics response that is closer to a Gaussian with tails, and a monochromator with mismatched slits gives a trapezoid. The direction of every result above is a property of any symmetric low-pass kernel, and the size of the Stearns recovery is a property of the triangle specifically.

The lights are constructions. The mercury lines here are Gaussians of stated width, and the real ones have pressure-broadened wings that a Gaussian does not have. A wing is exactly the part a slit does not remove, so a real tube is somewhat kinder to a coarse grid than this one.

And nothing here says what a sample with narrow structure would do, because there are none in the file. An interference filter is the sharpest reflectance anybody sells and it is still forty times wider than a mercury line.

The generalisation #

The habit worth carrying is about where a smoothing step sits in a chain.

A measurement chain that samples must filter first, and a chain that filters after sampling has already lost. That is Nyquist’s argument and it is ninety years old. What is specific here is that the filter is physical and nobody put it there for that reason — the slit exists because a monochromator needs one — so it never appears in the accounting as a benefit, only as a distortion to be corrected.

The reverse move is the failure this essay is about: replacing a measured spectrum with a formula, evaluating the formula at the same wavelengths the measurement used, and believing the two are interchangeable. They are interchangeable exactly when the spectrum is smooth on the scale of the slit, which is the same condition as everything else in this round and is checkable in one line.

Who found it, and when #

Stearns and Stearns published the three-term correction in 1988, in the journal Color Research and Application, from a paper industry that needed abridged spectra to agree with unabridged ones. The result that a triangular bandpass of one interval is nearly removable by a symmetric three-term filter is older in signal processing and was rederived there for colorimetry.

The CIE’s own technical report on interpolation, published in 2005, is careful about the distinction this essay turns on: abridging a spectrum by sampling and abridging it by averaging over a band are different operations with different errors, and a table that does not say which one produced it cannot be corrected. That care has largely not survived into practice, where a column of numbers at five-nanometre intervals is read as a column of values.

Where the ladder goes next #

The slit settles what a tabulated number is. The next question is what to do with the numbers between the tabulated ones, and the answer is not the one the arithmetic suggests: two of the three fill-in rules in common use leave the answer further from the truth than not interpolating at all.

After that the round turns to the condition under which none of this matters. There is a class of samples for which every grid, every slit and every origin give exactly the same colour, and it is larger and duller than it looks.