What the instrument reports

Every reflectance on this site is a function known exactly at every wavelength. Nothing measures one.

An instrument sees the sample through a slit of finite width, at a finite number of wavelengths, with finite noise. What it reports is a smoothed sample of a curve, not the curve — and the difference is usually negligible and occasionally expensive, with nothing in the file to say which case applies.

Two different losses are folded together in that picture, and they can be separated by moving one at a time.

Two more samples say that neither loss is a property of the interference filter this essay happens to have used.

The three things an instrument does to a sample #

It convolves. The monochromator’s bandpass is roughly triangular when the slits are matched, and its width is quoted as a full width at half maximum. Asking for the reflectance at 550 nm returns a weighted average over a band around 550.

It samples. An instrument reporting every 10 nm hands over 41 numbers across the visible range. Every colorimetric calculation downstream needs values on a finer grid, and the missing ones are produced by interpolation.

It adds noise, which is the least interesting of the three and the only one anybody puts on a datasheet.

The second is the one that goes unnoticed. Reported values and interpolated values are indistinguishable in the file: the same column of numbers, the same precision, no flag. A calculation performed on the reconstruction has no way to know which of its inputs were measured — and the colour it computes depends on all of them equally.

When it costs nothing #

The reassuring half, and it is genuinely reassuring.

Measure a featureless reflectance — a smooth ramp, no structure at all — through a 20 nm bandpass at 20 nm intervals and the colorimetric error is 0.0007 ΔE. Not small: absent. Convolving a straight line with a symmetric kernel returns the straight line, and interpolating between points on a straight line returns the line.

Most real object colours are close to that. A pigment absorbs across a broad band because molecular absorption is broad, which is why most reflectances are smooth; a dyed fabric, a painted wall, a printed ink all have reflectance curves with no feature narrower than tens of nanometres. For the overwhelming majority of samples the instrument’s bandpass is irrelevant, which is why it can be ignored almost always.

When it costs a great deal #

The exceptions are exactly the samples with spectral structure, and they are commercially concentrated in the places where colour matters most.

An interference pigment gets its colour from a thin-film stack rather than from absorption, so its reflectance oscillates — several maxima across the visible range, none of them wide. Measured through a 20 nm bandpass, the error is 2.97 ΔE. Between a 10 nm instrument and a 20 nm one, on the same sample, the disagreement is 2.34 ΔE.

That is the number with consequences. Neither instrument is wrong and neither has the truth; they are two measurements, both defensible, differing by more than twice the tolerance a supply contract typically specifies. Pearlescent automotive paint and most security printing work this way, and both are products held to tight tolerances by parties using different instruments.

The illuminant multiplies it #

The finding that reorganised this essay: the instrument’s error is not a property of the instrument.

The smoothing is a fixed operation on the reflectance, and it looks at first like a fixed penalty. It is not, because the reflectance is multiplied by the illuminant before anything is integrated. Smooth the sample and view it under a smooth light and the smoothing largely cancels. Smooth the same sample and view it under a lamp whose output is four mercury lines and three phosphor peaks, and the error lands wherever the lamp happens to have its power.

Measured: the same sample through the same pair of instruments disagrees by 0.88 ΔE under D65 and 6.30 ΔE under a triphosphor tube. A factor of seven, and the instrument did not change — the lamp in the room did.

That is worth stating as a rule, because it is not how measurement error is usually thought about. A tolerance quoted for an instrument is quoted under a smooth reference illuminant. The same instrument’s contribution to a disagreement under a discharge lamp is several times larger, and the multiplication is between two things neither party controls.

What the reconstruction actually does #

The interpolation deserves attention because it is where the invented values come from.

Between reported points, linear interpolation draws a straight line. For a smooth reflectance that is nearly right. For a curve with a maximum between two reported points it is systematically wrong in one direction: the peak is cut off, and the reconstruction always underestimates it.

That bias does not average out. A sample with several peaks, all clipped in the same direction, produces a reconstruction with systematically less structure than the truth — and a systematic error is far worse than a random one of the same size, because it cannot be reduced by measuring again.

There is a matching effect at minima, where interpolation fills in troughs. Together they mean the reconstruction is always smoother than the sample, never rougher, and the direction is known in advance.

The rule behind the factor of seven #

A pair of lamps is two points, and six of them give the shape.

Measuring the interference pigment through a 10 nm instrument and a 20 nm one, and asking how far the two disagree, under six sources: illuminant A 1.72, a white LED 1.97, D50 2.17, D65 2.34, a triphosphor tube 3.12, and a three-emitter source 4.05. A factor of 2.35 across the set, from the same two instruments reading the same sample.

The ordering follows the lamp’s own spectral roughness — the size of its second differences relative to its power — at a Spearman coefficient of 0.714 over the six. The smoothest source on the list is illuminant A, a Planckian radiator whose roughness is 0.0004, and it produces the smallest disagreement; the three roughest sources produce the three largest.

The correlation is not perfect, and the exception is the instructive part. The triphosphor tube is by far the roughest source at 0.597, three times the three-emitter’s 0.180, and it produces the smaller disagreement of the two. What matters is not how rough the lamp is but where its structure falls relative to the sample’s, and the three-emitter’s peaks happen to sit on the pigment’s oscillations where the tube’s mercury lines do not.

What a slit width costs, as a law #

The bandwidth is a dial, and the error against the truth follows a power of it.

On the same pigment under D65: a 5 nm bandpass gives 0.0000, a 10 nm one 0.633, 15 nm gives 1.623, 20 nm gives 2.967, 30 nm gives 5.857 and 40 nm gives 12.15. Fitted between neighbouring points the exponent runs between 2.0 and 2.3, so the error grows about as the square of the slit width. Doubling the slit quadruples the disagreement.

The zero at 5 nm is the useful end of that. The pigment’s oscillation has a period of eighteen nanometres, and a bandpass small compared with the structure costs nothing measurable — which is the smooth sample’s 0.0007 again, arrived at from the instrument’s side rather than the sample’s. What decides whether a bandpass is wide is never its width in nanometres; it is its width against whatever the sample has in it. And a quadratic law is steep enough that the journey from nothing at all to twice a supply tolerance takes a single doubling.

Why anybody uses a wide bandpass #

A fair question is why instruments are not simply built with narrower slits.

Light. A narrower slit passes less of it, so the signal falls and the integration time rises to compensate. An instrument measuring at 2 nm resolution takes considerably longer per sample than one at 20, and in a production environment measuring thousands of samples a shift, that is the entire economics.

There is also a resolution-versus-noise trade that does not go away. Halving the bandwidth halves the signal, which raises the relative noise, and past some point the noise in a narrow measurement exceeds the bias in a wide one. The optimum depends on the sample, which nobody knows in advance.

So the 10 nm instrument exists because it is a defensible compromise for most samples, and the 20 nm instrument exists because it is cheaper and faster and adequate for most samples. Both are correct engineering. The problem is not that either is badly designed; it is that the compromise was made once, in the instrument, and the samples arriving at it were not consulted.

What was computed here #

The slit is a triangular or Gaussian kernel of stated width, normalised to unit area. Measurement convolves the truth with it, samples at the reporting interval, and reconstructs by linear interpolation onto the working grid — the same three steps a real instrument and its software perform, in the same order.

The noise term is a fixed deterministic function of wavelength rather than a random draw, and that is a requirement rather than a shortcut: a build that drew random numbers would produce a different figure every time it ran, and a caption quoting a ΔE would be quoting a number that changed between builds.

Four assertions run in the gate.

Bandpass must cost a structured sample and not a smooth one — both halves, because if wide bandpasses wrecked everything nobody would ship one, and if they wrecked nothing there would be no essay.

Two real instruments must disagree by more than a supply tolerance. The pairing is 10 nm against 20 nm rather than something more extreme, because those are the two on the market; a comparison against a hypothetical perfect instrument would be a fact about an instrument nobody owns.

The illuminant must multiply the error by a large factor, which is the finding above.

And a perfect instrument on a smooth sample must return it exactly, to 10⁻³ ΔE. That one is the control. Without it, every number above could be measuring an error in the model rather than an error in the measurement, and there would be no way to tell.

A ten-nanometre instrument reporting every ten nanometres is the commonest specification in the trade, and an interference filter is the sample that punishes it.

What a colour is measured for #

An instrument reports a spectrum, and almost nobody wants a spectrum. What is wanted is one of three things, and they place different demands on the measurement.

A pass or fail against a standard. The dominant use by volume. Here only the difference between two measurements matters, and a systematic error common to both largely cancels — which is why two samples measured on the same instrument can be compared far more tightly than either can be compared to an absolute value. It is also why the industry’s answer to inter-instrument disagreement is to insist both parties use the same model.

A colour to be reproduced elsewhere, which requires the absolute value and inherits every error in full.

A spectrum to be used in a calculation — rendering the sample under a different illuminant, or predicting a metameric failure. This is the demanding case, because it needs the spectral detail rather than the three numbers, and it is exactly where the interpolated values do damage.

The third use is growing and the instruments have not changed. Predicting how a sample renders under a lamp requires multiplying its reflectance by the lamp’s spectrum wavelength by wavelength, and a reconstruction that is systematically smoother than the truth multiplied by a spectrum with holes in it produces an error neither party can audit.

The practical advice that follows is narrow and worth stating. If a measurement will only ever be compared against another from the same instrument, the bandpass hardly matters. If it will be carried into a spectral calculation, the bandpass and the reporting interval are part of the result, and they belong in the record alongside the numbers.

A coarse instrument on a structured sample is where the loss is largest, and it is the specification a cheap instrument actually has.

Where the model stops #

Three things a real instrument does that this model does not.

Geometry. A measurement is made at a specified geometry — 45°/0°, or diffuse with the specular component included or excluded — and the choice changes the answer substantially for anything glossy. Two instruments of identical bandpass at different geometries disagree by more than anything in this essay. Modelling it needs a bidirectional reflectance distribution, and this site has none.

Fluorescence. Everything here assumes light leaves at the wavelength it arrived at. It does not, for a great many real samples, and the instrument’s own internal light source then becomes part of the measurement — which is why brightened samples measured on two instruments with different lamps disagree even at identical bandpass and geometry.

Calibration drift and thermal effects, which are the largest source of disagreement in practice and the least interesting theoretically.

The essay’s claims are therefore a lower bound on inter-instrument disagreement, not an estimate of it. What it establishes is that bandpass alone is enough to exceed a supply tolerance on structured samples, which is a sufficient conclusion even though it is not the whole story.

A five-nanometre instrument is what a laboratory has, and it is the case that says how much of the loss is removable by buying a better instrument.

What the pictures cannot show #

The figures plot the truth against the reconstruction, and having both is the one thing a real measurement never has.

That is not a small caveat. The entire practical difficulty of instrument disagreement is that nobody can compute the numbers in these figures for a real sample, because computing them requires knowing the reflectance the instrument was trying to measure. What is available in practice is two measurements and a dispute, and no way to establish which is closer.

So this essay’s figures are a view from a position nobody occupies. The truth is available here only because it was constructed — and a constructed sample with a stated amount of structure is exactly what is needed to establish that the effect exists and how it scales, and is not a substitute for a measured one.

Every measurement on this site is an idealisation #

The uncomfortable implication of this essay is for everything else here.

The colour-matching functions are tabulated at 5 nm and every spectrum on this site is sampled at the same interval. Anything narrower than that — a mercury line, a laser primary, a sharp interference peak — is not representable, and what is drawn instead is the feature convolved with the sampling grid.

That is stated where it matters. The mercury lines in a fluorescent lamp are given an instrumental width rather than their true one, because their true width is picometres and the grid is nanometres; the site’s spectra begin at 380 nm, so an optical brightener’s excitation band is truncated and the computed effect is a floor.

Neither is a defect that could be fixed by better arithmetic. They are the same limitation this essay describes, arriving in a calculation rather than in an instrument: a spectrum represented by eighty-one numbers is a smoothed sample of a function, and every claim computed from one inherits the smoothing.

The difference between the site and the instrument is only that the site can say so. An instrument’s output file has no field for it, and neither has any of the interchange formats a measurement passes through afterwards — so the information is not lost at one point but never recorded at any.

That is the general shape of the problem this essay describes. Nothing here is a failure of measurement science, which has understood bandpass and sampling since spectrophotometry became routine. It is a failure of record-keeping: the conditions under which a number was produced are known at the moment of production and are not carried with the number, so every calculation downstream proceeds without them and produces an answer that looks exactly as authoritative as one made with them would.

Who found it, and when #

Bandpass correction has been understood since spectrophotometry became routine in the 1930s, and the standard correction — Stearns and Stearns, published in 1988 — is a simple three-point deconvolution that recovers a good deal of what the bandpass removed. It works well when the bandpass and the sampling interval match and less well otherwise, and a striking number of instruments do not apply it.

Inter-instrument agreement became a formal concern when colour measurement moved into supply-chain acceptance in the 1970s and 80s, at which point two parties with different instruments had a contract that depended on them agreeing. The literature on it is largely produced by instrument manufacturers, which is not a reason to disbelieve it and is a reason to notice what it does not compare.

The interference pigment problem became prominent in the 1990s with the spread of pearlescent automotive finishes, where a colour dispute involves a whole vehicle. It remains the standard hard case, and the standard answer is still that both parties must use the same instrument — which is a procedural solution to a measurement problem and an admission that the measurement problem is not solved.

Where this goes next #

The spectra whose structure makes this expensive are a lamp is not a blackbody. The samples that break the reflectance model outright are some paper is brighter than white. And what the disputed number is compared against once it has been measured is a tolerance is a shape.