A tolerance needs a second number

A specification says ΔE00 ≤ 1.0 and means it as a property of a pair of samples. It is a property of the pair and the person, and the two spectra decide how much of the person is in it.

That fact is already on this site, and it was left where it was found. At a fixed colorimetric difference of 1.0, the population’s ninety-fifth percentile doubles as a metameric black is added to one of the samples — from 1.13 to 2.32. Two pairs, the same number in the report, twice the risk.

What was not asked is the obvious follow-up. If the difference on its own does not say which pair is on the bench, what would?

The claim #

The missing field is a spectral distance, the instrument already has everything needed to compute it, and adding it changes decisions.

• A weighted root-mean-square difference between the two reflectances predicts the population’s ninety-fifth percentile, on pairs the calibration never saw, with three times less error than the colorimetric difference alone.
• The fitted rule has two coefficients and one of them comes out at one. The population’s spread equals the stated difference exactly when the two spectra agree, which nothing in the fit required.
• Each unit of the index adds 2.3 per cent of the stated difference to what the worst-off twentieth see.
• And it is not a metamerism index. Both of the CIE’s compare a pair under a second illuminant. Nothing here changes the light; the observer changes, and the illuminant stays where the specification put it.

The index #

The proposal is one number, and the design constraints on it are severe. It has to be computable inside a spectrophotometer from what the instrument has already measured — no population model, no second illuminant, no reference to anything the user would have to supply. It has to be zero when the two samples have the same reflectance. And it has to count a difference only where a difference could matter.

$$I = 100\sqrt{\frac{\sum_\lambda w(\lambda)\,(\rho_1-\rho_2)^2}{\sum_\lambda w(\lambda)\,\bar{\rho}^{\,2}}}$$

with w(λ) the sum of the three colour-matching functions under the illuminant, normalised to unit mean. That weight is the cheapest thing with the two properties wanted: it is essentially zero where no observer has any sensitivity, so a disagreement at 390 nanometres is not counted as a disagreement; and it is available from the same tables the ΔE was computed with.

The normalisation by the mean reflectance makes it relative, so a pair of dark samples and a pair of pale ones land on the same scale. What is being asked is how much of the shape the two disagree about, not how much light they return.

The pairs, and why they are constructed #

The calibration needs pairs at a known colorimetric difference whose spectral distance can be moved without moving that difference. Those do not occur naturally in useful numbers, so they are built.

A metameric black is a spectral function invisible to the reference observer by construction — orthogonal to all three of the rows a reflectance is projected against. Adding one to a sample’s reflectance changes its spectrum and changes nothing a colorimeter can report. The pair is then given a smooth tilt, solved by bisection, that brings the reference difference back to exactly the target.

So every row of a family is the same tolerance decision made about a more and more different pair of spectra, and assertTheDifferenceIsHeldFixed requires every one of them to be within two per cent of the target before anything is claimed. The families are generated from several base reflectances at several target differences, which is what stops the result being a property of one surface.

The coefficient nobody put there #

The rule fitted is

$$P_{95} = \Delta E_{00}\,(a + b\,I)$$

with two free coefficients and no constant term outside the product. The shape is chosen from the physics rather than from a search over model forms: a pair with no spectral difference is a pure lightness or chroma shift that everybody reads nearly identically, so the spread should be proportional to the difference and grow from there with how much the spectra disagree.

a comes out at 1.014.

Nothing in the fit required that. Two coefficients were free, thirty-five pairs went in, and the constant landed within one and a half per cent of unity — which is the statement that a pair whose spectra agree transfers to the whole population exactly. That is a check on the construction as much as a result: if the machinery had a systematic error in it, the constant is where it would have shown.

With a at one, b reads as a rate. Each unit of spectral index adds 2.3 per cent of the stated difference to the ninety-fifth percentile — so a pair at ΔE00 1.0 with an index of 25 is a ΔE00 1.59 pair for the worst-off twentieth, and a pair at ΔE00 2.0 with the same index is a 3.18.

The rule inverts into a sliding tolerance #

The proposal above adds a third line to a report and leaves the specification alone. Turning the rule around gives a second form of the same proposal that leaves the report alone and changes the specification instead, and it is the cheaper of the two to adopt.

If the ninety-fifth percentile is the stated difference times $(1.014 + 0.023\,I)$, then holding that percentile under a tolerance $T$ means holding the colorimetric difference under $T$ divided by the same multiplier. So a two-number specification can be written as one number that slides:

An index of 43 halves the usable tolerance. That is the sentence worth carrying, because it converts the field into something a specification writer can price: a pair whose spectra disagree by 43 units of the index has to be twice as close colorimetrically to carry the same risk, and a pair whose spectra agree can have the whole tolerance.

The second form has an advantage the third-line form does not. A report with three lines requires the reader to decide which one the contract is about, and a contract that names ΔE00 and prints a risk beside it has two numbers and one limit — which is precisely the arrangement that lets a supplier point at the passing one. A sliding tolerance has one number and one limit, and the slide is arithmetic the instrument does before printing anything.

It also makes the shape of the specification visible. A two-number rule is a boundary in the plane of stated difference against spectral index, and the boundary is a curve rather than a threshold — a hyperbola, since the product is what is bounded. So a specification with this field is a region rather than a line, which is the same conclusion a tolerance is a shape reaches in colour space and reaches here in a space one of whose axes is not a colour at all.

Two things about the table are worth stating rather than leaving implicit. The top row is not 1.00, because the fitted constant is 1.014 rather than one — a pair with identical spectra still loses a per cent and a half of its tolerance, which is the fit’s own residual and not a physical effect. And the slide is steepest where the index is smallest: the first ten units of index cost a fifth of the tolerance and the next forty cost another third, so the field does most of its work distinguishing nearly-identical spectra from slightly-different ones rather than at the metameric extreme.

That last point is the practical one. The extreme cases — a print against a display, a four-ink build against a spot ink — are already known to be fragile and are already treated with suspicion. The index earns its place in the ordinary range, where two samples of nominally the same colorant differ a little in formulation and nobody currently has any way to tell that from two samples that differ only in strength.

Tested where it was not fitted #

A field fitted and tested on the same pairs predicts its own construction. So the test set is built from different base reflectances, at differences between the calibrated ones, and read against a population drawn with a different seed — nothing shared with the fit but the two coefficients.

The root-mean-square error in predicting the ninety-fifth percentile is 0.218 with the index against 0.652 without it, a factor of three. The correlation is 0.966 against 0.813.

The bare number is not useless — a larger colorimetric difference does mean a larger spread, so a correlation of 0.81 is expected and would be higher still on a sample of pairs that varied more in ΔE00 than these do. What the index buys is the part of the variation that the difference cannot explain, which is precisely the part that decides whether a report transfers to a person.

At two units the decision is the one a great deal of packaging work is signed off on, and the family there is the one worth reading.

Whether it changes any answer #

A field that changes no decisions is a column of numbers. So the last question is asked in the form a specification is used in: accept or reject, at a stated tolerance, twice — once on the colorimetric difference and once on the risk the two-number field reports.

At tolerances of 0.5 and 3.0 nothing changes: the pairs are far from the line and both readings put them on the same side. At 1.5 and 2.0, seven of twenty-one decisions come out differently, and accuracy against the population goes from 67 per cent to 100.

That is the expected shape and it is worth saying plainly rather than dressing up. A second field can only change a decision near the line. Away from the line the first number decides, and it decides correctly. What the index does is tell a user which of two pairs reported identically is the one to look at again — and near a tolerance that is the whole of the job.

assertItChangesDecisions therefore requires two things: that some tolerance sees at least a fifth of the decisions change, and that the two-number reading is never the worse of the two at any tolerance in the sweep. The second is the one that would catch a field that was merely adding noise.

Tightening the tolerance while holding the colorimetric difference is the case a laboratory would call the safe one.

What it is not #

It is not a metamerism index. The CIE has two, and both compare a pair under a second illuminant — they ask what a change of light does, which is a different question with a different answer, and a pair can score well on one and badly on the other. Nothing here changes the light.

It is not a better colour-difference formula. The ΔE00 is computed exactly as it always is, and the field sits beside it rather than replacing it. A specification that adopted it would keep every existing limit and add one line.

And it is not a population model in disguise. The index itself contains no observers at all — it is a weighted distance between two measured reflectances — and the two coefficients that turn it into a risk are the only place a population enters. A user who disagreed with the population model could refit those two numbers against their own and keep everything else.

What a report would look like #

A spectrophotometer’s output today is a reflectance curve, a set of tristimulus values under one or more illuminants, and a colour difference against a standard. The proposal adds one line:

The first line is what is printed now. The second is computable from the two curves already on the screen. The third is the first times the rule, and it is the number a person would actually want, because it is a statement about what somebody looking at the pair will report rather than about what the instrument reported.

Two properties make this cheap in a way that most proposed additions to colour specification are not. Nothing new has to be measured — the reflectance curves are already there, at the same wavelength grid, for the same reason. And nothing has to be agreed internationally before it is useful: a laboratory can compute it from its own instrument’s existing output and use it internally to decide which borderline pairs deserve a second look.

Who found it, and when #

The dependence itself is old. Colourists have known since the trade existed that a metameric match is a fragile match and that a match made with the same pigments is not — the practice of “matching by formula” rather than by measurement exists precisely because two ways of reaching the same measured colour are not equally safe.

What has never been available is a number for it that a specification could carry. The CIE’s metamerism indices, published in 1971 and revised since, answer the illuminant half and are quoted rarely: they are extra work, they need a second illuminant to be agreed, and the number they produce is in units nobody else’s tolerance is in. The observer half has a much thinner literature, largely because a population model was not something an instrument could be expected to hold.

Both of those objections are answered by putting the population in the coefficients rather than in the instrument. The device computes a distance between two curves; the two numbers that turn it into a risk are published once, like a set of parametric factors, and can be revised without changing anything a machine does.

The fit is the part that turns all of this into something a specification could carry, and its constant is the number to check.

Why the difference alone cannot do it #

It is worth being precise about why no improvement to the colour-difference formula could recover this, because the natural response to a formula that fails is to fix the formula.

A colour difference is a function of two points in a three-dimensional space. Both samples in every pair here have been reduced to those three numbers before the formula sees them, and the reduction is a projection: three integrals against three fixed functions, discarding everything orthogonal to their span. Two pairs that differ in the discarded part arrive at the formula as literally identical inputs.

So no function of the six numbers, however sophisticated, can distinguish them. Not ΔE2000, not a better weighting of it, not a uniform space nobody has invented yet. The information required to tell the two pairs apart was removed one step earlier, by the projection that turns a spectrum into a colour — and it is removed for a good reason, because that projection is what an observer does.

The only place the information still exists is upstream of the projection, in the two reflectance curves. Which is why the field has to be a spectral quantity, and why the instrument is the only thing in the chain that could compute it.

Where it stops #

The pairs are constructed. Real disagreements between a sample and a standard are not metameric blacks plus a tilt — they are a pigment that ran short, a substrate that was not the substrate, a batch of ink from a different supplier. Whether the index behaves the same way on those is not established here and is the obvious first thing to check with a drawer of real samples.

The population is a model, with five variates sampled independently that are not independent in people. The coefficients inherit whatever that model gets wrong.

The weight function is a choice among several defensible ones. A weight proportional to the illuminant alone, or to ȳ alone, or to the derivative of the colour-matching functions, would all satisfy the two design constraints and would give different coefficients. What is claimed is that a weighted spectral distance predicts the residual; which weight is best is a further question and this one was chosen for being computable rather than for winning a comparison.

Where the ladder goes next #

The immediate extension is the same field computed for a change of illuminant rather than a change of observer, which is the CIE’s metamerism index in this essay’s form. Both are risks that a colorimetric agreement will not survive contact with something the specification did not record, and they are almost certainly correlated — a pair that is fragile to a person is probably fragile to a lamp, because both fragilities come from the same spectral disagreement.

If they are correlated strongly enough, one number covers both, and the specification gains one line rather than two. If they are not, the disagreement between them is more interesting than either.