What one number accepts

A contract says the delivery shall be within one colour difference of the specified colour, and both parties understand what that means. The instrument reports a number, the number is compared against one, and the delivery passes or does not. What nobody draws is the set of deliveries the sentence has just accepted.

The claim #

A stated tolerance is one number and the set it accepts is a three-dimensional shape whose size and proportions are decided entirely by which colour it is applied to.

• The volume within ΔE₀₀ 1 varies by a factor of 1.0 × 10⁵ across sixty-four colours on a lattice through the sRGB cube.
• The shape is never round. Its longest radius is between 3.3 and 29.9 times its shortest, with a median of 6.9.
• The smallest set belongs to a near-black at L* 3.6 and the largest to a light blue at L* 62.3.
• Doubling the tolerance multiplies the volume by between 8.5 and 8.9, against the 8.0 a locally quadratic metric would give, so the shape scales and the residual is the curvature.

What a tolerance is being asked to do #

The sentence in the contract has to survive a journey. It is agreed between two parties who are thinking about a colour; it is handed to an instrument that reports three numbers; and it is enforced by a formula that turns six numbers into one. Every step is fine and the composition of them is where the trouble is.

A tolerance is already known to be several tolerances in this collection: twenty-four pairs of surfaces built to sit exactly on a ΔE2000 tolerance of one read from 0.69 to 1.93 in the other five formulae, so a contract that does not name its formula accepts a different set of deliveries depending on which one the laboratory owns. That is a statement about the menu.

This is a statement about one item on it. Hold the formula fixed, name it, and agree it. The set of deliveries the agreed formula accepts still depends on the colour, and it depends on it by five orders of magnitude in volume.

The shape, measured #

Pulling the tolerance back is direct. Stand at a colour, choose a direction in tristimulus values, and walk outwards until the difference formula reports the tolerance. The distance walked is the radius in that direction. Ninety-six directions distributed evenly over the sphere give the volume; the longest and the shortest of them give the proportions.

The volumes run from 5.6 × 10⁻¹⁰ to 5.7 × 10⁻⁵ in cubed tristimulus units. Expressed as a linear scale that is a factor of about forty-seven in each direction, which is the more legible form: a delivery of a near-black may differ from its specification by about a fortieth of what a light blue may differ by, for the same number in the same contract.

The reason is the previous essay’s, arriving from the other side. The price of a deviation is high in the shadows because the compression is steep there, so a small deviation exhausts the tolerance quickly and the accepted set is small. In the light and chromatic parts of the space the price is low and the set is large.

That is not a defect. It is the whole purpose of a perceptual difference formula: the tolerance is meant to be a constant amount of visible difference — the ambition every uniform space has been built for — and a constant amount of visible difference is a varying amount of physical difference. The defect is that nobody says so, and a manufacturer reading a specification has no way to convert the contract into an instrument’s repeatability requirement, which is what they actually need.

What the shape is not #

The second half of the measurement is the one with no benign reading at all.

The accepted set is never round and it is often extremely far from round. The median ratio of longest radius to shortest is 6.9 and the worst in this lattice is 29.9. A delivery may sit thirty times further from its specification in one direction than in another and be accepted in both cases.

The direction matters because instruments do not err isotropically. A spectrophotometer’s bandpass errs along a particular direction in the spectrum, which maps to a particular direction in tristimulus values. A press drifts along its ink-density axes. A display drifts along its primaries. None of those is the direction a shell is longest in, and whether a given drift is comfortably inside a tolerance or straddling it is a fact nobody has computed because nobody had the shape.

Doubling the tolerance is a useful check on the whole construction. If the difference formula were exactly a quadratic form locally, the shell would scale linearly and its volume would go up by eight. The measured factor is between 8.5 and 8.9, so the shell is scaling almost linearly and growing slightly faster than it should, which is the same curvature that makes two departures fail to compose by their sizes showing up in a measurement that shares no code with it, and it is the same shape as the bowl this collection measured elsewhere.

That the two agree is worth having. The composition argument and the tolerance argument share a premise — that the formula behaves locally like a quadratic form and departs from it by a measurable amount — and they are computed by machinery with almost nothing in common.

Which direction is tightest, and it is the same one everywhere #

The shape has an orientation as well as a size, and the orientation turns out to be more regular than the size.

The shortest radius points the same way at every colour measured: a direction that raises X while lowering Y, which is very nearly the red-green axis of CIELAB. The metric is tightest there because ΔE₀₀ has no discount for a red-green movement — its chroma weighting acts on the magnitude of the chroma and not on which way it points, and the a* axis is where a small tristimulus change produces the largest chroma change.

The longest radius does not point the same way at every colour, and where it points is legible. On a neutral it is the achromatic direction — all three tristimulus values moving together — because a pure lightness change on a neutral is the cheapest thing the formula charges for at that lightness. On a saturated blue it is the Z direction, because ΔE₀₀’s chroma weighting divides by the chroma and a colour that already has a lot of it is charged less for more.

So a tolerance is tight in one fixed direction and loose in a direction that depends on the colour, and neither fact is stated anywhere in a specification. The consequence for a manufacturer is direct: a drift that happens to lie along the red-green direction consumes the tolerance an order of magnitude faster than one of the same physical size along the loose direction, and which one a given process produces is a property of the process rather than of the colour.

What it means as an instrument requirement #

The number a supplier actually needs is not the tolerance. It is how repeatable the measurement has to be for the tolerance to be enforceable, and that is the shell’s shortest radius.

At a near-black the shortest radius is 1.22 × 10⁻⁴ in tristimulus units. At a near-white it is 2.64 × 10⁻³. Those differ by a factor of twenty-two, and an instrument’s absolute repeatability — its stray light, its dark signal, its reference drift — does not.

Read as a fraction of the sample’s own signal the picture reverses: the near-black’s shortest radius is 3.1 per cent of its own luminance and the near-white’s is 0.30 per cent. Both readings are true and they answer different questions. The instrument’s error is closer to constant in absolute terms than in relative ones, which is why the absolute comparison is the one a laboratory should use, and it is the comparison that says the dark end is twenty-two times harder.

A specification that wanted to be honest about this would either widen the tolerance in the shadows or state it in a different quantity there — a lightness difference, or a density, both of which are what the trades that work in the shadows actually use. That the printing industry specifies black by density rather than by ΔE₀₀ is usually explained as tradition. It is also correct.

Where this bites in practice #

Three consequences follow and each is a thing a specification could carry and does not.

An instrument requirement cannot be derived from a colour tolerance. A supplier asked to hold ΔE₀₀ 1 needs to know how repeatable their measurement must be, and the answer is the shell’s shortest radius at their colour, not some average. On a dark colour that is a much tighter requirement than on a light one — and the shadows are where the compression stops compressing — and the specification looks identical in both cases.

A tolerance on a dark colour may be below the noise floor. The smallest shell in this lattice has a shortest radius of 1.22 × 10⁻⁴ tristimulus units, which is inside what an ordinary instrument’s repeatability produces on a black patch once stray light and dark signal are counted. A contract quoting one unit there is not a tight requirement; it is an unmeasurable one, and the honest response is to widen the number or to specify in a different quantity.

A tolerance is not symmetric between the two parties either. A supplier held to ΔE₀₀ 1 at a dark colour and at a light one is being held to two requirements that differ by a factor of twenty-two in what their instrument must resolve, and by a factor of about forty-seven in what their process must hold. Neither factor appears in the contract, and neither party can compute it from what the contract says.

And a tolerance is not a budget that can be divided. A chain with three stages, each allotted a third of a unit, is only meaningful if the three stages’ errors compose — and they compose by the cosine rule in a metric with an axis ratio of up to thirty, not by addition and not by quadrature.

The anisotropy and the composition problem are the same fact stated twice. A metric whose unit ball is thirty times longer one way than another is a metric in which the Euclidean intuition about directions fails badly, and the angles measured in the previous essay are what that failure looks like when two real departures are put into it.

What was computed, and how #

The shell is found by bisection along each of ninety-six directions distributed by the golden-angle spiral, which gives an even coverage of the sphere without a preferred axis. The volume is the mean cubed radius times the volume of the unit ball, which is exact for a star-shaped region sampled evenly in direction and is a slight underestimate for a very elongated one.

The sections drawn in the figures are computed separately and honestly: the plane is the one containing the longest and the shortest radius, walked in seventy-two steps. The census’s own directions cannot be drawn as an outline because a spiral over a sphere has no order in a plane, and drawing them as though it did would produce a picture of the sampling rather than of the shape.

The colours are a four-by-four-by-four lattice in sRGB with coordinates at 0.05, 0.3, 0.6 and 0.95, which spans the cube without sitting on its faces. A denser lattice moves the extreme values slightly and moves no conclusion; the factor of a hundred thousand is a property of the corner of the space rather than of the sampling.

What the standards already do about it #

The industrial standards do not quote one number, and the reason they do not is this measurement arrived at from the practical side fifty years earlier.

A textile or automotive tolerance is typically stated as three: a lightness tolerance, a chroma tolerance and a hue tolerance, often with the hue one much tighter than the other two. That is an ellipsoid with its axes along the cylindrical coordinates of CIELAB, and it exists because a single ΔE*ab number was found in practice to accept deliveries that looked wrong and reject deliveries that looked right.

ΔE₀₀ is the descendant of that experience: its three weighting functions are exactly an attempt to make one number behave like those three, by dividing each component difference by a term fitted to how large a tolerance that component needs at that colour. So the shape measured here is intentional and most of it is the point of the formula.

What is not intentional is where the shape is stated. The standards express it in CIELAB, which is the space the formula lives in and is not the space anybody’s process drifts in. A press drifts in ink densities, a display in primary drive levels, an instrument in spectral response — all of them nearer to tristimulus values than to a hue angle. Expressing the accepted set in tristimulus values is a change of coordinates and nothing else, and it moves the anisotropy from a factor of about three to a factor of up to thirty, because the cube root had been quietly removing some of it.

The formula’s own weighting functions are a partial inverse of the metric, and the coordinate change puts the rest of it back.

The check that the shape is a shape #

One thing has to be established before any of this is worth reporting, and it is easy to miss because it looks like a formality.

The accepted set is described here as a shape with a size, which presumes it is star-shaped about the specified colour — that a single radius exists in every direction, so that walking outwards crosses the tolerance once and never comes back. A difference formula with a non-monotone term could fail that, and ΔE₀₀ has non-monotone terms in it.

The bisection used here assumes the property rather than testing it, so it was tested separately: walking outwards in fine steps along a thousand directions at four colours and counting sign changes finds one crossing in every case. The set is star-shaped over the range measured, which is what makes a radius meaningful and a volume computable.

That check matters more than its result. The same construction applied to a formula with a hue-rotation term evaluated at a colour near the rotation’s own discontinuity would not be star-shaped, and the volume reported would be a number with nothing under it. Nothing warns of that; the bisection converges either way and returns whichever crossing it happened to bracket.

Where the model stops #

The shell is measured against ΔE₀₀ and the whole of it would move under a different formula. ΔE*ab gives shells that are much rounder in CIELAB by construction and no rounder in tristimulus values, because the cube root is the source of most of the anisotropy and every formula in the menu has it.

The volumes are in cubed tristimulus units, which is not a quantity anybody has an intuition for and is not comparable to anything outside this essay. The ratios are the transferable part.

And this is a tolerance around a specified colour, which is the shape a delivery contract takes. A tolerance between two arbitrary samples is a different object — a boundary through the space of pairs rather than a surface around a point — and the two should not be confused.

The generalisation #

The habit is about a scalar constraint on a vector quantity.

A specification that says within one unit has an implied ball in it, and the ball is only a ball if the unit is a Euclidean length in the coordinates the thing is actually delivered in. Whenever the unit is perceptual, learned, weighted or fitted, the accepted set is a shape, and its size and eccentricity are functions of the operating point.

The move is to draw the set once, at the extremes of where the specification will be applied. It is a few lines of bisection and it answers questions the specification cannot: how repeatable the instrument must be, which drift directions are dangerous, and whether the requirement is achievable at all in the corner of the space where it is tightest.

The failure mode is not that the specification is too tight or too loose. It is that it is both, in different places, while reading as one requirement — and the place it is tightest is invisible, because the number there is the same number as everywhere else.

Who found it, and when #

That colour tolerances are ellipsoids rather than spheres is the content of MacAdam’s 1942 measurement and of every uniform space built since; the industrial forms of it are the tolerance ellipses in textile and automotive standards, which explicitly quote separate lightness, chroma and hue tolerances rather than one number, precisely because one number does not describe the shape.

What is unusual here is the direction of the pull-back. Those standards express the shape in CIELAB, which is the space the formula lives in. Expressing it in tristimulus values is what a manufacturer needs, because tristimulus values are what an instrument reports and what a device drifts in, and the anisotropy is much larger there than in CIELAB — the cube root having removed some of it on the way.

Where the ladder goes next #

Every quantity in this round so far has had a derivative, which is what let it be priced by perturbing something. One does not. A file is written on a lattice, a lattice has no derivative anywhere, and the error it introduces can therefore be bounded and never propagated.