A tolerance is a region

A specification lists a tolerance per primary because a list is what a specification can hold. The object it is describing is not a list.

The claim #

The set of positions a display primary can occupy at a stated cost is a closed region, none of the three is round, and part of two of them lies outside the area a real primary can occupy.

• The red primary’s region is 15.1 times longer one way than another — 0.0021 in its narrowest direction and 0.0316 in its widest.
• Green is 6.6 times and blue 5.9, so a single ratio does not cover the three either.
• The areas differ by 4.5 times: green’s region is 1.32 × 10⁻³ square units of chromaticity, red’s 5.4 × 10⁻⁴, blue’s 2.93 × 10⁻⁴.
• Twenty-three of red’s forty-eight boundary directions and twenty-four of blue’s leave the realisable region. Green’s leave none.
• And the objective’s cheapest direction moves two primaries at once — red and green together, with blue carrying two per cent of it — which no per-primary tolerance can express at all.

What a free region is #

The design is the one this collection reaches when it asks for the three realisable primaries whose inverse is the best adaptation basis: red at (0.674, 0.326), green at (0.203, 0.743), blue at (0.137, 0.038), leaving 0.996 ΔE00 against a floor of 0.974 for a basis free to be any nine numbers at all.

Ask what happens if one primary moves and the other two are held. The adaptation cost rises, in every direction, at a rate that depends on the direction. Sweeping forty-eight directions in that primary’s own chromaticity plane and bisecting each outward until the cost has risen by one per cent traces a closed curve, and the curve is the tolerance.

It is not a circle and there is no reason it should be. The objective’s curvature over the six numbers is a 6×6 matrix with a condition number of 3,792, and restricting it to the two coordinates of one primary leaves a 2×2 with its own two eigenvalues. A level set of a 2×2 quadratic form is an ellipse, and a ratio of fifteen is what an eccentric one looks like.

The part that is not available #

The objective knows about adaptation and nothing else. In particular it does not know that a primary has to be a real light.

Every chromaticity inside the spectral locus is realisable — it is a mixture of real lights — and every chromaticity outside it is not. Twenty-three of the red region’s forty-eight boundary directions, and twenty-four of blue’s, land outside. Half of each of those two tolerances is a direction in which the cost does not rise and the primary cannot go.

Green’s region is entirely inside. That is not luck: green sits well inside the locus in the region where the curve bulges, so there is room in every direction, while red and blue sit close to the locus’s edges — red near the long-wavelength end where the locus is nearly a straight line, blue near the short-wavelength end where it turns sharply.

This is a second constraint of a completely different kind from the first, and the two have to be intersected rather than combined. A cost bound is a statement about performance; realisability is a statement about physics. Reporting the first alone gives a tolerance twice as generous as the truth on two of the three primaries.

Why one number per primary cannot work #

Three failures, and they are independent.

A region is not a radius. Quoting one tolerance per primary means quoting a circle, and a circle inscribed in an ellipse of ratio 15 wastes 93 per cent of the available room; a circumscribed one permits positions that cost far more than the bound. Neither is the tolerance.

The three primaries are not alike. Their ratios are 15.1, 6.6 and 5.9 and their areas differ by 4.5 times, so one specification cannot be reused across the three even if it were a shape rather than a number.

And the cheapest direction is not any primary’s. The flattest of the six eigen-directions has weights of 0.39 and −0.32 on red, −0.66 and 0.56 on green, and 0.00 and 0.02 on blue: it slides red and green together in opposite senses and leaves blue where it is. A design can move a long way along that direction — the condition number of 3,792 means sixty-two times further than along the stiffest — and no tolerance written per primary permits it, because per-primary tolerances treat the six numbers as independent and this direction is the statement that they are not.

The red region’s area and its radii do not describe the same ellipse #

The essay’s own account of the shape — a level set of a 2×2 quadratic form is an ellipse — makes the region’s area a consequence of its two extreme radii, and the three numbers published for red do not close.

An ellipse with semi-axes 0.0316 and 0.0021 has an area of 2.09 × 10⁻⁴. The area reported is 5.4 × 10⁻⁴, which is 2.59 times larger.

The gap is not a shape discrepancy. Working the other way, the ellipse consistent with the stated area and the stated ratio of 15.1 has semi-axes 0.05095 and 0.003374, and the published radii are 0.620 and 0.622 of those — the same factor on both axes, to three digits. The two descriptions differ by a uniform scale, not by an eccentricity, which rules out the region simply being non-elliptical: a non-quadratic level set would bulge in some directions and not others.

A uniform linear factor of 1.61 is, for a quadratic objective, a cost factor of 1.61² = 2.60 — so the area is the area of the region at a 2.6 per cent cost threshold, and the radii are the radii at one per cent. Nothing in the text settles which of the two was intended, and the areas are what the build’s own assertion compares. The ratios and the realisability counts are unaffected either way, since both are scale-free, so nothing in the argument turns on it; what turns on it is any later use of the areas as absolute room.

Most of the slack is not reachable one primary at a time #

The cheapest direction is not any primary’s is the third of the three failures, and the six eigenvalues put a number on how much it costs to ignore it.

Within a single primary, the widest-to-narrowest reach is 15.1 on red, 6.6 on green and 5.9 on blue — so the eigenvalue ratios of the three 2×2 restrictions are 228, 44 and 35. The full 6×6 problem’s condition number is 3,792, which is 16.6 times worse conditioned than the best of the three restrictions.

In the units a designer would use — how far the design can move — that is a factor of 4.1. The flattest direction over all six numbers permits a move 61.6 times the stiffest; the flattest direction available inside any one primary permits 15.1 times. So three quarters of the reach the objective allows exists only for moves that touch more than one primary, and a per-primary specification cannot express it, cannot permit it, and gives no sign that it is there.

That is a stronger statement than a circle inscribed in an ellipse wastes 93 per cent of the room, because the wasted room in that case is at least visible on a picture of one primary. This is room that does not appear in any per-primary picture at all.

The cheapest direction is three quarters green #

The direction itself is described as sliding red and green together, and its own components say the two are not equal partners.

Its weights are 0.39 and −0.32 on red, −0.66 and 0.56 on green, 0.00 and 0.02 on blue. As component lengths that is 0.505 for red against 0.866 for green — green moves 1.7 times as far — and as shares of the direction’s squared length it is 25.3 per cent red, 74.6 per cent green, and 0.04 per cent blue.

The blue figure is worth stating precisely because two per cent appears for it in the claim list. Two per cent is the size of blue’s largest component, 0.02, on a unit vector; blue’s share of the direction is the square of that, four hundredths of one per cent. Blue is not carrying two per cent of the cheapest direction. It is not in it, to four significant figures, and that is the sharper fact: the objective’s flattest direction is a rotation of the red–green pair with the blue primary held fixed.

Which makes the design advice more specific than the adaptation behaviour will follow along. The free room is overwhelmingly in green, in a direction coupled to red, and a manufacturer trading phosphor availability against adaptation has almost no latitude in blue on this axis and a great deal in the other two together.

The narrow direction, and what is in it #

The ratio of fifteen on the red primary is worth opening, because the two ends point at recognisable things and one of them is not available.

The widest directions, at radii of 0.0316 and 0.0283, run down and to the right from the primary at (0.674, 0.326) — and both are outside the spectral locus. The objective’s cheapest direction for the red primary is a direction a real light cannot occupy. The largest movement actually available is 0.0298, up and to the left, running along the locus towards shorter wavelengths.

The narrowest, at 0.0021, runs down and to the left, inward and towards the line of purples. That is the direction the objective minds most, it is a fifteenth of the widest, and it is entirely available.

So the two constraints are close to being in opposition on this primary: the direction performance permits is one physics forbids, and the direction physics permits freely is the one performance charges most for. Neither of the two is what a tolerance quoted as a single figure would describe, and a specification bounding only the gamut — which is what most of them bound — leaves the expensive direction unbounded and the cheap one irrelevant.

What was computed, and how #

The design comes from a search over the six chromaticity coordinates with two soft penalties — one for leaving the realisable region and one for falling below a stated share of the diagram — and the softness matters: an earlier version returned a large constant outside the feasible set, which gave the simplex nothing to walk down and reported the starting point unchanged at every gamut floor above sixty per cent.

The regions are traced by bisection: for each of forty-eight evenly spaced directions, a doubling search outward to bracket the rise and thirty-four halvings to locate it. Each boundary point is then tested for realisability separately, so the region and the physics are reported as two things rather than one.

The area is a shoelace sum over the boundary polygon, which is what a tolerance actually is if it is to be compared against another.

The assertion in the build makes three claims: that every region’s long axis exceeds its short by at least three; that every region closes, so that it is a tolerance rather than an unbounded direction; and that the three areas differ by at least a factor of two, so that one specification cannot cover them. The second is the one that would fail first if the design moved to a place where some direction cost nothing at all.

Why the design has so much room #

One per cent of 0.996 ΔE00 is a small budget, and the regions it buys are not small: red can move 0.032 in its widest direction, which is comparable with the distance between sRGB’s red primary and Display P3’s.

The reason is the finding this design came from: requiring an adaptation basis to be the inverse of three realisable primaries costs almost nothing, because the constraint barely moves the basis in the directions the objective can see — 0.068, against 0.891 for the constraint that requires it to hit three dichromat confusion points. A constraint that costs nothing is a constraint with slack in it, and the slack is exactly what these regions are made of.

The practical consequence is comfortable and worth saying plainly: a display’s primaries can be chosen for gamut, for phosphor availability, for power, or for whatever else drives the decision, and the adaptation behaviour will follow along. Nothing in this essay argues that anybody should design a display around its adaptation basis. It measures how much room there is not to.

What a wider gamut does to the room #

The design above is unconstrained in coverage: it minimises the adaptation residual and lets the gamut fall where it will, at fifty-four per cent of the diagram. A display that has to cover more is a display whose primaries are pushed outwards, and the question is what that does to the tolerances.

Requiring the coverage Rec. 2020 specifies costs very little in the objective — the residual goes from 0.996 to 1.018, which is two per cent — and this collection has measured that trade before. What it has not measured is the room left afterwards, and the answer is not the one the geometry suggests.

Traced around the wide design’s primaries — red at (0.686, 0.314), green at (0.137, 0.813), blue at (0.140, 0.037) — the regions get larger, not smaller: red’s area goes from 5.4 × 10⁻⁴ to 5.9, blue’s from 2.93 to 3.36, and green’s from 1.32 × 10⁻³ to 3.48 × 10⁻³, which is two and a half times. Pushed outward, the primaries sit where the objective is flatter, and there is more room rather than less.

What changes is how much of that room exists. Green’s free region went from having none of its forty-eight boundary directions outside the realisable area to having twenty-four, because green has been pushed from well inside the locus to sitting against it. Red stays at twenty-three and blue falls slightly, from twenty-four to twenty-one.

So the effect is real and it is not a shrinking. A wide-gamut design has more tolerance and less of it is available, and the two move in opposite directions on the primary the coverage requirement pushes hardest. A specification quoting an area would report the wide design as the more forgiving of the two, which on green it is not.

Where the model stops #

A chromaticity is not a primary. A real primary is a spectral power distribution with a width, and two lights at the same chromaticity are different objects for everything except colorimetry — for metamerism, for how a camera sees them, for how they behave across a population of observers. The six numbers here are what a primary’s chromaticity is, and a display specification also has to say what the spectrum is.

The objective is adaptation alone. A display’s primaries decide its gamut, its power efficiency, its metameric behaviour and its stability, and adaptation is one term among several. A tolerance from this objective is a bound on one of the things that could go wrong.

And a level set is not a manufacturing tolerance. The region here is the set of positions whose cost is within one per cent; a manufacturing tolerance is a distribution over positions with a yield attached. Turning the first into the second needs the shape and a target, and this supplies the shape.

The generalisation #

A tolerance is the level set of an objective, so it has the shape of that objective’s curvature — and a specification that lists one number per parameter has assumed a sphere.

The assumption is not usually stated and is nearly always wrong. Where the parameters interact at all, the level set is an ellipsoid with axes that are combinations, and the two errors that follow are opposite and both expensive: a conservative specification inscribes a sphere and throws away most of the available room, while a generous one circumscribes it and admits designs that miss the bound.

The way out is not to publish an ellipsoid, which no manufacturing process can use. It is to publish the objective, or a quadratic form standing in for it, so that a design can be checked rather than pattern-matched against a box. That is what a colour difference formula already is, in its own domain — a shape, agreed on, that anybody can evaluate — and the same move is available wherever a tolerance is currently a list.

The second half is the intersection. A level set of an objective is not a feasible region, and where the two differ the honest report is both. Half of two of the regions here is unreachable, which is invisible from the objective and obvious from the diagram, and the only reason it is reported at all is that the two were computed separately and compared.

Who found it, and when #

Tolerancing as a level set is standard in optical and mechanical design, where the objective is a wavefront error or a fit and the tolerance ellipsoid is a routine output of the analysis. Display specifications have not generally adopted it: the published primary tolerances in the broadcast and computer display standards are boxes or circles in chromaticity, quoted per primary, and the objective they are meant to bound is usually left implicit.

Where the discipline does think in shapes is colour difference, and it took a long time. A tolerance for a printed colour began as a box in three coordinates, became a sphere in a uniform space, and is now — in the specifications that take it seriously — an ellipsoid with a stated orientation, because the shape of what people notice is not round. The same argument applies one level up, to the tolerance on the device rather than on its output, and it does not appear to have been made there.

Where the ladder goes next #

Every measurement here has taken the shape of an objective and found that the shape carries the argument: which constraint is expensive, which parameter a design can be wrong about, which direction a budget should be spent in. What remains unmeasured is the shape of the objectives this collection has not written down — the ones a real display is actually designed against, of which adaptation is one term — and joining them is a larger job than any of these.