Two tolerances do not meet in a tolerance

Two requirements met separately and two requirements met together are different problems, and the difference is geometric.

The claim #

An intersection of tolerance regions is not the smallest of them and is not their average. It is smaller than any, its longest direction is generally none of theirs, and a specification written as a list of separate limits describes a set that contains points satisfying nothing.

• Regions here are long and thin. A camera dye’s adaptation region has a ratio of about three between its longest and shortest directions; a display primary’s runs to 15.1.
• They cross at an angle rather than nesting. The adaptation region for a display’s red primary has its longest direction at 330 degrees and its gamut region at 0 — thirty degrees apart, and the realisability boundary at 143.
• The intersection is a fraction of the smallest part: five per cent for a display’s red primary, sixteen for its blue, seventy-four for its green.
• More than one requirement holds every boundary, in different directions, so which specification decides how far this can move has no answer without a direction attached.
• And a list of tolerances is a box. A box drawn round an intersection contains corners that satisfy neither requirement, which is what a supplier reading a specification will build to.

What a tolerance region is here #

A tolerance is a region rather than a radius, and the regions in this essay are level sets: the set of positions a device parameter can take before a stated requirement gets a stated amount worse.

For a camera dye that plane is a centre wavelength and a bandwidth, both in nanometres. For a display primary it is a chromaticity — two coordinates of the same kind. In both cases the two axes are commensurable, which is what makes the shape of a region mean something rather than being a statement about a choice of units.

Every region here is star-shaped about the same centre — the designed device — so the intersection can be computed exactly: sweep all of them along the same forty-eight directions, and the intersection’s radius along a direction is the smallest of theirs along it. No polygon clipping, no approximation, and the result is itself star-shaped and can be intersected again.

Why an intersection is so much smaller #

The arithmetic is elementary and the consequence is not obvious until it is drawn.

Take two ellipses about the same centre, each three times longer than it is wide, with their long axes forty-five degrees apart. Each has an area proportional to three; their intersection is a rounded square whose radius in every direction is the smaller of the two, and its area is under half of either. Nothing about that is special to ellipses or to forty-five degrees. The more elongated the regions and the larger the angle between them, the smaller what survives.

On the devices here the numbers are: five per cent of the smallest part for a display’s red primary, sixteen for its blue, seventy-four for its green. The green is the outlier and the reason is visible in the figure: its adaptation region is the least elongated of the three, at a ratio of 6.6 against red’s 15.1, so it has less to lose to a crossing.

A specification that quotes the smallest requirement’s tolerance is therefore wrong by a factor of up to twenty on this device, and wrong in the dangerous direction: it permits parts nothing accepts.

There is a second effect on top of the loss of area, and it is the one that costs a manufacturer most. The intersection’s longest direction is generally neither region’s. Two long axes thirty degrees apart leave a survivor whose long axis is between them, so the cheapest way to be wrong — the direction a process can drift furthest in — is a combination of parameters that neither requirement, read alone, identifies.

That matters because a manufacturing process does not drift in an arbitrary direction. A deposition thickness, a phosphor loading, a temperature: each moves the device along its own line through the design space, and whether that line is cheap or expensive depends on the intersection’s orientation rather than on either requirement’s. A process aligned with one requirement’s cheap direction can be aligned with the intersection’s expensive one.

The box a specification writes #

The practical failure has a shape and it is worth drawing in words.

A specification says the red primary shall be within ±0.004 in x and ±0.006 in y, and separately the display shall cover at least ninety-five per cent of the intended gamut. Both sentences are true statements about the design. Together they describe a rectangle, and the corners of that rectangle are the points furthest from the centre in the diagonal directions — which is exactly where two crossed regions have least room.

The bottom row of that figure is the honest set of numbers and it is still a rectangle. The half-extents of a region along its two axes are a bounding box, and a bounding box always contains the region rather than being contained by it. For the camera’s blue dye the box is ±3.1 nanometres of centre by ±7.3 of width; the region inside it is a narrow diagonal sliver, and the box’s corners are outside every requirement.

The repair is not more numbers. It is a different kind of statement — a region, or a quadratic form, or a small table of allowed combinations — and specifications do not usually have a place to put one.

It is worth putting a number on how bad the box is rather than only asserting it. For the camera’s blue dye the region’s area is a fraction of its bounding box’s — the box is ±3.1 by ±7.3, an area of about 90 square nanometres, and the region inside it is a sliver of roughly a third of that. So two thirds of what the specification permits is outside what the device tolerates, and every one of those points is in a corner.

For a display’s red primary the same comparison is worse, because the region is more elongated: the bounding box of the intersection is several times its area. The general statement is that the box-to-region ratio grows with elongation and with the angle between the crossing regions, which are exactly the two properties that make a tolerance interesting. It is the same arithmetic that makes a difference in one space a different difference in another: a set described in one set of coordinates and read in another loses whatever the coordinates were not aligned with.

Which requirement holds which part #

The second thing a list cannot express is that the answer to what limits this changes with direction.

For a display’s red primary, the boundary is held by whether a light of that colour exists at all in forty-eight per cent of directions, by the adaptation cost in forty-two, and by gamut coverage in ten. For the green, it is adaptation in sixty-nine and gamut in thirty-one. For the blue, realisability in fifty and adaptation in thirty-one.

One requirement never holds any part of any boundary: the share of real surfaces the display can show. It is slack everywhere, by a wide margin, which is worth knowing before anybody spends effort tightening it — and is a null result of exactly the kind the camera’s dyes produced for throughput and separation one essay earlier.

The realisability boundary is the strangest of the four because it is not a cost at all. A chromaticity outside the spectral locus is not a colour, so no emitter of any kind produces it: it is a hard wall rather than a level set, and it holds half of the red primary’s boundary and half of the blue’s. Half of a tolerance region lying outside what a real primary can occupy was the previous round’s finding, and this is the same fact with the other three requirements drawn on top of it.

What was computed, and how #

Four costs, each written so that smaller is better and each normalised to its own value at the designed device, so that a one per cent rise means the same thing in all four and no weighting has to be invented. Inventing one would be the ordinary thing to do and would make every number here a statement about the weights.

The forty-eight directions are shared by every region of a given parameter, which is the design decision that makes the intersection exact rather than approximate. A direction in which a requirement does not reach the stated rise inside the search’s reach is reported as capped rather than as a large number: an unbounded direction is a real answer, and quoting a boundary the search invented would be a fictional one.

The area of a star-shaped region comes from the shoelace formula on its boundary polygon, which is exact for the polygon and approximates the region to the resolution of forty-eight directions. That is fine for a ratio between two regions sampled the same way, and it is why every number here is reported as a ratio.

The one place a list is right #

There is a case where the list is correct, and naming it is the fastest way to see why the other cases are not.

If the regions nest, the list is exactly right. When one requirement’s region is entirely inside another’s, the intersection is the smaller region, the binding requirement is the same in every direction, and quoting its two half-extents describes it as well as anything does.

Nesting happens when two requirements are essentially the same requirement at different strengths. A gamut coverage of ninety-five per cent and a gamut coverage of ninety-seven nest by construction. So do a colorimetric residual and a stricter version of itself. A specification whose requirements nest is a specification with one requirement in it and some redundancy, which is a reasonable thing for a document to contain and is not what any of the devices here have.

The test is one line and needs no new arithmetic: compare each region’s area with the intersection’s. If the intersection equals the smallest, they nest. Here the ratios are 0.05, 0.16 and 0.74, and none of them is one.

Where the model stops #

Four requirements is not all of them. A display’s primaries are chosen for power efficiency, for the availability of an emitter, for stability over temperature and time, and for cost; a camera’s dyes for manufacturability and for how they age. None of those is here, and every one of them would cut the intersection further.

The regions are two-dimensional slices of a six-dimensional problem. Moving one primary with the other two held is the right question for a tolerance on that primary and is not the whole design space; a display whose three primaries all drift together is a different object, and its tolerance is a region in six dimensions whose two-dimensional shadows are wider than these.

The requirements are not independent of one another. Gamut coverage and realisability are related — a primary pushed outward for coverage is pushed towards the locus — and treating them as two separate regions to be intersected is correct as geometry and slightly redundant as engineering. Nothing here decomposes them into independent factors, and a design study would want to.

And a level set is not a failure boundary. A one per cent rise in an adaptation residual is not a defect; it is a stated budget, and where to set it is a decision this collection cannot make. What survives the choice is the shape, which scales with the budget, and the ordering of which requirement binds where, which was checked at one per cent and at twenty and does not move.

The generalisation #

The rule is stateable in one sentence and is violated in almost every specification document.

A conjunction of requirements is an intersection of sets, and a list of intervals is a box that contains it. Writing the requirements separately is not merely incomplete: it is wrong in the permissive direction, because the box’s corners are the places where two elongated regions have least room and a list gives them away for free.

The second half is about what to do instead, and it is more awkward. The honest object is a region, and a region is hard to put in a document — which is presumably why nobody does. The cheapest useful improvement is not a region but a diagonal: state the two tolerances and the one combination that is not allowed, which for a crossed pair is the corner. That is one extra sentence and recovers most of the loss.

The elongation has a cause in each case and the causes are worth separating. A gamut region is long because area is a product: moving a vertex along the line joining it to the opposite edge changes the area, and moving it across that line barely does. An adaptation region is long because the objective’s curvature over a display’s six numbers has a condition number of 3,792, so a level set of it is nearly degenerate. Two different mechanisms, two different long directions, and no reason for them to agree — which is precisely why the intersection is small.

And the third half is a reading habit. When a specification says and, ask what the requirements’ regions look like and whether their long directions agree. If they nest, the list is nearly right. If they cross, the list is wrong by a factor that grows with how elongated they are — which, on a device whose objective has a condition number in the hundreds, is a large factor.

What this says about the camera #

The camera in the previous essay is the cleanest case, because its four requirements are genuinely different in kind and three of its regions are enormous.

Throughput and separation reach the edge of the search in every direction on every dye, so they contribute nothing to any intersection: geometrically they are the whole plane. Adaptation and the Luther residual are the two that cross, and they cross at a substantial angle — adaptation is nearly blind to a dye’s width while the Luther residual cares about it a great deal, so their long axes are close to perpendicular.

Two perpendicular long thin regions intersect in something close to a rectangle, which is the one case where a list of tolerances is nearly right. The camera is therefore the device where the specification-as-a-list works best, and the display — whose adaptation and gamut regions are thirty degrees apart rather than ninety — is where it works worst.

That is a useful thing to be able to say in advance rather than after building the wrong specification: the angle between two requirements’ cheap directions predicts how badly a list will describe them.

Who found it, and when #

The geometry is elementary and the practice is old: interval arithmetic, tolerance stack-up analysis and worst-case-corner analysis are standard in mechanical and electronic engineering, and the failure of a box to describe an intersection is precisely why Monte Carlo tolerance analysis replaced corner analysis in circuit design.

What has not travelled is the picture. Optical and colorimetric specifications are still written as lists of intervals — a chromaticity tolerance, a coverage requirement, a stability figure — and the intersection they describe is left to be discovered by whoever builds to all of them at once. The arithmetic that would draw it is a bisection along forty-eight directions, which is a minute of computer time.

Where the ladder goes next #

Four requirements have been drawn in the coordinates a specification is written in. Nobody manufactures in those coordinates.

What a maker of a display primary controls is a peak wavelength and a bandwidth, and the map from those two numbers to a chromaticity is strongly anisotropic — on the red primary its condition number is over eleven thousand. Carrying a tolerance region across that map changes its shape, changes which direction is cheap, and shows that most of a region drawn in chromaticity is a colour no single-peak emitter makes at all.