A tolerance's own unit rounds the corners

A margin costs a press its corners took the gamut of a coated CMYK press and shrank it by a safety margin — kept only the colours at least that far from anything the press cannot print — to see whether gradients held inside the shrunken press would tear at a narrow place. There was no narrow place: the press stayed in one piece until almost nothing was left of it. What a margin did cost was concentrated at the corners. A colour on a flat face of the press moves exactly the margin to get inside; the solid yellow, the sharpest corner, moved 3.7 times it, like the tip of a spike thirty-one degrees across, and every saturated end colour in the census moved at least 1.4 times.

That margin was a distance in CIELAB, because a distance transform on a CIELAB grid gives one, and the gradients held inside the shrunken press were held there by projection, the method a projection has no reason to detour had found free of the penalty’s traps. Nobody states a margin that way. A print buyer says keep every colour two ΔE₀₀ inside what the press can do, and ΔE₀₀ is not a distance in CIELAB: at a saturated yellow it counts several CIELAB units across chroma as one. The essay predicted that in the tolerance’s own unit the chromatic corners would round towards a face, because ΔE₀₀’s compression of chroma is exactly the blunting a spike needs, and that the dark corners — the yellows with half black in them, which already moved further in ΔE₀₀ than the solid ones — would stay sharp.

Half the price at the corners, and one spike left #

Measured in ΔE₀₀ against a margin stated in ΔE₀₀, six of the nine end colours move between 1.06 and 1.34 times the margin at one, two and four units. In CIELAB against a two-unit CIELAB margin every one of them moved further than a cube’s corner does. Every end colour’s ratio falls to between two fifths and four fifths of its CIELAB value, and to about half at the median. The solid yellow still moves 1.9 to 2.1 times the margin — the one corner in the census sharper than a cube’s in the tolerance’s unit. The dark corners round as well, against the prediction. What the tolerance’s unit costs instead is CIELAB: two ΔE₀₀ keep 68 per cent of the press where two CIELAB units kept 80, and move the solid yellow 15.7 CIELAB units inward, twice as far. The press stays in one piece.

• The chromatic corners round, as predicted, all but the sharpest.
• The dark corners round too, which the prediction said they would not.
• The price moves from the corners to the volume: a ΔE₀₀ margin is a thicker skin in CIELAB, thickest where chroma is high.

Every corner, in both units #

The end colours are the ten gradients’ ends from the census the earlier essays used: saturated overprints of two inks at full strength in stated proportions, two of them with half black. Each is moved to the nearest colour of the eroded press, and the distance moved is divided by the margin. A colour on a flat face scores one; a colour at the tip of a cone scores one over the sine of its half-angle, so the score says how sharp the corner behaves.

The squares, in CIELAB, sit between 1.7 and 4.6. They reproduce the earlier essay’s figures to within the grid — the solid yellow at 3.9 times a two-unit margin against its published 3.7 — because they are measured the same way this essay measures everything, to cell centres, so that the two units are compared like for like.

The circles, in ΔE₀₀, sit mostly between 1.06 and 1.34, and for every end colour at every margin the circle lies to the left of the square — at 0.41 to 0.76 of it, and at about half for the median end colour. Six end colours stay inside that band at all three margins: the yellow-greens with three quarters and half of yellow in them, the two yellows with a quarter and three quarters of cyan, the red of magenta and yellow, and the half-black green. Those are corners a tolerance in ΔE₀₀ barely charges for.

Three do not. The green of solid cyan and solid yellow moves 1.6 times a one-ΔE₀₀ margin and 1.3 at the larger ones — a small, sharp tip on a corner that is otherwise blunt. The half-black yellow moves 1.86 times at one unit and falls to 1.32 at four. And the solid yellow moves 1.91, 2.04 and 2.10 times, rising with the margin where its CIELAB ratio fell.

The corner each move implies #

Read as cones, the same numbers become angles. In CIELAB every end colour is sharper than a cube’s corner, from the solid yellow’s 30 degrees to 65 degrees for the yellow with a quarter of cyan. In ΔE₀₀ every one opens, to between 59 and 140 degrees, and all but the solid yellow open past a cube’s 71. The angles are an inference from one number each — a corner of a voxel gamut is a ridge or a wedge rather than a cone — but they give the move a shape.

Why the yellow opens least is a matter of which way its spike points. At the solid yellow a ΔE₀₀ unit is about five CIELAB units along chroma, about two across hue and about one and a half along lightness. A spike pointing straight out in chroma would be shortened more than twice as much as it is narrowed, and its angle would open by more than the yellow’s does. The yellow’s tip also points up the lightness axis — it is the light tongue of the gamut, at L* 89, the one place the press prints both light and saturated — and along lightness ΔE₀₀ shortens it by only about one and a half, less than it narrows it across hue. That reading is an inference, not a measurement of the spike’s direction; what is measured is that the yellow opens from 30 degrees to 59, and nothing else in the census stays under 71.

The dark corners were not different #

The prediction singled out the half-black yellows because in CIELAB they had already moved further in ΔE₀₀ than the solid ones: a two-unit CIELAB margin moved the half-black yellow 4.2 ΔE₀₀, the solid yellow 2.4. The expectation was that a margin stated in ΔE₀₀ would find those corners sharper still, since at low lightness ΔE₀₀ counts a CIELAB unit nearly in full.

They round. The half-black green moves 1.30, 1.32 and 1.29 times its ΔE₀₀ margin. The half-black yellow is sharp at one ΔE₀₀, 1.86, and blunts as the margin grows, to 1.57 and then 1.32. What the earlier numbers had measured was not that the dark corners were sharp in ΔE₀₀ but that a CIELAB margin was a large margin there — the CIELAB move it forced was worth many ΔE₀₀ units. A margin stated in the tolerance’s own unit does not ask those corners for more than it asks the face beside them.

What a tolerance margin costs in CIELAB #

The same number removes more of the press in ΔE₀₀. A margin of one keeps 82 per cent against 88; two keep 68 against 80; four keep 46 against 66. The skin a ΔE₀₀ margin removes is thicker in CIELAB wherever chroma is high, because there a ΔE₀₀ unit is several CIELAB units across. A tolerance has a grain found that a ΔE₀₀ tolerance pulled back into tristimulus values is a long thin shape with a consistent direction; pulled back into CIELAB at a saturated colour it is long along chroma and short across it, and a margin is that shape swept round the boundary. It is the same anisotropy a tolerance in the wrong coordinates found when a display primary’s chromaticity tolerance was pulled back to the emitter’s own knobs, and that what one number accepts found in the volume a single delivery tolerance admits: a round tolerance in one space is a long thin one in the next.

It is still one piece. At four ΔE₀₀, with under half the press left, the kept colours are one six-connected set. The earlier essay found no waist in the press for a CIELAB margin to cut, and a ΔE₀₀ margin does not find one either: a ΔE₀₀ ball is a stretched CIELAB ball, and stretching the margin along chroma trims the saturated tongues from their tips inward rather than pinching them across.

Every end colour moves further in CIELAB under the ΔE₀₀ margin, and the most saturated furthest: the solid yellow 15.7 units against 7.8, the red of magenta and yellow 11.1 against 6.2, the cyan-yellow green 10.3 against 6.2. Even the least saturated end colours — the yellow-greens with half and three quarters of yellow, and the half-black green — move about two thirds further. This is the corner price moved, not removed. A buyer who states a margin in ΔE₀₀ pays nearly the same at every colour in the tolerance’s own unit, and pays for it in CIELAB distance and volume at exactly the colours the CIELAB margin had charged most. The fifth ink buys a corner found that the volume an extra ink adds lands in those same saturated corners, which makes them the colours a margin in either unit takes from a four-ink press first.

Where the skin lies #

At L 50 half the plane is four ΔE₀₀ deep or more*, a broad core with a skin of even-looking bands round it. At L 85 six per cent is*, a sliver down the middle of the narrow light tongue towards yellow.

Side by side, the shapes are the same and the skins are not, and cell by cell the difference can be read as how many CIELAB units one ΔE₀₀ of depth is worth. Across the L 50 plane it is 1.9 to 2.4*, and there it does not grow with chroma — it is slightly larger nearer the neutral axis than at the plane’s saturated edge. Across the L 85 plane it grows from 1.3 near the neutral axis to 2.4 at the tip of the yellow tongue.* So a margin in the tolerance’s unit is a skin about twice as thick in CIELAB through the middle of the press, and in the light tongue it is thinnest at the pale tints and thickest at the saturated yellow — the colour whose corner the CIELAB margin had already charged most.

What this means for a margin in a specification #

State the unit, and expect the unit to decide what the margin costs. The same “two units inside the press” is a fifth of the press or a third of it, and a corner charge of four times or of twice, depending on whether the two are CIELAB or ΔE₀₀.

A ΔE₀₀ margin is fairer in the sense a buyer means. Every saturated end colour except one sits within a third of the margin in ΔE₀₀, so a buyer who asks for two ΔE₀₀ of safety gets roughly two at every colour, not two at a face and eight at a corner. A press is charged for the direction it barely moves found a tolerance charging a press for the direction in which the press is steadiest; here the same tolerance, used as a margin, charges the press evenly across its corners, and is correspondingly expensive in the one unit the press’s gamut is computed in.

The solid yellow still needs its own treatment. At twice the margin it is the one colour a ΔE₀₀ margin moves disproportionately, and it is also the colour a CIELAB margin moved most. Whatever unit a margin is stated in, the light yellow tongue is where a press’s gamut and a tolerance disagree most.

How the erosion was made exact #

A ΔE₀₀ margin is not a separable transform, so each printable cell’s depth — the smallest ΔE₀₀ from its centre to any face between a printable and an unprintable cell — was found by a local search over those faces. The search radius has to be large enough that no face beyond it can come within four ΔE₀₀, and the bound follows from how ΔE₀₀ is built: a step in lightness, chroma and hue is divided by $S_L$, $S_C$ and $S_H$ evaluated at the pair’s mean, and a rotation term can reduce the sum. The first version of the bound assumed that term could shrink a distance by at most $\sqrt{2}$, and took $S_L$ at the cell’s own lightness; a check against an unrestricted search on fifteen hundred cells found depths wrong by up to four hundredths of a unit. Near a hue of 275 degrees at high chroma the rotation term reaches 1.73 and can shrink a distance by a factor of 2.7, so a blue cell needs nearly twice a yellow’s radius. With the bound taken at the worst hue the radius can reach, the local search agrees with the unrestricted one on every sampled cell to float precision.

The press is the coated CMYK set at a 320 per cent ink limit used throughout, on a one-unit CIELAB grid of 439,844 printable cells with 61,236 boundary faces. The depths were computed for 306,119 cells; the rest are more than four ΔE₀₀ deep by the bound and did not need searching. An end colour’s move is to the nearest cell centre of the eroded press, as ΔE₀₀ measures nearness for the ΔE₀₀ margin and as CIELAB distance measures it for the CIELAB one.

What this leaves out #

ΔE₀₀ is not a metric. It does not obey the triangle inequality, and a margin in it is a set defined by a formula, not a ball in a space. That is why the erosion needs a direct search, and why “the nearest colour inside” has a meaning here only as the smallest formula value. The straight line is not the shortest gradient used ΔE₀₀ the other way, as a local rule for the cost of a small step integrated along a path. A margin defined by that path length would be an erosion by a true metric, and it is not the set measured here; how far the two differ at four units is not measured either.

The margin is fixed in ΔE₀₀ and the corner analysis reads it as a cone. The yellow’s opening angle and its direction are inferences from one ratio each.

The grid is one CIELAB unit, which at the saturated yellow is a fifth of a ΔE₀₀ unit along chroma. The one-unit margin’s ratios carry most of that quantisation; the two- and four-unit ones little.

Still open: whether the yellow’s spike points along lightness #

The solid yellow opened from 30 degrees to 59 where every other end colour opened past 71, and the reading offered for it is that its tip points up the lightness axis as well as out in chroma, so that ΔE₀₀ shortens it less than a pure chroma spike. That is a statement about the tongue’s direction, and it can be measured without any margin at all: fit the axis of the press’s light yellow tongue from its cross-sections at successive lightnesses, and compute the ratio of ΔE₀₀ length to width along that axis rather than along chroma.

The prediction is that the axis leans towards lightness by thirty to forty-five degrees, and that ΔE₀₀’s scaling along that tilted axis accounts for the 59-degree opening to within ten degrees. If it does not, the yellow’s persistence as a spike in ΔE₀₀ is something about the formula’s hue weighting at yellow — its hue function is near its lowest there, which makes $S_H$ small — rather than about the gamut’s geometry, and a margin stated in ΔE₀₀ would treat every yellow tongue that way, on any press.

A unit decides where a price falls #

The habit is about the unit a quantity is stated in.

The earlier essay measured a margin in the only unit a distance transform supplies, and found the price of a margin in the corners. That was true and it was a finding about CIELAB as much as about the press. Re-stated in the unit the margin is actually specified in, the corners stopped being where the price fell, and the price went to the volume — to the saturated skin a ΔE₀₀ unit makes thick.

The failure mode is to report a cost in a convenient unit and read it as a cost in the real one. A price computed in one unit and quoted in another has not been quoted wrongly; it has been moved, and where it lands depends on how the two units stretch against each other. Ask in which unit a buyer states the quantity before asking what it costs.