A tolerance cannot cross a condition

An earlier group of essays here established that a change of light is exactly a 3×3 matrix on tristimulus values, and that the part of it a gain can remove is whatever is diagonal in the axes the gain is applied along. That result is the reason the obvious repair here looks so promising.

Two measurement conditions differ. They differ by a known amount on a known set of patches. Fit the matrix, publish it, and let anybody convert.

The claim #

A change of measurement condition is not a change of light, even though the only thing that changed is the light. It adds a term proportional to the substrate showing through rather than to what the patch reflects, and a linear map on three numbers cannot produce a term of that shape.

• The best least-squares 3×3 over the whole patch set leaves 23 per cent of the mean disagreement — ΔE00 0.85 from 3.71 — and that is the best possible matrix, not a published one.
• It makes five of the seventeen patches worse. The solids, which barely disagreed to begin with, go from ΔE00 0.5 to 2.0.
• The reason is the shape of the extra term. The luminescent light comes from the paper under the ink, so it is largest where there is least ink and zero where there is most.
• That is neither multiplicative nor additive in the ordinary sense. It is additive with a coefficient that runs the opposite way to the patch’s own reflectance.
• And the test set matters. On six white sheets alone a 3×3 succeeds, because six near-white samples span barely one direction and a matrix has nine parameters. The first version of this test used exactly that set.

The test set that fails to test anything #

The first attempt at this measurement used the six stocks: an unbrightened sheet, four brightened papers and a laundered shirt. The best 3×3 between the two conditions reduced their mean disagreement from 5.7 to 0.6, leaving 11 per cent, and the obvious conclusion was that a matrix works.

It does not. What that result measures is the dimensionality of the sample set.

Six white sheets differ from each other along essentially one direction — more or less blue at nearly the same lightness — so the map from one condition to the other has to be right along one direction and is unconstrained in the other two. A 3×3 has nine free parameters. Fitting nine parameters to a one-dimensional family and finding a good fit says nothing whatever about matrices.

The repair is a sample set with chromatic range, and the honest one is the set the question actually arises about: patches printed on the brightened stock. Four inks at four coverages plus the bare sheet gives seventeen patches spanning the printable gamut, and every one of them sits on the same substrate — which is the arrangement a press profile is built from.

What the extra term actually looks like #

Take one patch: a tint of cyan at coverage a on a brightened sheet. Under the ultraviolet-excluded condition its radiance is the ordinary halftone arithmetic — an area-weighted mixture of bare paper and inked paper.

Restore the ultraviolet and a second term appears. The fluorophore is under the ink, so on the covered fraction the excitation has to pass through the ink film on the way in and the emission has to pass through it again on the way out; on the uncovered fraction neither does. The extra light is therefore

(1 − a) · e + a · e · t(λ)·(excitation through t)

which for a strongly ultraviolet-absorbing ink is very nearly (1 − a) · e.

So the term is proportional to the paper showing through. A bare sheet gets all of it. A twenty per cent tint gets four fifths. A solid gets essentially none, because printing ink absorbs hard below 400 nanometres — the vehicle does, quite apart from the pigment, which is why every ink behaves much the same there and why a heavy tint kills a sheet’s glow whatever colour it is.

That is the shape a matrix has to reproduce, and it cannot. A linear map M sends a patch’s tristimulus values v to Mv; the correction it applies is (M − I)v, which is proportional to v. The correction actually needed is proportional to 1 − a, and 1 − a is not a linear function of v — a dark cyan solid and a light cyan tint differ in v by a factor of several, and in 1 − a by a factor that runs the other way and is not proportional to anything.

Which patches the fit damages, and why that is the finding #

The mean tells one story and the per-patch numbers tell a better one.

Before any correction, the bare sheet disagrees by ΔE00 7.28, a twenty per cent cyan tint by 6.07, a seventy per cent tint by 2.25, and the cyan solid by 0.52. The gradient is the whole mechanism in one column: the more ink, the less disagreement, because the ink is blocking the excitation.

After the best 3×3, the bare sheet is at 0.96 and the cyan solid is at 2.00. Magenta goes from 0.48 to 2.17. Yellow from 0.05 to 1.16. Black from 0.37 to 1.54.

The correction has spent the solids to buy the tints. It has to: a least-squares fit minimises a sum, the light patches carry most of the error, and the only way a proportional correction can help them is by applying a correction proportional to their values — which lands on the solids too, where nothing was wrong.

That is the result worth carrying out of the essay, and it is invisible in the mean. A workflow that applied this matrix would fix a problem it had on the paper and create one it did not have on the solids, and its validation report — mean ΔE00 down from 3.7 to 0.9 — would look like a success.

The gradient is linear in the paper showing, to within a per cent #

The mechanism is argued from the physics and asserted from a column of four numbers. Those four numbers are enough to test it, and it passes about as cleanly as a four-point test can.

If the extra term is proportional to the uncovered fraction, the disagreement on the cyan ramp should be a straight line in 1 − a. Fitting one to the four published values:

The least-squares line is ΔE00 = 6.93(1 − a) + 0.39, with R² = 0.997. Anchored instead at the two ends — which uses no fitting at all — it reproduces the twenty per cent tint to 2 per cent and the seventy per cent tint to 13.

So the essay’s structural claim is not only argued, it is measured, and the measurement is already on the page. The intercept is the second half of it: 0.39 units of disagreement survive at full coverage, which is the part of the extra light that got through the ink film twice — small, non-zero, and exactly what the two-attenuations term in the arithmetic predicts.

That matters for the argument’s force. A correction proportional to a patch’s tristimulus values has to be proportional to something that runs with the ink; this runs against it with a correlation coefficient of 0.999. The two families are not merely different — they are very nearly orthogonal on this ramp.

The fit nearly breaks even on the patches it is judged by #

The per-patch column is the essay’s strongest evidence and it can be added up, which sharpens it.

Across the five patches the essay itemises, the correction buys 6.32 units and pays 5.45 — a net gain of 0.87. All of its real work is on the tints in between, which are not printed. The five patches a reader is shown are very nearly a wash.

And the damage is remarkably uniform: the four solids rise by 1.48, 1.69, 1.11 and 1.17 — a spread of half a unit around a mean of 1.36, on patches whose original errors ranged over a factor of ten, from 0.05 to 0.52. The correction applies almost the same absolute penalty to every solid regardless of how wrong it was, which is the signature of a term proportional to something the solids all share rather than to their individual errors.

Read as ratios the same fact is more striking. Yellow goes from 0.05 to 1.16 — a factor of 23 — and it is the patch that was closest to perfect before the correction. The fit damages most, in relative terms, exactly what it had least reason to touch.

Two numbers about the fit’s overall gain #

The mean falls from 3.71 to 0.85 across seventeen patches, which is a total of 48.6 units of error removed. The bare sheet alone accounts for 6.32 of that — 13 per cent from one patch of seventeen, and it is the patch furthest from anything a job is printed on.

That is worth setting beside the failure. A correction that removes half its error from the tints, an eighth from the bare substrate, and adds error to every solid is not a correction that has half worked; it is one that has redistributed error from the region a profile is least interested in to the region it is most interested in.

Why six white sheets were not too few #

The essay’s account of the first attempt — nine parameters fitted to a one-dimensional family — is right in substance and worth stating precisely, because the naive count does not support it.

Six samples with three coordinates each give eighteen equations for nine unknowns, which is comfortably overdetermined. A fit that good is not a case of more parameters than data.

What makes it vacuous is the rank rather than the count. Six near-white sheets vary along essentially one direction, so of the eighteen equations only about six carry independent information and the remaining twelve are near-duplicates of them. The matrix is then determined along one direction and free along two, and a free direction costs nothing on a set that never goes there.

The printed set leaves 23 per cent against the white sheets’ 11 — a factor of 2.2 — and that factor is the price of asking the matrix to be right in three directions instead of one. The right diagnostic for a fit like this is not how many samples there are but how many directions they span, and it is the one the first attempt did not run.

What distinguishes this from a change of light #

The comparison with the earlier result is exact and it is worth setting out, because both are “the light changed” and only one is a matrix.

A change of illuminant multiplies the whole spectrum before it meets the sample. Every sample’s radiance is scaled band by band by the same factor, and on a three-dimensional set of reflectances the resulting map on tristimulus values is exactly a 3×3 — proved, and checked at 10⁻¹³.

A change of measurement condition on a fluorescent substrate does that and adds a term that did not pass through the sample’s reflectance at all. The added light is the substrate’s, modulated by whatever is on top of it, and it enters the tristimulus values as an addition rather than as a multiplication.

The distinction has come up twice before on this site and each time the additive term was the one nothing could remove. The interface reflection is a pedestal — additive, sample-independent, and no gain touches it. A gloss pedestal takes ΔE00 5.05 to 4.50 under the best fitted gain, which is eleven per cent, while every multiplicative change in the earlier census gives up ninety per cent to a gain that is not fitted at all.

This one is worse than a pedestal, because a pedestal at least has the same size on every patch. This term’s size runs inversely with the patch’s ink coverage, so it is not even a constant offset that could be subtracted.

What the four conditions differ by is a lamp rather than a formula, and the short-wave half of each is where the whole disagreement lives.

What would work instead #

Nothing here says the conversion is impossible — only that it is not a 3×3.

The correct conversion needs the coverage, which the tristimulus values do not carry and the separation does. A press knows how much of each ink it laid down, and the separation that produced those numbers is not unique; a correction applied in the device’s own coordinates, with the area coverages as inputs, can produce a term proportional to 1 − a because 1 − a is one of its arguments. That is a four-dimensional to three-dimensional map rather than a three to three, and it exists.

Or the substrate can be measured under both conditions and the ink model can carry the rest. If the halftone arithmetic is being computed anyway — which it is, in any profile-building tool — then the fluorescent term can be computed from the substrate’s two measurements and the coverage, rather than fitted.

Or the measurement can simply be repeated. Building the profile under the condition the job will be judged under costs a re-measurement and no modelling at all, and it is what the printing standards recommend — the same conclusion the instrument’s own condition forces on everybody who reads it carefully.

None of those is a matrix published in a table that anybody can apply to anybody’s numbers — the same disappointment no matrix is right everywhere delivers about profiles, which is the thing the industry would most like to have and the thing this essay says does not exist.

What was computed, and how #

The patch set is the bare sheet plus four coverages of each of four process inks on the same brightened substrate: seventeen samples, each evaluated bispectrally under both conditions.

The halftone arithmetic is two states area-averaged in radiance — covered and uncovered — which is the Demichel treatment this collection already uses. The addition is that the fluorophore sits under the film, so the covered fraction’s excitation is attenuated once on the way in and its emission once on the way out.

The ink’s short-wave absorbance is one stated number for all four inks rather than a fitted band per ink, and it is the only parameter in these essays that could not be checked against something published. The justification is physical — the vehicle absorbs there, not the pigment — and the consequence of getting it wrong is quantitative rather than structural: a weaker ultraviolet absorber would make the gradient shallower and the matrix’s failure smaller, and would not make the term proportional to v.

The fit is a straight least-squares on tristimulus values, solved by normal equations, minimising squared error across all seventeen patches. That is the strongest form of the claim available: not that the published transforms fail, but that the best one does.

The assertion is written as two conditions. The fit must leave more than 15 per cent of the mean, and it must make at least three patches worse. The second is the one that carries the argument, because a residual can always be blamed on a hard problem and a regression on patches that were already right cannot.

Whether a linear correction could absorb the difference is a question with an answer, and the answer is the best 3 × 3 there is.

Where the model stops #

Seventeen patches is a small profile. A real press characterisation is several hundred, including overprints, and the fit would have more to work with. It would also have more solids and more near-solids, which is the region the fit damages, so the direction of the result is not in doubt.

Overprints are absent. A cyan-plus-magenta patch has two ink films over the substrate and blocks the excitation twice; including them would sharpen the gradient rather than soften it.

And no dot gain is modelled. The coverages here are nominal areas rather than what a press would actually lay down, which shifts every patch along the gradient without changing its shape.

Who found it, and when #

The problem is stated in the printing standards as a recommendation rather than as a theorem: measure under the condition the work will be judged under, and do not convert. That recommendation is old and its justification in the documents is empirical.

The structural reason — that the fluorescent term is additive and coverage-dependent — is straightforward once the halftone arithmetic and the bispectral arithmetic are written on the same page, and they usually are not. Halftone models live in the printing literature and bispectral models in the metrology literature, and the crossing point is a place where each side assumes the other’s variable is fixed.

What is new here is only the demonstration in the form that makes it decisive: the best matrix, fitted on the actual patch set, reported per patch rather than as a mean, with the regressions named.

The generalisation #

The pattern is a correction whose functional form is chosen from the wrong family, validated on a mean.

A least-squares fit will always find the best member of whatever family it is given, and will always report an improvement if the family contains anything better than the identity. What it will not report is that the family is wrong, because “the best fit is poor” and “the problem is hard” produce identical summary statistics.

The two diagnostics that separate them are cheap and neither is a mean. Look at whether any sample is made worse — a correction from the right family, well fitted, should improve nearly everything. And look at whether the residual has structure: here it is monotone in ink coverage, which is a variable the correction had no access to and which therefore names what is missing.

A residual that correlates with something the model does not see is not noise. It is the model’s missing term, labelled.

Where the ladder goes next #

If the term cannot be corrected, the remaining questions are about the substrate rather than the arithmetic. A proof cannot glow is the same impossibility in the printing shop’s currency: the direction that cannot be reached with ink is the direction the substrate is already in.

The other direction is the device that has to make the same decision with no measurement at all. A camera cannot record the excitation, so it faces this correction problem with one fewer input than a spectrophotometer has, and its white balance is precisely a diagonal matrix — the weakest member of the family that has already been shown to fail.