A profile interpolates light

A profile is a table established the shape of the object: a lattice of patches that were printed and measured, with interpolation everywhere else, and an error of exactly zero at every patch for ever. A profile is a fit between its nodes measured how large the error elsewhere is and how it falls with the patch count — twenty-seven times the work for eight times the accuracy.

Both measure a size. Neither asks which way the error points, and a signed quantity that has only been measured as a magnitude is usually hiding something.

It is a bias, and the bias is the whole of it #

A profile’s interpolation error between its nodes is not a scatter about the truth. It is a systematic lightening, its mean signed lightness error is as large as its mean colour difference, and it points the same way at every lattice size.

• At a nine-step lattice the mean signed lightness error is +0.136 against a mean colour difference of 0.133. At three steps it is +1.239 against 1.333, with 400 of 400 samples lighter.
• It lightens because a press’s response is convex in ink coverage. The first drop of ink removes more light than the last, so a straight line between two points on the curve lies above it.
• The space the table is held in decides the size by a factor of four and not the sign. Held in CIELAB, as an ICC profile holds it, the mean error is 0.133; held in unencoded tristimulus values, 0.536. The encoding is straightening the device’s curve, which is the opposite of what the same encoding does to an average taken over an image.
• A table allowed to be wrong at its own patches is two and a half times better between them, at every lattice size, and its bias falls from +0.206 to −0.0003.
• The repair costs nothing measured. It is arithmetic on patches already printed, and what stops it is the procedure a profile is validated by.

Why light, and why always #

A profile’s forward table holds what a device produced at each lattice point. Between two lattice points a colour engine interpolates, which is to say it assumes the device’s response is a straight line there. It is not.

Ink on paper is strongly convex in coverage. A press laying down twenty per cent cyan removes a large share of the light that twenty per cent of the sheet would have returned; going from eighty to a hundred removes very little, because the eighty is already most of the way to the ink’s own density. So the curve of light returned against coverage falls steeply at first and flattens, and a chord between two points on it sits above it everywhere in between.

Above means more light, and more light means lighter. That is the whole of the sign, and it is a property of every subtractive marking process rather than of this press: dot gain makes it stronger, an ink limit does not remove it, and the same argument runs on a display’s transfer function with its own curvature.

The figure above is that argument checked rather than assumed. At three steps a side every one of 400 samples between the nodes came out lighter than the press prints. At nine steps 369 of 400 did, and the mean signed error is still the whole of the mean difference. At seventeen steps the individual samples are more mixed — 286 of 400 — and the mean is still a bias: the lattice is fine enough that the curvature within a cell is small compared with the other things going on, and not fine enough for the curvature to stop being the mean.

The encoding is on the other side here #

There is a result earlier here that looks like it should contradict this one.

An average on the stored values found that an average taken in an encoded variable lands far from the average of the light, and always darker, because the encoding is concave and a convex combination of a concave function’s values lies below the function of the combination. A profile’s table is also an encoded quantity — an ICC A2B table holds CIELAB — and interpolating it is also an average in an encoded variable. The two arguments point opposite ways.

Interpolating the encoded table is four times better — 0.133 against 0.536 — and both err light. So the encoding is not adding a bias here; it is removing most of one that the device put there.

The reason the two cases differ is what is being interpolated against. An image average interpolates between two colours with nothing in between: the true answer is the average of the light, the encoding’s concavity is the only nonlinearity in the problem, and it is the whole of the error. A profile interpolates against ink coverage, where the device’s own convexity is much the larger nonlinearity, and the encoding’s concavity partly cancels it. One operation, two places in the chain, opposite verdicts — and what decides is whether anything else in the problem is curved.

That is worth stating as a rule, because the temptation after reading the first result is to say that averages belong in linear light. A profile’s table held in linear light would be four times worse.

A table that is allowed to be wrong #

The bias has an obvious repair and it is one nothing in the file format forbids.

A table is built by setting each node to what the device produced there — which is a choice, and the chart decides the profile is the essay about the other choice made at the same moment. That is the only arrangement under which the profile is exactly right at its own patches, and it is not the arrangement that minimises the error anywhere else. Choosing the node values to minimise the error between the nodes gives a different table, wrong at the patches by a little and right elsewhere by a lot.

The ordinary table’s mean error is 0.221 and its worst is 0.456, with a bias of +0.206. The refitted table’s mean is 0.095 and its worst 0.331, with a bias of −0.0003. Its worst case is now at the nodes, which is where a validation procedure looks, and it is smaller than the error the ordinary table carried between them.

That last point is the trade stated exactly. A refitted table is never as wrong anywhere as the ordinary table was somewhere. It exchanges being perfect in a measure-zero set of places for being better in all the others — the trade the objective nobody chose describes wherever a fit is scored on the points it was built from — and the price it pays at the patches is smaller than the price it was paying between them.

The gain is 2.33, 2.48 and 2.53 at five, nine and seventeen nodes — the same factor throughout, which is the useful part. Doubling a lattice costs twice as many measured patches per axis, and on the cyan ramp it buys about four times the accuracy. The refit costs no measurement at all and buys a consistent two and a half, so it is worth roughly two thirds of a lattice refinement, everywhere, for free.

What the bias costs, and what it does not #

A systematic error invites two guesses about what it does, and both are wrong in an instructive way.

The first guess is that it bands. A ramp through the table is exact at each node and lightened between them, so the error across a gradient is a ripple whose period is the node spacing — eight periods across a nine-node ramp, however many pixels the gradient covers. A thousand-pixel gradient at an ordinary desk distance puts that ripple at about a third of a cycle a degree, and its amplitude is 0.06 of a colour difference on the cyan ramp. That is far below anything a reader could see, and colour thrown away on purpose is the essay about what a visible spatial artefact actually costs. The ripple is real and it is not the problem.

The second guess is that a shift of everything in one direction is invisible because a reader adapts to it. That is nearly true of a whole picture and it is not true of the thing a profile is for. A profile exists to make a delivered colour match a stated one, and the statement is a number an instrument will check. An instrument does not adapt.

So the cost is a line in a budget. A delivery tolerance is checked by measuring patches and comparing, a delivery tolerance is three tolerances takes it apart, and a profile’s interpolation appears there as a contribution of about a tenth of a colour difference at a nine-step lattice. That contribution is not a spread that cancels against the chain’s other errors — it is a shift that adds to any other shift pointing the same way, and the other shifts in this chain point the same way: an average on stored values darkens, a profile lightens, and neither is an error whose expectation is zero.

Refitting removes one of them for nothing. That is the whole of the practical case, and it is smaller than the paragraph before it would suggest — which is why the argument has to be made in a budget rather than in a picture.

Which ink, and how much curvature it has #

The four process inks do not have the same curvature, and the bias follows it.

Magenta carries the largest error, 0.0788, with a bias of 0.0664 — 84 per cent of it. Cyan is 0.0562 with a bias of 0.0511 — 91 per cent. Black is 0.0422 with 0.0278.

Yellow is the exception and it is instructive. Its mean error is 0.0452 and its bias is 0.0084 — under a fifth. Yellow removes least light of the four inks, so its ramp is the least curved in lightness, so there is least for a straight line to miss. What is left is a genuine scatter: the small chromatic wanderings of a real ink’s ramp, which point in no particular direction.

So the bias is a lightness phenomenon and it scales with how much light the ink takes away. An ink that barely darkens a sheet interpolates almost without bias, and an ink that darkens it a great deal interpolates almost entirely bias. That is a prediction about a fifth or sixth ink in an extended set: an orange is closer to magenta, a green closer to cyan, and a light cyan in a photographic set should behave like yellow.

Why the repair is not made #

The refit is arithmetic on patches that have already been printed and measured. Nothing about it is expensive, nothing about it needs a press, and no part of the ICC specification says a table’s entries must be the measurements themselves.

What stops it is the check.

A profile is validated against its own target: print the patches, measure them, put them through the profile, and compare. A profile is a table put it plainly — a profile checked against its own target reports perfection for ever, because the error at every patch is zero by construction. The refitted table fails that check, by up to a third of a colour difference on a five-node ramp, while being two and a half times better everywhere a job actually lands.

So the procedure that made the bias invisible is the same procedure that makes its repair look like a regression. That is not a coincidence of this defect; it is what a validation set drawn from the training set does to any fit, and best on the average, undefined at the edge is the neighbouring case where the same habit hides a different failure.

A check that would see the difference is not hard to specify. Print a second, smaller target whose patches deliberately fall between the first one’s nodes, and validate against that. It costs one sheet, it is the only measurement in this essay that needs a press at all, and it would report the ordinary table as biased and the refitted one as not.

How the tables were built and refitted #

The press is the four-ink model used throughout here, on a coated stock, with its own dot gain and trapping. A forward table is the lattice of coverages printed and measured as CIELAB, and it is interpolated quadrilinearly — what a colour engine does, up to the difference between multilinear and tetrahedral interpolation, which does not bear on the sign.

The error between the nodes is sampled at 400 coverages drawn pseudo-randomly, deliberately not at the nodes, with the signed lightness difference kept beside the colour difference. The tristimulus comparison converts the same table’s entries to tristimulus values, interpolates there, and converts back, so that the only thing changed is the space the interpolation happens in.

The refit is a coordinate descent on the node values against the mean squared colour difference over a dense test set, with shrinking steps, so the cost falls monotonically and a refit that fails to improve returns the ordinary table unchanged. It is deliberately the simplest thing that could work: the claim is that some refit helps, not that this one is optimal, and a better optimiser would only widen the gap.

What this leaves out #

The refit is demonstrated on one-dimensional tone ramps rather than on the four-dimensional lattice a real profile holds. The mechanism is the same — a straight line across a curved cell — and the arithmetic is not: a four-dimensional refit has many more free parameters, its cells are interpolated quadrilinearly rather than linearly, and whether the gain is still about two and a half is a computation nobody here has run.

Nothing is said about the inverse table. An ICC profile holds both directions, the B2A table is built by inverting the forward one, and an inversion of a biased table carries the bias through with a sign the inversion decides. A second conversion is not a repeat is the essay about what happens when the two are composed.

And the press is a model. Its convexity is the real mechanism and its exact curvature is this model’s, so the sign of everything here is safe and the numbers are not portable to a particular machine.

Still open: whether a refitted table survives being inverted #

The awkward part of the repair is the half of the profile this essay did not touch. A colour engine converting an image into a device’s space uses the B2A table, which is built by inverting the A2B one — and an inversion is not a linear operation, so a forward table whose bias has been removed does not obviously give an inverse table whose bias has been removed.

Two outcomes are possible and they suggest different repairs. If the inverse of a refitted forward table is itself nearly unbiased, then the refit is a single change made in one place with the whole chain improved. If it is not, then the two tables need refitting against different objectives — the forward one against the error a soft proof makes and the inverse one against the error a separation makes — and a profile would carry two tables that are not each other’s inverse, which is a larger change to what a profile is than anything proposed here.

The measurement is the same shape as this one: build both tables from the same lattice, refit each, and compare the round trip at coverages that fall between the nodes.

A magnitude is half a measurement #

The habit is about a quantity that has been summarised as a size.

An error is a signed thing, and the first summary anybody reaches for is its magnitude — a mean absolute error, a root mean square, a worst case. All three throw the sign away, and all three are the right summary when the error really is a scatter. When it is not, they hide the most useful property the error has, because a bias can be removed and a scatter cannot.

The move is to keep the sign in one coordinate and put the mean of the signed quantity beside the mean of the magnitude. If the two are the same number the error is a bias; if the first is near zero the error is a scatter; and anything between is a mixture whose proportions are now known. The whole measurement is one extra column in a loop that was already running.

The failure mode is to refine when the error is a bias. Adding patches to a table is the obvious response to it being wrong between the patches, and it is the expensive response to a defect that a free rearrangement of the patches already present would have removed.