A resize with a negative weight in it

An average on the stored values found that a half-and-half blend of black and white taken on the numbers a file stores lands 18.7 colour differences from the blend of the light, and that the stored-value blend is the darker of the two on every pair it tried. The second finding was not a coincidence. The encoding is a concave function of the light, which is why the midpoint is not half. Every blend is a weighted average with positive weights, and a positive-weighted average of a concave function’s values is always below the function of the average. That is Jensen’s inequality, and it has a condition: the weights must all be positive.

The guarantee has a condition #

A resize with negative weights can make the stored-value result lighter than the light’s own, and on ordinary edges it does; on edges between full-scale values the clip removes the evidence.

• Linear interpolation, whose weights are all positive, is never lighter on any of six pairs of colours.
• Bicubic resizing is lighter on 8 samples of the edge between skin and its shadow, by up to 6.1 colour differences, and on 7 samples of dark text on a page.
• A three-lobe Lanczos kernel is lighter on the skin edge by up to 12.2 colour differences — nearly twice the whole half-and-half blend’s error for that pair.
• On black against white, red against green and blue against yellow, neither kernel is ever lighter, because the overshoot a negative lobe produces pushes a stored value past the ends of its range and the clip takes it away.

Where the negative weights are #

A resize computes each output sample as a weighted sum of nearby input samples, and the weights come from a kernel evaluated at the distances. Linear interpolation’s kernel is a triangle: every weight between nought and one, adding to one. Its output is always between its inputs, and it is a convex combination.

Sharper kernels are not triangles. Bicubic interpolation with the usual parameter has small negative lobes between one and two samples from the centre, and a Lanczos kernel with three lobes has larger negative lobes between one and two and a positive ripple between two and three. The negative weights are what make these kernels sharp: they subtract a little of each sample’s neighbours, which steepens an edge. They are also what makes them ring, overshooting above the bright side of an edge and below the dark side.

A resize with negative weights is not an average. It is a linear combination whose coefficients sum to one, and Jensen’s inequality says nothing about it. The sign of the difference between the stored-value result and the light’s own is no longer fixed, and where the negative weights dominate a sample it can flip.

A positive average minus another #

The failure can be described almost as exactly as the guarantee.

Any set of weights that sums to one, some of them negative, splits into its positive and negative parts. Write s for the total size of the negative weights. The resize is then 1 + s times a positive average A of the samples under the kernel’s positive part, minus s times a positive average B of the samples under its negative lobes. Jensen’s inequality says each of A and B, taken on stored values, is darker than it would be on the light. The resize adds the first and subtracts the second, so their two darkenings enter with opposite signs, and where the subtracted one is the larger — where a negative lobe reaches across an edge into the other colour — the balance tips and the stored-value result can come out lighter. The encoding bends the final comparison further, so this is the mechanism rather than an exact accounting; what it shows is why no inequality survives, since the sign now depends on which of two darkenings is bigger.

The size of s says how much room there is for the flip. For the bicubic kernel at a position between samples the negative weights total about an eighth; for the three-lobe Lanczos kernel, about a quarter. The skin edge’s worst lighter sample is 6.1 with bicubic and 12.2 with Lanczos, in rough proportion to the negative weight each kernel carries. Linear interpolation has s equal to nought, B vanishes, and the guarantee returns.

Beside the edge #

Across a sharp edge, each kernel’s output on the stored values and on the light can be compared sample by sample, and the difference has a shape that follows the kernel.

On the edge between skin and its shadow, linear interpolation makes the stored-value result darker at every sample that differs, by up to 7.3 colour differences, which is the familiar effect. Bicubic makes it darker on 4 samples and lighter on 8, by up to 6.1 on the lighter side. Lanczos makes it darker on 11 and lighter on 8, by up to 12.2.

The lighter samples are where the kernel’s negative lobe falls on the edge’s far side. There the output is a sum with a substantial negative contribution from samples of the other colour, and a negative-weighted term of a concave function pushes the inequality the other way. The overshoot and undershoot the kernel produces are larger on the stored values than on the light at some samples and smaller at others, and which way depends on which side of the edge the sample sits and which lobe dominates it.

Why black and white never show it #

The same kernels on an edge between black and white — the edge every test chart and every rendering check uses — are never lighter, at any sample.

Black and white are at the ends of every channel’s range. A kernel that overshoots above white or below black produces a stored value outside the file’s range, and the clip brings it back, on both the stored values and the light. On such an edge the clip is applied at 8 samples with either lobed kernel, and it removes exactly the overshoot that would have flipped the sign. Red against green and blue against yellow behave the same way, because each has every channel at nought or one.

So the edges that test charts use are the edges on which the old guarantee still appears to hold. The pairs that break it are mid-tones — skin, foliage, grey text on an off-white page — where the ringing stays inside the range and nothing hides it.

Six edges, three kernels #

A census over the six pairs the stored-value essay used puts the pattern in one place.

Linear interpolation: darker on 4 samples of every pair, lighter on none. Bicubic: lighter on 8 samples of sky against foliage, 8 of skin against shadow, 7 of text, and on none of the three full-scale pairs. Lanczos: lighter on 8, 8 and 5 of the same three, and on none of the others. The clip intervenes on 8 samples of every full-scale pair with either lobed kernel and on between none and 5 samples of the mid-tone pairs.

The worst lighter sample on each mid-tone pair is 2.0 to 2.3 colour differences on sky against foliage, 1.8 to 3.7 on text, and 6.1 to 12.2 on skin. The skin edge is the worst case because both its colours sit in the middle of the range, where the ringing is largest relative to what the clip allows and where the encoding’s curvature is still substantial.

What a blend already costs #

The resize result has to be read against the blend result, because both are errors a pipeline makes routinely and one is much better known.

A half-and-half blend on the stored values costs 18.7 on black against white, 19.3 on red against green, 16.1 on text and 6.4 on skin against its shadow, and always darker. A Lanczos resize of the skin edge produces a lighter error of 12.2 beside the edge — twice the size of the blend error on the same pair, in the opposite direction. So on mid-tone edges a sharp resize taken on stored values does not merely soften the darkening that a blend produces. It replaces it with a ring of lighter and darker errors of comparable or greater size.

Sharpening has the largest lobe of all #

A sharpening filter is a lobed kernel whose negative weight is the point rather than a side effect. An unsharp mask adds a multiple k of the difference between a picture and a blurred copy of it, which is the picture times 1 + k minus the blur times k: a positive centre and a negative surround whose total negative weight is k itself. A sharpening strength of one puts eight times bicubic’s negative weight into every edge, and sharpening is commonly applied to encoded values, because that is where a picture is when it is prepared for display.

By the argument above, stored-value sharpening of a mid-tone edge produces a lighter halo, and with several times a resize’s negative weight there is every reason to expect it larger than a resize’s. On black text on white it produces nothing lighter at all, beyond what the clip removes. An edge’s colour is fragile before sharpening ever reaches it — a grey edge arrives coloured from the sensor — and a lighter halo is then laid over a colour that was already not quite right.

The clip hides the sign, not the error #

Clipping makes the black-and-white edge look safe, and it is worth being exact about what it does. On a full-scale edge a lobed kernel overshoots past white on the bright side and past black on the dark side, in the light and in the stored values alike, and the clip returns both to the end of the range. What it removes is the overshoot, and with it the only samples where the stored-value result would have been the lighter one. The darker samples across the edge are untouched, so the familiar darkening survives and the guarantee appears to hold.

The mid-tone pairs show the other half. On them the clip intervenes on at most five samples, so most of each kernel’s ringing survives into the comparison, and every lighter sample the census counts is on those pairs. That makes the clip a property of the test rather than of the pipeline: a resampler checked on black text on white has been checked on the one kind of edge where lighter errors cannot appear, and it passes a check for them whichever way it resamples.

What a pipeline can do #

The consequences are about where a resize is done, and they are narrower than the usual advice. A resize is rarely the only operation a picture meets on its way to a reader, and a second conversion is not a repeat, so where in the chain the resize sits is part of the answer.

Resize in linear light, and clip afterwards. The usual advice to resample in linear light removes the systematic darkening of a convex blend. For a lobed kernel it also removes the ringing that stored-value resampling adds on top of the kernel’s own, because the overshoot is then a property of the light alone and the only question left is whether to keep it.

Test with mid-tones, not with black and white. A resampling check on a black-and-white edge cannot see the lighter samples, because the clip removes them on both sides of the comparison. A check on a mid-tone edge — skin against shadow is the harshest of the six here — sees both signs.

And treat a lighter halo around a mid-tone edge as a resampling artefact. Sharpening filters are lobed kernels too, and colour thrown away on purpose found that a video pipeline’s chroma resampling already produces large errors at edges of near-equal luminance. A halo that is lighter than both of its sides on a resized image is the negative lobe acting on stored values.

Or give up the lobe where lightness matters more than sharpness. A thumbnail of a colour swatch, a resized gradient or a small portrait gains little from a lobed kernel’s sharpness and pays for its ringing on exactly the mid-tones it contains. A tent or a Gaussian kernel keeps every weight positive, and with it the guarantee.

What was computed, and how #

Each pair is a step between two sRGB colours across thirty-two samples, magnified four times by evaluating the kernel at each output position’s distance from the input samples, with the input extended by repetition at its ends and the weights normalised to sum to one. The same resize is done once on the stored values and once on the linear light the stored values encode.

Both results are clipped to the unit interval, the linear result is re-encoded, and the two are compared in CIELAB under D65: the difference in lightness says which is darker, and ΔE₀₀ says by how much. A sample counts as lighter or darker only if the two differ by more than a twentieth of a colour difference. A sample counts as clipped if either result left the unit interval before clipping.

The kernels are the triangle, the Keys bicubic with parameter −0.5, and a Lanczos kernel with three lobes.

Where the measurement stops #

One dimension and one magnification. A two-dimensional resize applies the kernel along both axes, which multiplies its negative lobes into positive corner weights and changes the pattern near corners; a reduction rather than a magnification applies the kernel at a coarser spacing and aliases. Each input sample is also treated as a point, where a real pixel integrates over an area — a pixel has an aperture too — which softens an edge before any kernel sees it.

The six pairs are the ones already measured for blends, and they include only one skin pair. A census over the edges in real photographs would say how often a mid-tone edge of this kind occurs and how large its ringing typically is.

And the comparison is in colour difference, not visibility. A lighter sample one output pixel wide beside an edge is a spatial feature, and a difference has no size found that the visibility of a difference depends on how finely it is spread.

The habit #

The habit is about an inequality whose condition is quietly dropped.

Jensen’s inequality, a maximum principle, a monotonicity result: each holds for a class of operations — averages, diffusions, positive filters — and each is borrowed for a neighbouring class that looks similar and violates the condition. A sharpening filter looks like a smoothing filter with different numbers.

The move is to check the condition, not only the conclusion. Here the condition is one line: are all the weights positive.

The failure mode is to rely on the conclusion and test it where the violation cannot show. A rule tested only on the cases that clip is a rule tested only where it cannot fail.

Who noticed it first #

That resampling should be done in linear light is long-standing advice in image processing, and that lobed kernels ring and overshoot is standard signal processing. That clipping at the ends of the range hides overshoot is familiar to anybody who has resized a black-and-white chart.

That negative weights flip the sign of the stored-value error beside mid-tone edges, and that the flip reaches twice the size of the ordinary blend error on skin, are measured here; the sources consulted here discuss the darkening of stored-value blends and the ringing of lobed kernels separately.

Still open: the two-dimensional case #

A real resize is two-dimensional, and the product of two one-dimensional kernels has positive weights at its corners where both lobes are negative. Whether that restores the darker sign near corners, or produces a lighter ring around a mid-tone patch that is worse than either edge’s, is one more loop over a second axis and would say what a resized photograph actually shows.