Colour thrown away on purpose

The last thing that happens to a colour before it reaches a screen is that most of it is deleted. Video is stored and transmitted with one luminance sample per pixel and one pair of colour samples per two or four, so three quarters of the colour information in a 4:2:0 signal is discarded before anything is compressed.

This is not a compromise forced by bandwidth. It is a deliberate exploitation of a measurement about the visual system, it is nearly free most of the time, and where it fails it fails spectacularly.

The claim #

Chroma subsampling costs exactly nothing on an edge that carries no chroma, and an amount governed by how much chroma the edge carries otherwise. Whether the edge also carries luminance is very nearly irrelevant — which is not what the arrangement’s usual justification implies, and is what the table below had to be read carefully to see.

The measurement #

Take an edge — sixteen samples, one colour on the left and another on the right — encode it as luma and two colour-difference planes in the usual way, average the colour planes over blocks, decode, and measure the difference at every sample.

The first row is exact. Not small — zero, at every sample, at every ratio — because an achromatic edge has no colour difference to average and the luma plane, which is not subsampled, carries the whole of it.

The last row is a colour difference of forty-three, which is not a subtle artefact: it is a visible stripe of a third colour along the boundary.

What actually governs it, which is not the luma step #

The first and last rows invite a reading that the middle of the table refuses. Black-to-white has the largest luma step and costs nothing; red-to-green at matched luminance has the smallest and costs 25.9; so the loss looks like it is about whether the edge is carried by luminance.

It is not. Blue-to-yellow has a luma step of 0.430 — larger than four of the other five rows — and costs 38.9, the second worst in the table. Red on white has the second largest step and costs 16.8. Sorting the six rows by luma step does not sort them by error at all; the rank correlation is −0.37, which on six points is noise with a sign.

Computing the quantity the scheme actually averages — the distance between the two colours in the CbCr plane, which is what a block mean is taken over — sorts them almost perfectly:

Rank correlation with the chroma step: +0.94. The one inversion is the last pair, and it is small.

So the claim has to be stated in terms of what is thrown away rather than what is kept. An achromatic edge survives because it has nothing in the discarded planes, not because it has a lot in the retained one — those sound alike and come apart on every row in the middle. The luma plane’s job is to be untouched, not to be large.

That matters for predicting where the artefact will appear, which is the practical use of any of this. A rule of thumb built on the luma step says a high-contrast boundary is safe, and high-contrast coloured boundaries — a red logo on white, a blue sky against a yellow building — are among the worst cases in the table. The rule that works is the plain one: how far apart are these two colours in hue and chroma, and never mind how far apart they are in lightness.

Why the ordinary case costs more than it should #

There is one row that does not fit the proportionality, and it is the row the essay singles out as the one that matters.

Dividing each error by its own chroma step gives the price per unit of what was discarded: 60 for red on white, 69 for blue to yellow, 88 for red to green, 90 for magenta to green — and 184 for skin against a wall, two to three times everything else.

The reason is in the ruler rather than in the codec. CIEDE2000 divides a chroma difference by the chroma it was measured at, by design, because that is what the discrimination data say — so a displacement of a given size in CbCr counts for far more near the neutral axis than out among the saturated colours. The skin-and-wall edge is the only near-neutral pair in the table, and it is charged accordingly.

Which is the quantitative form of why subsampling artefacts are noticed on faces. It is not that the error is large — the chroma step across that edge is a fifth of the saturated rows’ — but that it lands in the part of the space where both the formula and the observer are most sensitive to it. A codec spending its samples on a measurement of spatial resolution is blind to that, because the eye’s chromatic acuity is not the quantity that varies here; the eye’s chromatic discrimination is, and the two are different measurements that happen to concern the same channels.

Why it is nearly free #

The justification is a real measurement about the visual system and it is worth stating precisely, because the popular version of it is wrong in an interesting way.

The eye’s spatial resolution for luminance detail is much higher than for chromatic detail. The usual account attributes this to there being fewer S cones, which is true but is not the reason: the resolution limit for the red-green channel is also far below the luminance limit, and there is no shortage of L or M cones.

The actual mechanism is at the second stage. Cone responses are recombined into one achromatic and two chromatic channels, and the chromatic channels are constructed by differencing neighbouring receptors — which is a low-pass operation, because a difference of two nearby signals cannot carry detail finer than their separation. The luminance channel is built by summing, which is not.

So the eye itself subsamples chroma, before any codec does, and the codec’s saving is genuinely aligned with what a person can see.

Why it is not free #

The alignment is between the codec and the visual system, and the failures come from a mismatch between two other things.

The codec averages Y′CbCr, and the eye’s channels are not Y′CbCr. The luma coefficients are a fixed weighted sum of gamma-encoded R, G and B, chosen in the 1950s for compatibility with monochrome receivers and adjusted since. That is not the achromatic channel of the visual system, so an edge with zero luma step is not necessarily an edge with zero achromatic contrast — and where the two disagree, the codec has put an edge entirely into the planes it is about to throw away.

The averaging is done on gamma-encoded values, which is not where averaging is a physically meaningful operation. Two chroma samples averaged in an encoded space do not give the chroma of the average colour, so the reconstructed intermediate is not on the line between the two — which is why the error at the boundary is a third colour rather than a blend.

And the sampling grid is arbitrary relative to the picture. An edge landing on a block boundary survives exactly; an edge one sample over is averaged across two colours. Half of all edges are in the bad case, and nothing in the picture chose which.

The case that is not exotic #

The dramatic rows above are saturated colours against each other, and it is fair to object that pictures do not usually contain those.

The row worth attending to is the fourth: skin against a wall, ΔE00 = 10.65. Two ordinary, unsaturated colours with a luma step of 0.127 — a face in front of a neutral background, which is the most photographed arrangement in the world.

Ten units is not a stripe of a wrong colour, but it is a visible fringe on a boundary the eye is exceptionally attentive to, and it is the reason skin tones are where subsampling artefacts are noticed. The other common case is red text on white — 16.8 at 4:2:2 and 24 at 4:1:1 — which is why any format used for computer-generated material carrying fine coloured detail specifies 4:4:4, and why a video call sharing a screen looks like a video call.

What was computed, and how #

Linear RGB to gamma-encoded, then BT.709’s luma coefficients and the two colour-difference signals; the chroma planes averaged over blocks of the stated width, held constant across the block on reconstruction, and the result taken back to linear RGB and to CIELAB.

Two choices are worth naming because they set the size of the numbers.

The reconstruction is nearest-neighbour, which is the worst case and was what consumer decoders did for a decade. Bilinear upsampling roughly halves the peak error and moves it, and does not remove it — the information is gone at the encoder.

And the edge is placed at an odd sample deliberately. An edge landing on a block boundary is reproduced exactly; the default here puts it one sample over, which is the case half of all edges are in. A default of “on the boundary” would have reported that subsampling costs nothing, which is a real result about a particular alignment and a false one about pictures.

What the notation means #

The three-number notation is opaque enough to be worth unpacking, because it encodes a piece of history rather than a scheme.

The numbers describe a block four samples wide and two high. The first is the luma sampling — always 4, meaning every pixel. The second is how many chroma samples are in the block’s top row, and the third how many are in the bottom row.

• 4:4:4 — chroma at full resolution. No loss, and what everything in a colour-managed still-image pipeline uses.
• 4:2:2 — chroma at half resolution horizontally, full vertically. Broadcast and professional acquisition.
• 4:2:0 — half horizontally and half vertically, the zero meaning “the bottom row has none of its own, take the row above”. Almost all delivered video and every video call.
• 4:1:1 — quarter horizontally, full vertically. An older acquisition format, and the coarse case in the table above.

A 4:2:0 signal carries one colour sample per four pixels, so three quarters of the colour information is gone before the codec starts. The measurements above are one-dimensional and therefore describe the horizontal half of that; the vertical half doubles the affected area on any edge that is not horizontal, and turns a stripe into a patch.

The other thing video throws away #

Chroma resolution is the loss with a mechanism behind it. There is a second, unrelated loss in the same signal that has no perceptual justification at all and is worth naming because it is often blamed for the artefacts above.

Video code values are studio range: in eight bits, black is code 16 and white is code 235, with the values above and below reserved for headroom and footroom that analogue equipment needed. That is 220 code values where 256 are available — a loss of 15 per cent of the code space, retained for compatibility with equipment that no longer exists.

Unlike subsampling, this costs precision uniformly and buys nothing. It is also the single most common source of colour faults in a video pipeline, because a full-range signal interpreted as studio-range comes out with crushed blacks and blown highlights, and the error is a mismatch of conventions rather than a loss of information.

The two losses are constantly confused, and the distinction is the one this essay is built on: one was chosen by measuring an observer, and the other was inherited.

Where this model stops #

One dimension only. 4:2:0, which is what almost all delivered video uses, subsamples vertically as well, so the artefact is a two-dimensional patch rather than a stripe and is correspondingly more visible on a diagonal edge. A one-dimensional row cannot show that, and this essay does not pretend to.

No compression. Everything above is the format, before any transform coding. A codec then quantises the chroma planes further, and the two losses compound.

The colours are stated as linear RGB rather than computed from spectra, which is a departure from this site’s practice and is deliberate: the subject is what happens to code values in a pipeline, and code values are the content. The conversion to CIELAB for the measurement is exact.

And nothing here models a real decoder’s chroma reconstruction. Modern decoders interpolate, some are adaptive, and a few use the luma edge to steer the chroma reconstruction — which recovers a good deal of what the format discarded, by assuming that colour edges coincide with luminance edges. That assumption is the same one the whole scheme rests on, applied twice.

The generalisation #

A system that discards what an observer cannot resolve is safe exactly to the extent that its measure of “what the observer resolves” is the observer’s measure, and every such system is built on an approximation of it.

Chroma subsampling’s approximation is that the achromatic channel is BT.709 luma. It is a good approximation and it is not the same function, so there exist edges with no luma contrast and considerable perceived contrast, and those edges are the ones that break.

The pattern is general and this site has met it repeatedly. A camera’s colour matrix approximates the eye’s matching functions by a linear combination of the sensor’s, and the residual is where it fails. A colour difference formula approximates discrimination by a metric, and there are pairs it ranks wrongly. An appearance model approximates adaptation by a transform with stated parameters, and the parameters are categories.

In every case the useful question is the same and it is rarely asked at the time: what does the approximation not see, and how does one construct an example? For subsampling the construction is trivial — hold luma fixed and move chroma — and the answer is a forty-three unit error in a format that has been shipping since 1994.

Who found it, and when #

The idea is older than digital video and belongs to the design of colour television. The problem in 1953 was to add colour to a broadcast signal that monochrome receivers had to keep receiving, and the solution — a luminance signal compatible with the old standard, plus a colour subcarrier squeezed into a narrow band beside it — was constrained by bandwidth rather than by perception.

That the narrow band was tolerable was justified by exactly the argument above, and it was tested extensively on observers before the standard was adopted. The two colour-difference signals of the American system were even given different bandwidths from each other, on the grounds that the eye resolves orange-cyan detail better than green-magenta detail — a refinement that was correct, was dropped by later systems for simplicity, and is a more careful piece of vision science than most of what replaced it.

Digital video inherited the arrangement wholesale. The 4:2:2 and 4:2:0 notations date from the 1980s and the CCIR 601 standard; they encode a decision made for a 1953 analogue bandwidth constraint that no longer exists, retained because the perceptual justification turned out to be sound and because the saving is still worth having.

It is one of the few places in this subject where an engineering compromise forced by circumstance turned out to be aligned with a measurement of the visual system, and it survived the circumstance by half a century.

Where the ladder goes next #

This is the top of the delivery ladder, and it sits on three cones, two axes — the second-stage recombination that makes the whole scheme work — and on the mosaic is not the observer, which is the retina’s own version of spending samples unevenly.

Beside it, a grey edge arrives coloured is the capture-side twin: the same trade made by a sensor’s mosaic, with the artefacts appearing at the other end of the pipeline.

Below, the field’s whole argument is that every stage of delivery is a map that loses something chosen deliberately — an ink limit, a rendering intent, a separation policy, and here a sampling grid. This one is the only stage where what was thrown away was chosen by measuring the observer first.