A black level is multiplied by the balance

The operations between a sensor and a picture come in two kinds, and the order is not in the documentation was about how the multiplicative ones — a gain per channel, a matrix — fail to commute with the nonlinear ones. There is a simpler failure that precedes all of them, and it involves the one operation every converter performs before the white balance and never mentions: subtracting the black level.

A pedestal error comes out coloured #

An equal offset left in the raw channels is multiplied by three different white-balance gains and becomes a tint, in the colour the lamp is weakest in, and it is largest in the shadows.

• A pedestal error of three thousandths of white turns a two per cent grey 2.4 colour differences away under daylight, 2.3 of them chroma, and a seventy-two per cent grey 0.21.
• Under a tungsten lamp the same error turns the two per cent grey 2.1 away, and at one per cent of white the tint is 8.3 against 6.6 under daylight.
• Pushed four stops in editing, a one per cent grey carrying a two-thousandths error moves from 1.4 colour differences to 8.9.
• An offset in proportion to the lamp’s own raw white is exactly grey after the balance, to the last bit, which is why flare from a grey scene lightens shadows without tinting them and flare from foliage or sky does not.

Why a sensor has a pedestal #

A photosite counts photoelectrons, and the count carries noise that is as likely to be below the true value as above it. Near zero light a count cannot be negative, so a sensor adds a fixed pedestal to every reading — several hundred counts on a fourteen-bit file — and the true signal is the reading minus the pedestal. The pedestal is the same in all three channels, because it is added in the electronics after the colour filter has done its work.

A raw file is not yet a picture, and a converter subtracts the pedestal first, before the white balance, before the matrix, before anything. The value it subtracts is estimated, from masked photosites at the sensor’s edge or from a number the manufacturer wrote into the file, and an estimate can be wrong. A warm sensor drifts; a long exposure accumulates dark current; a value rounded to a convenient number is rounded. What survives the subtraction is a small offset, equal in counts in every channel.

Why the balance turns it into a colour #

A white balance divides each raw channel by that channel’s response to the lamp, so that the lamp itself comes out with three equal values. The three divisors are not equal, because a silicon sensor under any real lamp responds unequally to its three dyes.

Under daylight the three gains are 1.00, 1.16 and 1.85. Under a tungsten lamp they are 1.00, 1.81 and 6.15. An offset equal in counts in the three raw channels comes out of the balance unequal in exactly those proportions. An equal addition of a little light has become an addition of a little blue light, and the matrix and the tone curve carry that on to the picture.

The mathematics is two lines. Subtract a constant and then multiply by a gain, and the constant is scaled by the gain; multiply first and subtract afterwards, and it is not. An offset and a gain commute only when the offset is zero or the gain is the same in every channel, and a white balance exists because the gains are not the same.

The same gains applied to noise #

An offset is a fixed addition. Noise is a random one, and the balance multiplies it by exactly the same gains.

Read noise is added in the electronics, after the colour filter, so it is the same size in counts in the three raw channels. Under a tungsten balance it comes out with the blue channel’s noise 6.15 times the red channel’s and the green’s 1.81 times — the gains above, applied to a random addition instead of a fixed one. That is part of why correcting colour costs noise, and why a shadow under tungsten light is noisiest in blue.

The two can still be told apart on a picture, and the difference is what averaging does to each. Noise averages away over an area and a pedestal error does not. Blurring a lifted shadow removes its speckle and leaves its cast untouched, which is a test anybody can run in an editor: a tint that survives a heavy blur is not symmetric noise. Noise whose negative readings were set to zero before the blur is not symmetric, and clipped noise does not average away — so the test separates a zero-mean error from everything else rather than noise from a pedestal.

The exact case #

There is one kind of offset that survives the balance without becoming a colour, and it is worth finding because it shows which property of the pedestal is responsible.

An offset added to each raw channel in proportion to that channel’s response to the lamp is, after the balance divides by that response, the same number in all three channels — exactly, with a spread of zero in floating point. On a two per cent grey it moves the colour by about a colour difference, and almost all of that is lightness: a lightening and very nearly nothing else, the remainder being the colour matrix’s own small departure from keeping the white exactly white.

That identifies the culprit precisely. It is not that an offset is present; an offset shaped like the lamp’s white is harmless to hue. It is that a pedestal is equal in counts, which is the one shape the balance is guaranteed to distort.

Under a warmer lamp #

The size of the tint follows the spread of the gains, and a tungsten lamp spreads them much further than daylight does.

At three thousandths of white the two per cent grey is 2.1 colour differences away under tungsten against 2.4 under daylight — slightly less, most likely because this pipeline does not adapt the tungsten rendering, so the grey it starts from is already warm and part of a blue shift moves it towards neutral. At one per cent of white it is 8.3 against 6.6. The tint’s hue moves from −77 degrees under daylight towards −110 under tungsten, which is to say further into the blue, following the channel with the largest gain.

The growth is close to linear in the error — 0.86 colour differences at a thousandth of white, 1.66 at two thousandths, 3.77 at five — because a small offset is a small perturbation of a colour that is itself small. A thousandth of white is about sixteen counts on a fourteen-bit file, which is a size a warm sensor over a long exposure can drift by, and on a deep shadow it is already most of a colour difference.

A push carries the tint with it #

A shadow that is too dark is routinely brightened in editing, and a raw editor does that by multiplying the raw values before the pipeline runs — which is the correct place for an exposure change, as a stop is not a stop afterwards found. It is also the place that multiplies the pedestal error by the same factor.

A one per cent grey with a pedestal error of two thousandths of white is 1.4 colour differences from the right grey. Pushed one stop it is 3.1; two stops, 4.6; three, 6.5; four stops, to a lightness of 45, it is 8.9. A tint that was hidden in a shadow at L* 4 — where the eye is least able to see colour — has been moved to the middle of the scale and enlarged.

That is the familiar complaint about lifted shadows in underexposed raw files: they come up with a colour cast, usually magenta or green or blue, that no white balance setting removes. The cast is not noise and it is not the balance being wrong. It is a pedestal error, multiplied by the balance and then by the push.

Dark current is a pedestal that grows #

A pedestal taken from a file’s metadata is a number for a cool sensor and a short exposure, and the photosites do not keep to it.

Heat frees electrons in silicon as light does. This dark current accumulates in proportion to the exposure time and rises steeply with the sensor’s temperature, and it collects in every photosite whatever colour filter sits above it — so it is equal in counts in the three channels, which is the one shape the balance is guaranteed to distort. A long exposure on a warm sensor is a machine for producing exactly the error measured here, sized by the exposure. Nor is it uniform across the frame: photosites near a warmer part of the sensor collect more, and a gradient of pedestal error becomes a gradient of tint.

A night scene under tungsten light, underexposed and pushed in editing, meets every multiplier at once: the longest exposure for the largest dark current, the lamp with the largest blue gain, and the push that carries both into the middle of the scale. The masked photosites at the sensor’s edge collect dark current over the same exposure as the image area, which is why a per-frame estimate from them removes most of it where a fixed value cannot.

Flare is an offset nobody can subtract #

The sensor is not the only source of an offset. Light scattered inside a lens adds a fraction of the whole scene’s light to every pixel, before the sensor records anything, and no converter can measure it to take it out.

Flare has the colour of the scene’s average, and the balance treats it by the same arithmetic. Flare from a scene that averages grey is in proportion to the lamp’s white, and it lightens a two per cent grey by 1.0 colour difference with 0.01 of chroma — the exact case above. Flare from a scene of foliage adds 1.6 of chroma and from open sky 2.9, because those scenes’ averages are not the lamp’s colour and the balance cannot make them so.

It is one contributor to the green cast in a forest’s shadows and the blue in shadows under open sky — alongside the larger fact that a shadow has its own illuminant — and it is the one offset in the chain with no measurement to subtract.

The top of the range has the mirror image #

The pedestal is a problem at the bottom of the range. At the top a channel stops rather than shifts — it saturates at the photosite’s capacity or at the file’s largest value — and a limit reaches the balance by the same route as an offset. A clip applied equally in counts in the raw channels is multiplied into three unequal limits, so the channels of a clipped highlight leave the balance at different heights, which is part of why a blown highlight turns rather than simply going white.

What a converter could do #

The consequences are narrow and practical.

Estimate the pedestal per channel, and per frame. Masked photosites exist at the edge of most sensors precisely so that the black level can be measured on the frame rather than assumed, and a converter that uses a fixed value from the file’s metadata is using a number that was right when the sensor was cool.

Subtract before balancing, and say so. The order is forced by arithmetic — no converter can do otherwise and get a neutral black — but it is the kind of step whose position one step has no choice described: forced, and therefore unrecorded, and therefore the place a residual error hides.

Treat a shadow cast as a black-level diagnostic. A tint that grows with a push and has the hue of the lamp’s inverse is a pedestal error, not a white balance error, and correcting it with a balance slider moves every other colour in the picture. A balance estimated from the highlight is untouched by the error, because in a highlight three thousandths of white is a negligible share of the signal, so the balance and the cast can be diagnosed separately.

Record the pedestal that was subtracted. A converter that writes the black level it used into its output lets a later edit tell a pedestal error from a white-balance error; one that discards it leaves only the picture, in which the two look alike until a push separates them.

What was computed, and how #

The pipeline is a modelled raw converter: a silicon sensor with three dye filters and an infrared-cut filter, a white balance dividing each channel by its response to the lamp, a colour matrix fitted to a set of surfaces under that lamp, a per-channel tone curve and a clip. Greys are flat reflectances under D65 or CIE illuminant A, captured by the sensor model.

A pedestal error is added to the raw values as a fraction of the largest channel’s response to the lamp, either equally in the three channels or in proportion to each channel’s response. The colour difference is ΔE₀₀ between the pipeline’s output with and without the offset. A push multiplies the raw values, offset included, by a power of two before the pipeline runs. Flare adds a fraction of the raw response to a stated scene reflectance.

Where the measurement stops #

The pipeline is a model, and a real converter’s matrix, curve and gamut handling will change the sizes. The mechanism does not depend on any of them: it is the balance’s unequal gains acting on an equal offset, and every converter has both.

The pedestal errors are illustrative sizes rather than measured ones. How far a particular sensor’s black level drifts with temperature and exposure time is a property of that sensor, and the result here is what a given drift costs rather than how likely it is.

And noise is not modelled. A real shadow’s colour is also affected by the noise’s own asymmetry near zero and by any clipping of negative values, which add their own biases on top of the offset’s.

The habit #

The habit is about an additive error meeting a multiplicative correction.

A correction that multiplies — a gain, a flat-field, a normalisation — is designed to remove a multiplicative error, and it does. Any additive error present at the same time is scaled by the correction instead of being removed, and if the correction’s gains differ between channels, the additive error acquires the pattern of the gains.

The move is to ask, of any offset in a system with channel-dependent gains, whether its shape matches the gains. An offset shaped like the reference is harmless; an offset of any other shape is converted into the reference’s inverse.

The failure mode is to diagnose the result as a problem with the correction. A cast that follows the balance was put there by what the balance was given, and adjusting the balance to remove it breaks everything the balance was right about.

Who noticed it first #

That black-level subtraction must precede white balance is in every raw processing specification and every converter’s implementation, and coloured casts in lifted shadows are among the most discussed faults in raw processing. Veiling glare and its dependence on scene content are standard in lens design.

That the cast takes the hue of the lamp’s inverse, that an offset in proportion to the lamp’s white is exactly neutral after the balance, and how large the tint is for a given error under two lamps, are consequences worked out here on one modelled sensor.

Still open: what a real sensor’s pedestal does #

The sizes above are what a stated error costs. What a real sensor’s black level actually does — how far it drifts between the masked pixels and the image area, with temperature, over an exposure of seconds or minutes — is a measurement on real dark frames, and it would turn the cost curve here into a statement about when a lifted shadow can be trusted.