Four places to clamp are two pipelines

Clipped noise does not average away found that a converter’s clamp at zero turns a raw reading’s symmetrical noise into a one-sided pedestal, that the pedestal is coloured because the three channels reach zero at different signals, and that where in the pipeline the clamp sits decides how much of it survives. It ended on a procedure rather than a result: which place a real converter uses is not in its documentation, and a black frame and a dim grey card at a high amplification would place each one.

A procedure is a claim that a measurement identifies something, which is the question where a camera is blind to itself asks of a camera’s own calibration. Before running this one on a converter it is worth asking what it could identify at best.

Two of the four are the same pipeline #

A clamp at zero commutes with the white balance and with the tone curve and not with the colour matrix, so the four positions a converter could clamp at are two pipelines — and the proposed measurement separates those two, although not on the card it names.

• Two of the six pairs are identities, to machine precision, for every raw input including negative ones.
• The line between the two groups is the colour matrix, the only step that mixes channels. A clamp commutes with a positive scale per channel and with a monotone curve that fixes zero, and the matrix is neither.
• The measurement separates them by 1.98 colour differences against its own noise of 0.057 at twenty thousand pixels.
• The best card is not the black frame. The separation is 1.63 on black, peaks at 1.98 on a card returning half a per cent of the white, and has fallen to 0.06 by a twelfth.
• Ten thousand pixels is enough and a hundred is not, and at a low amplification the whole effect is 0.045 — so the high gain in the original suggestion was load-bearing.

Why two of them collapse #

The clamp this is about is the one at zero, which is what turns symmetrical noise into a pedestal. The three other steps of a raw pipeline are a white balance, a colour matrix and a tone curve, and the clamp can sit in any of four places among them.

A white balance is a positive scale applied channel by channel. Scaling a number and then taking the larger of it and zero gives the same answer as taking the larger first and then scaling, because the scale is positive and so preserves the sign. So a clamp before the balance and a clamp after it are one pipeline.

A tone curve is monotone and fixes zero. Applying it and then clamping gives the same answer as clamping and then applying it, for the same reason one step further along: the curve cannot turn a negative into a positive or a positive into a negative. So a clamp before the tone curve and a clamp after it are one pipeline too.

A colour matrix mixes the channels, and that is exactly what breaks the commutation. A black level is multiplied by the balance is the neighbouring case, where an offset and a gain fail to commute for the same reason in the other direction. A matrix row is a weighted sum with negative weights in it, so a channel that was positive can come out negative and a channel that was negative can be outvoted by two positive ones — and whether the clamp ran before or after decides which. One step has no choice establishes that the matrix is the step with the fewest degrees of freedom about its position; here it is the only step whose position can be seen at all.

So the answer to where does this converter clamp has two possible values, not four, and a measurement that returned four would be reporting something that is not there.

The commutation arguments above are each two lines and all three are worth checking rather than believing, because each rests on a property of a step that an implementation could quietly lack. A white balance that clamped its own output would not commute; a tone curve fitted with a small negative lift at the bottom — which some film emulations carry — would not fix zero and would not commute either. The check is the identity table: run both pipelines over a grid that includes negative inputs, and require a difference of exactly zero rather than a small one. A commutation that holds to three decimal places is a commutation that does not hold, and the distinction matters here because a pair that nearly commutes is a pair a measurement with enough pixels could separate.

The signature, and the card nobody proposed #

With the question narrowed, the measurement can be asked whether it answers it.

The separation is 1.63 colour differences on a black frame, rises to 1.98 on a card returning half a per cent of the white, and falls to 0.06 by a twelfth. The measurement’s own noise — found by measuring the same pipeline twice with different pixels — is 0.057, so the peak clears it by a factor of thirty-five.

The black frame is not the best card, and the reason is about what a clamp acts on. A clamp at zero bites on noise that straddles zero. On a black frame every channel’s noise straddles zero at once, so all three are clamped alike and much of what the matrix would have done to their differences is gone before it runs. On a dim card the channels are at different distances from zero — the lamp drives them unequally, and at a hundredth of the white the starved channel is still straddling while the strong one is clear — so the clamp acts on some and not others, and the matrix’s mixing has something asymmetric to work on.

That is a concrete correction to the procedure and it costs nothing to adopt: print the card at half a per cent rather than relying on the black frame, or better, print a ramp of them and read the peak.

The pedestal the two pipelines deliver #

The separation is a number and it is worth knowing what it is a separation of, because the two pipelines fail differently rather than by different amounts.

A clamp before the matrix acts on each channel at its own distance from zero. Under an incandescent lamp the blue channel is starved and its noise straddles zero while the red channel’s does not, so the clamp lifts blue and leaves red — and the matrix then mixes that asymmetric lift into all three outputs. A clamp after the matrix acts on the matrixed values, whose distances from zero are a different set, and lifts a different combination.

Both deliver a black frame with a cast in it and the casts are not the same cast. That is why the statistic is a colour difference rather than a lightness difference, and it is why the effect survives the tone curve: a curve applied per channel cannot remove a difference between channels, and a contrast control is three controls is what it does to one instead.

The practical form of that is a test a user could apply to a delivered picture rather than to a card. A deep shadow in an ordinary photograph, averaged over a flat region, has a mean; if that mean is neutral the converter is not clamping where the noise reaches, and if it has a cast the direction of the cast says which side of the matrix the clamp is on. It is the same measurement without the card, with worse control and no extra equipment.

How many pixels, and how much gain #

The other two things a procedure needs to state are how much of the frame it needs and under what conditions.

The noise falls as the square root of the count — 5.36 colour differences at a hundred pixels, 0.42 at ten thousand, 0.021 at a million — while the separation stays at 1.98. So a hundred pixels identifies nothing, a thousand is marginal, and ten thousand identifies everything identifiable. A flat card occupying a modest part of one frame holds far more than that.

At a low amplification the whole effect is 0.045 of a colour difference and at a very high one 1.98 — a factor of forty-four. That is because what distinguishes the two pipelines is what the clamp does to noise, and a large sensor at a low amplification has almost no noise reaching zero on a card that is not quite black. The high gain in the original suggestion is not a convenience for making the effect easier to see; it is the only condition under which the question has an answer.

A coloured card does not help #

One intuition is worth testing because it is reasonable and would cost a print run.

If the distinguishing step is the colour matrix, a coloured target ought to separate the two pipelines better than a grey one — the matrix does more to a colour than to a neutral, so putting a clamp before or after it ought to matter more.

The two curves are close and the grey one is slightly higher: 1.98 against 1.77 at the best card. The intuition is wrong because of what the clamp acts on. The clamp acts on the noise, not on the signal, and the noise is the sensor’s rather than the card’s: at these levels it is dominated by read noise, which is the same whatever the card returns. A coloured card changes only what the noise is added to, and moves the three channels’ distances from zero around — which here happens to be slightly unhelpful.

So the procedure stays as it was named, with one card changed: dim greys at a high amplification, in one frame, with the best reading at half a per cent of the white.

What the procedure would actually tell a user #

Three things follow, and the third is the one worth carrying.

It returns one bit. A converter clamps before the colour matrix or after it, and that is the whole of what any measurement of any scene can recover. A user expecting to learn the clamp’s exact place in a four-step list will be disappointed by a correct answer.

The bit is worth having. The earlier work priced the difference between the two arrangements, and a shadow’s cast is delivered along with its detail in one of them and not the other. A converter’s choice here is not a detail of implementation; it is the difference between a black frame that averages to neutral and one that averages to a colour.

And what the bit cannot tell anyone is whether the clamp should be there at all. A converter that runs its noise reduction on signed raw values has no clamp until much later and delivers neither pedestal — which is the arrangement two converters and one highlight found them differing over at the other end of the range; that is a third pipeline, distinguishable from both of these, and the same measurement finds it. The order is not in the documentation is the standing problem, and this narrows it: the part of the order that matters for shadows is one bit, it is measurable in one frame, and nobody publishes it.

How the pipelines and the signature were computed #

The four placements insert a clamp at zero into the fixed order of the other three steps, and the ceiling at one is applied last in every case, so that only the floor’s position varies. That is a different set from the twenty-four permutations of four steps measured elsewhere here: the ceiling and the floor are different operations in different places, and treating them together is right when neither bites and wrong here.

The identity check runs both pipelines over a grid of raw values from a fifth below zero to a half above the white, in every combination of the three channels, and reports the largest difference in the delivered value. Negative inputs are included deliberately: a clamp’s whole subject is what happens at the ends, and a grid of positive values would report every pair as identical.

The signature is the delivered mean of a stated number of noisy readings of one card, with the signal the card’s reflectance times the lamp’s raw white and the noise the sensor’s at that signal. The measurement’s own noise is the same signature computed twice with different pixels, which is what a second frame would give.

What this leaves out #

The pipeline is four steps. A real converter has a dozen — a black-level subtraction, a demosaic, a noise reduction, a lens correction, a local tone map — and each is another place a clamp could sit and another step it might or might not commute with. What survives is the argument rather than the count, in the way the order is not in the documentation intends: any step that is a positive per-channel function commutes with a floor at zero, any step that mixes channels does not, and the number of distinguishable positions is one more than the number of mixing steps.

The noise is Gaussian and independent per channel. A real sensor’s read noise has a shape, and neighbouring photosites are correlated through the demosaic; both change the pedestal’s size and neither changes which pairs are identities.

And the identity is exact for this clamp. A converter that clamps at a small positive value rather than at zero — which some do, to leave headroom for a later subtraction — has a clamp that does not commute with the tone curve, because the curve does not fix that value. That is a fifth pipeline and the measurement would separate it.

Still open: what a real converter answers #

Everything here is about what the measurement could say. What it does say is one frame’s work and nobody has taken it.

The procedure is now fully specified: a flat card returning about half a per cent of the raw white, photographed at the highest amplification the camera offers, with ten thousand or more pixels of it in frame, delivered through the converter with every optional correction off and nothing else changed. The statistic is the delivered mean, and the prediction is a colour: one pipeline delivers a neutral mean and the other a mean with a cast in it, in the direction of the channel the lamp starves.

Two outcomes are interesting for different reasons. If converters differ, the bit is worth publishing and a user comparing two of them on shadow rendering has a mechanism rather than an impression. If they all agree, the question is why — the arrangement is not forced by anything, the specification does not name it, and unanimity would suggest they share more code than their documentation implies.

A procedure is a claim about identifiability #

The habit is about what to do with a proposed measurement before making it.

A procedure names an apparatus and a statistic and implies that the statistic pins the thing. That implication is checkable in advance, and checking it is cheaper than the measurement: enumerate the hypotheses, compute what each would produce, and look for pairs that produce the same thing. A pair that does is not a limitation of the apparatus and cannot be fixed by a better one.

The move is to run the enumeration on the model before running the procedure on the world. It produces three useful things at once — the number of answers the measurement can return, which are indistinguishable, and where in the range the measurement is most sensitive — and the third is usually not where the proposal said to look.

The failure mode is to make the measurement and report the count of hypotheses. A procedure that separates two pipelines and is described as placing a clamp among four positions will eventually be run by somebody who concludes that two converters differ when they do not.