The order is not in the documentation

Every description of raw conversion contains the same list. White balance, then a colour matrix, then a tone curve, then whatever the encoder does with values it cannot hold. The list is correct and it is drawn as a chain of boxes, and the arrows between the boxes are treated as decoration rather than as content.

The claim #

The four operations do not commute, so the arrangement is a colour decision of a size comparable to the profile’s own error, and it is written down nowhere.

• Exchanging the white balance and the colour matrix costs 9.21 colour differences at the mean and 13.24 at the worst patch.
• Exchanging the matrix and the tone curve costs 0.60 and 1.54 — smaller, and larger than a delivery tolerance.
• Exchanging the tone curve and the clip costs exactly nothing, and that is a theorem rather than a small number.
• The twenty-four arrangements produce five distinct outcomes, three of them identical to the documented one and the furthest 64.8 colour differences away.
• And the matrix that is best depends on which arrangement it will be used in: refitting it through the pipeline moves the picture by 0.71 at the mean.

Why they cannot commute #

Two operations commute when performing them in either order gives the same answer, and there is a short list of reasons that can happen.

Two linear maps commute when they share an eigenbasis. A white balance is a diagonal gain in the camera’s own three channels; a colour matrix is a general three-by-three fitted to take those channels somewhere else. A diagonal matrix and a general one share an eigenbasis only if the general one is itself diagonal in those coordinates, which would mean the camera needed no colour correction at all — that is the Luther condition, and silicon does not satisfy it.

A nonlinearity and a linear map commute essentially never, because the nonlinearity acts on each channel separately and the linear map mixes them. The single exception is a nonlinearity applied identically to channels that the matrix leaves proportional, which is to say a neutral.

And a clip does not commute with anything, because it is not invertible: information the clip removed cannot be put back by whatever follows it.

So the question is not whether the order matters but how much, and the answer needs a camera, a set of surfaces and a difference formula rather than an argument.

What each exchange costs #

The measurement runs the site’s own silicon sensor over thirty surfaces — twenty-four coloured patches and a six-step grey scale, which is the shape of a real chart — fits the matrix on them, and then swaps each adjacent pair in turn.

The white balance against the matrix is the large one. Nine colour differences at the mean and thirteen at the worst patch, which is not a subtlety: it is a picture with a visible cast. Balancing after the matrix means applying a per-channel gain in display coordinates rather than in the camera’s, and the two bases are as far apart as the camera is from Luther.

The matrix against the tone curve is the small one and it is not small. Six-tenths of a colour difference at the mean and 1.54 at the worst patch, which is above the tolerance in most delivery contracts. Applying the curve before the matrix means the matrix mixes compressed channels, and mixing compressed values is not compressing mixed ones.

The curve against the clip is exactly zero, and it is worth being clear about why. The curve is monotone and maps the unit interval onto itself. Clamping a value to that interval before applying such a function gives the same answer as clamping afterwards, for every input, exactly — which is the kind of statement this collection tests rather than asserts. That is the only commutation anywhere in this pipeline and it is a theorem about monotone functions rather than a measurement.

The twenty-four, and the five #

Four steps have twenty-four arrangements and the natural expectation is twenty-four answers.

There are five. Three arrangements are byte-identical to the documented one, and they are the three that differ only in where the clip sits among steps that cannot push a value out of range. The remaining twenty-one fall into four groups, at 0.60, 9.21 and two values within three hundredths of 64.8.

The clustering is the interesting part. The two large groups differ from one another by two hundredths of a colour difference, which means that once the balance has been moved to the wrong end of the chain, nothing else about the arrangement matters — the picture is already sixty-five colour differences away and the remaining permutations are rearranging the deckchairs.

That is a useful shape for a specification. A converter needs to get one thing right, and the one thing is that the balance precedes the matrix. Everything else in the chain is worth between nothing and 1.54.

The matrix that follows from the order #

A consequence that does not appear until the arrangement is treated as a variable: the nine numbers a converter ships are fitted against an objective, and the objective has the arrangement inside it.

Every camera profile in existence is a linear least squares in tristimulus values: one closed-form solve, one answer, and an objective nobody chose. Measured through the curve and the clip, the same objective has no closed form — because nothing downstream of the three numbers is linear — and descending from the linear answer improves the rms from 1.27 to 1.01 colour differences.

A quarter better is worth something and the interesting number is the other one. The two matrices render the surfaces 0.71 apart at the mean and 1.27 at the worst, which is to say they are genuinely different pictures rather than the same picture scored differently. A refit that improved the score and moved nothing would be a curiosity; this one moves the pixels.

And the refit’s answer depends on the arrangement it was fitted for, so a matrix is only a matrix relative to a pipeline. Shipping one with a raw file, as camera manufacturers do, is shipping a number whose meaning depends on software the manufacturer does not write — which is one more thing a photograph is not a measurement of.

What sixty-five colour differences looks like #

The largest group is worth describing rather than only quoting, because a number that size stops being an error and becomes a different picture.

Sixty-five colour differences at the mean, over a set of ordinary surfaces, is the distance between a photograph and a photograph of something else. The mechanism is simple once the arrangement is written out: with the balance last, the matrix is applied to unbalanced camera values, so it is asked to convert a signal in a basis it was not fitted for, and the result is then given a per-channel gain in display coordinates that cannot undo it because the matrix has already mixed the channels.

The white balance is the only step in the chain that carries information about the scene, and it is also the only one that has to be applied in the camera’s own coordinates. The matrix, the curve and the clip are all properties of the equipment; the balance is a property of the light in the room. Putting a scene-dependent gain after an equipment-dependent mixing is the arrangement that fails, and it fails for a reason that generalises well beyond cameras.

That also explains the clustering. Once the balance is in the wrong place the picture is wrong by an amount set by how far the light is from the fit illuminant, and the remaining permutations move it by a fraction of a colour difference on top of that. The three groups near 64.8 differ by under a hundredth, which is the curve’s contribution measured against a sixty-five unit error — invisible, and correctly so.

The exchange nobody would make, and the one everybody might #

Two of the three exchanges are hypothetical and one is not.

Nobody balances after the matrix. It is the one arrangement whose wrongness is visible in a thumbnail, and any converter that did it would have been fixed within a day of shipping. Its nine colour differences are here as a bound rather than as a hazard: they say what the chain’s largest available mistake is, and they say it is a mistake nobody makes.

The matrix against the curve is the live one. Both arrangements are defensible and both are implemented. Applying the matrix first is the colorimetric reading: convert to the destination’s coordinates while the values still mean linear light, then render. Applying the curve first is the rendering reading: get the tones where they should be while the values are still in the camera’s own channels, where the curve’s shape was designed, then convert.

Neither is wrong and they differ by 0.60 colour differences at the mean and 1.54 at the worst patch. That is the size of the disagreement between two competent converters that both follow the documentation, and it is not attributable to their matrices or their curves, both of which could be identical.

The clip’s placement is the third, and the theorem above says it is free — until something goes over the ceiling, which is the subject two rungs further along.

What this does to a reproducibility claim #

The practical consequence is about what it means to say that a colour workflow is reproducible.

A raw file plus a matrix plus a curve is usually treated as a complete specification of an image: hand those three to any converter and the same picture comes out. That claim is what makes a raw file an archival format rather than a proprietary one, and it is the claim this measurement bounds.

The three artefacts do not determine the picture. They determine it to within the arrangement, and the arrangement is worth up to sixty-five colour differences in principle and 1.54 in the range of arrangements a competent converter would actually choose. A tolerance of 1.54 on an archival format is not catastrophic and it is not nothing: it is larger than the tolerance most delivery contracts name, and it is invisible to every check anybody performs, because both converters agree with the specification.

The repair is one line of metadata and it does not exist. There is no field in any raw format, in any profile format, or in any of the metadata standards consulted here that names the order the steps are to be applied in.

What a converter could say and does not #

Three things would settle this and each is one line in a specification.

The arrangement. Four step names in an order. It fits in a tooltip, it costs nothing to state, and it would let one converter’s output be reproduced by another — which at present it cannot be, because the arrangement is inferred from behaviour rather than read.

Where the clip is. The clip is the one irreversible step, and the next rungs of this ladder are about what its position does to a highlight. The curve-and-clip commutation above means the answer is often it does not matter, and the exception is exactly the case a photographer cares about.

And what the matrix was fitted against. A matrix fitted in tristimulus values and used through a curve is being used for something it was not optimised for, and the size of that mismatch is the 0.71 above.

Varying the curve’s strength separates the two mechanisms cleanly. The balance-against-matrix number does not move at all when the curve softens, because that exchange is between two linear maps and the curve plays no part in it. The matrix-against-curve number scales with the curve’s strength, and goes to zero as the curve goes to the identity.

That separation is the check that the measurement is measuring what it claims. An effect that moved with a parameter it should not depend on would be an artefact; these two move exactly as their mechanisms say they should.

Five outcomes from twenty-four arrangements #

The collapse from twenty-four to five is worth taking apart, because the structure of the collapse is more informative than the extremes.

Three arrangements are byte-identical to the documented one. Two of them move the clip: the clip after the curve and the clip before the curve are the same operation, by the theorem above, and a clip placed before the matrix is also a no-op here because no balanced value in this set is over the ceiling. The third is the documented order itself.

Three sit at 0.60: the curve before the matrix, with the clip in each of the three places that make no difference.

Two sit at 9.21: the balance after the matrix, with the curve still before the clip.

And sixteen sit within three hundredths of 64.8, which is every arrangement in which the balance comes last or nearly last. That group’s internal spread is 0.025 colour differences across sixteen arrangements, which is a stronger statement than its size: once the largest error is present, the others are not merely small but nearly identical.

That is a structure worth naming because it is the shape a dominant term produces. The chain has one operation whose placement dominates, and the moment it is wrong the rest of the chain’s freedom is spent. A budget assembled by adding the three exchange costs — 9.21 plus 0.60 plus 0 — would predict 9.81 for the worst case and is out by a factor of six and a half, because the exchanges are not independent and the worst arrangement is not the sum of the worst exchanges.

Three adjacent swaps do not generate the group. They generate it in the sense of permutations and they do not compose in colour, which is the same failure of additivity this round has now met on four different objects.

The sensor’s own contribution #

One question a reader should ask is how much of this belongs to the camera rather than to the arithmetic, and it has a clean answer.

The balance-and-matrix exchange is a statement about how far the camera’s matrix is from diagonal, which is a statement about how far the sensor is from the Luther condition. A colorimetric sensor — one whose channels are the colour-matching functions — has a matrix that is exactly the identity up to a scale, and for it the two operations would commute exactly.

So the 9.21 is a measure of the camera’s own non-colorimetry, expressed in a new way. It is not the same measure as the profile’s residual: a sensor could have a small residual and a very non-diagonal matrix, or the reverse. The exchange cost is a property of the matrix’s off-diagonal terms and the residual is a property of what the matrix cannot fix, and the two are close to independent.

That gives the number a use beyond this essay. A manufacturer wanting one figure for how much their sensor’s colour depends on the pipeline can compute the balance-and-matrix exchange over their own chart, and it requires nothing they do not already have.

What was computed, and how #

The sensor is this collection’s own silicon model: a quantum efficiency curve, three colour-filter dyes and an infrared-cut filter, all as stated formulae rather than tabulated. The surfaces are twenty-four coloured patches from the imaging library plus a six-step grey scale, and the grey scale is load-bearing — an unconstrained least squares over coloured patches alone leaves the matrix’s row sums free, because nothing in such a set has all three balanced channels equal, and the first version of this calculation produced a matrix that made the camera’s own white forty per cent too bright, which is the fit’s sample set deciding the answer in a form nobody had met.

With the grey scale in the fit set the matrix takes the camera’s white to the display’s white to within six parts in a thousand, and nothing imposed that: it is what the fit does once the fit is asked about neutrals.

The tone curve is a smooth S applied in the encoded variable at a stated strength, which is what a converter’s default curve is. Nothing here depends on its exact shape: every result scales with the strength and the two linear-map results do not depend on it at all.

The comparison is made in CIELAB after the whole chain, which is where a viewer’s judgement would be made, and the difference formula is ΔE₀₀.

Where the model stops #

Four steps is a simplification of a real converter, which also does noise reduction, sharpening, lens correction, highlight reconstruction and a local tone map. Each of those adds arrangements, and the count grows as a factorial. A grey edge acquiring colour is one of them measured on its own.

The matrix is fitted here rather than shipped, so the absolute numbers belong to this sensor and this fit set. The ordering of the three effects — balance-and-matrix large, matrix-and-curve small, curve-and-clip zero — is structural and would survive any camera.

And nothing here says which arrangement is right. Balance before matrix is right, because a white balance is an adaptation and an adaptation belongs in the receptors’ own coordinates. Where the curve goes relative to the matrix is a genuine design question with defensible answers on both sides, and this essay measures the cost of the disagreement rather than resolving it.

The generalisation #

The habit is about a diagram whose arrows carry information.

A pipeline drawn as boxes with arrows between them presents its steps as the content and its order as the layout. When the steps are linear the presentation is nearly honest — order still matters, but a reader who assumes otherwise is out by a bounded amount. When any step is a compression, a threshold, a clamp or a lookup, the arrows carry as much as the boxes.

The move is to state the order as part of the specification rather than as part of the picture. It is four words and it makes the process reproducible.

The failure mode is that two implementations of the same documented process produce different results and neither can be shown to be wrong, because the document they both satisfy does not distinguish them. A specification that admits sixty-five colour differences of freedom is not a specification, and the reason it looks like one is that its freedom is in the diagram rather than in the text.

Who found it, and when #

That raw conversion is a sequence of choices rather than a decoding is well understood by the people who write converters, and the differences between them are a permanent subject of argument among photographers. The usual explanation for why two converters disagree about the same file is that they use different matrices and different default curves, and both are true.

The order does not appear in those discussions and does not appear in the documentation of any converter consulted here. Its absence is probably because the order feels like an implementation detail rather than a parameter — which is exactly the reading this measurement contradicts.

Where the ladder goes next #

Four of the pipeline’s operations can be arranged twenty-four ways. A fifth cannot be arranged at all: the reconstruction has to come first, not by convention but because a colour matrix needs three numbers and a mosaic site has one — and what happens when it is nevertheless moved is measurable.