A chart decides what a camera scores

Two laboratories measuring the same camera with the same method and different charts will report numbers a factor of five apart, and both will be right.

The residual being measured here is measured against an observer rather than against nothing, and the standard supplies two observers rather than one.

The control rebuilt for that same ten-degree observer is what says the transfer of the construction is a property of the condition rather than a property of any particular sensor.

The claim #

A camera profile’s reported error is a statement about the chart it was measured on, and the chart’s saturation is the axis that decides it.

• Holding the fit fixed and sweeping the test chart’s chroma from 0.10 to 1.00 moves the reported mean error from 0.373 to 2.002 ΔE*₀₀ — a factor of 5.4.
• As an elasticity that is 0.672, over the same span this collection uses for every other sensitivity in it.
• The census of chromatic adaptation gives 0.687 for the same quantity over a completely different construction of a test set, with no shared code.
• The worst patch is 2.08 times the mean, which is the same factor of about two that shows up wherever a mean over surfaces is reported.
• So the chart’s chroma range is the specification, and everything else about the chart is second order.

What a camera profile is #

A camera has three channels and so does an observer, and a camera is not an observer because its three sensitivities are not a linear combination of the colour-matching functions. The standard repair is a 3×3 matrix from the camera’s raw responses to tristimulus values, fitted by least squares over a set of surfaces — a set that is a declaration like any other.

Two arguments decide everything about the number that comes out: which surfaces the matrix is fitted on, and which surfaces it is scored on. Passing the same set for both is how a camera profile is usually evaluated, and it is a measurement of the fit rather than of the camera — which this collection established two rounds ago and is not the subject here.

The subject here is the second argument alone. Hold the matrix at whatever it is, and ask what changing the scoring set does.

The sweep #

The test surfaces are Gaussian reflectance bumps of stated chroma — a parameter that scales the depth of the band, from a flat grey at 0 to as saturated as a smooth single-lobed reflectance gets at 1. The fit is held at chroma 0.25 throughout, so the matrix never changes.

A factor of 5.4 on the mean and 6.5 on the worst patch, with the camera untouched.

Both ends are defensible chart designs. A chart of pastel patches at chroma 0.10 is what a portrait photographer’s targets look like, and one at the saturated end is what a delivery tolerance is checked against; a chart at chroma 1.0 is what a printing-ink evaluation looks like. Neither is wrong, and a number quoted from one has no useful relation to a number quoted from the other.

The cross-check #

The number that makes this worth an essay rather than a caution is that it turns up twice.

The adaptation census is a mean residual after a von Kries gain, computed over a lattice of cosine-combination reflectances, reported in ΔE*₀₀. The camera profile is a mean fitting error, computed over Gaussian bumps, reported in ΔE*₀₀. They share no code path: different libraries, different surface constructions, different physical questions, different reasons for existing.

Their elasticity to the saturation of the test set is 0.687 and 0.672.

Two per cent apart. That is not a coincidence and it is not a shared bug — there is no shared code to carry one. It is what happens when two quantities are both a failure to handle spectral structure, reported in a compressive colour-difference metric:

• the physical mismatch scales roughly linearly with modulation depth, which would give an elasticity of one;
• CIEDE2000’s chroma weighting grows with chroma, so a fixed physical mismatch on a saturated sample earns a smaller reported difference;
• and the two combine to about 0.7, whatever the mismatch is a mismatch of.

So the elasticity is a property of the pairing between “a spectral mismatch” and “a perceptually weighted difference metric”, not of either measurement. That is the kind of statement two instruments agreeing can support and one cannot.

What it means for reporting #

Quote the chart’s chroma range with the error. Not the patch count, not the manufacturer, not the illuminant it was measured under — those matter and they matter less. A profile error with no chroma range beside it is a number whose meaning is a factor of five wide.

Do not compare two published camera errors measured on different charts. The between-chart variation swamps the between-camera variation, in the way a test set’s own construction swamps what it is measuring for any two cameras of the same class. This is the ordinary reason two laboratories disagree about the same device, and the ordinary fix — agree on a chart — works precisely because the sensitivity to everything else is small.

And when charts must differ, correct with the elasticity. An error measured at chroma c₁ and wanted at c₂ scales as (c₂/c₁)^0.67, which is a one-line correction and is right to within a few per cent across the whole range measured here. That is far better than the alternative, which is comparing the numbers unadjusted.

Why the worst patch is twice the mean #

The right-hand column of the sweep runs at 1.71 to 2.08 times the left, and the ratio grows slowly with chroma.

That is the same factor of two the adaptation census shows between its published means and its worst surfaces, and for the same reason: a mean over a set of surfaces summarises a distribution whose upper tail is where the model’s failure concentrates. Here the concentration is on the patches whose spectral shape is furthest from anything the camera’s three sensitivities can resolve — which, in a chart of single-lobed bumps, are the ones whose lobe falls where the camera’s and the observer’s sensitivities most disagree.

A profile reported with a mean and a maximum is reporting the distribution; a profile reported with a mean is reporting a third of it. The ISO conditions for camera characterisation ask for both, for exactly this reason, and the reason is worth knowing rather than complying with.

Why the fit’s chart matters differently #

One distinction to keep separate, because conflating the two is easy.

Sweeping the scoring chart with the fit fixed is what this essay measures: it changes what is reported about a fixed device. Sweeping the fitting chart changes the device’s profile itself, which is a different intervention with a different signature — a matrix fitted on pastels and used on saturated inks is a worse matrix, which is a fit measured on its own training set, not merely a differently-scored one.

Doing both at once, so that fit and test are at the same chroma, gives 0.361 at 0.10 and 2.082 at 1.00 — nearly the same numbers as holding the fit. So on this camera the fit’s chart barely matters and the scoring chart is nearly all of it, which is a useful and slightly surprising result: the least-squares 3×3 is stable to what it is fitted on and the reported error is not stable to what it is scored on.

The reason is that a 3×3 has nine degrees of freedom against a spectral mismatch with far more, so it lands in nearly the same place whatever smooth surfaces it is shown. A fit can be exact and empty is the same observation from the other side.

What the chart’s own description is worth can be read across the whole census rather than at the one setting this essay’s sweep uses.

The control that says this is a mismatch #

The sweep on its own is consistent with a less interesting explanation: that CIEDE2000 simply reports larger numbers for more saturated samples whatever is going on, so any error measured on a saturated chart would be larger.

The check is a sensor that has nothing to be wrong about. A Luther-condition sensor — one whose three spectral sensitivities are an exact linear combination of the colour-matching functions — has a 3×3 that is not a fit but an identity, and its profile error is zero for every surface under every light. Sweeping the chart’s chroma on it gives zero at every chroma.

So the elasticity is measuring a mismatch and not a metric artefact. A metric-only effect would appear on the control as well, and it does not, because there is nothing for the metric to weight. That is the assertion this collection’s own habit demands: the machinery is handed input it must give a different answer for, and it does.

The same check has a second use. The difference between the real sensor’s curve and the control’s flat zero is the mismatch, chart by chart, so the sweep is a decomposition of what the Luther condition’s violation costs as a function of what it is measured on. At chroma 0.10 the violation costs 0.37 ΔE*₀₀ and at 1.00 it costs 2.00, and both are the same violation.

What a standard could ask for #

Two lines, and neither needs a new measurement.

A reported profile error should carry the mean chroma of the chart it was measured on. That is one spectrophotometric number per chart, computed once by whoever makes the chart, and it turns an uncomparable figure into a comparable one via the exponent measured here.

And it should carry a maximum as well as a mean. The worst patch runs 1.7 to 2.1 times the mean across the whole sweep, so a mean alone understates what a specification has to tolerate by about half — the same factor of two that turns up wherever a mean over surfaces is published.

Neither is a new burden. Both quantities already exist inside every characterisation any laboratory runs, and are discarded before publication.

Where the model stops #

The chart here is a constructed family of Gaussian bumps, not a real chart. A real target — a twenty-four-patch colour chart, a printed ink set — is a fixed collection with a fixed and unstated chroma distribution, and the sweep says how much a change in that distribution is worth rather than what any particular target’s value is.

Applying the elasticity to a real chart therefore needs the chart’s chroma distribution measured, which is a spectrophotometric measurement anybody with the chart can make and which nobody publishes. That is the actionable gap: the correction exists, the constant is measured, and the input it needs is a number charts do not carry.

And the elasticity is local. At chroma above about 0.9 the bumps begin to clip against the physical reflectance bounds, and the relationship flattens; below 0.1 the patches are nearly grey and the reported error approaches the small residual a 3×3 leaves on neutrals. The 0.67 is right across the middle of the range and is not a law.

What the exponent is not #

Three readings the sweep does not support, and each is available enough to be worth refusing.

It is not that saturated charts are better. A chart at chroma 1.0 reports a larger error, and a larger error is not a more informative one — it is the same mismatch weighted differently. What a saturated chart does buy is discrimination between cameras, because two devices that differ by 0.1 ΔE*₀₀ on pastels differ by 0.5 on saturated patches and the second is easier to measure against noise.

It is not that the exponent is a property of cameras. It comes out at 0.67 on the silicon sensor modelled here and at essentially the same value on the others, because it is a property of how a spectral mismatch scales with modulation depth and how CIEDE2000 weights the result. A sensor with a wildly different failure mode — an infrared leak, say, which acts on the surfaces’ long-wavelength tails rather than on their visible modulation — would not follow it.

And it is not a correction that can be applied blind. Rescaling by (c₂/c₁)^0.67 assumes both charts differ only in chroma. Two charts that differ in chroma and in how their patches are distributed round the hue circle differ in a second way this exponent knows nothing about, and the delivery half of this collection shows how large a hue dependence can be.

The prediction about the infrared leak, tested #

The second refusal above is a prediction rather than a caution: a sensor whose failure acts on the surfaces’ long-wavelength tails rather than on their visible modulation would not follow the exponent. That can be run, and it is the sharpest available test of what the exponent is a property of.

Four sensors, the same fit set, the same ten scoring charts from chroma 0.10 to 1.00, and an ordinary least-squares slope of log error against log chroma over all ten points rather than a ratio of the two ends:

The prediction holds. Removing the infrared cut moves the slope from 0.690 to 0.826, a fifth of the way to linear, and it degrades the power law itself — the residual variance quadruples, from R² 0.9937 to 0.9714. Both are the signature the refusal describes: a leak adds an error that does not scale with visible modulation, so it dominates at the pale end, and the curve stops being a straight line on log axes because two mechanisms of different exponents are being added together.

Moving the cut rather than removing it separates level from slope cleanly. A 700 nm filter instead of a 665 nm one nearly doubles the error at every chroma — 0.716 against 0.373 — and moves the slope by three per cent. That is the useful discrimination: the sensor’s design sets how much mismatch there is, and the exponent describes how the reported figure scales with the chart, and those two are almost independent as long as the failure is a visible-band mismatch.

The control says what the other three rows are measuring. A sensor satisfying Luther exactly reports 6.8 × 10⁻¹¹ at chroma 0.10 and 1.6 × 10⁻¹⁰ at 1.00 — floating-point residue, four orders of magnitude below anything else in the table and eleven below the numbers this essay is about. Whatever slope is fitted through those two values is a slope through arithmetic noise, and the fact that it comes out near 0.36 rather than near 0.69 is the point: nothing about the sweep is an artefact of CIEDE2000 or of how the charts are constructed, because both survive here with the mismatch removed and the effect does not.

Why the fit is stable and the score is not #

The asymmetry in the middle of this essay is worth a section, because it is the opposite of what most people expect and it has a clean reason.

The matrix barely moves when its fitting chart changes. Fitting at chroma 0.10 and at 1.00 gives matrices whose reported errors on a common test set differ by a few per cent, so a camera profile is a robust object.

The reported error moves by a factor of five when its scoring chart changes.

The reason is dimensional. A 3×3 has nine degrees of freedom, and it is being fitted to a mismatch — the difference between the camera’s three sensitivities and three colour-matching functions — that lives in a function space with hundreds. A least-squares fit to a vastly over-determined problem lands in nearly the same place whatever smooth surfaces it is shown, because the leading structure of the mismatch dominates every sample. The residual, by contrast, is what the fit could not take, and how much of that a particular chart exposes is exactly what the chart’s chroma decides.

So the two facts are one fact: the fit captures the part of the mismatch every chart agrees about, and the score measures the part they disagree about. A stable fit and an unstable score are the signature of a model with too few parameters for its problem — which a 3×3 emphatically is, and which a fit that is exact and empty is the extreme case of.

Who found it, and when #

That a camera’s characterisation error depends on its target is not news to anybody who has done one, and the standards say so obliquely by specifying the target. What is not usually done is to measure the dependence and publish a correction constant.

Its appearance here was a test of whether this round’s thesis reached a second part of the collection at all. The adaptation census’s elasticity to its test set’s saturation had been measured; the question was whether that was a fact about the census or a fact about the question. The answer coming back at 0.672 against 0.687, from machinery that shares nothing, is what turned a finding about one file into a finding about a kind of measurement.

Where the model stops, twice over #

Two limits already stated and one that is not.

The chart is constructed and its chroma is a parameter rather than a measurement, so the sweep gives an elasticity and not a value for any real target. The elasticity is local, flattening above chroma 0.9 where the bumps clip and below 0.1 where the patches are nearly grey.

The third limit is the one that matters most for using the result. The sweep holds the illuminant fixed at D65. A camera’s mismatch with the observer is a mismatch of spectral sensitivities, and how much of it a given surface reveals depends on the light as well as on the surface — exactly as it does for a fourth reflectance dimension, where a narrowband source and a smooth one disagree about which surface structure matters. So a characterisation under a tungsten lamp or a fluorescent tube would have its own exponent, probably not 0.67, and measuring it is the obvious next thing.

Where the ladder goes next #

The same shape appears a third time in the delivery half of the collection, where a whole colour-management budget is computed over one ramp of one hue.