Three curves for one space

Six of the seven arguments in this round’s audit are measurements of an eye. The seventh is not a measurement at all, and printing its zero beside the other six is the clearest available statement of what the six are measurements of.

The claim #

Three curves spanning one subspace are one observer, and any invertible linear recombination of them is the same observer exactly.

• Measured at 8 × 10⁻¹⁵ ΔE₀₀ through an arbitrary three-by-three, which is the floating-point floor rather than a small residual.
• It follows from what a match is. Two lights match when three integrals agree, and a set of integrals that agree still agree after any invertible mixing.
• So x̄ȳz̄ and a set of cone fundamentals are the same observer in different clothes, and every reported disagreement between them is a disagreement about the space.
• And the exactness has a condition which the next essay is about: it holds when the white is divided out before the rotation is undone, and fails by 8.11 ΔE₀₀ when the division happens in the rotated coordinates.

What an observer is, formally #

A colour match is an identity between three numbers. Two spectral power distributions φ and ψ match for an observer with sensitivities l₁, l₂, l₃ when

∫ φ lᵢ = ∫ ψ lᵢ for i = 1, 2, 3.

That is three linear functionals applied to a function, and what it says about φ − ψ is that the difference lies in the null space of all three. Colour is exactly three-dimensional because there are three functionals; everything in the null space is invisible, and that null space is what a metameric black is.

Now take any invertible three-by-three T and form mᵢ = Σⱼ Tᵢⱼ lⱼ. The three new functionals have the same null space, because a vector annihilated by all of l₁, l₂, l₃ is annihilated by any combination of them, and invertibility gives the converse. The set of matching pairs is unchanged.

So an observer, considered as a predictor of matches, is not three curves. It is the three-dimensional subspace of functionals they span, or equivalently the codimension-three null space they annihilate. Curves are coordinates on it.

The measurement is a check on the arithmetic rather than on the theorem. The theorem is not in doubt and the point of computing it is to check that the arithmetic in this collection implements it rather than something adjacent.

Take the reference observer’s three cone absorptances. Apply

T = [[0.72, 0.31, −0.04], [−0.19, 1.14, 0.06], [0.03, −0.08, 0.97]]

to get three different curves — the first is mostly the long-wavelength cone with a third of the middle mixed in, the second is mostly the middle with a fifth of the long subtracted, and neither resembles a receptor. Compose the tristimulus matrix with T⁻¹ so that the same three tristimulus values come out. Compute a colour.

The answer differs from the unrotated computation by 8.1 × 10⁻¹⁵ ΔE₀₀. Not nearly: at the floor, which for a quantity of order ten in double precision is where an exact identity lands after a few hundred floating-point operations.

That is the same standard of evidence this round applies to its other identities, and it matters here because a small number would have meant something quite different — that the implementation was doing something almost but not quite equivalent, which is the commonest way an identity gets broken in code.

Why the literature keeps reopening it #

Given a theorem this clean, the recurring debate about whether to work in x̄ȳz̄ or in cone fundamentals needs an explanation, and there are three good ones.

The first is that the two are not the same space. The 1931 functions and the Stockman–Sharpe fundamentals are not related by any three-by-three, because they are different measurements: the 1931 set descends from Wright’s and Guild’s matching data with a luminance constraint imposed, and the fundamentals descend from dichromat matching and are known to differ from the 1931 set in the short wavelengths. A real disagreement about the subspace is a real disagreement about which pairs of lights match, and the correction has existed since 1951 without being adopted.

The second is that a basis has numerical consequences. Two bases for one space give identical exact arithmetic and different conditioning. A matrix inverted in one basis and in another can differ by orders of magnitude in its sensitivity to noise, and a camera profile fitted in one basis is not the fit made in another.

And the third is that everything downstream is nonlinear. The identity is about the three numbers. CIELAB takes cube roots of them, an adaptation transform takes ratios of them, and an appearance model takes both — and none of those operations commutes with a change of basis. That is where the exactness ends, and it ends immediately.

Placing the theorem in that figure beside seven measured identities is worth a sentence of defence, because they are different kinds of claim. The six sample-side rows and the gain row are statements that a particular quantity is zero, checked numerically. The basis row is a statement that an operation changes nothing, and it would be true of any observer, any light and any sample.

It is drawn there anyway because the figure’s job is to show what makes a departure vanish, and a change of basis vanishes for a reason worth putting beside the others. Seven of the eight rows empty a factor of the pairing; the eighth says the pairing was never there, because nothing was changed.

Where the exactness ends, precisely #

The identity is stated carefully above and the careful part is easy to skip: the rotation is undone before anything nonlinear happens.

Divide the white out in the rotated coordinates — which is what an analyst would do who had only ever seen the rotated curves and believed them to be receptors — and the answer moves by 8.11 ΔE₀₀ on a red pigment under daylight. Nothing has changed about the observer; what has changed is which three ratios are being held fixed.

That is a large number for an operation that looks like bookkeeping, and it is the whole subject of the next essay. Stated here in its narrowest form: a linear identity survives any linear operation and none of the nonlinear ones the field routinely performs, and every published colour pipeline performs at least two.

What this settles and what it does not #

It settles the question of whether a computation should be done in one basis or another, for anything that is purely linear. It should not matter and it does not, and an implementation where it does has a bug.

It does not settle which curves to publish, and there the arguments are real and are about people rather than mathematics. Cone fundamentals are interpretable — a value is a probability that a photon is caught by a named pigment — and x̄ȳz̄ are not, because x̄ has two peaks and none of the three corresponds to anything. Against that, the 1931 functions have a century of tabulated data and industrial specification behind them, and ȳ is a photometric quantity in its own right.

And it does not settle what happens when the two sets disagree about the space. The 1931 functions are known to be wrong in the blue by a factor approaching ten in ȳ at short wavelengths, which is not a rotation of anything. A reader who takes the theorem above as licence to treat the two sets as interchangeable has read it as saying more than it does.

The residual is what a rotation cannot fix #

That figure is the same theorem read as a diagnostic, and it is a use of it worth having.

This collection’s analytic observer is three cone absorptances carried to tristimulus values by one fitted three-by-three. The fit is a least-squares search over exactly the group the theorem says is free, so whatever residual is left is not a basis problem. The rotation that could have removed it has already been applied and the best it can do is 2.3 per cent root-mean-square on ȳ and 16.4 on z̄.

That is a much stronger statement than “the model does not match the table”. It says the model’s three curves span a different subspace from the tabulated set’s, and no amount of recombining them will bring the two together. The disagreement is about which pairs of lights match, which is the only kind of observer disagreement there is.

The same logic settles a question about the ten-degree functions that gets asked in the other direction. The 1964 set is not a rotation of the 1931 set — if it were, the two would predict identical matches and there would be no reason to publish both — and the gap between them is 2.65 ΔE₀₀ over this collection’s own surfaces, which is a subspace difference and not a coordinate one.

A short way to test any claim about bases #

The theorem gives a one-line test for whether a claimed disagreement between two sets of colour-matching functions is real, and it is worth stating because it is quick.

Fit the best three-by-three between them and look at the residual. If the residual is at the numerical floor, the two sets are the same observer and any reported difference is an artefact of coordinates, of conditioning, or of a nonlinear step somewhere downstream. If the residual is not at the floor, the two sets disagree about which lights match, and the size of the residual is the size of the real disagreement.

Applied to the pairs this collection carries: the 1931 and 1964 sets fail the test comprehensively, as they must. This collection’s template and the 1931 set fail it at 16.4 per cent on z̄. And any two derivations of the same underlying data — a set of fundamentals published in cone units and the same set published in energy units, say — pass it exactly, and a great deal of confusion in the literature is about pairs of that kind.

That figure is the theorem’s complement and is the quickest way to see what it excludes. A rotation mixes the three curves; it cannot move one of them along the axis, because a shifted curve is not in the span of the unshifted three. Any change that leaves the span alone is invisible and any change that leaves it is not, and the span is exactly what “which pairs of lights match” means.

The same test disposes of a question that gets asked about the ten-degree functions. Two sets of curves are the same observer if and only if each is in the other’s span, and a fit that leaves a residual has established that they are not. There is no third possibility and no partial credit.

What was computed, and how #

The rotation is arbitrary and is chosen to be badly conditioned enough to be a real test: its determinant is 0.74 and its inverse has entries of order two, so an implementation that lost precision in the composition would show it.

The tristimulus matrix is composed with the inverse of the same rotation, which is the step that makes this a change of basis rather than a change of observer. Omitting it would compare an observer against a different observer and return a large number, and that is the version of the experiment that a careless implementation performs.

The assertion this figure family carries requires the residual below 10⁻⁹ ΔE₀₀ and it measures 8 × 10⁻¹⁵. The gate is deliberately several orders looser than the measurement, because a gate set at the measured value would fail the first time anybody changed the order of two multiplications.

One more use of the theorem is worth recording because it saves work rather than settling an argument. A collection that wants to change the basis it computes in — from x̄ȳz̄ to cone fundamentals, say — can do so with no consequence at all for any linear result it has published, and with consequences for every nonlinear one. That is a much sharper statement than “it might matter”, and it says exactly which figures would have to be recomputed: the ones with a division or a cube root in them, which in this collection is nearly all of them and not quite all.

The exceptions are the ones worth knowing. A gamut boundary is a linear object and does not move. A metameric black is defined by a null space and does not move. A Luther condition is a statement about spans and does not move. Whether a camera can be a colorimeter is therefore a question no change of basis can affect, which is why it can be answered once and for all rather than per pipeline.

Where the model stops #

The theorem is about linear functionals and an eye is not one. Everything downstream of the three integrals — adaptation, the opponent stage, the nonlinearity that produces lightness — is outside it, and the identity says nothing about any of them.

It also assumes the three sensitivities are linearly independent, which is true of any real observer and fails for a dichromat, where two of the three coincide and the subspace is two-dimensional. A confusion point is what that failure looks like on a diagram, and none of the arithmetic here applies to it unchanged.

And the rotation is exact in infinite precision. In finite precision a badly enough conditioned T would degrade the result, and the one used here is chosen to be mildly awkward rather than pathological.

One consequence for how this collection is built is worth recording, because it was arrived at by accident. Every observer in this round is carried to tristimulus values by one fitted three-by-three, shared by all of them, and that choice was made for bookkeeping — so that a difference between two rows would be a difference in what the cones caught rather than in the arithmetic.

The theorem says the choice was free. Any invertible matrix would give the same answers for the linear part, so nothing was lost, and what was gained is that a reader can compare two rows without wondering whether two fits differed. A convention that is provably free is worth adopting even when nothing forces it, because it removes a question rather than answering one.

The generalisation #

The habit is about telling a coordinate change from a model change.

Most quantities that get argued about are computed in coordinates somebody chose, and a surprising number of published disagreements are about the coordinates. The test is always the same: find the group of transformations the answer should be invariant under, apply an arbitrary member of it, and check that the answer does not move. If it does, there is a bug or a hidden nonlinearity. If it does not, then any remaining disagreement is about the object.

The failure mode is to argue about coordinates for years without running the test, and the reason it happens is that the two kinds of disagreement feel identical from inside. Both present as two groups getting different numbers from the same data. The invariance test separates them in an afternoon and is almost never the first thing anybody does.

Who found it, and when #

Grassmann’s laws of 1853 are the statement that colour matching is linear, and the invariance under change of primaries follows immediately from them. Every textbook derivation of the x̄ȳz̄ functions from r̄ḡb̄ is an application of it: the 1931 functions were obtained from Wright’s and Guild’s data by exactly such a rotation, chosen to make all three functions non-negative and to make ȳ equal the luminous efficiency function.

That choice is the reason the theorem is worth stating rather than assuming. The 1931 basis was picked for two properties that have nothing to do with vision — positivity, which was about the desk calculators of the day, and the luminance constraint, which was about photometry — and a century of practice has treated the resulting curves as though they meant something individually.

Where the ladder goes next #

The identity has a condition on it, and the condition is worth more than the identity. It holds in the eye’s own coordinates and fails by eight to thirteen colour differences in the ones everybody computes in, which is what the next essay measures.