The camera has its own metamers

Metamerism is taught as a fact about the eye. Two spectra, one colour; the collapse from a function to three numbers has an enormous null space, and every vector in it is invisible.

Nothing in that argument mentioned eyes. It mentioned a $3 \times N$ matrix, and a camera has one.

The claim #

A camera has metamers of its own, they are not the eye’s, and both directions of disagreement can be constructed rather than searched for: pairs a photograph loses, and pairs a photograph invents.

The second direction is the one nobody expects and the one that causes the practical trouble. A camera merging two colours produces a photograph that is missing something. A camera separating two colours a person cannot separate produces a photograph with a difference in it that was not in the room, and there is nothing in the file to indicate which of the two happened.

Why it needs no new machinery #

The construction this site has used since the expansion phase is a projection. Given a $3 \times N$ matrix $A$, the metameric black of any candidate $f$ is

$$b = f - A^{\mathsf{T}}(AA^{\mathsf{T}})^{-1}Af$$

which satisfies $Ab = 0$ by construction. Add $b$ to any spectrum and $A$’s three integrals are unchanged.

Nothing in that formula knows what $A$ is. Hand it the colour-matching functions and it produces a spectrum the eye cannot see; hand it a camera’s sensitivities and it produces one the camera cannot see. The function did not need modifying and neither did anything downstream of it — which is the practical payoff of having built the machinery around the matrix rather than around the observer.

The only real work in either direction is scaling. A reflectance must stay inside $[0,1]$, so the metameric black is scaled to the largest multiple that keeps both members of the pair physical, and a construction that requires negative reflectance is arithmetic rather than a demonstration.

The first direction: a lost distinction #

Project onto the null space of the sensor, add the result to a base reflectance, and the two surfaces produce the same raw triple.

Measured: the two raw responses agree to 0.0004 per cent — which is the projection being exact rather than an approximation — and to the standard observer the pair is ΔE00 2.4 apart. Two and a half is comfortably visible; it is more than twice a typical paint tolerance.

The photograph of these two surfaces contains one colour. There is no processing that recovers two, because there is no residual difference in the file to work with. Whatever the converter does — matrix, profile, lookup table, machine learning — it applies the same operation to the same input and gets the same output.

This is the failure that people describe as “the camera got that shirt wrong”, and the description is not quite right. The camera did not get one shirt wrong; it got two shirts identical, and only one of them looks wrong to whoever is complaining.

The second direction: an invented one #

Now project onto the null space of the observer instead, and add that to the base. The two surfaces are now a metameric pair for the eye — asserted, not promised: ΔE00 = 1.5 × 10⁻¹³, which is floating-point noise — and the camera records them 9.7 per cent apart in raw.

Both of those constructions depend on which observer is standing in for “the eye”, and the standard offers two of them rather than one.

The light is the third argument in the same construction, and changing it moves the pair just as surely as changing the observer did.

Nine point seven per cent of the raw magnitude is not subtle. After a colour matrix it is several ΔE, which means the photograph shows two visibly different colours where a person standing in the room sees one.

This is the harder failure to argue about, because the evidence is on the camera’s side. The photograph does show a difference; the difference is real, in the sense that the two spectra genuinely differ; and it is invisible to every human observer. Anybody adjudicating a colour dispute from a photograph is at the mercy of it.

Why both directions have to exist #

They are two consequences of the same theorem and neither is optional.

Luther’s condition fails exactly when the two null spaces differ. If $\ker S \not\subseteq \ker A$ there are stimuli the camera merges and the eye separates — direction one. If $\ker A \not\subseteq \ker S$ there are stimuli the eye merges and the camera separates — direction two. And since both are $N-3$ dimensional subspaces of the same $N$-dimensional space, neither can contain the other unless they are equal.

So the two failures come together, and the only sensor with neither is the one that satisfies the condition. The control sensor built for that purpose demonstrates it: run the same search against it and the largest raw gap over the whole space of constructions is $10^{-16}$. It has no metamers of its own, because its null space is the observer’s.

What it looks like in practice #

Constructed metamers are the clean demonstration and they are not what anybody meets. What people meet is the same phenomenon with an ordinary pair of surfaces and a small effect.

The spectra most likely to trigger it are the ones with structure finer than a sensitivity curve’s width, because a smooth spectrum is where two smooth kernels most nearly agree. In descending order of trouble: displays and LED lighting, whose primaries are narrow bands; fluorescent lamps, with mercury lines a few nanometres wide; saturated synthetic dyes, especially in the blue and violet; and fluorescent materials, which are outside the reflectance model altogether.

The classic professional case is fabric. Two dye formulations matched to the eye under a specified illuminant, photographed for a catalogue, come out different — and the correction that fixes one camera makes the other worse, because the two cameras have different null spaces from each other as well as from the eye.

Which direction is worse #

They are not symmetric in consequence, and the asymmetry is not the one people expect.

A lost distinction is recoverable in principle. If two surfaces record identically, a second photograph under a different illuminant will usually separate them — the two spectra differ, so multiplying by a different lamp changes the two integrals differently, and the pair is only metameric with respect to one product. That is exactly how illuminant metamerism is diagnosed for the eye, and the same trick works on a camera. It costs a second exposure and it works.

An invented distinction is not recoverable at all, because there is nothing wrong to detect. The two surfaces really do differ spectrally; the camera really did measure a difference; every check one could run confirms that the file is a faithful record of what arrived at the sensor. The photograph is correct and the conclusion drawn from it is false, and no amount of re-photographing helps, because a second exposure will show the same real difference.

The only remedy is to know that a person would not see it, which requires knowing the reflectances — which is the measurement nobody photographing anything has.

This inverts the usual intuition about instruments. An instrument that misses something is understood to be limited; an instrument that reports a difference is understood to be sensitive. Here the second is the more dangerous property, because its output is indistinguishable from a correct one.

Both directions, sized #

The pair drawn above is one construction, and the extremes of each direction are worth having.

Lost: the largest pair the camera merges is 15.33 ΔE00 to the eye, with a raw separation of exactly zero. Two surfaces a person would call obviously different colours, recorded as one triple to the last digit.

Invented: the largest pair the eye merges is separated by the camera by 0.0967 in relative raw units, with a colour difference of exactly zero. Two surfaces nobody can tell apart, recorded as measurably different.

The second exposure really does work on the first, and by a wide margin. The camera-blind pair’s raw separation under a second lamp is 0.046 under D50, 0.112 under a three-emitter source, 0.149 under illuminant A, 0.176 under a white LED and 0.212 under a triphosphor tube — the last being twenty-one per cent of the pair’s own mean raw signal, which no sensor noise conceals. The tube is the best diagnostic on the list, for the reason that makes tubes difficult everywhere else: its spectrum is nothing like the one the pair was constructed against.

And the two directions are of the same size in the units that matter. The invented distinction is 0.0967, and the separation a second exposure under illuminant A buys for the lost one is 0.149. A camera’s spurious difference is therefore about two thirds of a real difference a diagnostic exposure would uncover, in the same units, on the same sensor.

That is the practical form of the asymmetry. It is not that the invented differences are small and the lost ones large. Both are of a size an instrument reports confidently, and only one of them can be checked.

Running the construction the other way round says what the camera can see that the eye cannot, and the ten-degree observer is the harder case for it.

The version that is not about metamerism at all #

There is a much more common cousin of direction two, and it is worth separating so the constructed cases do not get the blame for it.

A camera’s null space and the eye’s differ, which is this essay. A camera’s white balance and the eye’s adaptation also differ, which is not — that is a difference in how each responds to the illuminant rather than in what each integrates. The two produce similar-looking complaints: a photograph shows a colour difference nobody in the room noticed. But one is a kernel mismatch and cannot be corrected, and the other is a choice about the adapting white and can be.

Telling them apart is straightforward in principle: a white-balance error moves everything in the frame in one direction, while a metamerism error moves particular surfaces and leaves their neighbours alone. In practice the two are usually happening at once, which is why the argument about a photograph’s colour is rarely settled.

Where the model stops #

These constructions are extremal, and the world is not. The pairs above were found by searching a family of sinusoidal candidates for the one whose projection gives the largest admissible separation. Real reflectances are not drawn from that family, and the everyday effect is much smaller — a fraction of a ΔE on ordinary surfaces, several on awkward ones. The construction establishes that the failure exists and bounds how bad it can get; it does not estimate how bad it typically is.

The sensor is modelled and the numbers move with it. A different dye set has a different null space and therefore a different worst case. What does not move is the existence of both directions, which follows from the condition failing rather than from how it fails.

And this is entirely about the visible band. The infrared problem of the previous three essays is separate: it is a common-mode signal rather than a null-space mismatch, it is removed by a component rather than by a matrix, and a perfectly filtered camera still has every metamer described here.

The generalisation #

The useful abstraction is that two projections of the same space disagree in two directions at once, and the second direction is the one that gets forgotten.

Whenever one instrument, model or summary replaces another, the question usually asked is what the replacement misses. That is direction one and it is the easy half. Direction two — what the replacement distinguishes that the original does not — is equally guaranteed whenever the two are not linear redescriptions of each other, and it produces a different kind of error: not an omission but a spurious signal, indistinguishable in the output from a real one.

The examples multiply once the shape is visible. A summary statistic that separates two datasets a domain expert considers equivalent. A classifier keying on a feature that is real, present, and irrelevant. A search index distinguishing two documents a reader would call the same. In each case the system is not malfunctioning; it is projecting onto a different subspace, and the extra distinction is as much a property of its kernel as the missing one.

The protection is the same in every case and it is the one this essay is built on: construct the disagreement rather than wait for it. The null space of a stated projection is computable, and a deliberately constructed worst case is worth more than a large sample of typical ones, because typical cases are drawn from wherever the two projections happen to agree.

Under a tungsten lamp the same construction gives a larger answer, and the direction is the one the mechanism predicts.

Three observers and a machine #

It is worth putting this beside the other disagreements this collection has measured, because the ordering is not the one a reader would guess.

The 1931 and 1964 standard observers disagree because one was measured on a two-degree field and the other on ten. Individual people disagree with each other by more than the two standards do, which is the uncomfortable fact that gets left out of most accounts. And a camera disagrees with all of them by more again.

But the kind of disagreement is identical in all three cases. Each pair of observers has two null spaces that are not the same subspace, so each pair has stimuli one merges and the other separates, in both directions. Observer metamerism between two people and sensitivity metamerism between a person and a camera are one phenomenon with one algebra.

That is a reassuring conclusion and a slightly deflating one. Reassuring, because a camera is not a defective member of some other category; it is another observer, and the machinery built for observers applies to it unchanged. Deflating, because it means the target a camera is being asked to hit is not a fact about the world but an average of seventeen people, and “accuracy” in this field always names an agreement between two instruments rather than a correspondence with anything.

Who noticed, and when #

Camera metamerism has a much shorter history than observer metamerism, because for most of the twentieth century the taking device was film and film’s failures were dominated by other things — reciprocity failure, dye stability, the coupler chemistry — that were larger and more urgent.

Electronic sensors made the comparison clean. The CIE’s 1995 sensitivity metamerism index is the formalisation, and the term used in the literature for direction one is usually camera metamerism while direction two is often called spectral sensitivity metamerism or simply folded into the same index, which slightly obscures that they are separate failures.

What the modern lighting transition has done is make both worse without anybody changing a sensor. Thermal illumination is smooth and forgiving; the narrow-band sources that have replaced it are exactly the stimuli on which two different kernels most disagree, and the complaints about cameras rendering LED-lit scenes badly are this essay’s subject arriving as a customer-support issue.

What a photograph can still be used for #

None of this makes photographs useless as colour evidence, and it is worth saying where the line falls, because the field’s closing essay depends on it.

A photograph is reliable about relations within a frame far more than about absolute values. Two surfaces photographed side by side under one light, in one exposure, through one lens, have been through identical processing; if the file says one is darker than the other, it is, and the camera’s null space cancels out of the comparison unless the pair happens to be one of the constructions above.

A photograph is unreliable about absolute colour for every reason this field has listed: the white balance was guessed, the matrix was fitted on somebody else’s surfaces, the tone curve was chosen to look good, and the sensor merges and invents distinctions relative to an eye.

And it is least reliable exactly where it is most often used as evidence — matching a repair to an original, adjudicating whether a delivered item matches a sample, judging a paint under a light that was not recorded. Every one of those is a question about absolute colour under an unstated illuminant, asked of an instrument whose kernel is unpublished.

The practical rule that falls out is the one professionals already use without usually deriving it: put the reference in the frame. A known chart photographed beside the unknown converts an absolute question into a relative one, which is the kind the instrument is good at. It does not repair the metamerism — a chart cannot fix a null-space mismatch — and it removes almost everything else.

Where the ladder goes next #

Downward, this rung sits on Luther said when it would work, which is the theorem this essay demonstrates, and on two spectra, one colour, which is the same construction with the observer’s matrix in it.

Upward, the field turns from what cannot be fixed to what is done instead. A camera profile is a fit is the matrix somebody chooses given all of the above, and no matrix is right everywhere is the measurement of how much that choice can be made to matter.