Where a camera is blind to itself

The nine numbers an observer’s matches leave free are a modelling freedom nobody manufactures. A camera’s six are a purchase order.

The claim #

The adaptation objective over a camera’s six dye parameters has a condition number of 623, so the design can move twenty-five times further in its cheapest combination than in its dearest. The cheapest is the three bandwidths together; the dearest is where the blue dye sits.

• The six eigenvalues run 1.90 × 10⁻², 4.02 × 10⁻³, 9.95 × 10⁻⁴, 5.13 × 10⁻⁴, 1.01 × 10⁻⁴, 3.06 × 10⁻⁵, all in nanometres, so no weighting has to be invented to compare them.
• The stiffest direction is the blue dye’s centre, at a weight of 0.96 of a unit direction, with nothing else above 0.20.
• The flattest is the three widths, which carry 97 per cent of it between them.
• There is no invariance here and no zero eigenvalue. A camera with a different green is a different observer, so the spread is a manufacturing tolerance rather than a redundancy in the model.
• And the objective is the same one the observer’s nine numbers were scored on, so the two are directly comparable: nine numbers with a rank of six against six numbers with a rank of six.

What the six numbers are #

The sensor modelled here is three Gaussian dye transmittances with an infrared cut, which is the crudest thing that can honestly be called a colour filter array and is the one this collection uses throughout. Each channel has a centre wavelength and a width; three of each is six numbers.

Optimised against the same census of illumination changes the observer’s basis is scored on — subject to staying within a stated distance of the Luther condition, without which the search walks to three narrow lines that adapt superbly and cannot tell a metamer from its partner — the design comes out at centres of 610.1, 542.0 and 449.4 nm with widths of 34.9, 25.6 and 30.2 nm, leaving 0.974 ΔE00.

That number is the same as the observer’s unconstrained floor to four figures, which is a result in its own right: three ordinary dyes reach the optimum a basis free to be any nine numbers reaches.

What has not been asked is what the objective’s shape is around that design, which is the question a manufacturer would ask first.

Six eigenvalues, no zeros #

Differentiating twice over the six parameters and decomposing gives eigenvalues spanning 1.90 × 10⁻² down to 3.06 × 10⁻⁵: a condition number of 623, and a reach ratio — how far the design can move for the same cost — of 25.

There is no null space. That is worth pausing on, because the observer’s nine numbers do have one: three of them rescale a row of the basis and the model cannot see them. A camera’s parameters have no such symmetry. Scaling a dye’s sensitivity is not one of the six numbers; moving a dye is a different dye, and the objective notices.

So the spread here means something different. In the observer’s case a flat direction is a redundancy — a direction along which the model is provably blind, and along which nothing is lost by fixing a convention. In the camera’s case a flat direction is a tolerance: a combination of six manufacturable quantities the objective barely notices, which is exactly what a specification should be loose about.

The six numbers are three dyes with a centre and a width each, and the tolerance on any one of them is a region rather than a bar.

Where the blue dye sits #

The stiffest direction carries a weight of 0.96 on the blue dye’s centre wavelength, 0.20 on the blue width, 0.19 on the green centre, and essentially nothing on the other three.

How well a camera adapts is decided by where its short-wavelength dye is put.

The reason is the one that keeps recurring in this collection. An adapted observer’s gain in each channel is the ratio of the two whites read in that channel, and across a census of real illuminants the short-wavelength channel’s ratio is the one that swings — a tungsten lamp against daylight moves the blue reading by a large factor and the red one by a small one. A sensor whose blue channel is in the wrong place gets the largest gain wrong, and the largest gain is where the error lives.

The same structure appears in the observer’s basis, at 97 per cent the short-wave row for the stiffest direction of the nine-parameter problem. Two different objects, one built and one modelled, with the same answer about which end of the spectrum decides the answer.

The blue dye’s own region is worth drawing on its own, because it is the one place in this design where the objective’s tolerance and the physics of a real filter disagree about where the dye may be put.

That is the exception rather than the rule, and the green dye is the comparison that shows it: the same construction, the same rise, and a region that lies entirely inside what a filter can be made to do.

Where the bandwidths are not #

The flattest direction is 0.56, 0.48 and 0.63 on the three widths, with the three centres carrying between −0.16 and 0.09. Squared, the widths carry ninety-seven per cent of it.

So the design can widen or narrow all three dyes together — by twenty-five times as much as it can move the blue dye’s centre, for the same cost in adaptation — and the objective barely registers.

That is a striking thing to be able to say about a colour filter array, because bandwidth is a quantity manufacturing has strong opinions about. A wider dye passes more light and improves the signal to noise. A narrower dye is more selective and improves colour separation. Neither of those considerations is in this objective at all, which is the correct reading: the adaptation objective does not care about the widths, so the widths are free to be decided by everything else.

The converse is the useful engineering sentence. A tolerance budget spent on holding the dyes’ centre wavelengths — particularly the blue one — buys twenty-five times what the same budget spent on holding their widths does, against this criterion.

What was computed, and how #

The Hessian is 22 second differences over six parameters, at a step of a quarter of a nanometre — small compared with any feature of the dye curves and large compared with the objective’s own numerical noise. The decomposition is a one-sided Jacobi singular value decomposition, which here is convenience rather than necessity: with no zero eigenvalue to resolve, forming the Gram matrix would have been safe.

The parameters are all in nanometres, and that is the reason the eigenvalues can be compared at all. A Hessian over parameters in different units has eigenvalues in different units, and their ratio is a statement about the choice of units rather than about the objective. Comparing a camera’s six with a display’s six would be exactly that mistake — one set is in nanometres and the other in chromaticity — and this collection does not.

The assertion in the build makes three claims: that the condition number exceeds a hundred, so the design is genuinely far more particular about some combinations than others; that the stiffest direction is dominated by the blue dye’s centre; and that the flattest is dominated by the three widths together. The second and third are the ones a specification would be written from, and both would fail loudly if the objective or the design moved.

What the Luther constraint is doing to this #

The design is not unconstrained: it is required to stay within a stated distance of the Luther condition, which asks that the sensor’s sensitivities be a linear transformation of the observer’s. Without it the objective’s optimum is degenerate in an uninteresting way — three narrow lines.

That constraint is a surface, and the curvature reported above is the objective’s in the full six-dimensional space rather than along the surface. So a direction that is flat for the objective may be blocked by the constraint, and a tolerance quoted from the objective alone would be too generous. The honest thing is to walk both directions and see.

At the design the Luther residual is 0.2282 against a ceiling of 0.28. Walking along the flattest direction — all three dyes widening or narrowing together — the residual runs 0.193, 0.206, 0.228, 0.257, 0.293 at steps of six, three, zero, minus three and minus six nanometres. Along the stiffest direction it runs 0.242, 0.227, 0.228, 0.248, 0.282 over the same range.

The feasible region is not symmetric, and it is not large. In one sense along each direction the constraint is not merely satisfied but improved; in the other it is exhausted after about five nanometres. So the reach ratio of 25 describes a level set of the objective and overstates what a design can actually do, in one direction of each pair, by a factor this essay is not able to summarise in a single number — because the binding side depends on which direction is being walked.

What survives is the ordering, which is what a tolerance budget is written from. The blue dye’s centre is expensive whichever way it moves; the three widths together are cheap in both senses and blocked in one. A specification that holds the centres tightly and the widths loosely is right about the objective and needs a separate bound on narrowing.

The number the essay says it cannot give #

The Luther section reports the walks, observes that the constraint bites in one direction of each pair, and declines to summarise the effect — by a factor this essay is not able to summarise in a single number, because the binding side depends on which direction is being walked. The four readings on each walk are enough to give it, once one thing is stated that the essay leaves implicit.

Interpolating each walk to the ceiling of 0.28: the flat direction becomes infeasible at −4.92 nanometres and the stiff direction at −5.82. So the constraint is not merely asymmetric — it is tighter on the flat direction than on the stiff one, by sixteen per cent, which is the opposite of what the objective’s curvature would suggest.

The missing ingredient is what the same cost means, since a reach ratio is a ratio of distances at equal objective rise and the essay never names the rise. Fixing it at one per cent of the objective — 0.0097 ΔE00 on a floor of 0.974 — the stiff direction reaches 1.01 nanometres and the flat one would reach 25.2, which is the unconstrained ratio of 25. The constraint caps the flat direction at 4.92, so the effective ratio is 4.9.

The reach ratio is not 25 in any usable sense. Twenty-five is what the objective alone allows, and the Luther constraint removes four fifths of it at the ordinary tolerance and more at a looser one. The remaining advantage runs from about seven to about two depending on how much adaptation performance a design is willing to give away, and it falls as the budget grows because the cap does not move while the stiff direction’s reach does.

That is a substantially weaker conclusion than the headline and it leaves the essay’s engineering advice intact. A tolerance budget on the centre wavelengths still buys about five times what the same budget on the widths buys — a real and useful factor, arrived at from the same four readings the essay already prints, and one a specification can be written from. What it cannot support is the sentence in the claim section: the design can move twenty-five times further along it for the same cost is true of the objective and false of the design.

Ninety-four per cent, not ninety-seven #

One arithmetic correction, in the section the tolerance advice rests on. The flattest direction’s three width components are 0.56, 0.48 and 0.63, whose squares sum to 0.941 — the widths carry 94 per cent of the direction, not 97.

The three centre components then account for 0.059, which is consistent with the −0.16 to 0.09 range quoted for them. The stiffest direction’s three named weights check out exactly: 0.96, 0.20 and 0.19 square to 0.9977, so the eigenvector really is almost entirely those three and the nothing else above 0.20 is doing no rounding.

Three points of concentration is not much and the direction of the error is the flattering one, which is why it is worth fixing in an essay whose whole subject is which numbers a specification should trust.

The observer’s spread is wider, not narrower #

The comparison section puts the observer’s condition number at 890 against the camera’s 623 and reads the two as comparable. In the currency the essay uses everywhere else — reach rather than curvature — they are further apart than that suggests.

A reach ratio is the square root of a condition number, so the observer’s six visible numbers span a reach ratio of 29.8 against the camera’s 24.9. The observer’s design space is a fifth more anisotropic than the camera’s, which is a mild difference and runs the way one would expect: the observer’s parameters are an unconstrained basis and the camera’s are six numbers already pinned near a constraint surface.

And the camera’s 24.9 is the figure that the Luther cap then reduces to about five, while the observer’s 29.8 has no equivalent cap — nothing constrains a modelled basis to lie near anything. So the two objects meet at the same objective floor and, once the constraints each actually lives under are applied, are not comparably anisotropic: the modelled one keeps its full spread and the manufactured one keeps a fifth of it.

Nine numbers and six, on one objective #

The two objects are worth putting side by side, because the objective is identical and the comparison is therefore free.

The observer’s nine have a rank of six: three of them rescale a row of the basis and nothing downstream can see them. The six that remain span a condition number of 890, and the design — such as it is, since nobody designs an observer — reaches 0.974 ΔE00.

The camera’s six have a rank of six: nothing is invisible. They span a condition number of 623, and the design reaches 0.974 ΔE00, which is the same number to four figures.

So a camera built from three ordinary dyes reaches the floor a basis free to be any nine numbers reaches, and it does so with a comparable spread of curvatures and no redundancy at all. The observer’s extra three numbers buy nothing — they are exactly the three the model cannot see — and the six that are left behave very like a camera’s six.

That is a satisfying place for two lines of argument to meet, and it is worth being careful about what it does not say. It does not say a camera is an observer: the sensor here satisfies the Luther condition only to within a stated distance, and a sensor that satisfied it exactly would be colorimetric and would still be free to choose its axes. What it says is that the adaptation objective, pointed at either object, finds about the same floor and about the same shape.

Where the model stops #

Three Gaussians are not a camera. Real colour filter arrays have asymmetric, multi-lobed transmittances with long tails, and a real sensor’s response is the dye times the silicon’s quantum efficiency times the microlens times the infrared cut. The six numbers here are a caricature chosen so that the design space is small enough to see.

The objective is adaptation alone. Nothing here scores colour separation, noise, the metamers a camera has of its own, or how well a fitted matrix behaves away from the light it was fitted under. A real design is a compromise among all of those and this curvature is one term of it.

And a tolerance from a Hessian is a local statement. The reach ratio describes a level set at a stated rise; a manufacturing tolerance is a distribution, and turning one into the other requires knowing how far out the quadratic model holds. Here it holds well enough over the ranges quoted, and it would not over ten times them.

How far out the quadratic holds is a question the figure can be asked directly, by drawing the same three regions at three times the rise and seeing whether they merely scale.

The generalisation #

Where a design’s objective has a wide spread of curvatures, the tolerance budget belongs in the stiff directions and nowhere else — and the stiff directions are usually combinations rather than single parameters, which is why a specification listing one tolerance per quantity cannot express them.

That is the practical half. The methodological half is smaller and applies further. A curvature over parameters in one unit is a comparison; a curvature over parameters in mixed units is a choice of units. The six numbers here are all nanometres and the comparison is free. A design mixing a wavelength, a density and an angle has no such luck, and the eigenvalues of its Hessian mean nothing until somebody has decided what a nanometre is worth in degrees — which is a decision about the design, not a property of it.

The habit that follows is cheap: before reading a condition number, check that the parameters share a unit. Where they do not, either scale by the manufacturing tolerance on each — which makes the eigenvalues dimensionless and the comparison meaningful — or report the directions rather than the numbers.

Who found it, and when #

Sensitivity analysis of camera spectral sensitivities is not new — the effect of dye placement on colour reproduction error has been studied since the 1980s, and the standard tools are the Luther condition, the various quality factors derived from it, and direct simulation over reflectance sets.

What is not standard is treating the design’s own parameters as the space and taking a second derivative in it. The usual analysis perturbs one parameter at a time and reports a sensitivity per parameter, which is the diagonal of this matrix and misses everything off it — and everything here is off it. The flattest direction is three widths moving together; no one-at-a-time analysis would find it, because moving any single width is a mixture of the flat direction and several stiff ones.

A one-at-a-time sensitivity study is a diagonal, and a diagonal of a matrix with a condition number of six hundred is not the matrix.

Where the ladder goes next #

The other manufactured object with six numbers over it is a display’s three primaries, and its curvature has a different shape: the flattest direction moves red and green together and leaves blue alone. Turned into a picture, that is a region on the chromaticity diagram rather than a tolerance per primary — and part of every one of those regions is unreachable.