A straightened channel repeats the one beside it

Two channels estimate what one can only sort gave a phone’s ambient-light sensor narrow channels and asked them a practical question: how far off will the camera’s colour be under this lamp? Across fifty-four lamps — a census of fourteen and two constructed families of twenty — it read each channel’s departure from what the camera predicted and fitted a straight line from departure to the camera’s error, scored by leaving each lamp out in turn. A channel at 450 nanometres estimated the error to ±0.16 ΔE₀₀; 450 and 500 together to ±0.14; guessing the mean, ±0.30.

A channel at 570 nanometres did something odd. It ranked the lamps’ errors almost as well as 450 — a rank correlation of 0.83 against 0.87 — and estimated them worse than the mean, ±0.57. The reason was one lamp: the three-emitter source reads 2.1 at 570 nanometres, three times the next lamp and a hundred times a smooth one, while its camera error is ordinary. Left out, 570 nanometres was nearly as good as 450. The relation between reading and error is steep at small readings and flat at large ones. The essay’s closing section asked the obvious next thing: pass the reading through a logarithm, a square root or a saturating curve with one fitted constant, keep the extreme lamp in, and see. The prediction was that a transformed 570 nanometre channel reaches 450’s ±0.16, and that 450 with a transformed 570 beats 450 with 500 — because 570 is where the camera’s and the observer’s sensitivities differ most.

Rescued, and redundant #

Through a logarithm the 570 nm channel estimates the camera’s error to ±0.226, through a square root ±0.266, through a saturating curve whose constant is chosen inside each held-out fold ±0.267 — all better than the mean’s ±0.304 and less than half the raw channel’s ±0.572. None reaches the 450 nm channel’s ±0.164. Put beside it, a log-transformed 570 nm channel leaves the estimate at ±0.165; a 500 nm channel takes it to ±0.143, and adding 570 to both takes it nowhere. The two lamps 570 nm reads highest, the three-emitter source and the triphosphor tube, have camera errors of 1.75 and 2.37, and the transformed channel misses the tube by 0.72; 500 nm, by 0.43.

• The transform works on the channel it was aimed at. The one extreme lamp no longer drags the fit.
• The prediction still fails on both counts: 570 nm does not reach 450 nm, and does not add to it.
• Straightened, 570 nm sorts the lamps as 450 nm does — a rank correlation of 0.82 between the two readings — so what it adds, 450 already carries.
• The case a transform cannot fix is two lamps with high readings and different errors, and it is the case that matters.

What straightening does #

On a logarithmic axis the lamps nearly line up. The smooth radiators and the violet-pumped LEDs read hundredths or thousandths at 570 nanometres and give the camera errors near 1.2; the cyan-filled LEDs and the phosphor LEDs read about a quarter and give errors of 1.7 to 2.2; the tubes and the three-emitter source read most. The logarithm turns the reading’s steep-then-flat relation to the error into something close to a line, and the estimate from it is off by ±0.226, left out one lamp at a time.

The logarithm is not a mysterious choice. A narrow channel has to be read on its own defined the departure as how far a channel’s reading sits from what the camera’s three channels predict it should read under a smooth lamp; a structured lamp departs by an amount set by how much of its power sits in lines or gaps near the channel, and that spans orders of magnitude. The camera’s error does not. A channel whose reading spans a thousandfold, estimating a quantity that spans a factor of two, is naturally read on a logarithmic scale.

The two circled lamps are the difficulty. The three-emitter source reads 2.12 and the triphosphor tube 0.73, half a decade apart at the top of the axis. Their camera errors are 1.75 and 2.37 — the larger reading with the smaller error. The logarithm makes the three-emitter source harmless by pulling it in; it pulls the tube in too.

Three transforms and four estimates #

Every transform rescues the channel. The logarithm to ±0.226, with its worst miss the triphosphor tube at 0.63; the square root to ±0.266, worst the three-emitter source at 1.32; the saturating curve r/(r + c) to ±0.267, worst again the three-emitter at 1.43. The square root compresses too little to tame a reading a hundred times the typical one, and the saturating curve’s constant, as the section after next shows, cannot be chosen reliably.

None reaches 450 nanometres. The best, the logarithm, is 38 per cent worse than the 450 nm channel alone, and its worst miss is only slightly smaller than 450’s own 0.74.

Beside 450 nanometres none of them helps. 450 with a log-transformed 570 is ±0.165 against 450 alone’s ±0.164 — a hair worse, because a second input costs a degree of freedom and buys nothing for it; with a square root, ±0.189; with the saturating curve, ±0.205. 450 with 500 is ±0.143, and adding a transformed 570 to that pair gives ±0.146. Two channels estimate what one can only sort had already found the 500 nm channel the useful second one; 570, straightened, does not displace it.

The two lamps a transform cannot separate #

The three-emitter source and the triphosphor tube are the two lamps the 570 nm channel reads highest, and they are the pair on which the whole question turns. The three-emitter source is three narrow emitters, one of them near 570; its reading there is enormous, and the camera renders it with an error of 1.75, a little above the phosphor LEDs’. The triphosphor tube is three narrow phosphor bands with gaps between; its 570 nm reading is a third as large, and the camera’s error under it is 2.37, the worst of all fifty-four lamps.

From 450 nanometres alone, the tube is missed by 0.74 and the three-emitter source by 0.06: the 450 channel reads them similarly, 0.62 and 0.81, and the estimate puts them both near two. With a log-transformed 570 nanometre channel added, the tube is still missed by 0.72. A monotone transform keeps the order of the readings, and the order says the three-emitter source should have the larger error; the fit, balancing both lamps and fifty-two others, can do nothing for either. With 500 nanometres added instead, the tube is missed by 0.43. The tube’s spectrum is nearly empty between its blue and green lines, and the 500 nm channel reads 1.90 there against the three-emitter source’s 0.87 — a reading that goes the same way as the errors do.

What the 450 nm channel leaves #

The question whether a second channel adds anything is the question whether it predicts what the first one leaves. Against the 570 nm reading, the 450 nm channel’s residuals lie in a nearly flat band: lamps whose error 450 underestimates and lamps whose error it overestimates sit at every 570 nm reading alike. Their rank correlation with the logarithm of the 570 reading is 0.12. The triphosphor tube stands alone above the band at the high end, and no line through the rest of the cloud reaches it.

Against the 500 nm reading the residuals’ rank correlation is 0.16 — not much larger, but carried by the right lamps. Split by family, adding 500 nm cuts the structured lamps’ residual from ±0.34 to ±0.28 and the cyan-filled LEDs’ from ±0.095 to ±0.077; adding a log-transformed 570 leaves the structured lamps at ±0.33 and makes the cyan LEDs worse, ±0.108. The phosphor LEDs and the cyan-filled LEDs are where the 500 nm channel sees something the others do not: whether the gap between the blue pump and the phosphor has been filled.

The LEDs show it most plainly. The plain phosphor LEDs read 0.24 to 0.27 at 570 nanometres and give the camera errors of 1.9 to 2.2; the cyan-filled LEDs read a median of 0.21 there and give a median error of 1.67. At 570 the two families are nearly the same lamp, since both put their long-wave power in one broad phosphor band. At 500 the plain LEDs read 0.35 to 0.47 and the cyan-filled ones a median of 0.22, falling to 0.02 where the fill is heaviest — the channel is reading the fill itself, and the fill is what lowers the camera’s error. A 570 nm term, asked to account for the cyan family’s lower errors, has only readings a little below the plain LEDs’ to work with, and its straight-line share of that difference is the wrong size for the tubes, which is why adding it makes the cyan residual larger rather than smaller.

A constant that looks settled #

Fixed at 0.2 for every fold, the saturating curve estimates the error to ±0.190 — nearly as well as 450 nanometres, and better than the logarithm. That number is not available to anyone building the estimator. The constant 0.2 was chosen by looking at how every one of the fifty-four lamps is scored, including the one being held out, which is exactly what leaving one out is meant to prevent.

Chosen honestly — inside each fold, from the fifty-three lamps the fit is allowed to see — the constant does not settle. Fifty-three folds choose 0.2. The fold that holds out the three-emitter source chooses 1, because without that lamp in view the curve that best fits the rest bends much later — and that fold’s prediction for the three-emitter source, made with a curve that barely saturates, misses it by 1.43. The held-out error is then ±0.267, worse than the logarithm, which has no constant to choose. A one-parameter transform fitted on a census whose largest readings come from two lamps is, in effect, fitted to those two lamps, and a new lamp with a large reading is a new vote on where the curve should bend.

Two channels that sort alike #

Straightened, 570 nanometres sorts the lamps the way 450 does. Across the fifty-four lamps the rank correlation between the log of the 570 reading and the 450 reading is 0.82; the smooth radiators and the violet-pumped LEDs sit low on both axes, the phosphor LEDs, the cyan-filled LEDs and the tubes high on both. The 500 nm channel correlates with 450 at 0.79, about as much — the difference is not that 500 is more independent of 450 in general, but that it disagrees with 450 in the particular places where 450’s estimate is wrong.

That is also why the prediction’s reasoning went astray. It placed the second channel where the camera and the observer differ most, on the grounds that the error arises there — a camera’s error under a lamp being, as a matrix is fitted under one light put it, the price of nine numbers fitted under another. It does, and that is why a channel there ranks the error well. But the lamps that put power where camera and observer disagree are the structured lamps, and a channel at 450 nanometres has already identified them. The lamps two channels miss are smooth to the camera found the same kind of redundancy between channels chosen for different reasons; what a second channel has to supply is not a better view of the same lamps but a view that separates lamps the first one lumps together.

What a sensor designer should take from it #

A channel’s value to an estimate is what it adds, not what it predicts. The 570 nm channel predicts the camera’s error well, once straightened, and adds nothing to a 450 nm channel. The 500 nm channel predicts it less well alone and cuts the estimate’s error by an eighth beside 450. A designer choosing a second channel by its own rank correlation with the error would choose 570 and gain nothing.

A transform chosen on a census is a claim about the census’s extremes, which is the lesson a chart decides what a camera scores drew about test charts, one level up: the set chosen decides the number, and here it also decides the curve. The logarithm, with no constant, is the safe choice here, and it is safe because it asks nothing of the two lamps that decide every fitted constant. A camera maker whose lamp set is dominated by a few very structured sources should expect any fitted curve to be a fit to those sources.

How the estimates were scored #

The fifty-four lamps, the camera model with its two-matrix profile blended at each lamp’s colour temperature — the blend two matrices do not reach a white LED found failing under exactly these LEDs and tubes — the ambient sensor and each narrow channel’s departure are the collection’s, as two channels estimate what one can only sort built them. An estimate is a least-squares line or plane from one, two or three channels’ departures to the camera’s mean error, with the 570 nm departure r used as read, as $\log r$, as $\sqrt{r}$ or as $r/(r + c)$. Every estimate is scored by leaving one lamp out, fitting on the other fifty-three and predicting it; the saturating curve’s c is chosen inside each fold from 0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5 and 1 by leaving each of the fifty-three out in turn. The fixed-constant sweep uses the same c for every fold. Rank correlations are Spearman’s; residuals by family are root mean squares of the held-out misses within each family.

What this leaves out #

The sensor is modelled, as it was in the essays this continues: gaussian channels on a silicon response, spectra stopping at 780 nanometres, calibration taken as perfect. A measured phone sensor would change the departures’ sizes and the camera’s errors, and the argument needs only that a transformed reading’s order is the reading’s order.

The census decides the extremes. Fifty-four lamps with two very structured sources at the top is a particular census; a lamp set with many tubes and few three-emitter sources would make the 570 nm channel’s high readings mean a large error, and the logarithm would then help rather than merely rescue.

The estimate is linear in its inputs. A model that let the 570 nm reading’s effect depend on the 450 nm reading — an interaction — could in principle separate the two extreme lamps, which differ at 450 as well; with fifty-four lamps and one pair to learn from, it would be fitting that pair.

Still open: whether a ratio of two channels reads the gap directly #

The 500 nm channel helps because it sees whether the stretch between a lamp’s blue emission and its green is empty, and the triphosphor tube’s is emptier than anything else in the census. But a single channel’s departure measures that gap against the camera’s prediction, which folds in everything else about the lamp.

The calculation is an estimate from the ratio of the 500 nm reading to the 450 nm one, used as a single input and beside the 450 nm channel, scored by leaving each lamp out. The prediction is that the ratio, which cancels whatever the two channels share, separates the triphosphor tube from the three-emitter source and the phosphor LEDs by a larger margin than either reading alone, and that one ratio-based input does as well as the two channels as separate inputs — ±0.14 — with one fewer number to fit. If it does, the second channel’s job is not to see more of the spectrum but to put a denominator under the first.

A second opinion that agrees is not a second opinion #

The habit is about what a second input to an estimate has to do.

The 570 nm channel was proposed as a second input because it is placed where the error arises, and it does track the error — once a transform removes one lamp’s leverage, it tracks it about as well as the first channel does. The first channel was already tracking it. A second input earns its place by being right where the first is wrong, and the lamps on which the 450 nm estimate is wrong are lamps the 570 nm reading cannot rank correctly either.

The failure mode is to judge a candidate input by how well it predicts the target, rather than by how well it predicts what the existing inputs leave. Two channels that agree with each other and with the answer are one channel measured twice.