Relatives, not twins, teach a sensor a tube

An ambient light sensor on a display has one job in a dark room: to say what colour the room’s light is, closely enough that the display can declare its black in that colour and keep a dark picture’s shadows from taking a tint. A room sensor must read the lamp to a hundred kelvin put the tolerance at about 0.0045 in the CIE 1960 plane at a tenth of a candela, and no calibration reads the lamps and the LEDs together found that a three-channel sensor cannot meet it for lamps and LEDs at once, however its matrix is fitted.

Seven channels end the trade and lose the tube gave the sensor more channels, spread evenly from 445 to 650 nm, and fitted each sensor’s matrix to the room’s forty-seven lights with every light held out of its own fit. Seven channels read every lamp, mixture and LED within a unit; the trade between the two families dissolved. One light went the other way. The triphosphor tube, whose phosphors emit in narrow bands at 543 and 611 nm, was read within a unit only by the coarsest sensors, three and four channels, and beyond it by every finer one — 2.5 units at seven channels, thirty-three at twelve. Held out of its own fit, nothing left in the census resembled it, and a matrix with more freedom extrapolated further from it.

That essay ended by naming the remedy as the census rather than the sensor. A real census of room lights would hold several triphosphor tubes, sold at different colour temperatures from the same three phosphors in different proportions. The prediction was that four such siblings, at 3,000, 3,500, 4,000 and 5,000 K, bring the tube within a unit at seven and eight channels and stop its rise past eight, while the halophosphate tube, still alone, rises past eight as before. The alternative was that the tube stays beyond a unit, which would mean its narrow bands fall between fine channels whatever the census teaches, and a fine sensor cannot read narrow-band light at all.

The tube comes in from five channels #

With four triphosphor siblings in the census, the tube held out of its own fit is read within a unit by every sensor from five channels to twelve: 0.66 at five, 0.48 at six, 0.04 at seven, 0.19 at eight, 0.18 at ten and 0.06 at twelve, where alone it was 1.7, 1.4, 2.5, 2.4, ten and thirty-three. Sensors of seven and eight channels then read all fifty-one lights within a unit. With three siblings only — 3,000, 3,500 and 4,000 K, every one warmer than the tube’s 5,260 K, the nearest by sixty mireds — the tube is still read within a unit from five channels on. The siblings cost the other families under a quarter of their worst error from five to eight channels, and at twelve they raise the lamps’ worst error from 0.0005 to 0.0025.

• The prediction holds, at seven and eight channels and past them: the tube’s rise is not shrunk but gone.
• The alternative fails. Fine channels read narrow-band light perfectly well once the census holds narrow-band light.
• A twin is not needed. Relatives on one side only, none nearer than sixty mireds, do nearly as well as a family that includes a near twin.

Three curves share their start and part at five channels. Alone, the tube’s tint rises from 1.7 units at five channels to thirty-three at twelve; with siblings, it falls to a few hundredths. At three and four channels the siblings change little, because those sensors read the tube within a unit already — their nine and twelve numbers cannot follow a narrow band’s position, and so cannot misread it. From five channels on, the matrix has enough freedom to follow features, and whether it follows them towards the tube or away from it is decided by whether the census has shown it any tube.

The curve with three warmer siblings tracks the one with four, a little above it: 0.85 at five channels against 0.66, 0.29 at eight against 0.19, 0.47 at ten against 0.18. The fourth sibling, at 5,000 K and within ten mireds of the tube, helps by a factor of two or three at the finest sensors and is not what brings the tube in.

Same bands, other proportions #

Each sibling is the census tube with two of its phosphors turned up or down. The warmest has its blue band at 0.43 of the tube’s and its three europium reds at 1.64 times; the 5,000 K sibling, at 0.95 and 1.035, is nearly the tube itself. The terbium green at 543 nm and the four mercury lines stand where they were, and every band sits at the same wavelength in all five spectra. Each sibling was solved for, by Newton’s method in mireds and Duv, to sit within a kelvin of its stated temperature and at the tube’s own Duv of −0.0035, slightly below the Planckian locus as triphosphor tubes are.

What a sibling teaches the matrix is where a triphosphor tube’s bands are, not what proportions they come in. A held-out tube at 5,260 K differs from the 4,000 K sibling in its ratio of blue to red, which every channel sees as a smooth change in the balance of signals — the kind of change a linear map handles by interpolation. What it does not differ in is where its green band falls on the slope of the channel at 540 nm, and that position, which the census alone could not supply, is the same for the whole family.

Seven channels read every light #

Counted over the enlarged census, the lights beyond a unit fall from thirty-seven at three channels to twenty at four, six at five, one at six — the 2,200 K lamp — and none at seven or eight. At ten and twelve channels they rise again to one and two, and those are the census’s two other tubes, the halophosphate and the broadband, which are still alone.

The count at three and four channels is higher with the siblings than without, by four and two. The extra lights beyond a unit are siblings: at three channels the 3,000 and 3,500 K tubes, held out, are read at 2.2 and 1.1 units, because a coarse sensor reads a tube by its overall balance of signals and the warmest siblings are where that balance is furthest from anything else in the census. A coarse sensor reads every tube roughly; a fine one reads a tube well or badly according to its family.

What the siblings cost #

A least-squares fit has only so much freedom, and four lights that pull it towards narrow-band spectra take some of it from the lights that were there before. From five to eight channels the cost is small: the LEDs’ worst error moves from 0.0023 to 0.0028 at six channels and the lamps’ from 0.0040 to 0.0047, and at seven and eight neither moves by more than a fifth. No lamp, LED or mixture changes side of a unit at any count from five.

At twelve channels the cost is real. The lamps’ worst error rises from 0.0005 to 0.0025, fivefold, and the LEDs’ from 0.0007 to 0.0012. Both stay well inside a unit, but the pattern is the earlier essay’s in reverse: at twelve channels the matrix has more freedom than the census can pin, and the siblings, which have the most unusual spectra, pull the extra freedom their way. The finest sensor is the one most changed by whatever is added to its census — for better on the lights added, and for worse on the others.

That is the cost two stages catch the tube and bill the lamps they misplace found in another instrument: a camera’s error estimate, given the halophosphate tube’s siblings and a line to fit them, caught the tube and charged the other lamps for it. There the charge was paid through misclassification; here it is paid through the matrix itself, and at seven or eight channels it is small enough to be no charge at all.

Every sibling, held out #

Each of the five tubes, held out, is read by a fit that still contains the other four. From six channels every one is within a unit, and at seven and eight every one is within 0.41. The warmest sibling is the slowest to come in: 2.2 units at three channels, 1.1 at four, 1.00 at five, then 0.17 at six. It is the family’s edge — every other triphosphor tube is cooler — so it alone is read by extrapolation within its family, as the 2,200 K lamp was the edge of the Planckian lamps and the last of them read.

The 4,000 K sibling, between two others, is read best of all at every count from seven, under a tenth of a unit. This is the shape a family’s line runs through its neighbours found for halophosphate tubes in the camera’s estimate: a family whose members differ in one quantity lies along a line, and a member between two others is read from that line by interpolation. Here the quantity is colour temperature and the line runs in a seven-dimensional signal space, but the principle is the same.

The tubes still alone #

The prediction’s control holds. The halophosphate tube is read within a unit only from six to eight channels in both censuses, and past eight it rises as before — to 1.05 and 1.29 at ten and twelve with the siblings, against 1.11 and 1.12 without. The broadband tube, read from five channels to ten in both, reaches 1.56 at twelve with the siblings against 1.21 without.

Neither is helped, and at the finest sensor both are slightly worse, for the reason the lamps are: the siblings take freedom from the fit that the lone tubes were using. Relatives help their own family and no one else’s. A census that wanted to read every tube would need every tube’s family in it, and new LEDs miss the line their cousins drew is the warning about how far a family’s line carries even then: the cousins teach the line, and a new member is read well only so far as it lies on it.

Two instruments, one lesson about families #

A run of essays on a different instrument reached the same conclusion from the other side. They estimated a camera’s colour error under a lamp from an ambient sensor’s narrow channels, one straight line from a few channel departures to the error, and their trouble was always the lamp the census held nothing like. The tube a third channel misses has no family found that no channel rescued the halophosphate tube, and the reason was the one here: the tube’s error depended on something only the tube had.

The repairs there went through classification — decide a lamp’s family first, then use the family’s own line — and classification turned out to be where the cost lay. A classifier that abstains gives up the tube found a cautious classifier abstaining on exactly the lamp the families were built to catch, and radiators and LEDs are one family to the sensor found that two families the census had kept apart were, to the sensor, one: merged, every member was estimated better.

The matrix here classifies nothing. It is one linear map from seven signals to a chromaticity, and the family enters it only as four more rows in a least-squares fit. That is why the siblings cost so little at seven channels and why no light is ever misplaced: there is no decision to get wrong, only a fit that the family’s members bend slightly towards themselves. A linear map with enough channels can hold several families at once, if the census shows it each one, which is what the camera’s single line could not do and what its classifiers were trying to supply. The merge of radiators and LEDs was the same observation made the other way round — that a family is whatever the instrument sees as one line, not whatever the census named — and the triphosphor siblings behave as one family here in the only sense the fit can register: held out in turn, each is read from the others.

What a sensor maker should take from this #

Train a fine sensor on families, not on examples. A seven-channel sensor fitted to lamps, LEDs, mixtures and a family of four triphosphor tubes reads every one of fifty-one lights within a unit with each held out. The same sensor trained on one triphosphor tube reads that tube at two and a half units.

The family need not bracket the light. Three siblings all warmer than the tube, from sixty to a hundred and forty mireds away, read it within a unit from five channels on. What a sibling carries is where the family’s bands lie, and any member carries it.

Stop at seven or eight channels. Past eight, every addition to the census costs the lights already in it, and the lone tubes rise however the census is enlarged. Seven or eight channels are where the census’s lights are read best and the census’s composition matters least.

How the siblings were built #

The census tube’s spectrum is its five phosphor components — a blue band at 450 nm, 52 nm wide; the terbium green at 543 nm, 18 nm wide; europium reds at 611, 626 and 651 nm — and the mercury lines at their fixed strengths. A sibling scales the blue band by one factor and the three reds by another, and both factors are found by Newton’s method on their logarithms so that the sibling’s correlated colour temperature is the one stated and its Duv the census tube’s, −0.0035. The enlarged censuses are the forty-seven lights with four siblings at 3,000, 3,500, 4,000 and 5,000 K, and with the first three only. Each sensor is the earlier essay’s: n Gaussian channels centred evenly from 445 to 650 nm, each 1.25 times its spacing wide at half height, its matrix the least-squares fit at unit luminance to every light but the one read, and every light’s tint the moon picture’s worst ΔEITP in the shadows at a tenth of a candela with the floor declared in the sensor’s chromaticity.

What this does not reach #

One family, built rather than measured. The siblings are the census tube with two phosphor loadings changed. Real triphosphor tubes from different makers use slightly different phosphors, at slightly different wavelengths, and a census of real tubes would teach a family less tidy than this one.

Noiseless channels. The finer the sensor, the larger the matrix’s weights, and noise in each channel’s signal would be multiplied by them. The seven-channel sensor’s readings here are exact; a real one’s would scatter, more than a three-channel sensor’s would.

The halophosphate and broadband tubes have no relatives. Their families were not built, and they are what remains beyond a unit past eight channels.

Still open: whether noise undoes the seventh channel #

A sensor’s matrix reads each channel’s signal with a weight, and a fine sensor’s weights are larger and of both signs, because it reads chromaticity from small differences between neighbouring channels. Every real channel carries noise — shot noise at low light, drift with temperature — and the matrix multiplies that noise by its weights.

The calculation is the seven-channel sensor fitted to the enlarged census, every channel’s signal perturbed by independent noise of a quarter, a half and one per cent of its own size, and each light read a hundred times — the share of readings beyond a unit, and the noise each sensor amplifies, for three, five, seven and twelve channels. The prediction is that the seven-channel sensor’s readings stay within a unit for every light at a quarter of a per cent and that at one per cent it is no better than five channels, because its weights amplify channel noise about three times as much. If seven channels stay ahead at one per cent, the gain the census found survives the instrument it would be built as.

A twin and a relative #

The habit here is about what a validation set has to contain.

The earlier essay’s held-out fit was fair to every light and still failed one, because the census held no light like it. The obvious repair was a twin: put something nearly identical to the tube in the census, and hold the tube out with its twin left in. A twin works, and it proves nothing beyond itself — a census of twins reads its members because each one is nearly in its own fit.

The failure mode is to repair a held-out failure by adding a near copy of the failing case, and to count the repair as general. The test is to add relatives that are not copies — here, siblings on one side of the tube only, sixty to a hundred and forty mireds away — and see whether the repair survives. It did here, which says the sensor learned the family’s shape and not the tube.