Two stages catch the tube and bill the lamps they misplace

The tube a third channel misses has no family asked why a phone’s ambient-light sensor, reading narrow channels at 450 and 500 nm, could not estimate the camera’s colour error under a halophosphate fluorescent tube. Its estimate missed that tube by 0.445 ΔE₀₀ where it estimated the census’s fifty-four lamps to ±0.141, and no third narrow channel between 560 and 660 nm helped. The answer was the census: it held one halophosphate tube, and a lamp left out of a census that holds nothing like it is estimated from nothing. Given four siblings — the same tube with its main phosphor band moved from 558 to 598 nm — a 640 nm channel read the family’s error well. But one straight line serving every lamp then served the LEDs worse, and the census’s held-out error rose to ±0.162.

The proposal it ended on was to stop asking one line to serve every lamp. Recognise a lamp’s family first — radiators, white LEDs, halophosphate tubes and the rest — then apply that family’s own line, fitted on its own members, each lamp left out of both steps. The prediction was that the tube’s miss falls to about 0.2 while the fifty-four lamps’ error stays near the ratio’s ±0.14, because the LEDs keep their own line. And it named how that could fail: if the classification misplaces lamps, the gain goes to them as extra error, and the census will say how many a family needs before its line is worth having.

The stages work when the family is known and fail at finding it #

Given each lamp’s family, the two stages do better than predicted: the halophosphate tube is missed by 0.011 ΔE₀₀, the LEDs are estimated to ±0.111 against ±0.127 under the ratio alone, and the fifty-four lamps to ±0.133 — better than the ratio’s ±0.141. Found instead, by the nearest other lamp in the sensor’s 450, 500 and 640 nm departures, the family is right for the tube, which is still missed by 0.011, and wrong for five lamps — the pink-tinted radiator, both other tubes, the three-emitter source and one violet-pumped LED. Those five are missed by ±0.41, the lamps placed right by ±0.107, and together the census comes back to ±0.162, no better than the one line the stages were meant to replace. The failure the prediction named is exactly what happens.

• The idea is sound. A family’s own line fits its family, and a classifier that knew the families would give every family a better estimate than one line does.
• The classifier is the whole cost. Five misplaced lamps of fifty-four spend everything the other forty-nine gain.
• Finding a family by its centre fails outright: the white LEDs, pumped from the violet to the cyan, have no centre that describes them.
• The tube’s small miss is an interpolation. Its siblings bracket it, and four lamps fit a line of three numbers almost exactly.

The bars are the result by family. The halophosphate tube’s bar collapses under both two-stage designs. The LEDs’ bars are shortest with the family given and longer with it found, because a violet LED placed among the radiators is estimated by the radiators’ line. The three-emitter source is badly estimated by everything — it is a family of one, and falls back to the census’s line — and the radiators do worse under their own line than under the ratio, for a reason the pink-tinted radiator explains below.

Two stages, each lamp left out of both #

The census is the earlier one: fifty-four lamps — fourteen named and forty generated LEDs — with the camera’s colour error under each, the error two matrices do not reach a white LED found doubling away from daylight, and the sensor’s narrow channels read as departures from what the camera’s own three channels predict; plus the halophosphate tube’s four siblings, which join the lamps an estimate may be fitted on but are not themselves scored. Each lamp is scored by an estimate built entirely without it.

Stage one assigns a family. The families are what the lamps are: radiators and tinted radiators; white LEDs, including every generated violet-pumped and cyan-filled one; the halophosphate tube and its siblings; the triphosphor and broadband tubes; the three-emitter source. A lamp’s family is found from its three departures — at 450, 500 and 640 nm — either as the family whose centre, the mean of its members’ departures, is nearest, or as the family of the single nearest other lamp.

Stage two applies the family’s line. The camera’s error is fitted on the ratio’s departure and the 640 nm departure over the family’s other members, and applied to the lamp. A family with fewer than four other members — the other tubes, the three-emitter source — cannot fit three numbers with anything to spare, and the lamp is estimated instead by the census-wide ratio line, fitted without it.

With every lamp in one family and the ratio’s line applied, the construction returns the earlier essay’s ±0.141 to the digit.

With the family given, every family but one gains #

Telling the estimate each lamp’s family is the best a classifier could do, and it sets the ceiling on what the stages can buy. Over the fifty-four lamps the error falls to ±0.133. The LEDs, forty-four of the fifty-four, gain most in absolute terms: ±0.111 against ±0.127 under the ratio, because their own line need not bend towards the tubes and radiators. The halophosphate tube is missed by 0.011, against 0.445 with no siblings in the census and 0.215 under one line with them.

The radiators lose, from ±0.157 to ±0.215, and nearly all of it is one lamp — the same lamp the lamps two channels miss are smooth to the camera would predict, a smooth one whose trouble no narrow channel sees. The pink-tinted radiator is a smooth lamp whose error, 1.62 ΔE₀₀, is far above its family’s other members’ 1.14 to 1.20, and nothing in its narrow departures says so — the earlier essay found the same lamp becoming the worst miss once the tube was caught. Under the census-wide line it borrows from the LEDs, whose errors are larger; under its own family’s line it is estimated from five radiators that all err less than it does, and is missed by 0.51.

Found by the nearest lamp, five lamps land in the wrong family #

A real sensor is not told the family. Found by the single nearest other lamp in the three departures, the families are right for forty-nine lamps and wrong for five: the pink-tinted radiator, placed among the LEDs; the triphosphor tube, placed with the three-emitter source; the broadband tube, among the LEDs; the three-emitter source, among the LEDs; and one violet-pumped LED, at 465 nm with the broadest band, among the radiators. The halophosphate tube is placed correctly, next to its siblings.

The map shows why the five are where they are. The white LEDs spread across most of the plane: a violet pump moves the 450 nm departure one way, a cyan fill the 500 nm one, and a warm phosphor the 640 nm one, so the family occupies a sprawl rather than a cluster, and a lamp from a small family near the sprawl’s edge has an LED as its nearest neighbour. The two other tubes and the three-emitter source are families too small to have a member near every lamp that belongs with them. The triphosphor tube’s europium lines put it near the three-emitter source, whose red emitter sits nearby; the broadband tube’s smooth coating puts it among the LEDs, whose phosphors are smooth too. Each placement is reasonable on the evidence the sensor has — a lamp is not a blackbody made the point that a lamp’s label says little about its spectrum, and here its spectrum’s few narrow readings say little about its label.

The misplaced five pay for the other forty-nine #

The five misplaced lamps are missed by ±0.413; the forty-nine placed correctly, by ±0.107. The second number is better than any design in the census, and the first is worse than all of them. The violet LED at 465 nm estimated by the radiators’ line is missed by 0.72; the pink-tinted radiator estimated by the LEDs’ by 0.44; the three-emitter source by 0.34. Together they bring the census’s error to ±0.162, the same as the single line with a 640 nm channel that the two stages were designed to improve on.

That is the prediction’s own failure clause, and it happened in its sharpest form: the gain went to the misplaced lamps as extra error, and nearly all of it. The number of lamps a family needs before its line is worth having is not the question the census answers most usefully. Two channels estimate what one can only sort found an estimate better than a verdict because an estimate degrades gracefully where a verdict flips; a family assignment is a verdict, and it flips. The halophosphate family worked with four siblings. What decides the outcome is how many lamps are misplaced into families whose lines are wrong for them, and five of fifty-four was enough.

A family’s centre cannot hold the LEDs #

The simpler classifier is worse by far. Finding each lamp’s family by the nearest family centre misplaces forty-one of the fifty-four lamps, and the error is ±0.515. The white LEDs’ centre, the mean of forty-four lamps spread from violet-pumped to cyan-filled, lies where no LED is; most violet LEDs are nearer the radiators’ centre and most cyan-filled ones nearer the halophosphate family’s. A family whose members differ in the very departures the classifier reads has no centre, and a centroid rule sorts its members into whichever compact family is closest.

A narrow channel has to be read on its own found that pooling a narrow channel into one statistic with the others lost what it alone could see; the same lesson arrives here for families. The sensor’s departures place a lamp in a space where each family is a shape, not a point, and a rule that treats families as points misreads the largest of them.

The tube’s small miss is an interpolation #

The tube’s miss of 0.011 is the most striking number here, and it needs reading carefully. The four siblings put the tube’s main band at 558, 568, 588 and 598 nm, and the census’s tube sits at 578, between two of them. Its family’s line is fitted on four lamps with three numbers, which leaves one number’s worth of freedom for noise, and the camera’s error falls smoothly as the band moves up. A line fitted on points either side of the tube is interpolating, and interpolation along a smooth trend is easy.

That is what the stages buy, stated honestly. They let a lamp be estimated from relatives, and for a lamp whose relatives bracket it, the estimate can be very good. A halophosphate tube whose band sat outside the siblings’ range, or a family whose members differed in two ways at once, would be estimated by extrapolation, and four siblings would not support it as well.

What a sensor designer should take from this #

A family’s own line is worth having only if the family can be recognised reliably. Here it could for the halophosphate tube and for forty-eight other lamps, and the five it could not undid the gain. A classifier that knew when it was unsure — and fell back to the census-wide line for a lamp whose nearest neighbours disagree — would keep most of the stages’ gain; this census has not tested one.

The white LEDs are one family to the camera and several to the sensor. Their departures spread because their pumps and phosphors differ — the camera, a fourth observer with its own broad channels, sees them as a single kind of error — and a classifier built on those departures should treat them as several families or as a region, not a point.

The tube can be caught. One ratio reads a peak and a trough built an estimate that treats every structured lamp alike; the halophosphate family shows that a lamp’s error can follow a quantity — here the position of its main phosphor band — that a long-wave channel reads well, once the estimate is allowed to use it for that family alone.

How the estimate was built #

The fifty-four lamps, the camera’s error under each, and the narrow channels’ departures are those of the earlier essays on this sensor: each channel a 10 nm triangle whose share of the sensor’s signal is compared, in the logarithm, with a linear prediction from the camera’s three channels fitted on smooth training lamps. The siblings are the halophosphate model with its main band moved, their departures computed through the same channels. Families are assigned by what each lamp is. The centroid classifier uses the mean of each family’s other members’ three departures; the neighbour classifier, the single nearest other lamp by Euclidean distance in the three departures, unscaled. A family’s line is ordinary least squares with an intercept on the ratio’s departure in magnitude and the 640 nm departure signed, fitted on the family’s members other than the lamp scored, used when there are at least four of them; otherwise the census-wide line on the ratio’s departure alone, fitted on the census’s other fifty-three lamps.

What this leaves out #

The classifier is the simplest one. A nearest-neighbour rule on unscaled departures is a baseline; scaled departures, several neighbours with a vote, or a rule that abstains when neighbours disagree would misplace fewer lamps, and the census does not say how many fewer.

The families are given by what the lamps are, not discovered. A sensor would have to learn them, and the families it learned might not be these.

The siblings are one tube’s family, and a tidy one. Real halophosphate tubes vary in more than the main band’s position; with a family less regular than this one the tube’s small miss would not survive.

Still open: whether a classifier that can abstain keeps the gain #

The five misplaced lamps were placed confidently and wrongly. A lamp whose nearest neighbours belong to different families is one the classifier should not trust itself about, and the census-wide line is a safe estimate for it.

The calculation is the two-stage estimate with a classifier that looks at a lamp’s three nearest neighbours and abstains — falling back to the census-wide line — unless all three share a family, scored as before. The prediction is that the pink-tinted radiator, the two small-family tubes and the three-emitter source abstain, because their neighbours are mixed, that the violet LED at 465 nm still lands among the radiators, and that the census’s error falls to about ±0.135, most of the known-family ceiling, while the halophosphate tube keeps its small miss. If the error stays near ±0.16, abstention costs the LEDs their own line more often than it rescues the misplaced, and a family’s line needs a better map of the lamps than the sensor’s own departures provide.

A two-stage method is as good as its first stage #

The habit is about where to look when a method built from two parts fails.

The two stages were proposed for the second stage — a family’s own line — and the second stage did everything asked of it. The measurement that decided the result was the first stage’s accuracy, which the proposal assumed rather than stated. Five errors in the first stage outweighed forty-nine improvements in the second, because an error in the first stage puts a lamp under a line built for something else, and such a line can be arbitrarily wrong for it.

The failure mode is to evaluate a pipeline by the stage that was designed and assume the one that was not. The test is to run it twice — once with the first stage perfect, once as it would be — and read the gap between them as the first stage’s price.