The bin's curve holds between its points and breaks beyond them

A camera’s colour error under a white LED depends on what the LED is made of, and the run of essays this one continues has estimated that error from an ambient sensor’s narrow channels before the photograph is taken. The converter gives the red phosphor away set out the two stages: read the LED’s red phosphor from the sensor’s 600 nm departure, then read the camera’s error from the estimated recipe — red phosphor, colour temperature in mireds and their product — by a line that each pump’s LEDs draw. Beside the locus the converter still gives the red away rebuilt every LED a Duv of 0.003 below and above the Planckian locus, where real LEDs are binned, and found the first stage robust and the second a third to a half worse, because the camera’s error itself moves with the Duv.

Across a bin, the camera’s error bends with the temperature put the Duv into the line. One term bought little; the error rises across the locus at 2,700 K and falls at 4,000 and 6,500 K, so the Duv had to enter as a slope and a curve for each temperature, ten numbers a pump. That line read the LEDs beside the locus within ±0.056 ΔE₀₀ below and ±0.048 above, each LED left out of its own fit.

Every curve in that line was fitted through three Duv values: −0.003, nought and +0.003. Three points determine a parabola, and a parabola fitted through three points passes close to them whatever the quantity it describes does in between. Leaving one LED out at a time tests the line on a recipe it has not seen, but not on a Duv it has not seen; every held-out LED sat at one of the same three values as the LEDs that trained it.

The prediction was that new LEDs built at a Duv of ±0.0015, inside the bin, are read within ±0.06 by the ten-term line, because the error’s course across the bin is smooth, and that LEDs at ±0.006, outside it, are read worse than ±0.10, because a curve fitted on ±0.003 carries its bend out with it. If the inner LEDs had been read worse than ±0.08, the curves would have been lines drawn through three points rather than the error’s course, and a bin would need more Duv values before its line could be trusted anywhere.

Inside and outside, read on LEDs the line never saw #

Built at the four new Duv values, the grid’s thirty recipes give 119 LEDs — one at −0.006 cannot be built within the converter range the grid allows — and the ten-term line, learned on the eighty-nine LEDs at nought and ±0.003, reads those at −0.0015 within ±0.045 and those at +0.0015 within ±0.043. Those at −0.006 it reads within ±0.153 and those at +0.006 within ±0.161. A slope per temperature, three Duv terms fewer, reads the inner LEDs at ±0.053 and ±0.049 and the outer ones at ±0.093 and ±0.110. With no Duv term at all the outer LEDs come within only ±0.142 and ±0.249.

• Both halves of the prediction hold. Inside the bin the curve reads better than the ±0.06 asked of it, and better than at the bin’s own edges, where it was fitted; outside it, worse than ±0.15 on both sides.
• The curve is the best line inside the bin and the worst of the four with any Duv in it outside. It beats the slope by about a tenth at ±0.0015 and loses to it by a half at ±0.006.
• The alternative the prediction named does not hold. The inner LEDs are read well, so the curve is not merely passing through its three points.

Read across the seven Duv values the four lines draw four shapes. The line with no Duv term rises steeply on both sides of the locus, as an offset ignored must. One Duv term flattens the slope on one side and not the other. The slope per temperature is the flattest line in the figure: it is never the best at any Duv inside the bin and never far from it, and at twice the bin’s edge it is the best of all four. The curve is a valley, deepest inside the bin and climbing faster than any other line beyond it.

A reader who saw only the three fitted values would choose the curve, since it reads both sides of the bin best. LEDs built on the locus still fall off the line found the camera’s error a V about the locus across freely varied recipes, and a V read at its arms and its point looks like any curve one likes. A reader who saw only the four new values would choose the curve inside and the slope outside. The two readers would agree about the inside and disagree about which line describes the error, and the disagreement is the subject of the rest of this essay.

The error itself does not leave its parabola #

The obvious account of the outer failure is that the camera’s error turns more sharply beyond ±0.003 than within it, so that any curve learned inside the bin falls short outside. The census can test that directly, because twenty-nine of the thirty recipes are now built at all seven Duv values and the error’s course across them can be read without any line at all.

Averaged over each temperature’s recipes, the camera’s error at 2,700 K runs 1.878, 1.876, 1.896, 1.931, 1.984, 2.057 and 2.274 ΔE₀₀ from −0.006 to +0.006. The parabola through its three values at nought and ±0.003, carried out to ±0.006, misses the mean there by 0.015 below the locus and 0.020 above. At 4,000 K it misses by 0.001 and 0.004, and at 6,500 K by 0.001 and 0.001. At the two cooler temperatures the parabola through three points predicts the error at twice their spread to a few thousandths.

So the obvious account is wrong. The error’s average course is smooth across the whole range, curved most at 2,700 K and nearly straight at 6,500 K, exactly as the three fitted values said, and it stays on the parabola they define. Whatever goes wrong at ±0.006, it is not that the error bends more there.

That rules out one cause and sharpens the question. A line that has a separate slope and curve for each temperature should be able to follow a mean course that is itself a parabola for each temperature. If it misses that course by 0.17 at twice the bin’s edge, its curve terms are not describing the course. They are describing something else.

A bend borrowed from the locus #

The ten-term line does not learn each temperature’s course alone. Every pump’s line shares four terms across all three temperatures — a constant, the estimated red phosphor, the mireds and their product — and only the Duv terms are set temperature by temperature. Where the shared terms cannot place a temperature’s error exactly, some of the misfit is left for the Duv terms to absorb, and the only Duv terms that can absorb an offset at the locus without moving the bin’s two edges unequally are the curves.

Read by its mean miss at each temperature, the ten-term line leaves the LEDs at ±0.003 within a thousandth or two of nought at every temperature, and the LEDs on the locus — each left out of its own fit — +0.068 out at 4,000 K, −0.105 at 6,500 K and −0.033 at 2,700 K. On the new LEDs it misses by +0.042 at 4,000 K and −0.046 at 6,500 K inside the bin, and by −0.17 and +0.19 at ±0.006, on both sides. The miss at every temperature is symmetric in the Duv’s sign, and it changes sign between the locus and the outer LEDs.

That is the signature of a parabola laid through a gap. If the line’s curve at a temperature is set so that the bin’s edges are read exactly and the locus is left off by an amount r, the miss at any Duv d is r times one less the square of d over 0.003: three quarters of r at ±0.0015 and minus three times r at ±0.006. With the locus offsets the line leaves held out, that rule gives +0.051 and −0.20 at 4,000 K and −0.079 and +0.32 at 6,500 K; the readings are +0.042 and −0.17, and −0.046 and +0.19 — the same sign everywhere and two thirds to nine tenths of the size. The held-out locus offsets overstate what the fitted line carries, since leaving an LED out of its own fit moves the line away from it, so the rule’s figures are upper bounds and the readings sit under them.

The mechanism is now plain. The curve does not bend because the error bends. It bends because the line, with its shared terms fixed, puts the locus LEDs and the bin’s edges on different footings at 4,000 and 6,500 K, and the cheapest way to read both edges well is a parabola that passes through them and misses the middle. Across a bin, the camera’s error bends with the temperature noticed the price inside the bin — the ten-term line read the locus itself at ±0.073, worse than the ±0.062 of a line with no Duv in it — and took it for the cost of the repair. It was the repair’s mechanism.

Worst where the error is straightest #

The temperature-by-temperature reading confirms it from the other side. At ±0.006 the ten-term line reads the 2,700 K LEDs within ±0.095 below the locus and ±0.122 above, the 4,000 K LEDs within ±0.175 and ±0.177, and the 6,500 K LEDs within ±0.188 and ±0.194. The slope per temperature reads 2,700 K at ±0.119 and ±0.145, 4,000 K at ±0.084 and ±0.082, and 6,500 K at ±0.041 and ±0.070.

If the curve had failed because it carried a real bend too far, it would fail most where the bend is strongest, at 2,700 K. It fails least there, and that is the only temperature at which it beats the slope outside the bin. At 6,500 K, where the error’s mean course is nearly a straight line and the three fitted values asked for almost no curvature, the ten-term line has its largest curvature of the three — because 6,500 K is where its locus offset is largest — and its worst reading. The curve’s error outside the bin follows the line’s locus offset, not the error’s bend.

The slope per temperature fails in the opposite pattern. Its outer misses are largest at 2,700 K, where the true course bends and a straight line through three points falls under it, and smallest at 6,500 K, where the course is straight. That is the failure of a line that is too simple for the quantity it describes, and it grows slowly. A family’s line runs through its neighbours met the second kind among lamp families, where a line fitted on six radiators close together, extended past them, missed the triphosphor tube by 7.6. The curve’s is the failure of a line whose extra terms describe something other than the quantity, and it grows with the square of the distance.

Inside the bin, every LED within a tenth #

Of the sixty LEDs at ±0.0015, the ten-term line misses none by more than 0.078 and the slope per temperature none by more than 0.099; both lines miss low at 2,700 and 6,500 K and high at 4,000 K, the curve by about two hundredths at 2,700 K and four or five at the cooler two. Read LED by LED the inner reading is as good as the summary says, with no stray recipe carrying the root mean square.

The inner LEDs lie where the borrowed bend is small. At ±0.0015 a parabola through the bin’s edges and the locus misses by three quarters of its locus offset, and the locus offset is under a tenth, so the inner reading inherits a few hundredths of it and no more. The prediction held for a reason it did not name. The error’s course is smooth, as it said, but what keeps the inner LEDs close is that they sit between two places the line was made to fit, and a parabola between its points cannot stray far from them whatever it is describing.

That is also why the held-out test at three Duv values could not see the problem. Each left-out LED sat at a Duv every other LED shared, so its reading tested the line’s grasp of recipes and never its grasp of the Duv. The new LEDs at ±0.0015 are a test of the Duv inside the fitted range, which a parabola passes easily; the LEDs at ±0.006 are the first test outside it.

The first stage holds everywhere #

The red phosphor is read within ±0.097 at −0.006, ±0.104 at −0.0015, ±0.107 at +0.0015 and ±0.114 at +0.006, against ±0.110 and ±0.117 at ∓0.003 and ±0.120 on the locus. The first stage reads the new LEDs as well as the old, and the LEDs furthest from the locus a little better than those on it. The converter gives the red phosphor away found the 600 nm departure reading the converter’s peak, which the red phosphor pushes shorter, and a Duv term in the first stage lets that reading follow a converter rebuilt for a different chromaticity.

So the whole of the outer failure is in the second stage, and in one part of it: the Duv’s curvature terms. Two stages catch the tube and bill the lamps they misplace showed how a two-stage reading can hide its first stage’s mistakes in its second; here the first stage makes none worth the name, and the second makes its own. The estimated recipe going into the error line is as good at ±0.006 as anywhere, and the slope per temperature turns it into a reading within ±0.11. Only the curve, given room to bend by the locus offset, throws it away.

What a calibration engineer should take from this #

A line validated by leaving one LED out at a time has been tested on recipes, not on Duv. Every calibration LED of a bin sits at a handful of Duv values, and a held-out score there says nothing about LEDs between or beyond them. The test that does is LEDs at new Duv values, and here it changed which line is best outside the bin. New LEDs miss the line their cousins drew met the same gap in recipes, where a line that learned what a channel means from LEDs varied only in their pump misread new LEDs varied in their phosphors.

A curve term can carry a misfit rather than a curvature. When a line’s other terms are shared across groups and cannot place each group’s level exactly, the curve terms absorb the difference and look like a bend. The check is to compare the line’s implied curvature with the quantity’s own course across more Duv values than the line was fitted on; here they differ at 6,500 K by an order of magnitude.

Inside a bin of ±0.003, either line will do; beyond it, the simpler one. Real binning tolerances run from about ±0.002 to ±0.006 depending on the maker and the grade. A sensor calibrated on LEDs at the centre and edges of a tight bin and then used on a loose one is in the situation the outer LEDs model.

How the LEDs were built and read #

The grid is thirty white LEDs: three blue pumps at 440, 452 and 465 nm, four red phosphor strengths from nought to one, and three correlated colour temperatures, 2,700, 4,000 and 6,500 K. Each recipe is solved for the conversion and the yellow phosphor’s peak that place it at a target chromaticity a stated Duv from the Planckian locus at its temperature, by Newton’s method in the CIE 1960 plane to a ten-millionth; a solution whose phosphor peak falls outside the range the grid allows is not built. For each built LED the camera’s colour error is the mean ΔE₀₀ of the census’s test colours as the modelled camera sees them under the LED against the eye, and the sensor’s departures are read from its narrow channels.

The four lines are the ones the earlier essay compared. The first stage reads the red phosphor from the 600 nm departure, the mireds and their product, with the Duv as one more term in every line that has one; the second stage reads the camera’s error within each pump from the estimated red, the mireds and their product, with the Duv entering not at all, as one term, as a slope for each temperature — the Duv, its product with the mireds over 250 and with their square — or as that slope and the same three terms in the square of the Duv. Every line is fitted by least squares on all eighty-nine LEDs at nought and ±0.003 and read on the 119 new LEDs; the values at the fitted Duvs are those LEDs read with each left out of both stages, as before. Spreads are root mean squares.

What this does not reach #

Real bins are not grids. The census builds every recipe at every Duv; manufactured LEDs scatter across a bin with recipes the maker chose for other reasons, and the Duv and the recipe are correlated in ways this grid does not represent.

The camera and sensor are one model each. The error’s shape across the bin, warm whites rising and cool falling, is a property of this camera’s dyes against the eye, and a different camera could reverse it.

The red phosphor’s misreading is not followed into the error. Its ±0.10 enters the second stage through the estimated red, and how much of each LED’s miss it accounts for is not separated here.

Still open: whether a level per temperature frees the curve #

The ten-term line borrows its bend from the gap it leaves between the locus and the bin’s edges, and the gap is there because its shared terms cannot place each temperature’s level. A term that gives each temperature its own level — one more number a pump, the square of the mireds — would let the locus LEDs be placed without bending the Duv’s course to do it.

The calculation is the ten-term line with the square of the mireds added to each pump’s error line, read on the eighty-nine LEDs with each left out and on the 119 new LEDs as here. The prediction is that the locus LEDs come within ±0.05, below the ±0.062 of a line with no Duv in it, and that the LEDs at ±0.006 come within ±0.08 on both sides, better than the slope per temperature’s ±0.093 and ±0.110 — because with the level placed, the curve terms are left to describe the error’s own course, which stays on its parabola. If the outer LEDs stay beyond ±0.10, the curve terms are absorbing something other than a level, and the bin’s error line should be learned with no curvature at all.

A held-out score that tested the wrong variable #

The habit here is about what a validation varies.

Leaving one LED out at a time is a sound test and it passed, and it was a test of the variable it held out: the recipe. The Duv sat at the same three values in every training set and every test, so the score could not report how the line behaved at any other Duv. The new LEDs varied the Duv and found the line’s best performance and its worst in the same model, a few thousandths of Duv apart.

The failure mode is to read a held-out score as a statement about every input the model takes, when it is a statement only about the inputs the hold-out varied. The test is to name, for each input, whether any test point took a value no training point took — and where none did, to build one.