Held-out validation — where it appears
Named by 33 essays across 4 fields — each of them below, with the objects they name alongside it.
The chart decides the profile
A camera's colour matrix is nine numbers fitted to a set of patches somebody chose, and the number that comes with it is the error on those patches. On a chart with no chromatic range that number is 0.19 ΔE00 and the matrix delivers 1.65 — and a second matrix, indistinguishable on the chart, delivers 2.03.
A matrix is fitted under one light
A camera's colour matrix is nine numbers determined by a chart photographed under a particular illuminant, and the error it quotes is the error under that illuminant. Fitted under a tungsten lamp and used under a cold sky it delivers eight times as much — and the two-matrix scheme every real profile uses turns out not to be a compromise at all.
A fit can be exact and empty
Every fitted object here reports one number, the residual on the data it was fitted to, and every one of them has two more that nobody publishes — how many of its parameters the data actually determine, and how large the family of equally good answers is. The third column is where the failures live.
A profile is a fit between its nodes
A printer profile is a table, exact at every patch that was printed and interpolated everywhere else — and everywhere else is where every job lives. Going from a hundred and thirty-five patches to three and a half thousand is twenty-seven times the work for eight times the accuracy, and the exponents say that is as good as it gets.
The chart was measured by an observer too
A camera profile is fitted so that the camera's numbers reproduce the chart's measured tristimulus values. Those values were computed through the 1931 observer, so the fit inherits every departure in this round — and the fit's own residual, at 0.19 ΔE₀₀ on the chart, is fifteen times smaller than the term it cannot see.
A corner is corrected by one row
A grey-card gain map makes the corner of a frame exactly right on grey and leaves coloured patches 1.86 colour differences wrong at twenty-five degrees. A three-by-three matrix fitted at that position halves it — and six of its nine numbers do nothing, because the moved filter edge is in one channel. The three that matter rebuild the lost red from green and blue, they carry to another lamp better than a gain map in eleven cases of twelve, and in the twelfth, a row fitted under tungsten and used in daylight, they leave the grey 9.2 off.
One row for every lamp costs the lamps that lose least
A corner correction fitted under one lamp is right under that lamp and can be badly wrong under another, so a converter unsure of its lamp was offered a single row fitted under several at once. Pooled over four lamps, one row holds every lamp's corner between 1.33 and 1.56 colour differences — no lamp worse than a grey-card map, and none near the 4.91 a wrong row can leave. The price falls on the white LED and the fluorescent tube, which it leaves four times worse than their own rows, because they lose the least red at the corner and the pooled row is built to repair the lamps that lose the most. Two rows, one per class of lamp, keep almost all of both.
Two channels estimate what one can only sort
A narrow sensor channel was fitted to call lamps smooth or structured, and the label turned out to be a proxy for the thing a camera cares about: its own colour error under the lamp. Scored against that error directly, the best channel is still the one at 450 nm — the band where camera and observer differ most comes close and no closer. Read as an estimate rather than a verdict, a second channel at 500 nm, useless as a vote, earns its place: together they predict the camera's error to within 0.14 ΔE00 over fifty-four lamps, and deciding when to repair from that estimate raises a quarter of the false alarms the label raises.
A straightened channel repeats the one beside it
A phone's narrow sensor channel at 570 nm ranks a camera's colour error under a lamp almost as well as the one at 450, and estimates it worse than guessing the mean, because one lamp reads ten times higher than any other. Read through a logarithm, the channel is rescued: its estimate improves from ±0.57 to ±0.23. It still does not reach the 450 nm channel's ±0.16, and put beside that channel it adds nothing at all, where a 500 nm channel cuts the error to ±0.14. Straightened, 570 nm sorts the lamps as 450 nm already does; and the two lamps it reads highest have the most different errors, which no transform of one reading can tell apart.
One ratio reads a peak and a trough
Two narrow sensor channels, at 450 and 500 nm, estimate a camera's colour error under a lamp to ±0.14 ΔE₀₀; the proposal was to read their ratio as one number instead. Taken as one channel's departure over the other's, the ratio is worthless — no better than the mean, because for smooth lamps both departures are nearly nothing. Taken as the departure of the two readings' ratio from what the camera predicts for it, one number estimates the error to ±0.141, as well as the two channels as two. The reason is a sign: under every structured lamp the channels depart in opposite directions, a peak at 450 and a trough at 500, and the ratio adds them; where a cyan fill lifts 500, it subtracts them, as the error falls. And no five per cent calibration error moves any of these estimates by a hundredth.
The tube a third channel misses has no family
Two narrow sensor channels read as one ratio estimate a camera's colour error under fifty-four lamps to ±0.141 ΔE₀₀, and their worst miss is the halophosphate tube, whose trouble lies in its long-wave phosphor. A third narrow channel in the long-wave gap was predicted to halve that miss near 610 nm. No channel does, at any centre or width: the best buys under a tenth of the whole error and leaves the tube's miss above three quarters of what it was. At 610 nm the camera already predicts the tube's share almost exactly. The tube's error follows where its main phosphor band sits, and a 620 or 640 nm channel does read that — inside the tube's own family. The census has one halophosphate tube, and a lamp left out of a census that holds nothing like it is a lamp no channel can be fitted to.
Two stages catch the tube and bill the lamps they misplace
A narrow-channel estimate of a camera's colour error missed the halophosphate tube because the census held nothing like it, and a line that learned the tube's family from four siblings cost every other lamp. The proposed cure was two stages: recognise a lamp's family, then apply that family's own line. Given each lamp's family, it works as predicted and better — the tube missed by 0.011 ΔE₀₀, the fifty-four lamps estimated to ±0.133 against the ratio's ±0.141. Found by the nearest other lamp in the sensor's departures, the family is right for the tube and wrong for five lamps, and those five, missed by ±0.41, take the whole census back to ±0.162. The gain is real; the classifier is where it is lost.
A classifier that abstains gives up the tube
Found by the nearest other lamp, a lamp's family cost a two-stage estimate of a camera's colour error more than it bought, because five lamps were placed confidently in the wrong family. The proposed repair was a classifier that abstains — falls back on the census-wide line — unless a lamp's three nearest neighbours agree. It abstains on the wrong lamps. The misplaced ones have three agreeing neighbours, all white LEDs; the lamp it gives up is the halophosphate tube the two stages were built to catch, whose third neighbour is a cyan LED, and its miss returns from 0.011 to 0.445. Two agreeing neighbours look better than knowing every family. Chosen without looking at the lamp being estimated, the rule is worth exactly what the ratio alone is worth.
A family's line runs through its neighbours
The halophosphate tubes differ from one another by one number — where their main phosphor band sits — and so lie along a line in the three departures an ambient-light sensor reads. The proposal was to classify a lamp by the family whose line passes nearest, which would keep the tube a vote of neighbours gives up. It keeps the tube, under every version that does not rescale the distances. It also places eighteen white LEDs on the radiators' line, because that line — fitted to six radiators close together — runs along the LEDs' own and, extended, straight through the violet-pumped ones; and the radiators' estimate, extrapolated to them, misses the triphosphor tube by 7.6. Clamped to the stretch each family spans, the lines still cost ±0.285; given to the halophosphate family alone, its line claims a cyan LED and costs ±0.179. The one line classifier that comes in under the ratio line's ±0.141 does so by two thousandths, having placed the tube with the LEDs.
Radiators and LEDs are one family to the sensor
An ambient-light sensor's three narrow departures see the collection's six smooth radiators and forty-four white LEDs along one line: the radiators' own short line lies on the LEDs' long one. Estimating a camera's colour error family by family, the census had treated them as two, and every classifier that tried to tell them apart paid for it. Merged into one family of smooth broadband sources, found by the nearest other lamp as before, the two-stage estimate comes to ±0.127 over fifty-four lamps held out — below the ratio line's ±0.141, and below the ±0.133 the separate families managed when every lamp's family was simply given. The halophosphate tube is still caught. The radiators, which the prediction said the merge would cost a little, are estimated better inside it than inside their own family, because a three-coefficient line fitted over five radiators was worse for them than the census-wide ratio line.
New LEDs miss the line their cousins drew
An ambient-light sensor's three departures can estimate a camera's colour error under a lamp, and merging radiators and white LEDs into one smooth family brought that estimate to ±0.127 over fifty-four lamps, each left out in turn. Every one of those lamps had relatives in the pool. Fourteen lamps the census never saw test what the relatives were worth. The prediction was that new smooth lamps would land on the line and tinted incandescents fall off it by four tenths or more. The radiators and daylights land on it, within ±0.13 each. The tinted lamps do too — the pink-glass incandescent, placed among the LEDs by its nearest neighbour, misses by 0.12. The lamps that fall off are the new white LEDs: ±0.20, half again the census's error, two of them by a quarter. Their misses follow the sensor's 640 nm channel at a slope of −0.72, explaining four fifths of their scatter, because the line learned what that channel means from LEDs varied only in their blue pump. Adding the fourteen to the pool does not help.
LEDs varied in their phosphors fall off every line
The merged smooth family estimates a camera's colour error from an ambient-light sensor's departures to ±0.127, and five new white LEDs, varied in their phosphors where the census's forty-four vary their pump, missed by ±0.196 because the family's line had learned almost nothing about the sensor's 640 nm channel. Rebuilt on forty-eight LEDs varied in pump, converter and red phosphor, the line learns it: its 640 nm weight goes from −0.13 to −0.74, most of the way to the −0.85 the new LEDs implied. The prediction was that the new LEDs would then land within ±0.13. They miss by ±0.229, worse than before, crossing to the other side of the line, and the census's own held-out error more than doubles, to ±0.294. No pool of LEDs restores it, and no line in the sensor's departures reads the grid's own LEDs better than ±0.29. Across the grid the camera's error is a V about the Planckian locus, rising away from it on the green side and on the pink — and every one of the census's LEDs sat on the green side.
LEDs built on the locus still fall off the line
A camera's colour error under a lamp can be estimated from three narrow channels of a small sensor, read as departures from a smooth lamp of the same colour, and the estimate held for the census's white LEDs, which varied only their blue pump. Rebuilt on LEDs varied freely in pump, converter and red phosphor, it failed: the error was a V about the Planckian locus, and no line in the departures read it. A maker does not let the white fall where it falls, so the proposal was to build the LEDs as a maker does — the converter tuned for each pump and red phosphor until the white sits on the locus — and the prediction was that the V would go and the line return, the census's held-out error to within ±0.15. Thirty of thirty-six such LEDs can be built. On the locus the V is gone and the line is not: the LEDs alone are missed by ±0.25 to ±0.28, the census's lamps by ±0.27. Taken one pump at a time, the departures read the LEDs pumped at 452 nm to ±0.05 and those at 440 and 465 nm no better than ±0.17. Within one pump the LEDs are readable, and across pumps they are not.
A channel at each pump cannot see the recipe
Thirty white LEDs built onto the Planckian locus defeat a line in a small sensor's three departures, except within one pump: the 452 nm LEDs are read to ±0.047 by the 450 nm channel, the others no better than ±0.15. The proposal was a channel at each pump — 440 and 465 nm beside the 450 — and the prediction that each pump's LEDs would be read from their own channel within ±0.08. The 465 nm LEDs' own channel halves their miss, to ±0.135; the 440 nm LEDs' moves it from ±0.171 to ±0.149. No line in any three of the five departures reads either pump better than ±0.1. Yet the camera's error is simple: a line in what each LED is made of — its red phosphor, its colour temperature and their product — reads every pump within ±0.07, the 440 nm LEDs to ±0.043. The error is easy; the channels cannot see the recipe that sets it. Knowing each LED's pump halves the pooled miss, to ±0.119, and two blue channels can tell only one pump of three from the others.
The converter gives the red phosphor away
A camera's error under white LEDs built onto the Planckian locus is simple in what each LED is made of — its red phosphor and its colour temperature — and a small sensor's narrow departures cannot read the red phosphor. The proposal was a channel at 610 nm, where a garnet converter falls and a red phosphor rises, and the prediction that it reads the red-phosphor strength within a tenth of its range. Within a pump it reads nothing: ±0.36 to ±0.43 at 440 and 452 nm against a spread of ±0.36, and no red channel from 600 to 680 nm does better than ±0.22. The reason is the locus. To keep the white's colour, each step of red phosphor pushes the converter's peak shorter, by 29 to 55 nanometres from none to the most, and in every red channel the two changes cancel. What survives is the converter: the 600 nm departure reads its peak to ±5 nm. Read across all thirty LEDs at once, the converter gives the red phosphor away — ±0.12 — and the error read through that estimated recipe comes within ±0.08 at every pump, the 440 nm LEDs at ±0.078, half their own channel's miss.
Seven channels end the trade and lose the tube
A three-channel ambient sensor fitted to a room's forty-seven lights trades its error on LEDs against its error on lamps, and a fourth channel moved that trade without closing it. The prediction for more channels was that six, spread from 445 to 650 nm, bring every lamp, mixture and LED under a ΔEITP unit with the lamps within 0.0025 in the CIE 1960 plane, and that nothing short of eight reads the halophosphate tube. Six channels leave the 2,200 K lamp beyond a unit and the lamps at 0.0040; seven bring every lamp, mixture and LED in, with the LEDs and lamps both under 0.001, and the trade is gone — both families improve together. The halophosphate tube is read from six. But the triphosphor tube, held out of its own fit, is read only by three and four broad channels and lost by every finer sensor, its tint growing to thirty-three units at twelve channels. A sensor learns the lights it was fitted on, and a finer one extrapolates further from a light with no relative in the census.
Relatives, not twins, teach a sensor a tube
An ambient sensor of seven evenly spread channels, fitted to a room's forty-seven lights, reads every lamp, mixture and LED within a ΔEITP unit with each held out of its own fit, and loses the triphosphor tube, which nothing else in the census resembles — thirty-three units out at twelve channels. The prediction was that four triphosphor siblings, the same phosphors mixed for 3,000 to 5,000 K, would bring the tube within a unit at seven and eight channels and stop its rise past eight. They do both: the tube reads at 0.04 with seven channels and 0.06 with twelve, and seven or eight channels then read all fifty-one lights. A twin is not what does it. Three siblings, every one warmer than the tube and the nearest sixty mireds from it, read it within a unit at every count from five. What the siblings cost is paid at the finest sensors: at twelve channels the Planckian lamps' worst error rises fivefold, while from five to eight it moves by under a quarter.
Beside the locus the converter still gives the red away
On the Planckian locus, a white LED's red phosphor pushes its converter to a shorter peak, so an ambient sensor's 600 nm departure, which reads the converter, reads the red phosphor too — within ±0.12 of a spread of ±0.36 — and through it the camera's colour error within ±0.074 ΔE₀₀. Real LEDs are binned a few thousandths off the locus, and the prediction was that a Duv of 0.003 moves the converter about as far as a fifth of the red phosphor does, so that the line learned on the locus misreads off-locus LEDs by a fifth of the range and their error falls back to ±0.12. The converter moves under two nanometres. The line reads the off-locus LEDs' red within ±0.11 and ±0.12, as well as it reads the locus's own held out — with a bias of about a twentieth that follows the Duv's sign. The error reads a third to a half worse, ±0.096 and ±0.107, and most of that is the camera's error itself moving off the locus with nothing in the recipe changed.
Across a bin, the camera's error bends with the temperature
White LEDs a Duv of 0.003 off the Planckian locus have their red phosphor read by an ambient sensor as well as LEDs on it, and their camera colour error read a third to a half worse, because the error itself moves with the Duv. The prediction was that one Duv term in the error's line, learned over all eighty-nine LEDs, brings those beside the locus back to the locus's ±0.074 ΔE₀₀ and costs the locus LEDs nothing. One term buys little: ±0.109 below the locus and ±0.085 above. The error turns across the locus one way at 2,700 K and the other at 4,000 and 6,500 K, and bends most where it is warmest, so the Duv has to enter as a slope and a curve for each temperature — ten numbers a pump, not five — before both sides come back, to ±0.056 and ±0.048. That line then reads the locus LEDs at ±0.073, worse than the ±0.062 a line with no Duv in it manages once it is learned across the bin.
The bin's curve holds between its points and breaks beyond them
A camera's colour error under a white LED was read from an ambient sensor across a manufacturing bin by a line with a slope and a curve in Duv for each colour temperature, fitted at three Duv values — on the locus and 0.003 either side — and reading the LEDs beside the locus within ±0.056 and ±0.048 ΔE₀₀. A curve through three points passes through them whatever happens between them. The prediction was that 119 new LEDs built at ±0.0015 are read within ±0.06, and those at ±0.006 worse than ±0.10. Both halves hold: ±0.045 and ±0.043 inside, ±0.153 and ±0.161 outside. But the error's own mean course stays on its parabola within 0.02 out to ±0.006. The line's bend is not the error's: it is the gap the line leaves between the locus LEDs and the bin's edges, and a gap spent as a curvature quadruples at twice the distance.
A level for each temperature lets the curve be the error's
An error line with a slope and a curve in Duv for each colour temperature read white LEDs inside a manufacturing bin within ±0.045 ΔE₀₀ and LEDs at twice the bin's edge only within ±0.16, though the camera's own error stays on its parabola there. Its curve was borrowed: the line's shared terms could not place each temperature's level, and the square of the Duv absorbed the gap. The prediction was that one more number a pump, the square of the mireds, brings the locus LEDs within ±0.05 and those at ±0.006 within ±0.08. It brings the locus to ±0.025 — three times better than the locus-only line that started this run of essays — and the outer LEDs to ±0.049 and ±0.058. The curvature the line now implies at each temperature is the error's own within a fifth; what it leaves is the warm whites' scatter at the bin's edges.
Seven channels carry their noise as lightly as three
An ambient-light sensor of seven narrow channels, fitted to a room's forty-seven lights and four triphosphor siblings, reads every one of the fifty-one within a ΔEITP unit of a dim moon picture's tint, each light held out of its own fit. A fine sensor was expected to read chromaticity from small differences between neighbouring channels, with large weights of both signs that multiply every channel's noise. The prediction was that seven channels stay within a unit at a quarter of a per cent of channel noise and are no better than five at one per cent. The first half holds. The second fails completely: at one per cent seven channels still read every light's every one of a hundred readings within a unit, while five lose twenty-two lights. The premise was wrong. Seven channels' matrix barely cancels — its contributions sum with one sign — and per unit of noise it moves the reported chromaticity no more than three broad channels do.
A tilt moves every light's reading alike
An ambient-light sensor of seven narrow channels reads all fifty-one of a room's lights within a ΔEITP unit of a dim moon picture's tint and keeps them there through one per cent of independent noise on every channel, because its matrix barely cancels. A drift shared by the channels does not average out: as a photodiode warms its response tilts, rising at long wavelengths and falling at short. The prediction was that a tilt reaching ±1 per cent at the ends of the range leaves the seven-channel sensor within a unit for every light and that the twelve-channel sensor, whose Z row cancels, loses more lights than seven at every tilt. Seven channels hold to ±2 per cent, the worst light at 0.56. Twelve loses two lights at every tilt — the two it misreads with no tilt at all — and none to the tilt. Every sensor's reported chromaticity moves by about seven ten-thousandths in u′v′ per per cent of tilt, every light nearly the same way.
The red phosphor tilts the warm whites and bends nothing
An error line with a level, a slope and a curve in Duv for each colour temperature reads white LEDs at 4,000 and 6,500 K within ±0.04 ΔE₀₀ across and beyond a manufacturing bin, but warm whites at twice the bin's edge only within ±0.087. The prediction was that the red phosphor sets a warm white's bend, so that one term more a pump — the square of the Duv times the estimated red — brings them within ±0.05. It moves them by a thousandth. The red phosphor enters the slope instead: the Duv times the red takes them to ±0.046 below the locus and ±0.067 above. And the rest points upstream. The first stage reads a warm white's red phosphor squeezed toward the middle of its range, the warm misses follow that misreading times the Duv at r = 0.79, and given the red each LED was built with the warm whites come within ±0.05 — though the same substitution doubles the error at 4,000 K.
The line had already undone the squeeze
An ambient sensor's first stage reads a 2,700 K white LED's red phosphor squeezed about a third toward the middle of its range, and the camera-error line's warm misses outside the bin follow that misreading times the Duv. The prediction was that giving the first stage the Duv times its 600 nm departure and times the mireds would undo the squeeze. It changes nothing: the squeeze belongs to the temperature, and a level and a departure slope for each temperature read the warm red four times better, within ±0.035. But the eleven-term error line does not move — no warm LED by more than 0.006 — because at one temperature a squeeze is a rescaling its red and red-times-mireds terms had already undone. Only the red-slope line, whose one coefficient a pump spans three temperatures, gains: to ±0.034 below the locus, within 0.002 of the red known. And the cold whites pay for it.
Two estimates of one red each read their own end
Two first stages read a white LED's red phosphor from an ambient sensor: one squeezes the warm whites' red a third toward the middle, the other reads it four times better, and a camera-error line with a red slope prefers the first for cold whites and the second for warm. The prediction was that a line handed both — the eleven-term line on the squeezed estimate, the red slope on the faithful one — keeps each end's gain. It does, within 0.004 at the cold end. But the second estimate is not needed: the squeezed red alone, its red slope multiplied by the mireds, reads every LED within a ten-thousandth of the split line. And the two estimates' difference is worth a term of its own. Added to the line, it reads the cold whites within ±0.016 at every Duv and on the locus within a thousandth, and moves the warm whites and the middle by nothing.
A dark offset pulls every light toward one colour
An ambient sensor of seven narrow channels reads all fifty-one of a room's lights within a ΔEITP unit of a dim picture's tint through independent noise and a gain tilt, because its matrix barely cancels. A dark current is different again: it adds the same signal to every channel whatever the light, and at the dim end of a room's lighting it is a larger share of a weak channel than of a strong one. The prediction was that an offset of a quarter of a per cent of the strongest channel costs the seven-channel sensor lights, while three broad channels move least. Seven channels hold every light to two per cent. Three move furthest. And the offset does something simpler than either sensor's design suggests: it draws every light's reading straight toward the one colour the sensor reports for equal signals in every channel.
A light's distance from the equal-signal colour sets when it goes
An uncorrected dark offset draws every light an ambient sensor reads straight toward the one colour its matrix reports for equal signals in every channel, and a seven-channel sensor holds the room's fifty-one lights within a ΔEITP unit to an offset of two per cent of each light's strongest channel. The prediction was that the first light goes near three per cent and is the warmest, that each light's crossing offset falls with its distance from the equal-signal colour, and that two readings a light predict its crossing within a fifth. The warmest goes first — the 2,200 K radiator — but at 2.1 per cent. The twelve 2,700 K LEDs go together, between 3.9 and 4.4. Crossing follows distance at r = −0.93, and a straight line through each light's tints at one and two per cent predicts its crossing within 8 per cent, always early. Below half a per cent nothing moves at all.
Named alongside it
The objects these essays reach for when they reach for this one.
Camera rawWhite LEDLeast-squaresOverfittingWhite balanceCalibrationIdentifiabilityLinear modelPredictionFluorescentModelling assumptionCorrelated colour temperature