Colour Science & Perception
what the eye settles, and what it cannot
The collection

Every essay — page 46

Page 46 of 54, continuing through the fields in the same order.

What light is What the eye does Matching and measuring Difference and uniformity What the brain does What a scene does What a camera does Where the model breaks What it takes to deliver it

SeriesObserversNamed objectsRefutationsSearch

What a camera does

A camera is three functions of wavelength collapsing a spectrum onto three numbers, which is what an eye is — and they are different three functions. Everything a photograph can and cannot establish about colour follows from that, and from the fact that silicon sees three hundred nanometres past where anybody is looking.

One infrared-cut filter, crossed at four angles. The transmittance of the same interference filter for light crossing it at the centre of a frame and at three steeper angles, from 560 to 760 nm. Its half-transmission edge is at 665 nm straight on and moves to 658, 646 and 630 nm, following the edge wavelength times the square root of one minus the squared sine of the angle over the square of the stack's effective index, 1.8. The deepest reds are what it takes away, and a lamp decides how much light there was in them.

The corner of the frame has another filter

A camera's infrared-cut filter is an interference stack, and light crossing it at an angle sees its edge moved towards the blue — from 665 nm at the centre of a frame to 646 at twenty-five degrees. The corner's red channel loses a tenth of its response under daylight and a sixth under tungsten. A grey-card gain map makes the corner exactly right under the lamp it was made under; under a tungsten lamp the same grey is 2.6 colour differences wrong, under a white LED 4.2, and even under its own lamp the coloured patches are not corrected, because a moved edge is not a gain.

7 figures A black level slightly wrong, through the balance, under daylight. A grey ramp from half a per cent to seventy-two per cent reflectance, with a pedestal error of a tenth, three tenths and one per cent of white left in all three raw channels before the white balance, against the same ramp with none. The horizontal axis is the grey's reflectance, logarithmic. At a three-tenths error a two per cent grey is 2.4 colour differences off, most of it chroma, and a seventy-two per cent grey 0.21. An equal offset in the raw channels is not equal after three different gains.

A black level is multiplied by the balance

Every raw value carries a pedestal that is subtracted before anything else, and a white balance is then a different gain in each channel. Subtracting a constant and multiplying by one commute only when the constant is zero or the gains are equal. So a pedestal left three thousandths of white too high becomes 2.4 colour differences in a two per cent grey, most of it chroma, in the colour the lamp starves — and pushing that shadow four stops in editing makes it 8.9. An offset in proportion to the lamp's own white is the exception, and it is exactly grey.

7 figures Noise clipped at zero, averaged over a shadow, under tungsten. A grey ramp from black to ten per cent reflectance under tungsten, captured at three illustrative noise levels, with every negative raw reading set to zero before the readings are averaged over an area. Each line is the colour difference between that average and the noiseless grey. At high gain a half per cent grey is 1.04 off and a black frame 0.79; at very high gain the worst is 2.95, at 1.0 per cent. The same readings averaged before any clip come back exactly, at every level. The tint is gone once every channel sits several deviations above zero.

Clipped noise does not average away

Noise on a raw reading is as likely to fall below the true value as above it, which is why averaging an area removes it. A converter that sets negative readings to zero keeps the upper half and throws the lower away, and the mean of what is left is the signal plus a pedestal. With no black level error anywhere, a half per cent grey under a tungsten lamp comes out 1.04 colour differences off at high gain, 8.98 after a four-stop push — and a blur that removes every trace of the noise leaves the tint where it was.

7 figures Three corrections for the corner of a frame, each made under daylight. The mean colour difference over twenty-four coloured patches between the centre of a frame and its corner, against the angle light arrives at, after three corrections each fitted under daylight, D65 and used under it: a grey-card gain map, a correction confined to the red channel's row, and a full three-by-three matrix. All three leave the grey exact. At 25° the gain map leaves 1.86, the red row 0.93 and the matrix 0.90; at 35°, 3.81, 1.95 and 1.81. Six more numbers buy almost nothing, because the moved edge is in one channel.

A corner is corrected by one row

A grey-card gain map makes the corner of a frame exactly right on grey and leaves coloured patches 1.86 colour differences wrong at twenty-five degrees. A three-by-three matrix fitted at that position halves it — and six of its nine numbers do nothing, because the moved filter edge is in one channel. The three that matter rebuild the lost red from green and blue, they carry to another lamp better than a gain map in eleven cases of twelve, and in the twelfth, a row fitted under tungsten and used in daylight, they leave the grey 9.2 off.

7 figures Two matrices blended by colour temperature, under fourteen lamps. For each lamp, with the neutral held exact as a converter holds it: the mean colour difference over twelve test surfaces with a matrix fitted under that lamp (the short bar) and with the tungsten and daylight matrices blended at the weight its correlated colour temperature gives (the long bar). Smooth lamps on or near the locus sit within 6 per cent of their own matrix. The lamps with lines or narrow bands in them sit a median of 2.2 times theirs, from 1.38 for a broadband tube to 5.2 for a three-emitter source.

Two matrices do not reach a white LED

A camera profile's two matrices, blended by the scene's colour temperature, are as good as a matrix fitted anywhere along daylight. Under a white LED or a fluorescent tube the same blend leaves colours twice as far off as a matrix fitted under that lamp, and no weight inside the profile's range repairs it. What decides it is not how far the lamp sits from the Planckian locus — a triphosphor tube sits nearly on it and fares worst — but whether its spectrum has lines in it, which a white balance reading cannot see.

6 figures Four ways to fill in a clipped highlight: a glossy surface with a reflection of the lamp, under tungsten. Twenty-four chart surfaces under tungsten, as a glossy surface with a reflection of the lamp, taken up a ramp until their raw channels reach the sensor's ceiling. Each line is the mean colour difference, at equal lightness, between the true colour and what one response to the clipped reading makes of it: clipping to white, carrying the clipped values through, filling the clipped channel from the surface's own ratio, and filling it from the surface's colour plus the lamp's. At 0.4, where most surfaces have one channel clipped, the four leave 5.87, 4.37, 5.56, 0.00; at 2, 2.90, 26.97, 8.18, 8.18.

Filling in a highlight is a claim about the surface

A converter that rebuilds a clipped channel has to say what the highlight was. A matt surface over-exposed keeps its own colour, and filling the lost channel from that colour is exact. A glossy highlight is the surface's colour plus a reflection of the lamp, and the same fill leaves it 7.6 colour differences too colourful — while a fill that solves for surface and lamp is exact. Neither works once two channels are clipped, and a tungsten lamp keeps a highlight in the one-channel band more than twice as long as daylight does.

8 figures The confusion matrix, and which corner the common lamps are in. Twelve fixtures sorted two ways. Down the page is what their spectra are; across is what flicker says. The two corners on the diagonal are 7 fixtures the classifier gets right. The 3 missed are structured lamps that do not flicker — a white LED and a warm LED on constant drivers, and a three-emitter fixture — and those are the lamps most modern interiors are lit by. The 2 false alarms are smooth lamps that do flicker: a halogen lamp on mains and a tinted radiator, both of which a photograph of a room is quite likely to contain.

Flicker sorts lamps the wrong way

A camera cannot see a spectral line in its own white, so the essay on the two-matrix profile named the classifiers a device might have instead, and the first of them was flicker. Flicker is measurable, and it measures the wrong thing. It sorts lamps by how their power is delivered while the matrix needs them sorted by how their spectrum is shaped, and the two are independent: a white LED on a constant driver is perfectly steady and strongly structured, and it is what most indoor photographs are lit by.

7 figures The statistic a converter would read, under each model. The log ratio of the two channels that are still open, against how bright the surface is, under the two models of what a highlight is. A matt surface over-exposed keeps its own ratio exactly — the line is flat, and it must be, because scaling every channel by the same amount leaves a ratio alone. A glossy surface carries a reflection of the lamp on top of its body colour, so its ratio slides towards the lamp's as the reflection strengthens: -0.086 of a log unit between a quarter of full scale and nine tenths. That slide is the whole of the evidence a converter has for choosing between them.

The converter can choose except where it matters

A converter rebuilding a clipped highlight has to say whether the surface was matt and over-exposed or glossy and carrying a reflection, and the evidence is whether its raw chromaticity slides towards the lamp's on the way up. Read through the sensor's own noise, the slide is clear on most of the chart from the few tens of pixels a specular highlight holds — and on the surfaces where it takes half a frame, confusing the two models costs more than the median. A ratio cannot see a common scale, and an exposure supplies one.

7 figures The four places a clamp could sit are two pipelines. Every pair of clamp positions, with the largest difference their delivered values reach over a grid of raw inputs that includes negative ones. Two of the six are exactly zero: a clamp at zero commutes with the white balance, which is a positive scale applied channel by channel, and with the tone curve, which is monotone and fixes zero. It does not commute with the colour matrix, which is the only step that mixes the channels — so the four positions collapse to two, before the matrix and after it, and no measurement of any scene can say more than which side a converter is on.

Four places to clamp are two pipelines

The essay on clipped noise ended on a procedure: a black frame and a dim grey card at a high amplification would place each raw converter's clamp. A procedure is a claim that a measurement identifies something, and this one identifies less than it looks. A clamp at zero commutes with the white balance and with the tone curve and not with the colour matrix, so the four positions are two pipelines — and the measurement separates them, on a card at half a per cent of white rather than on the black frame.

6 figures The same chart in two studios and in the room that holds both lamps. Each of the chart's twenty-four surfaces, with the pixels its slide needs at two standard deviations at high gain, on a logarithmic scale. Lit by a 3000 K radiator alone the worst surface needs 16,194; lit by daylight at 6500 K alone, 139,574. Lit by both, 50 and 50 per cent of the light, with a highlight of each lamp read together, the worst needs 10 and the median 6. The studios' hard surfaces sit at different places on the chart, and no surface is hard in both.

Two lamps decide what one lamp could not

Under one lamp a handful of surfaces cannot tell a glossy highlight from a matt over-exposure without most of a frame, because there the two differ by a scale a ratio cannot see. In a room lit evenly by a 3000 K lamp and daylight, every surface on the chart is decided from ten pixels. A body can sit at one lamp's white, not at two, and the rescue holds only while the second lamp carries a fifth of the light and its white sits far enough from the first.

7 figures How far each lamp's sensor reading is from what the camera predicts, with RGB + clear. Fourteen lamps, six smooth and eight structured, each scored by how far an ambient-light sensor with red, green, blue and clear channels reads from what the camera's white predicts through a map fitted on smooth radiators and daylights. The smooth lamps score at most 0.057 and the structured at least 0.133; the dashed line is the threshold at the gap's geometric middle, 0.087. The lights the map was fitted on score at most 0.0113. Every lamp falls on its own side of the line.

Two sensors disagree about deep red, not lines

A phone's ambient-light sensor and its camera read the same lamp differently, and the difference sorts fourteen lamps into smooth and structured without a single mistake — where flicker made five. It is not reading their lines. Almost all of it comes from the sensor's clear channel collecting deep red the camera's infrared cut throws away, so daylight with its far red trimmed is called structured and a white LED with a far-red emitter is called smooth.

7 figures The calibration error a narrow fourth channel survives, by where it is placed. For a red, green and blue ambient sensor with one more channel 10 nm wide, centred from 450 to 640 nm: the first calibration error at which some lamp of fourteen is misclassified. Dashed: the channel pooled into the whole residual, as the ambient sensor's disagreement was read; its best is ±1.50 per cent. Solid: the channel read on its own, as its departure from what the camera's white predicts for it; its best is ±4.0 per cent, at 450 nm. Crosses mark centres where the channel read alone does not separate the lamps at all. The two horizontal lines are the red, green and blue design (±1 per cent) and the design with a clear channel (±4 per cent).

A narrow channel has to be read on its own

An ambient sensor with a clear channel tells structured lamps from smooth ones by reading deep red, and one without it reads structure but breaks at one per cent of calibration error. A narrow fourth channel between the camera's peaks was proposed to have both. Pooled into the sensor's disagreement with the camera, it reads structure and breaks at one and a quarter per cent — no better than the design it was meant to rescue. Read on its own, as its departure from what the camera predicts for it, a channel at 450 nm holds to four per cent, the clear channel's figure, and far red does not move it.

6 figures