Two lamps decide what one lamp could not

The converter can choose except where it matters put a number on a decision every raw converter makes when a channel clips. A matt surface over-exposed keeps its own ratio between the channels that are still open; a glossy one carries a reflection of the lamp, so its ratio slides towards the lamp’s as the reflection strengthens. Read through the sensor’s noise, the slide was clear from a few tens of pixels on most of the chart. On a handful of surfaces it needed most of a frame, and those were not the cheap surfaces to get wrong.

The reason was a confounding rather than a shortage of precision. On the hard surfaces the two models differ in the open pair by a common scale, a ratio discards a common scale by construction, and an unknown exposure supplies one. That essay ended on a prediction about the ordinary indoor photograph, which is lit by more than one lamp: two lamps put two different whites into the reflections, the surfaces hard for one should differ from the surfaces hard for the other, and a room should have fewer surfaces hard for both than a studio has for either.

The prediction holds, and more strongly than it was stated.

Every surface is decided, on two conditions #

In a room lit half by a 3000 K radiator and half by daylight, reading both lamps’ highlights decides every surface on the chart from at most 10 pixels, with a median of 6. The same chart under the radiator alone had a surface needing 16,194, and under daylight alone one needing 139,574.

• Each highlight still has hard surfaces: ten for the radiator’s reflection and two for daylight’s, at a small highlight’s budget of sixty pixels. None is hard for both.
• A surface is hard for a lamp where its body sits at that lamp’s white in the two open channels, and a body sits at one white at a time.
• The weaker lamp must carry about a fifth of the light. From a fifth to four fifths the worst surface needs at most 27 pixels; with the radiator at 2 per cent it needs 1,618, and with daylight at 2 per cent, 301.
• The two whites must sit far enough apart in the open pair. Of fifteen rooms built from six lamps, the closest pair of whites leaves a surface needing 5,358 pixels at a very high gain and the farthest needs 26.
• Whether a lamp’s spectrum is smooth or full of lines does not enter. Only where its white sits in two channels does.

The same chart, the same sensor, a second lamp #

The model is the one the earlier essay built, changed in one place. The camera is the modelled silicon sensor; the surfaces are the chart’s twenty-four, spanning hue at one chroma; a matt reading is the body scaled, and a glossy reading adds a share of the lamp’s raw white to a fixed body term, both normalised to equal exposure. The statistic is the log ratio of the two open channels between a quarter of full scale and nine tenths, and the noise is the sensor’s photon and read noise at the counts those pixels carry.

What changes is the room. The body of every surface is lit by the mixture, so its raw colour is neither lamp’s. A curved glossy object under two lamps carries two highlights — a window in one place and a ceiling lamp in another — each a reflection of one lamp, in proportion to that lamp’s share of the light. Each gives its own slide towards its own white, with its own noise. A converter that has located both reads both, and because their noise is independent their signal-to-noise ratios add in quadrature.

Every one of the six lamps used here clips the red channel first, so the open pair is green and blue in every room. That simplifies the geometry without favouring the result: the two lamps’ reflections are read in the same two channels and differ only in the direction they push.

The figure above is the comparison the prediction asked for. The two studios have their worst surfaces at 16,194 and 139,574 pixels. In the room holding both lamps the dark curve never leaves single figures.

Hard surfaces move rather than disappear #

The first part of the prediction was that each lamp would still have its own hard surfaces, and it does.

Read alone, the radiator’s highlight leaves ten surfaces above the sixty-pixel budget, from surface 9 to surface 18, and its worst, surface 15, needs 9,110 pixels. Daylight’s highlight leaves two, surfaces 4 and 5, the worse needing 447. The two sets do not share a surface.

So a converter reading only one of the two highlights in this room is no better off than it was in a studio: it has a blind patch on the chart, and a different patch depending on which highlight it chose. The dark curve is what happens when it reads both. Where the radiator’s reflection says nothing, daylight’s is clear, and the reverse; the combined need is set by whichever highlight is informative, and on this chart one of them always is.

That is what the earlier essay meant by a second observation with a different geometry. The first observation was confounded with a scale. The second is confounded with a different scale, on a different surface, and two confoundings that do not coincide identify the model between them.

A body sits at one white at a time #

Why the two sets are disjoint is visible in the open pair itself.

A reflection adds a lamp’s raw white to the body. In the open pair that moves the body’s ratio towards the white’s ratio, by an amount proportional to how far apart the two already are. A body whose green-over-blue ratio already equals the lamp’s does not move at all, however strong the reflection, and on that surface a glossy highlight and a matt over-exposure differ only by how bright they are.

The radiator’s white sits at 1.16 on this scale and daylight’s at 0.46. The chart’s bodies, lit by the mixture, trace a curve round the hue circle that crosses each line in places. The radiator’s hardest surface, 15, sits at 1.17; daylight’s hardest, surface 4, at 0.41. Each highlight’s difficulty climbs as the body approaches its line, which is the same mechanism measured under one lamp: across the radiator studio’s twenty-four surfaces the rank correlation between the body’s distance from the lamp’s white and the log of the pixels needed is −0.94, and the hardest surface is the nearest.

A body has one ratio. It can sit on the radiator’s line or on daylight’s, never on both, so no surface can be blind to both reflections at once — provided the two lines are not on top of each other, which is the second condition and the one the six-lamp table measures.

The mixture moves the body curve as well. A surface that crossed the radiator’s white in the radiator’s own studio is lit by bluer light in the room, and its ratio drops. So the hard surfaces in the room are not the studio’s hard surfaces: surface 7 needed 16,194 pixels under the radiator alone and needs 9 here. Which surfaces are hard is a property of the room, as the earlier essay found it was a property of the lamp, and a table of hard surfaces made in a studio says nothing about a living room.

At a very high gain, two weak readings add #

The quadrature sum does something a set overlap cannot show, and the noisiest sensor state makes it visible.

At a very high gain the radiator’s highlight leaves seventeen surfaces over budget and daylight’s eight, and three surfaces — 21, 22 and 23 — are over it for both, needing between 62 and 86 pixels from either highlight alone. Counted as sets, the prediction that no surface is hard for both fails here.

Read together, those three need 36 pixels each, and the worst surface on the chart needs 38. Neither highlight is blind on those surfaces; each is merely short of the threshold, and two independent readings that are each a little short are, together, comfortably over it. The overlap that matters is not of surfaces above a budget but of surfaces where both slides are near zero, and there are none: the three surfaces over budget for both sit between the two white lines, where each reflection moves them modestly and neither moves them not at all.

That distinction is worth carrying beyond this chart. A converter that classified each highlight independently and then voted would report three surfaces undecided. A converter that combined the evidence before deciding reports none.

The second lamp must carry enough of the light #

The even room is the favourable case. A living room at night with one bulb and a television, or a studio with a window it forgot to cover, is lit mostly by one lamp.

From a fifth of the light to four fifths, the worst surface needs at most 27 pixels. Outside that band the weaker lamp’s reflection is proportionally fainter, its slide shrinks towards its noise, and the stronger lamp’s blind patch comes back. With the radiator carrying a tenth of the light the worst surface needs 83 pixels; a twentieth, 291; a fiftieth, 1,618, with four surfaces undecided. At the other end, with daylight carrying a tenth the worst needs 23, a twentieth 62, and a fiftieth 301, with seven undecided.

The two ends are not mirror images, and the reason is the studios they approach. A room lit almost entirely by daylight inherits daylight’s blind spot, which in its own studio needed 139,574 pixels; a room lit almost entirely by the radiator inherits a blind spot that needed 16,194. The faint second lamp is rescuing a deeper hole at one end than at the other, so it has to carry more of the light to do it.

So the rescue is a property of rooms where the second lamp is a real light source rather than a stray one. A window at a fifth of the ceiling lamp’s contribution is enough. A standby light across the room is not, and the photograph is back in the studio.

The two whites must sit far enough apart #

The second condition is that the two lamps disagree about green and blue, and the six lamps available make fifteen rooms to test it on.

The further apart the two whites sit, the fewer pixels the worst surface needs, at a rank correlation of −0.98 across the fifteen rooms. The closest pair is the 3000 K radiator with a warm white LED, whose whites sit 0.026 apart: a room lit by both is, in the open pair, a room lit by one lamp, and its worst surface needs 5,358 pixels with eleven undecided. The farthest is daylight with the same warm LED, 0.72 apart, needing 26.

At a very high gain only three of the fifteen rooms decide every surface, and all three pair daylight with a lamp whose white sits at least half a unit from it: the radiator and the two LEDs. At a high gain ten do, and the five that fail are the five closest pairs, all within 0.18 of each other. Two daylights do not rescue each other, and neither do two warm lamps. The ordinary case the prediction had in mind — a warm interior lamp and a cool window — is among the best cases the table contains.

Nothing in the figure distinguishes smooth spectra from structured ones. A triphosphor tube with daylight, 0.28 apart, needs 143 pixels, and 4000 K daylight with daylight, two smooth spectra 0.40 apart, needs 83: both sit on the same trend as every other room, placed by their separation alone. Two matrices do not reach a white LED found that a lamp’s spectral lines decide how well a colour matrix carries to it, and flicker sorts lamps the wrong way that a camera cannot easily see them. This decision is indifferent to them: the statistic reads the lamp through three channel sums, and two lamps with the same sums are the same lamp here, lines or none.

Light helps, and the room helps more #

The earlier essay’s sensor sweep showed that a brighter exposure moves the whole curve down and leaves the hard surfaces hard. The room does something different in kind.

At a low gain the two studios’ worst surfaces need 1,239 and 10,961 pixels, and the room’s needs one. At a high gain, 16,194, 139,574 and 10; at a very high gain, 69,148, 573,931 and 38. Gain multiplies every need by roughly the same factor, as it did in one studio. The room divides the worst need by about three orders of magnitude at every gain, because it removes the zero in the denominator rather than shrinking the noise in the numerator.

That is the difference between precision and identifiability, measured. An image does not determine the light is the collection’s standing example of the second kind of problem, where more of the same data cannot help. Here a different kind of data can: a second highlight is not more pixels of the first, it is a second equation with a different coefficient.

One caution about the studio numbers themselves. The earlier essay’s lamp was CIE illuminant A, at 2856 K, and its worst surface needed 413,718 pixels. The 3000 K radiator here puts the same chart’s worst at 16,194. A hundred and fifty kelvin moved the worst case by a factor of twenty-five, because the worst case is a body sitting almost exactly on the lamp’s line and a small change in the line moves how nearly it sits. The median barely moved. A worst case that sensitive to the lamp is a reason not to quote it as a property of the method.

How the room was built #

The six lamps are daylight at 6500 K, a 3000 K blackbody radiator, 4000 K daylight, a neutral and a warm phosphor-converted white LED, and a triphosphor fluorescent tube, each scaled so its raw white’s largest channel is one. A room at a stated share is the weighted sum of two of those spectra, and a surface’s body is the sensor’s response to that sum times the surface’s reflectance, normalised to its own largest channel.

A highlight of a lamp is the body at 0.55 plus a reflection of 1.4 times that lamp’s raw white, times the lamp’s share of the light, times the reflection’s strength at the reading’s exposure — the construction the one-lamp census used, with the share added so that a faint lamp gives a faint reflection. Each highlight’s slide is read between a quarter of full scale and nine tenths, with the sensor’s noise at the counts the two open channels carry. A highlight’s one-pixel signal-to-noise ratio is half the difference between the glossy and matt slides over the larger of their standard deviations; two highlights combine as the root of the sum of their squares; and the pixels needed are the square of two over that ratio.

The budget of sixty pixels is the earlier essay’s small specular highlight, and a surface counts as undecided when the combined need exceeds it.

What this leaves out #

Both highlights have to be on the surface and found. A flat glossy surface under two lamps shows at most one reflection from any one viewpoint, and a small curved one may show both in a dozen pixels. The combined need is a need per highlight, so a surface showing only the radiator’s reflection is back in the radiator’s studio. A grey edge arrives coloured is a reminder that a highlight’s pixels also come through the mosaic, which correlates the noise in the two open channels and makes every figure here somewhat optimistic.

The converter has to know which highlight is which lamp. The model assumes the two lamps’ whites are known, as the highlight is the white balance assumes one is. A mixed-light scene is exactly the case where a single white balance is wrong somewhere in the frame, and estimating two whites from one photograph is a harder problem than the decision it would feed.

The chart is one family of surfaces at one chroma, and the two models are the dichromatic pair. A metal’s highlight carries its own colour, a surface that is not a multiplication takes that model apart, and neither lamp’s reflection points the way this model says on a metal.

And the rendering of the rest of the picture under mixed light — the colour matrix, the white balance, the tone curve applied to a scene with two whites in it — is untouched. The question here is only whether a clipped highlight can be classified, and the answer to it does not make the room easier to photograph in any other respect.

Still open: whether one highlight and its surroundings are enough #

The rescue needed two reflections of different lamps on one surface. The commonest case in a real room is one highlight on a surface whose body is lit by both lamps, and that case carries partial information of its own.

A body lit by two lamps is not uniformly lit. The side facing the window is bluer and the side facing the lamp warmer, so the unclipped body pixels around a single highlight already trace a line between two body colours. A matt over-exposure must lie on that line, scaled; a glossy highlight leaves it towards one lamp’s white. Whether the gradient across a body supplies the second equation that the second highlight supplied here — and how many body pixels it takes when the lamps’ directions are close together — is a calculation on a surface with a shape, which this model does not have.

The prediction worth recording is that it will work where the two lamps light the surface from very different directions and fail where they do not, so that the second condition found here, distance between the whites, acquires a geometric partner: distance between the lamps.

A confounding is broken by a second equation, not a better first one #

The habit is about what to do when an estimator’s failure is a confounding.

The first response to an estimator that needs half a frame is to make it more efficient — better noise handling, a smarter threshold, a brighter exposure. Each of those divides the need by a constant. None of them touches a surface where the thing being measured is exactly zero, and the need there is dominated by how close to zero it is, which no amount of precision changes.

The move is to ask what second observation would have a different blind spot, and whether the scene already contains one. Here it did: the ordinary room has two lamps, and their reflections are blind in different places. The measurement then has two questions rather than one — whether the blind spots overlap, and whether the second observation is strong enough to count — and both had answers the one-lamp model could not have given.

The failure mode is to count an overlap of thresholds. At a very high gain three surfaces were over budget for both highlights and every one of them was decided when the evidence was combined, because being short of a threshold and being blind are different states. A mean is not a worst case, and a vote is not a sum; an estimator that decides each observation separately throws away exactly the information a second observation was there to add.