Silicon sees past the visible

Every plot of a camera’s spectral sensitivities looks like three overlapping humps between four hundred and seven hundred nanometres, tapering politely to nothing at both ends. It is a reassuring picture and it is drawn after the component whose job is to make it true.

The sensor underneath responds over more than twice that range, and over the last third of it the three channels are indistinguishable from one another.

The other two channels are the same stack with a different dye in it, and the part past the visible is the same part.

The third dye is a third shape in the visible and the same tail beyond it, which is what makes the three numbers converge rather than merely grow.

Two more channels with the filter fitted say that the cut is doing the same work in all three, which is why it can be one piece of glass.

The claim #

Silicon responds from roughly 350 to 1107 nanometres, and the organic dyes of a colour-filter array are all three transparent above about 800. Over the top three hundred nanometres of the sensor’s range there is therefore no colour information at all — three channels measuring one quantity.

Both halves are worth stating separately because they have different kinds of cause. The silicon range is set by solid-state physics and is not negotiable. The dye transparency is set by chemistry and is negotiable in principle and not in practice.

Where the two ends of silicon come from #

The long end is the band gap and it is exact. Silicon’s gap is 1.12 electronvolts at room temperature. A photon carrying less than that cannot promote an electron from the valence band to the conduction band, so it is not absorbed and produces no signal. Converting energy to wavelength,

$$\lambda_{\max} = \frac{hc}{E_g} = \frac{1240\ \text{eV·nm}}{1.12\ \text{eV}} \approx 1107\ \text{nm}$$

which is why the figures in this field stop at 1100 and not at some rounder number. The cut-off is not perfectly sharp — the transition in silicon is indirect, so absorption near the edge needs a phonon as well and falls off over a few tens of nanometres rather than instantly — but it is close enough to a wall that nothing past 1150 matters.

The short end is absorption depth and it is a device property rather than a material one. Blue photons are absorbed within a few tens of nanometres of the surface. That is very shallow, it is above the depletion region in most sensor designs, and carriers generated there recombine at the surface before anything collects them. So a silicon photodiode’s response falls away in the blue not because silicon fails to absorb blue light but because it absorbs it in the wrong place. Back-illuminated sensors exist precisely to move the collecting structure to the side the light arrives from, and they recover a good deal of that blue response.

Why the dyes give up #

A colour filter array is three organic pigments, patterned in a mosaic a few micrometres across and a micrometre thick, deposited photolithographically and expected to survive years of ultraviolet and heat without shifting.

Organic pigments absorb by electronic transitions between molecular orbitals, and the energies of those transitions are — almost by definition of what makes something coloured — in the visible. A dye that absorbs strongly at 900 nanometres would need a transition around 1.4 electronvolts, which requires a very extended conjugated system, and such molecules exist and are notoriously unstable. Nothing anyone would put on a sensor for ten years absorbs meaningfully in the near infrared.

So past about 800 nanometres each of the three dyes is doing what a piece of clear plastic does, and the measurement below is what that means numerically.

Above 850 nanometres the three transmittances agree to within 0.03 per cent of each other. Not approximately equal — equal to a level where the difference is far below any manufacturing tolerance, and where the ratios between channels, which is all colour ever is, carry no information whatsoever.

What that does to three numbers #

The consequence is not that infrared adds a little noise. It is that infrared adds the same large number to all three channels, and adding a constant to a triple is the operation that most efficiently destroys chromatic information.

Consider a surface whose channel readings under some illuminant are in the ratio 3 : 2 : 1. Add an infrared contribution of 10 to each and they become 13 : 12 : 11, which is very nearly neutral. The three integrals are still perfectly well-defined measurements; they are simply measurements of a quantity dominated by something that is the same in all three.

Under a 2200 K lamp — a dimmed incandescent, a candle, a sunset — the invisible fraction of an unfiltered sensor’s signal is above nine tenths. Under a 9000 K north sky it is far lower and still large. Both numbers depend on the surface as well, which is the subject of the next essay in this field and is the half of the argument that is easiest to leave out.

Why the lamp matters as much as the sensor #

The infrared share is not a property of the camera. It is a property of the product of the lamp’s spectrum, the surface’s reflectance and the sensor’s response, and the lamp is the largest of the three.

A tungsten filament at 2856 K has its Planck peak, by Wien’s displacement law, at

$$\lambda_{\text{peak}} = \frac{2.898 \times 10^6\ \text{nm·K}}{2856\ \text{K}} \approx 1015\ \text{nm}$$

which is not merely outside the visible band but outside it by more than the visible band is wide. A tungsten lamp is an infrared source that happens to leak some visible light — it is the reason its luminous efficacy is so poor — and pointing an unfiltered silicon sensor at a tungsten-lit scene is pointing it mostly at radiation nobody can see.

A daylight source is a very different case. Its peak is inside or near the visible band, its infrared tail is real but proportionally much smaller, and the same uncut sensor keeps substantially more colour separation under daylight than under tungsten. That difference is asserted rather than described: the library checks it, and a change that made the two agree would stop the build.

What it looks like when it goes wrong #

The failure has a characteristic appearance, and it is worth describing because it is the thing anybody who has removed a camera’s filter has seen.

Black synthetic fabric goes pale. Foliage goes white or pink. Skin goes waxy and loses its variation. Everything drifts towards a washed-out magenta — magenta specifically, because the red and blue dyes have slightly more residual leak in the near infrared than the green does in most designs, so the common-mode term is not quite common and the residue is red plus blue.

The measurement behind that description is the channel spread: how far the three raw values sit from their own mean, as a fraction of it. With the filter fitted, a set of saturated surfaces under tungsten gives a mean spread of 1.045. With it removed, 0.119. That is a factor of 8.8, and it is the honest size of what the filter is doing.

Two things are worth taking from that figure now and one is deferred. The measurable thing is that the channel spread collapses as the tail rises, so the damage is a joint property of the sensor and the surface. The deferred thing is which tail is realistic, and the answer is nearer the top of that range than the bottom — which is the next essay.

The one place the response is used deliberately #

Silicon’s infrared range is a nuisance in a colour camera and is the entire point of several other devices, which is a useful check on reading this essay as “infrared is a problem”.

Every television remote control emits at 940 nanometres, chosen because silicon detects it efficiently and nobody can see it. A phone camera with a weak filter shows the emitter flashing, which is the standard trick for testing a remote and is a direct demonstration of everything above. Autofocus assist lamps, proximity sensors, pulse oximeters and the structured-light projectors in face-recognition systems all live in the same band for the same reason.

Astronomical and scientific imaging removes the filter on purpose. A silicon CCD used for photometry wants every photon it can get and has no interest in matching a human observer; its “colour” comes from a filter wheel with narrow, named passbands, and the response beyond 780 nanometres is a legitimate measurement band rather than contamination.

The difference between the two cases is not the sensor. It is whether the instrument is trying to agree with an eye. That is worth stating plainly because it is the boundary this whole field runs along: a camera is an instrument that has been asked to impersonate an observer, and almost every awkward feature of colour imaging is a consequence of that request rather than of any limitation of the hardware.

Where the model stops #

These curves are a model and not a datasheet. The silicon response has the right shape for the right reasons and the dyes are Gaussians with a stated leak; a real sensor is measured with a monochromator and differs from this in detail. What survives the difference is the structure — a band-gap wall, a blue roll-off from absorption depth, and three dyes converging in the near infrared — because all three are consequences of the physics rather than of the model’s parameters.

Nothing here is about thermal infrared. The near infrared this essay is about is 780 to 1100 nanometres, which is reflected light: it comes from the lamp, bounces off the surface, and would be visible if human pigments happened to absorb there. It is not thermal emission. A room-temperature object radiates around 10 micrometres, nine times further out, where silicon is entirely transparent and a completely different detector technology is needed. A camera with its filter removed is not a thermal camera and cannot see anything in the dark that is not being illuminated.

And the mosaic has been ignored. Each photosite has one dye over it, so the “three channels” of this essay are three interleaved sub-images rather than three measurements at one point — which has its own consequence and is independent of everything above.

The generalisation #

The transferable form of this is about where an instrument’s sensitivity ends relative to where its selectivity ends, and the two are almost never the same place.

An instrument that responds over a wider range than it discriminates over has a common-mode signal, and a common-mode signal is the most damaging kind. Random noise averages down; a systematic offset in one channel can be calibrated out; but a large signal shared equally by every channel of a ratio-based measurement attacks exactly the quantity being measured, and it does so in a way that looks like a real reading rather than like an error.

This is a familiar shape elsewhere on this site. Veiling glare in a spectrophotometer does the same thing: light from outside the passband arriving at the detector, adding a common term, flattening the measured reflectance towards a constant. Stray light in the eye does something related. In every case the repair is a filter rather than a computation, because the information needed to compute the correction was destroyed by the same mechanism that requires it.

The design lesson generalises further: an instrument’s out-of-band response is part of its specification, and it is the part that gets omitted. Every sensitivity plot in this essay’s subject is drawn after the filter, which is convenient, honest about what the shipped camera does, and hides the fact that the filter is a component with a tolerance, an angular dependence and a failure mode.

The comparison the eye makes unnecessary #

It is easy to read all of this as a defect of silicon, and the comparison that shows it is not is the eye’s own long-wavelength limit.

Human vision stops around 780 nanometres for the same kind of reason silicon stops at 1107: the pigment’s absorption falls away, and a photon that is not absorbed does nothing. The L cone’s peak is near 564 nanometres and its sensitivity at 780 is down by something like five orders of magnitude. The eye also has a short-wavelength limit that is not the pigment’s — the lens absorbs the ultraviolet, which is why people whose lenses have been removed report seeing into it.

So both detectors have a window, and both windows are set by absorption edges. The difference is only that silicon’s happens to be wider, and — this is the part that matters — the eye has no equivalent of the dye problem, because its three pigments differ from one another right across its window. There is nowhere in the visible band where the L, M and S responses converge to a common curve.

That is a real structural advantage of a pigment-based detector over a filter-plus-broadband-detector one, and it is not usually stated. In the eye, selectivity and sensitivity end at the same place because they are the same mechanism. In a camera they are two separate components, and the gap between where one ends and the other does is the gap this essay is about.

The peak is robust and the percentage is not #

It peaks at 49.6 per cent near 6950 K is the visible-fraction figure’s summary, and the two halves of it behave very differently under the one choice it does not state — where the visible band ends.

Recomputing the maximum of a blackbody’s visible share over temperature, for four band definitions:

The temperature moves by 130 kelvin across all four and the percentage moves by sixteen points. So near 6,950 K is a statement about physics and 49.6 per cent is a statement about a convention, and the second needs its band quoted where the first does not.

That is worth having because the same figure carries the sibling essay’s claim that no thermal radiator gets half its power into the visible band. On this collection’s own grid the maximum is 49.0 per cent and the claim clears by a point; widen the band to the colour-matching functions’ full support and it fails outright at 55.5. The half is a property of the 380-to-780 window, and a reader taking the claim to a different convention would find it reversed.

The robust part of the figure is the one worth carrying anyway: the best a thermal source can do is near 6,900 K, which is close to daylight and far from any lamp anybody builds, so every thermal source in use is on the steep side of its own optimum.

What the two ends are worth, checked #

Two of the essay’s supporting numbers can be computed from outside its own machinery.

The tungsten case. For a 2200 K blackbody, 79.2 per cent of the power between 380 and 1100 nanometres lies past 780. Weighting for the fact that the three channels coincide out there and divide the visible band between them — which triples the infrared’s contribution to the summed signal — gives an invisible share of about 92 per cent. Above nine tenths is right. The same arithmetic gives 85 per cent at 2856 K against the 88.6 the neighbouring essay’s collapse factor implies, and 41 per cent at 9000 K, which is far lower and still large.

The cone’s long edge. On the standard pigment template at a peak of 564 nanometres, the L cone’s sensitivity at 780 is 1.8 × 10⁻⁵ of its maximum — four and seven tenths orders down, reaching a full five orders at about 800. Something like five orders of magnitude is right and slightly conservative.

The two absorption edges are not the same shape #

Both detectors have a window, and both windows are set by absorption edges. The difference is only that silicon’s happens to be wider — and the shapes are different in a way that decides which of the two numbers is a convention.

The pigment’s fall is an exponential tail. From its peak it is down 2.3 orders by 700 nanometres, 3.9 by 750 and 4.7 by 780, so the last eighty nanometres of the visible band cost two and a half orders of magnitude and the curve never actually stops — it just becomes unmeasurable.

Silicon’s fall is a wall. The response holds near its full value to within a few tens of nanometres of 1107 and then ends, because an indirect band gap softens the edge over that scale and no further.

So 1107 is a number and 780 is a convention. The band gap is a physical constant read off an energy; the long end of the visible band is wherever somebody decided the tail had become negligible, and the sixteen-point spread in the table above is exactly the cost of that decision. A camera’s window has a hard edge nobody chose and the eye’s has a soft edge somebody did, and the essay’s own grid sits at one particular reading of the second.

That also explains the asymmetry in how the two limits are usually stated. Nobody argues about where silicon stops. The argument about where vision stops is a hundred years old, is about a threshold rather than a mechanism, and is why the tables stop in three different places depending on which body published them.

Who noticed, and when #

That silicon responds into the near infrared was known from the beginning of solid-state photodetection in the nineteen-forties and fifties; the band gap was one of the first things measured about the material. Nobody discovered it in a camera.

What is interesting historically is the reverse case. Photographic emulsions are naturally sensitive only to blue and ultraviolet, and every extension beyond that had to be added: Hermann Vogel found in 1873 that a dye adsorbed onto silver halide extended sensitivity to the wavelengths the dye absorbed, which made orthochromatic and then panchromatic film possible, and infrared-sensitive plates followed in the nineteen-thirties. Film had to be pushed towards the red one dye at a time.

Silicon arrived with the opposite problem. It sees too far, and the entire history of colour sensor design at the long end is subtraction rather than addition — which is the subject of the filter that makes colour possible.

Where the ladder goes next #

Downward, this rung sits on a camera is a fourth observer and on a spectrum is not a colour, which is where the difference between a function of wavelength and three numbers is set out.

Upward, two rungs follow directly. The filter that makes colour possible measures what removing this signal is worth and what it costs. Most things are pale in the infrared supplies the half of the argument this essay has held back, which is that surfaces are not dark out there either — and that the two facts multiply.