The cost of a steep notch repeats every step

The line can be in the sample put a notch into a reflectance and found that a five-nanometre grid mis-sums it under any light once the notch is narrower than a couple of steps, and handles it easily once the notch is wider. The notch it used was Gaussian, and it named that as the place the measurement stopped: a real interference filter has a flat bottom and steep sides, which it expected to make the sampling worse at a given width. Worse turns out to be the wrong word. The sampling behaves differently in kind.

A width that is a whole number of steps #

A steep-sided notch costs almost nothing on a five-nanometre grid when its width is a whole number of steps and up to several colour differences when it is not, however wide it is — and the size of the oscillation is set by how quickly the edges rise, not by the width.

• Under a 6500 K thermal source, with edges rising in 0.44 nm, the notch costs 0.02 at 10 nm wide, 1.99 at 12.5 and 0.03 at 20. The pattern repeats every five nanometres from two to thirty.
• It does not fade with width. The worst width costs 3.94 near four nanometres, 2.89 near fourteen and 2.25 near twenty-four, where a Gaussian notch costs 0.01 from eight nanometres on.
• Soften the edges and the pattern goes. With edges rising in 2.2 nm the worst in a period is 1.33; rising in 6.6 nm, 0.08; at a whole number of steps the cost is about 0.02 whatever the edge.
• Under a fluorescent tube a steep notch wider than about twelve nanometres moves by six to nine colour differences as the grid slides, because its edge now crosses a mercury line.

Two edges and a grid #

A flat-bottomed notch is a reflectance that runs level, drops over an edge, runs level at the bottom and climbs back over a second edge. A sum on a grid evaluates it only at the grid’s points, so the sum sees level stretches exactly and sees each edge as a jump that happened somewhere between two samples.

Each sample stands for a cell five nanometres wide. A sample that falls in the notch’s floor counts its whole cell as notch, and a sample just outside counts its whole cell as not. So each edge adds or removes whatever fraction of a cell lies between it and the nearest sample, and the error an edge makes depends only on where in its cell it falls. For the notch above, a grid starting at 380 nm lands two samples on the floor and a grid starting half a step later lands three — the sum sees a notch ten or fifteen nanometres wide, and the true notch is twelve and a half.

The integral an edge’s error disturbs is the notch’s area weighted by the light and the observer, and a missing half-cell of a notch 0.66 deep at green wavelengths is a visible difference. Where the grid starts found the same kind of fault with a spectral line: the answer depends on where a sample happens to land against a feature narrower than a cell, and no averaging over the rest of the spectrum removes it.

Why whole steps cancel #

The two edges are the notch’s width apart. If that width is a whole number of steps, the two edges fall at exactly the same place within their cells, whatever the grid’s origin, so one edge’s extra fraction of a cell is the other edge’s missing fraction. The number of samples in the floor is the same for every origin, and the sum counts the notch’s width correctly.

If the width is a whole number of steps plus a half, the two edges fall at opposite places in their cells and their errors add. Between those two cases the error moves smoothly, so the cost against width is a wave with the step as its wavelength. At 10 nm the cost is 0.02; at 12.5, 1.99; at 15, 0.34; at 17.5, 1.59; at 20, 0.03. The whole steps are not quite equal — 15 and 25 nm cost 0.34 and 0.42 where 10, 20 and 30 cost two to four hundredths — because the light and the observer are not level across a wide notch, and the cancellation is exact only for weights that are. Nor is the worst width in a period at the half step: it sits a nanometre or so from the whole step, at four, six, fourteen, sixteen, twenty-four and twenty-six nanometres, where one edge has just crossed a sample and the other has not.

The amplitude falls slowly as the notch widens — 3.94 at four nanometres, 2.89 at fourteen, 2.25 at twenty-four, about a quarter less every ten nanometres. The error an edge makes is a fixed amount of reflectance times a fraction of a cell, so it does not shrink; what shrinks is its share of what the notch as a whole does to the colour, which grows with the notch’s width.

Beneath all of it is an exact null. A notch of no depth is a flat reflectance and costs nothing at any origin, to under 10⁻¹³, because the white it is judged against is summed on the same grid — a neutral has no grid. Every number above is a departure from that zero, and it belongs to the notch.

A Gaussian notch has no edge #

The notch the earlier measurement used had no such wave, and the reason is worth stating in the terms of a sum rather than of a picture.

A Gaussian notch two nanometres wide costs 2.14, five nanometres 0.27, and from eight nanometres on a hundredth. Its narrowest feature is its width. A sum that treats a smooth function as a string of cells misjudges each cell by an amount set by how sharply the function curves inside it, and a Gaussian’s curvature falls as the square of its width; so once the notch is a couple of cells wide, the error in each cell is small and gets smaller fast. That is the ordinary behaviour of a sum on a smooth function, and it is why the tabulation rule could state safety as a ratio of widths.

A function with a jump in it does not behave that way. The cell containing the jump is misjudged by a fraction of the whole jump, however wide the rest of the function is, and that error falls only in proportion to the cell’s width, not to its square. A flat notch is two jumps, and a notch fifty nanometres wide has jumps as abrupt as one five nanometres wide. The Gaussian’s safe region past eight nanometres does not exist for it at any width.

The difference between the two is the difference between the kind of object the rule was built on and the kind a filter maker builds. A pigment’s absorption band is smooth for chemical reasons; an interference filter’s edge is steep by design.

Half a cell at each end, again #

The error an edge makes has been met before on this grid. The endpoint term has a name found that a five-nanometre sum over a truncated range carries an error that falls only linearly with the step, and traced it to the half-cell at each end of the range: a sum that gives the first and last samples a whole cell each has counted half a cell beyond each end.

A notch’s edges are the same object moved into the middle of the spectrum. Each is a place where the function stops being one level and becomes another, and a sum that gives each sample a whole cell counts whatever part of the cell lies on the wrong side. The endpoint term was repaired by halving the weights of the two end samples, which works because a range’s ends sit on samples by construction. A notch’s edges sit wherever the filter maker put them, and no weighting fixed before the edges are known can know which fraction of which cell to take.

The grid slides, and the wave moves #

A calculation that cannot compute a reference can still slide the grid and watch the answer, which the line can be in the sample recommended as the one check a calculation can run on itself.

The spread across five origins follows a related wave. At a whole ten nanometres the spread is 0.30; at nine and eleven it is 2.46 and 2.40. At a whole number of steps every origin sees the same number of samples in the floor, so the answer barely moves; a nanometre either side, some origins catch one more sample than others, and the answer jumps between two values as the grid slides.

So the self-check works on a steep notch, and it says something the cost at one origin does not. A notch that happens to be very nearly a whole number of steps wide passes the check and passes it honestly: its computed colour is close to right on every origin. A notch half a step off a whole number fails the check, and the check does not care whether the notch is five nanometres wide or fifty.

The edge sets the size #

If the wave comes from the edges, softening the edges should flatten it at every width at once, and it does.

An edge rising from ten to ninety per cent in 0.22 nm gives a worst width in the period costing 3.09. Rising in 0.88 nm, 2.44; in 2.2 nm, 1.33; in 4.4 nm, 0.36; in 8.8 nm, 0.02. The cost at exactly ten nanometres is 0.02 at every one of those edges. The wave’s amplitude is a function of the edge alone, falling steeply once the edge is about as wide as the step, and where in each period the cheap widths fall is a function of the step alone.

That is the tabulation rule’s comparison moved to the right feature. A grid is not a resolution reduced the decision to the narrowest feature in the product of light and sample against the step, and which end to buy turned that comparison into a choice between refining the step and widening the range. For a Gaussian notch the narrowest feature is the notch. For a steep notch it is the edge, and a notch can be fifty nanometres wide and still have a feature a fraction of a nanometre wide at each end of it.

An interference filter’s edge is exactly that kind of feature. Its steepness is what it is sold on — the transition from blocking to passing is specified in nanometres or tenths of one — and a laser-safety filter or a fluorescence filter set is steep precisely so that a line on one side is blocked and light a few nanometres away is not.

A line lamp scrambles the pattern #

Everything above was computed under a light with no features. Under a fluorescent tube the notch’s edges and the tube’s lines are both narrow, and they interact.

For notches up to about ten nanometres wide the wave survives, costing 0.66 at five and 0.76 at ten against 3.40 at four and 3.69 at six. At about eleven nanometres the lower edge of a notch centred at 552.3 nm reaches the tube’s mercury line at 546.1, and from there on the spread across origins is six to nine colour differences at almost every width, whole steps included. With the softest edges the wave is gone, as it was under the smooth light, and from fifteen nanometres on the spread is still between five and a half and seven: the line’s own fragility, which the line can be in the sample measured on a Gaussian notch, does not care how steep the edge meeting it is.

So under a line lamp the governing feature is whichever is narrowest in the product — the edge, the line, or the edge crossing the line — and the cheap widths of the smooth-light wave are no protection once a line is involved.

What a notch filter’s datasheet does not say #

A filter’s transmittance is published as a table or a formula with a centre, a width and an edge steepness, and a colour computed from it on a five-nanometre grid inherits all three.

The width a filter is sold at is chosen for its application, not for anybody’s grid, so whether a given filter happens to sit near a whole number of five-nanometre steps is a coincidence. A filter twenty nanometres wide is nearly free; one twenty-two and a half nanometres wide costs a colour difference and a half; and the two might be the same design at two stated tolerances. Nothing in the colour computed at one origin says which case a filter is in — three numbers cannot see a line, and they cannot see where an edge fell either — and the case changes if the table is resampled onto a grid with another origin.

Resampling is where the damage is usually done. A filter’s maker tabulates it at a nanometre or finer, which holds its edges; a colour calculation that interpolates the table onto five-nanometre points before summing throws the edge’s position away, and a finer reading of a coarser table cannot restore it afterwards. The honest object is the fine table, summed as it stands or integrated through a slit — and the slit, as two slits are not one slit shows, has a condition of its own.

A notch chosen to fit the grid #

The wave suggests a repair that should be named so that it can be refused. A calculation could choose its step so that a notch of known width is a whole number of steps — a four-nanometre grid for a twelve-nanometre filter — and get the cheap end of the wave.

It works for one filter and nothing else. The cheap widths depend on the step, the step is shared by every sample in a calculation, and a scene with two filters of different widths, or one filter and a mercury line, has no step that suits all of them. It also depends on the width being known to a fraction of a step, which a manufacturing tolerance of a nanometre does not supply, and on the edges’ positions rather than the width alone, which the wave’s slow drift across widths shows is already an approximation.

The repair that does not depend on luck is resolution: a grid fine enough to put several samples across each edge, which for a half-nanometre edge is a step of a tenth of a nanometre. That is what the reference here uses, and it is fifty times finer than the grid it measures.

How the notches were computed #

The sample is a constant reflectance of 0.72 with a flat-bottomed notch 0.66 deep, centred at 552.3 nm, whose two edges are logistic functions of stated scale; the ten-to-ninety per cent rise of a logistic edge is 4.39 times its scale. The lights are closed-form spectra — a 6500 K thermal source and a fluorescent tube with mercury lines 1.2 nm wide — and the observer is given in closed form so that it can be evaluated anywhere.

The reference colour is the sum at two hundredths of a nanometre from 380 to 780 nm, with the white summed on the same grid; halving it again moves the answer by under two hundred-thousandths of a colour difference. The cost is ΔE₀₀ between the reference and the colour summed on a five-nanometre grid from 380 nm with its own white, and the spread is the range of that cost across grids offset by one to four nanometres. Widths run from two to thirty nanometres in half-nanometre steps.

What the measurement leaves out #

The edges are logistic and symmetric. A real interference edge is not a logistic: it has ripple beside it and a shape set by the layer design, and a filter’s two edges are rarely mirror images. The argument needs only that each edge is narrow and that the two are some distance apart, and the exact amplitude of the wave belongs to these edges.

The notch’s floor is flat. Real notch filters have a floor with structure of its own and an optical density that varies across it, which adds features narrower than the step inside the notch rather than only at its ends.

And the sample is point-sampled. An instrument integrating through a slit blurs the edges before they are tabulated, which softens them towards the regime where the wave vanishes — the reason the slit is what makes it legal for a lamp — and what that blur then does when the lamp has lines too is measured separately.

The habit: find the narrowest feature, not the named one #

A feature is described by the dimension that matters to whoever made it — a notch filter by its width, a band by its bandwidth, a line by its wavelength. The dimension that matters to a sampled calculation is the narrowest scale on which the function changes, and for a function with steep boundaries that is the boundary, however far apart the boundaries are.

The move is to ask how fast a tabulated function changes at its fastest, before asking how wide its features are called. A step function’s fastest change is its step.

The failure mode is to read a wide feature as a safe one. A fifty-nanometre notch with half-nanometre edges is two half-nanometre features, and a grid five nanometres apart sees each of them as a coin toss.

Where the pattern comes from #

That the rectangle rule’s error on a step function depends on where the step falls within a cell, and cancels for an interval that is a whole number of cells, is elementary numerical analysis; the same fact underlies the periodic error of any sampled boundary. Interference filters’ steep edges and the need to tabulate them finely are part of optical filter practice.

That a flat-bottomed notch’s grid cost in colour is periodic in its width with the step as its period, that its amplitude belongs to the edge’s rise, and how a mercury line scrambles the pattern, are computed here on one closed-form sample.

Still open: a filter’s own table, resampled #

The notches here are formulae. A published transmittance table for a real notch or band-pass filter, at a nanometre, resampled onto a five-nanometre grid by the interpolation a colour engine actually uses, would say where on the wave real filters fall, how much of the edge’s cost the interpolation adds beyond point sampling, and whether the computed colours of filter sets used in cameras and microscopes can be trusted on the standard grid at all.