Three audits and one shape

Three audits in one round, on three subjects with nothing in common, and each of them came out the same shape. That is either a fact about the world or a fact about how the round was conducted, and it is worth deciding which.

The claim #

All three of this round’s audits found a departure that is a pairing of two deviations, vanishes exactly when either factor is empty, and had been invisible because its parameter was imported rather than passed.

• The tabulation’s cost is a pairing of what the index leaves out with what the light puts there — and vanishes for a flat sample or a smooth light.
• The observer’s cost is a pairing of the observer’s deviation with the stimulus’s deviation from the adapting white — and vanishes for a flat sample or a gain.
• The scene solver’s cost is a pairing of the surface’s non-Lambertian part with the field’s non-uniformity, which the previous round had already established.
• And each parameter was a module-level constant rather than an argument, which is why none of the three had ever been varied.

The three, stated together #

Reading the three side by side makes the common structure visible in a way none of them shows alone.

The tabulation. X = k Σ R S x̄ Δλ is summed over an index with a step, a range and an origin. What each costs is a product of what the index omits and what the light has there: a range costs nothing under a lamp with no ultraviolet, and a step costs nothing on a light with no narrow feature. Two conditions, either sufficient.

The observer. The same integral against three curves, and the curves have seven arguments. What each costs is a product of the observer’s deviation with the stimulus’s deviation from its adapting white. A flat sample gives zero for every observer; a gain gives zero for every sample. Two conditions, either sufficient.

The scene. A radiosity solver’s unknown is one number per patch, and what that costs is a product of a surface’s departure from Lambertian with a field’s departure from uniform. A matt wall or a uniform field gives zero. Two conditions, either sufficient.

Three subjects — an index, a set of curves, a solver’s unknown — and one algebraic form.

Why the shape recurs #

The recurrence is not a coincidence and it is not deep. It follows from what all three departures are.

Each is a difference between two linear functionals applied to the same object. The tabulation’s is a sum over one index minus a sum over another; the observer’s is an integral against one set of curves minus an integral against another; the scene’s is a transport with one reflectance model minus a transport with another.

A difference of two linear functionals is itself a linear functional, and applying it to an object gives an inner product — the functional’s difference paired with the object. That is the pairing, and it is why both factors can independently be zero: an inner product vanishes when either argument does.

So the structure is a consequence of linearity rather than a discovery about colour. What is not a consequence of linearity is that the conditions are reachable — that there exist ordinary samples and ordinary lights that empty each factor — and that is the part worth having.

The algebra is trivial and the reachability is the finding. A flat reflectance is a grey card, a smooth light is daylight, and a Lambertian wall is a matt one; all three are things anybody has, and each of them makes a departure exactly zero.

What makes a choice invisible #

The second common element is about visibility rather than algebra, and it is the more useful of the two.

In all three cases the parameter that turned out to matter was imported rather than passed. LAMBDAS is a module-level constant. The observer is observer("1931"), called inside spectrumToXYZ with a default. The Lambertian assumption is not a parameter at all — it is a property of the solver’s unknown.

None of the three appears in any function signature. So an audit that examines a calculation’s arguments, which is what an audit naturally does, will not find them. The census’s own self-audit examined its declared inputs and found several that mattered, and it could not have found these.

That gives a rule with more general use than anything else in the round: read the imports, not the signature. A short list, in most code, of constants and modules, and each one is a decision that could have been otherwise.

The rule has a corollary that is uncomfortable. A parameter promoted from a constant to an argument becomes visible and therefore auditable, and the promotion is usually resisted because a constant is simpler. Simplicity of interface and auditability are in direct tension, and the tension is normally resolved in favour of the interface.

Three and not more, for a reason worth checking. A reader might reasonably ask whether the same shape would be found anywhere it was looked for, which would make it a fact about the method rather than about the subject.

Two things argue against that. The conditions are exact rather than approximate: eight of this round’s ten are at the floating-point floor, and an exact zero is a strong claim that a fitted structure would not produce. And the conditions are reachable: a method that found a pairing whose factors could only be emptied by unphysical inputs would have found a formal identity rather than a fact.

Against that, the shape is a consequence of linearity and colorimetry is linear nearly everywhere, so it is fair to expect the shape wherever the linearity holds. That is not a criticism of the finding; it is the finding’s scope.

Where it would fail is anywhere the departure is not a difference of linear functionals. An appearance model’s nonlinearity, a gamut mapping’s clipping, a halftone’s Yule–Nielsen exponent: none of those is linear, and a departure in one of them would not factor. The pairing is a property of the linear part of this subject, which is most of it and not all.

What the three audits found that was not shared #

The differences are as informative as the similarities and there are three.

Repairability. The tabulation’s terms are repairable in software — a rule change, a wider range — at a cost that is arithmetic. The observer’s are not repairable at all, because they are properties of people. The scene’s are repairable by a better solver, which now exists.

Size. The observer’s terms are two to three colour differences, the scene’s one to five, and the tabulation’s a tenth to half a unit. So the smallest of the three is the only one anybody can fix.

And who bears them. The tabulation’s fall on whoever computes; the scene’s on whoever renders; the observer’s on whoever looks. Only the last has no representation in any specification.

That third asymmetry is the round’s most consequential and it is the reason the landing essays are all about the observer. A term borne by somebody outside the calculation is a term nobody in the calculation is motivated to find.

There is a fourth recurrence worth adding to the list, and it is about presentation rather than structure. Every ladder in all four audits turned out to be a ladder of examples. The previous round’s four departures spanned a factor of seven on one choice of sample and the place index alone spanned thirty-three across materials; this round’s six observer departures reorder between two lights; and its two tabulation decisions reverse between two lamps.

That is a direct consequence of the pairing. A departure’s size is a product of two factors and a ladder holds one of them fixed, so a ladder is a slice and its ordering is a property of where the slice was taken.

The lesson is now four for four and can be stated as a rule: a bar chart of departures needs a distribution beside it, and the distribution should be over whichever factor the chart holds fixed. Doing so has changed the reading of every such chart this collection has drawn.

What an audit of this kind can and cannot reach #

The previous round drew a boundary and this one sits inside it unchanged.

A departure can be audited when there is a fuller model to compute the departed answer with. The tabulation had one — an analytic observer on any grid. The observer had one — a pigment template with seven arguments. The scene had one — a directional transport solver, which had to be written.

Where no fuller model exists, the departure cannot be sized, only named. This round names several: the interaction between the range and the observer’s blue-absorbing departures, the census under two observers, the room with a gloss finish below the solver’s boundary.

That boundary is a property of the method rather than of the subject, and it is worth restating because it is easy to mistake an audit’s silence for a small term. An audit reports what it could compute, and the list of what it could not is part of the report.

What the round changes about how this collection works #

Three consequences for practice, all of them about interfaces rather than about colour.

A parameter with no call site should be given one when it is plausibly load-bearing. LAMBDAS now has one in three modules, the observer has one in the audit’s machinery, and the scene solver has a roughness argument. None of those changes any published number and all three make the next audit cheaper.

Two audits should share a stimulus set. That cost nothing here and produced the round’s sharpest single result, which required both audits on one light.

And a condition should be classified rather than reported. Eight identities and two limits is a more informative statement than ten small numbers, and the classification takes one comparison against the floating-point floor.

A fifth recurrence is about what an audit owes afterwards, and all four have now paid the same debt in the same currency.

Each audit ended by asking what its departure does to the collection’s existing results, and each got the same answer in two parts: the comparisons survive and the absolute numbers do not. That division is not a coincidence either — it is the shared-term argument, and a shared term cancels from a difference and not from a value.

So the accounting after an audit is nearly mechanical once the shape is recognised. Sort the affected results into comparisons and levels; the first are safe to first order; the second carry the term in full; and the list of levels is usually short, because most published results are comparisons.

That is a considerably cheaper accounting than recomputing anything, and this round performed it three times — for the scene work, for the census, and for the census’s grid — at a cost of about a paragraph each.

What was computed, and how #

Nothing new is computed in this essay; it is a reading of the round’s three audits, whose numbers are in their own essays.

The claim that the pairing follows from linearity is an argument rather than a measurement, and it is checkable: the previous round computed the pairing directly and against a linearised prediction and found them equal to a part in a thousand billion for three of four departures.

The classification of conditions into identities and limits is made by comparing against the floating-point floor rather than against a tolerance, which is the discipline the previous round had to establish when three of its eight conditions were identities and five were limits.

One more thing the four audits share is worth recording because it is about how they were started rather than what they found. In every case the audit began from a sentence in a previous round’s closing essay naming something that had not been done — the model equation’s dropped arguments, the range that was a truncation, the observer whose substitutions already existed, the scene solver with no lobe.

None of those sentences was a plan. Each was an honest note about a limit, written because the round that wrote it could not go further, and each turned out to be the whole content of a later round. A shortfall recorded precisely is a slate for the next round, which is the argument this fleet’s own instructions make for keeping one and is not usually demonstrated four times in a row.

The corollary is that the value of a shortfall entry depends entirely on its precision. The observer is a limitation would have produced nothing; the substitutions exist and only the plumbing is missing named the work and turned out to be right about the plumbing and wrong about the difficulty.

Where the model stops #

The three audits are three, which is a small sample for a claim about a shape. The previous round’s audit of the sample makes four, and all four are in one collection computed by one set of machinery, so a common cause in the machinery cannot be excluded.

The linearity argument covers the departures that are differences of linear functionals and says nothing about the collection’s nonlinear parts, which include everything downstream of tristimulus values.

And the invisibility claim — that each parameter was imported rather than passed — is a fact about this collection’s code rather than a general one. Another implementation might have passed all three, in which case the audits would have been available years earlier and this essay would be about something else.

The generalisation #

The habit is about what to do when several unrelated investigations produce the same shape.

The first thing is to check whether the shape is a consequence of something shared — a common formalism, a common implementation, a common method — rather than a discovery. Here it is: linearity produces inner products, and this subject is linear.

The second is to notice that a trivial consequence can still be a useful finding, because what it licenses is not trivial. The pairing is arithmetic; that its factors can be emptied by ordinary objects is not, and it is what turns an algebraic identity into a diagnostic anybody can run.

The failure mode is to stop at the recognition. A shape that recurs is a template rather than a conclusion, and its value is that the next audit knows what to look for: two factors, two conditions, and a check that each one is exact rather than small.

A closing caution about the template, since it is offered as one. A shape that fits four cases will be reached for in a fifth, and the reaching is where a template stops being useful. The pairing holds because the departures are differences of linear functionals; a departure that is not will not factor, its conditions will not be exact, and looking for two factors will find two things that are not factors.

The check is cheap and it is the same check the round has used everywhere: impose each proposed condition and see whether the residual is at the floating-point floor or merely small. An exact zero confirms the structure; a small number means something else is going on and the template has been applied where it does not fit.

Who found it, and when #

The pairing structure was first stated in this collection by the round that audited the model equation, where four departures turned out to be inner products of a sample’s deviation with a light’s. This round found it three more times in three unrelated places.

The general observation that a difference of linear functionals is a functional, and that applying one gives an inner product, is elementary and is in every functional analysis text. Its use as an audit template does not appear to be written down anywhere, presumably because it is too simple to publish and too specific to be obvious.

Where the ladder goes next #

The shape’s most useful half is the conditions rather than the sizes, and that deserves its own statement: the conditions are the result, and a reader who remembers nothing else from the round should remember them.