A serif moves the error and adds little

Sharpened type errs on its dark side sharpened scanned letters two ways — by an unsharp mask applied to the stored, encoded values, as most software does, and by the same mask applied to light — and looked at the difference as the eye would see it on a printed page at forty centimetres. It had set out to find the error gathering at the letters’ corners and stroke ends, where the eye counts a corner’s error, not its peak had found a sharpened square’s corner seen at more than twice its edge. It found something else. The seen error lies on the darker side of each edge, 77 to 90 per cent of it, so black type’s outer corners, which point into paper, hold less than their share, and reversed type’s, which point into ink, hold two or three times theirs.

The explanation turned on a geometric fact about black type: its dark side is a stroke, too narrow at text sizes to wrap round a corner. The essay named the one feature of ordinary type that could change it. A serif adds, at every stroke end, a short bar across the stroke, and the join between serif and stem is an inside angle where ink wraps round a small notch of paper — the configuration that did gather error on 24-point letters, where the strokes are wide enough to wrap. It predicted that a serif face would not gather at its serifs either, because a serif is thinner than a stem and its notch smaller than the region the eye pools over at ten point. If the brackets did collect error, “it is the serif, not the corner, that a sharpening test for text should be aimed at.”

Brackets do not gather; reversed tips do #

Black serif type’s corner zones hold 0.74 to 0.84 of their share of the seen error at nine, ten and twelve point — no more than the same letters without serifs. Its serif brackets hold three quarters of their share, its serif tips half. In reversed type the serif tips hold 1.8 to 2.5 times their share and the brackets under half. Serifs lengthen the letters’ outline by a tenth and their total seen error by one to four per cent in both polarities.

• The prediction holds for black type: serifs do not give it corners that gather, and the result for sans-serif letters stands for book type.
• The brackets never gather in either polarity — the inside angle the proposal feared holds less than its share everywhere.
• Reversed serif type gathers at its serif tips, which point into the dark ground, exactly as reversed type’s stroke ends do.
• Serifs redistribute the error without adding to it: a tenth more outline, a few per cent more error.

The letters #

The earlier census drew four capitals — E, H, L and T — on a five-by-seven grid with a one-cell stroke, so that every corner was known exactly. A serif cannot be drawn on that grid. So the same four letters are drawn here on a grid three times finer, seventeen cells by twenty-one, with three-cell stems, in two versions. Without serifs they are the original letters at three cells a cell, and they reproduce the original result: at ten point in black type their corner zones hold 0.77 of their share of the seen error, against 0.78 on the coarser grid. With serifs they gain, at every free stroke end, a serif one cell thick — a third of the stem — reaching one cell past each side of the stroke: horizontal at a stem’s end, vertical at a bar’s.

Every corner of each letter is then labelled by the part it belongs to. A stroke corner is where a stem or bar ends without a serif; a junction is an inside angle where bar meets stem; a serif tip is an outer corner of a serif; a bracket is the inside angle where a serif meets its stroke. The seen error is divided among them by assigning each pixel’s error to the nearest point on the outline, and each part’s share of the error is compared with its share of the outline within two pixels of a corner.

Why the error has a side #

It helps to recall why sharpening on stored values errs at all, because the reason is what decides where a serif’s error can go. An unsharp mask is an average with a negative weight in it: it subtracts a blurred copy of the image from the image and adds the difference back. An average on the stored values showed that averaging encoded numbers rather than light is not a neutral shortcut, because the encoding is curved, and a resize with a negative weight in it showed which way that curvature pushes a result when some weights are negative. Beside an edge, both of the mask’s halos — the bright overshoot on the pale side and the dark undershoot on the dark side — come out lighter than the same mask applied to light. The corner of a resized patch is lighter than its edges found the same bias in a resize, and found that it concentrated where two edges met.

What moves the error to the dark side is the reader, not the mask. The eye pools each pixel’s error with its neighbours over a region a few pixels wide at print distance, so the two halos merge across the edge, and the pooled difference is then read on a lightness scale that is steep in the dark. The straight piece under the cube root priced that steepness: the same difference in light costs far more against ink than against paper. And the eye keeps the lightness errors is why this is the error that matters, since the eye’s spatial filtering discards the fine chromatic part of a difference long before it discards the lightness part.

So a corner gathers error in proportion to how much dark lies within the eye’s pool around it. For a serif, the question becomes whether the ink round a bracket — a stem on one side, a serif a third as thick on the other — is enough dark to wrap the angle, and whether a serif tip’s ground is. The first was the proposal’s worry, and the second is what it did not ask.

Where the error sits in a serif letter #

In black type the error follows the strokes. It lies along the ink side of every edge, as the earlier essay found for sans-serif letters, and the serifs — short vertical bars at the ends of the E’s three arms — add a little more ink edge for it to follow. The circles round the serifs’ tips and brackets hold little of it. In reversed type it spreads into the dark ground round the letter, gathering at every outer corner, and the serif tips, which are outer corners pointing into the ground, collect it most.

The T shows the difference most plainly. Its stem ends at the baseline in a horizontal serif, making two small inside angles and two outer tips at the foot. In black type the foot’s error is spread along the serif’s ink; reversed, it concentrates at the two tips, where the ground wraps round the serif’s ends.

Which corners gather #

At ten point in black type only the junctions hold more than their share, and only just — ×1.07. Stroke ends hold ×0.82, serif brackets ×0.74, serif tips ×0.49. The junctions are inside angles where a bar meets a stem, and they hold a little more than their share because their dark side is the full width of the stem, wide enough at ten point for some error to wrap into the angle. The brackets are also inside angles, and they hold less, because their dark side is the serif, a third of a stem thick, and the notch is smaller than the eye’s pooling at this size. The proposal’s reasoning is right, measured.

Reversed, the picture inverts. Serif tips hold ×2.37 and stroke ends ×2.19; junctions ×0.25 and brackets ×0.40. Every outer corner points into the dark ground, and the error the eye sees is on the dark side, so outer corners gather and inside angles give it up. A serif adds outer corners — two at every stroke end — and in reversed type each one gathers. At ten point the serif tips are twelve per cent of the outline and hold 28 per cent of the seen error.

Nine point differs in one detail. There the unserifed stroke ends of the serif letters hold ×1.04 of their share rather than ×0.82, and the junctions ×1.16. Both are inside the band of the sans-serif letters at the same size, whose junctions hold ×1.08, and neither moves the pooled figure above one; the serif parts themselves — brackets ×0.73, tips ×0.63 — stay under their share.

Across text sizes #

Black type stays under one at every size, with serifs as without: 0.84 with serifs against 0.91 without at nine point, 0.74 against 0.77 at ten, 0.80 against 0.94 at twelve. The serifed letters sit slightly lower, because serifs add corner zones that hold less than their share. Reversed type stays above one: 1.23 against 1.38, 1.22 against 1.34, 1.16 against 1.40. The serifed letters again sit lower, for the opposite reason — the brackets add corner zones that hold very little — and the serif tips’ gathering is diluted by them in the pooled figure.

The brackets hold under their share at every size in both polarities: 0.73 to 0.77 in black type and 0.33 to 0.43 reversed. The tips hold about half their share in black type and about two and a half times it reversed at nine and ten point, falling to 1.8 at twelve, where the serifs are larger and their error is less concentrated at their corners. That fall with size is the trend the eye counts a corner’s error, not its peak found for all corners: at larger sizes the eye’s pooling covers less of a corner, and a corner’s error is seen more as an edge’s.

Moved, not added #

Serifs lengthen the four letters’ outline by ten per cent and raise their total seen error by one to four. In black type at ten point, one per cent; in reversed type at nine point, four. The serifs add edge, and sharpening on stored values errs along every edge, so some added error was expected. But the error they add is far less than the edge they add, because a serif’s edge is short and much of it is inside a bracket, where the error is weakest. A serif redistributes a letter’s error more than it adds to it, and in black type it redistributes it away from the corners.

What this means for sharpening scanned text #

A sharpening test for black text needs no special serif case. The earlier finding — that black type’s error is on its dark side and its corners carry less than their share — holds for book type with serifs. A test pattern of sans-serif letters measures what a page of a serif face will show.

A test for reversed text does need it. Reversed type’s outer corners gather the error, and a serif face has twice as many outer corners at its stroke ends as a sans face. A reversed serif headline, white type on a dark ground, is where sharpening on stored values will show most, and the error will be at the serifs’ tips. A halftone is a luminance object found that only the eye’s luminance channel resolves a print’s fine structure, and the same channel carries this error: a serif tip’s error is a lightness error at a scale the luminance channel sees.

And the cure is the same in both cases. Sharpening on light rather than on stored values removes the difference entirely, as the earlier essays showed; the serif question is only about where the error sits when the cure is not applied.

How the letters were measured #

The letters are unions of cells on a seventeen-by-twenty-one grid, with a margin of one cell each side, stems three cells wide and serifs one cell thick and one cell beyond the stroke each side. Each letter is rendered at the stated cap height on a 300-dot-an-inch page — a cap height of 0.7 of the point size — by exact area coverage, mixed in light, encoded as sRGB, and sharpened by an unsharp mask of radius 1.5 pixels and amount one, on the stored values and on linear light. The colour difference between the two results is filtered by the collection’s three spatial channels at a print’s geometry, forty centimetres, and read as CIEDE2000 per pixel. Every outline vertex where the outline turns is a corner; it is convex when one of the four cells meeting there is filled and concave when three are, and it belongs to a serif when any of those cells is a serif cell. Each pixel’s error goes to the nearest point of the outline, and each part’s share is the error within two pixels of its corners over the total, against those corners’ share of the outline’s length.

What this leaves out #

The serifs are slabs. Real book faces have bracketed serifs — a curved fillet between serif and stem — and hairline serifs thinner than a third of a stem. A fillet removes the sharp inside angle altogether, which can only reduce a bracket’s error further; a hairline serif is thinner, which reduces its tips’ error in black type and concentrates it in reversed.

The eye is the collection’s model of it, contrast-sensitivity functions measured with gratings at threshold and applied here to errors well above it — the caveat every threshold was measured with a grating attaches to every such calculation. The finding rests on the same two properties as the earlier census — pooling over a few pixels at print distance and a lightness scale steep in the dark — and a different curve would move the ratios without, on either property, reversing which side of a bracket is darker.

The letters are rectilinear. Rounds — O, C, S — have no corners, and their error is the edge’s; the census says nothing new about them.

And the page is at print distance. On a screen, at 60 centimetres and a coarser pitch, a ten-point letter is a few pixels tall and the eye’s pooling covers all of it; there the distinction between corner and edge disappears.

Still open: whether a hairline serif gathers in black type #

The serifs here are a third of a stem, and in black type their tips hold half their share. A hairline serif, a tenth of a stem, is common in display faces, and its tips are corners whose dark side is so thin that the sharpening halo might spill round them onto the paper, as it does round reversed type’s corners.

The calculation is this census with serifs a tenth of a stem thick, on a grid fine enough to draw them — ten cells to a stem — at display sizes from 18 to 48 point, where hairlines are used. The prediction is that hairline serif tips in black type begin to gather above about 24 point, where the tip is large enough for the eye to see its corner as a corner, and that the effect grows with size — the opposite of the text-size trend, because the hairline’s thinness rather than the letter’s size decides how much of the halo wraps round. If the tips gather, a display face’s hairlines are the one place in black type where sharpening on stored values makes its error at corners.

A feature that adds geometry need not add error #

The habit is about measuring what a new feature does to a quantity before assuming it adds to it.

A serif adds corners, edges and inside angles, and every one of those was already known to carry sharpening error in some configuration. The natural expectation was that serifs would add error in proportion to what they add of the letter’s outline, or worse. Measured, they added a tenth of the outline and a few per cent of the error, and in black type they moved it away from where the eye counts it most.

The failure mode is to price a feature by the geometry it adds rather than by where the quantity sits on that geometry. Error on a sharpened edge sits on the dark side; a feature that adds edge mostly where the dark side is thin adds very little of it.