Keying hair and transparent edges: what a chroma key can and cannot do

Flyaway hair, smoke, a glass, a hand moving fast — things that are half backdrop and half subject. Here is what happens to them, what the edge slider changes, and where the honest answer is "not with this tool".

Why anything semi-transparent is hard

A chroma key makes one decision per pixel: measure the distance from the key colour in the Cb/Cr plane, compare it to a threshold, write an alpha. A strand of hair over green is not "70% hair" — it is one colour at one distance. Past the threshold it is opaque, inside it clear. Nothing in the image carries coverage.

So semi-transparency has to be recovered rather than measured. A pixel that is half dark hair over bright green and a pixel that is entirely mid-green sit at two different distances, and the maths cannot tell which is which.

What this keyer writes by default: a binary matte

Start here, because it decides what the rest of the page means. The edge slider starts at 0. Feather is then zero, the two thresholds are equal, and the loop sets hard = hi - lo < 1e-6. Every alpha written is 0 or 255, and the value reaching the file is alpha >= 1 ? a : Math.round(alpha * a).

At default settings there are no semi-transparent pixels at all. Hair comes out as a hard silhouette: each strand survives whole or vanishes whole. That is not an export bug and not the PNG dropping data — the keyer never computed an intermediate value.

What the edge slider buys you

Two things, depending on direction.

Positive edge is feathering. At edge 100 with a colour key the ramp spans 46 units of Cb/Cr distance (60 for a neutral backdrop such as a white wall). With the default tolerance of 25 the lower threshold is 29.5, so the ramp runs 29.5 to 75.5: closer is fully clear, further is fully opaque, and only pixels inside that band get partial alpha. That band is the only place a soft edge can exist.

Strands blended with the screen can land in the band and come out soft. But the band is narrow, defined by colour distance rather than coverage, and it moves the whole edge — it does not detect hair. Push it far enough and you feather areas that are not edges at all.

Negative edge is trimming, the opposite trade. It raises the lower threshold so more pixels go fully clear. At edge −100 it lifts it by 62 units, and on the spill-green fixture that eats 6,902 of the subject's 21,341 pixels: IoU drops from 1.000000 to 0.889322. That is the price of killing a fringe — real subject goes with it.

Smoke, glass and motion blur

Same mechanism, worse odds. Smoke and glass refract and accumulate rather than blend. A pixel of smoke is green tinted toward grey; a pixel of glass is green shifted, often with the backdrop distorted through it. Neither sits reliably in a narrow band around the key, so some lands in the ramp and some does not: patches that key out next to patches that stay solid grey. No slider fixes that, because the distance is not consistent across the area.

Motion blur is the same problem in time. Every frame is keyed independently, with no temporal or spatial filtering, so a soft edge along a moving hand is re-decided each frame.

What spill suppression does here, and what it does not

Spill suppression changes colour, never alpha. It runs on pixels above zero alpha and pulls the key channel toward the brighter of the other two — for a green key, green toward max(red, blue) by the slider fraction, 50 by default. On hair carrying green bounce that genuinely helps.

It cannot make hair transparent, because it never touches the matte. A strand the key called opaque stays opaque, just less green. It also skips pixels already cleared, so it does nothing on the background side of an edge.

When this does not work

• Flyaway hair at default settings. The matte is binary, so strands are present or gone. You need the edge slider above zero before any partial alpha exists.
• Hair against a shadowed screen. The backdrop itself is failing to key, and no edge setting recovers it. On shadow-green, tolerance 25 leaves 21,201 of 55,459 background pixels behind (IoU 0.617718); it takes tolerance 40 to reach 1.000000.
• Backlit or strongly rim-lit hair. A bright halo between the strands and the screen shifts the colour of every edge pixel, differently on each strand.
• Smoke, fog, haze or fire. Density varies across the frame, so no single threshold band follows it.
• Glass and clear plastic. A window or a glass refracts the backdrop instead of blending with it, moving those pixels to a colour the key was never pointed at.
• Fast movement. Each frame is keyed alone with no temporal filtering, so the blend along a moving edge is re-decided every frame.
• Anything near the key colour. A green shirt, a green reflection in glass, green-tinted hair dye: on flat-green, raising tolerance far enough to chase them turns the whole subject transparent at once — at tolerance 90 all 21,341 subject pixels are gone and IoU falls to 0.722122.
• Detail thinner than one pixel. The camera has already blended it into its neighbours. No keyer separates what the sensor never resolved.

What to do instead

Most of this is won on the shoot day, not at the slider. Light the subject separately from the backdrop, put distance between them so bounce falls off, add a hair or rim light on the far side so strands separate by brightness rather than hue, and avoid backlighting that pushes screen colour into the hair. If a shot needs smoke or glass in frame, plan for it not to key cleanly.

For the few shots that need a hand-built matte — a hero product frame someone will see full screen — the right tool is a rotoscope brush in a real editor. This tool is a colour key and does not pretend otherwise.

If you want a soft edge for compositing, the transparent PNG export carries 8-bit alpha, but at default settings there are only two values in it. The tolerance page covers where the thresholds sit, and the artifact guide covers telling a soft edge from a fringe. For footage, the export settings page explains what each format keeps.

Where the numbers come from

Every figure here comes from the synthetic 320×240 fixtures in the chroma-key-reference repository, also published as a preprint (DOI 10.5281/zenodo.22916767). They are flat rectangles of solid colour scored against a binary ground-truth matte by mean IoU, which measures how well a backdrop separates from a solid subject.

They contain no hair, no smoke, no glass, no motion blur and no soft edges, so they say nothing about how those come out. Every claim on this page about hair, smoke and glass is mechanism read off the algorithm, not a measurement. We did not test those cases and will not quote a number we did not produce.