Chroma key vs luma key: which one this tool uses, and when it switches

A chroma key measures colour. A luma key measures brightness. This keyer uses one or the other depending on the colour you key out, without telling you which. Here is the rule, what changes when it flips, and the shots where neither one separates anything.

The one line that decides it

Every parameter starts with one test on the key colour: neutral = Math.sqrt(ku*ku + kv*kv) < 7. If the colour you asked it to remove has essentially no colour in it, there is nothing to measure in the Cb/Cr plane, so the pixel loop switches to luminance — the only thing that can tell a white wall from a white shirt.

There is no toggle. The mode follows the key colour, and the presets show how wide the gap is. Green #00b140 measures Cb −26.6337 / Cr −79.3117, magnitude 83.6643. Blue #0047bb is 69.9803 / −44.9322, magnitude 83.1633. A white wall #f2f2f2 is 0.0000 / 0.0000, magnitude 0.0000, flagged neutral — and so are black and mid grey. Broadcast green and blue sit further out again at 136.1412 and 129.1750. Nothing lands anywhere near the threshold of 7: a backdrop is either clearly coloured or clearly neutral.

What each one measures

Chroma. For each pixel: du = Cb(pixel) - Cb(key), dv = Cr(pixel) - Cr(key), d = sqrt(du*du + dv*dv). Under the threshold it is background, over it subject. Hue and saturation decide it. The practical upside: anything genuinely colourless — a grey laptop, a black cable, white text — sits at the origin of that plane, far from a green or blue key, and is safe.

Luma. d = |luma(pixel) - luma(key)|, where luma is 0.299R + 0.587G + 0.114B. Hue stops mattering entirely. The distance is absolute, so a pixel brighter than the key by the same amount counts just as much as one darker. Key a white wall at luma 242 with the default tolerance of 25 and the radius is 37.5: everything from 204.5 upward is treated as backdrop, including a shirt brighter than the wall behind it.

What changes when it flips

Three constants change together. The threshold scale goes from 118 to 150, feathering from 46 to 60 and trimming from 62 to 70 — luminance spans a wider range than chroma distance. At the default tolerance of 25 that puts a colour key's threshold at 29.5 and a neutral key's at 37.5. A number you tuned on a green screen does not transfer to a white wall.

The other thing to know: the test runs on the key colour, not on your footage. Point it at a warm off-white wall and it may have just enough chroma to fail the neutral test, so you get a colour key with a very small chroma magnitude — a tight, fragile threshold that follows tint instead of brightness. That is the mode you did not want, and there is no way to ask for the other one.

What the neutral fixture shows

The neutral-white fixture is a flat #f2f2f2 field with a #9a9a9a rectangle on it — the two differ by 88 luma and by no chroma at all. At the shipped defaults it scores IoU 1.000000: all 49,579 backdrop pixels cleared, none of the 27,221 subject pixels touched. Automatic key detection read #f2f2f2 exactly, with a maximum channel delta of 0.

Sweeping tolerance across it gives a cliff, not a curve. Every value from 0 to 55 stays at 1.000000. At 60 it drops to 0.645560 and stays there through 100. That is arithmetic: the radius is (tolerance/100) * 150, so tolerance 55 gives 82.5, under the 88-luma gap, and tolerance 60 gives 90, past it. One step over the margin removes the entire subject, and no further setting brings any of it back.

For contrast, a green-dominant baseline scored 0.000000 on the same fixture even when handed the best possible threshold. That is the whole argument for the fallback: a colour rule has nothing to grab on a white wall.

When this does not work

• A subject close to the backdrop in brightness. On a neutral backdrop, brightness is the only separation you have. This fixture had 88 luma of margin; a pale object on a pale wall has far less, and one tolerance step past the margin took out all 27,221 subject pixels at once.
• Same brightness, different colour. A luma key cannot see colour. A grey prop on a grey backdrop is not recoverable at any slider position.
• Anything brighter than the wall. The distance is absolute, so on a wall keyed at 242 a highlight reaching 250 luma is inside the default radius and gets keyed out along with the wall.
• Shadow or uneven light on a neutral backdrop. Not measured. Our brightness sweep covers a green screen only, with no numbers for a wall carrying a gradient. Treat it as untested.
• You cannot choose the mode. There is no switch. The key colour decides, and changing it changes what gets removed.
• Off-white backdrops land in the wrong mode. A cream or warm-grey wall can fail the neutral test and be keyed by colour instead, with the tighter 29.5 threshold and a matte driven by tint.
• Automatic key detection can flip it for you. It takes the median of the outer 5% border of the frame. If the wall does not reach the edge of the shot, the median reads whatever else is there, and both the key colour and the mode change with it.
• Every number here is synthetic: 320×240 flat rectangles of solid colour scored against a binary ground truth by mean IoU. No real walls, no fabric, no clothing, no shadows on neutral backdrops. Everything said above about those is mechanism, not a measurement.

Which one you are getting

Green or blue screen, you get a chroma key — that is the default and it is what the tool is built for. White, black or grey backdrop, it switches to brightness on its own. Neither needs a decision from you. What decides the outcome is the separation: hue distance on a colour screen, brightness distance on a neutral one, and brightness is the easier of the two to lose.

If you are working without a screen at all, the white background page covers what that fallback is actually good for. The tolerance page explains where the threshold sits in both modes, the blue vs green page covers picking a coloured screen, and the artifact guide helps when the result is close but not clean. Start from the tool if you just want to drop a file in.

Where the numbers come from

Every figure on this page comes from the synthetic 320×240 fixtures in the chroma-key-reference repository, also published as a preprint (DOI 10.5281/zenodo.22916767). They are flat rectangles of solid colour scored against a binary ground-truth matte by mean IoU, and the threshold arithmetic is read straight from the keyer source.

They are not real photographs of real walls. The neutral-white result describes one grey rectangle on one flat grey field. Claims about clothing, shadows on neutral backdrops and uneven room light are mechanism, not measurement — we did not test those.