Chroma key artifacts — what each edge problem actually is
A dark rim, a green rim, holes in the subject, stair-stepped edges, an outline that crawls when the video plays. They look alike in a thumbnail and have four different causes. Here is how to tell them apart.
Where all of them come from
The keyer converts every pixel to YCbCr, measures how far it sits from the key colour in the Cb/Cr plane, and turns that distance into an alpha. At the default edge of 0 the two threshold edges coincide, so alpha is only ever 0 or 1 — a binary matte. Nothing in that loop looks at the neighbouring pixel or the previous frame.
So every artifact on an edge is one of three failures: backdrop pixels that stayed opaque, subject pixels that got erased, or subject pixels that kept the wrong colour. Which one it is decides which slider helps.
The dark rim: backdrop that survived
The outline hugs the subject, is grey rather than green, and thickens on the side that fell into shadow. Sometimes it is a whole patch of screen that refused to go.
Those pixels are the screen, only darker. Distance in Cb/Cr survives a brightness drop far better than distance in RGB, but not completely: a backdrop at 55% brightness sits 85.0941 units from the key in RGB and 37.8083 away in Cb/Cr, while default tolerance of 25 opens a band of only 29.5 units. So it stays solid.
On the shadowed fixture at tolerance 25, 21,201 of 55,459 backdrop pixels survive, a recall of 0.617718. At 35 it is down to 12,720 (0.770641), and the fixture reaches 1.000000 only at tolerance 40. Raising tolerance is the fix; the better one is lighting the screen so it never needs the extra room.
The green rim: a colour problem, not a matte problem
The subject is intact and the cut is clean, but hair, skin and light clothing carry a green cast, strongest at the outline. This is not a hole in the matte. Light bounced off the screen onto the subject, those pixels really are partly green, and the keyer kept them correctly.
On the spill fixture at tolerance 25 the matte scores 1.000000, precision 1.000000 — nothing erased, nothing left behind. Chasing the tint with tolerance instead erases the rim with the subject: at tolerance 80, 7,206 of 21,341 subject pixels are gone, precision 0.885008, and the rest are still tinted, because the tint is inside the subject rather than on its boundary.
Two controls address it: the spill slider pulls the dominant channel down toward the brighter of the other two, and a negative edge trims the outer band. At edge −100 the trim costs 6,902 subject pixels, IoU 0.889322. There is no sweep for spill strength — every edge-sweep row was measured with it off — so this page can say what the slider does, not by how much.
Holes in the subject
See-through patches in a person, thin fabric gone translucent, hair thinning out. The opposite failure: subject pixels fell inside the band.
It is not gradual. On the flat-green fixture the score holds 1.000000 to tolerance 85, then at 90 the key erases all 21,341 subject pixels at once and IoU drops to 0.722122 — one step between a clean matte and an empty frame. A trim does the same by another route: edge −100 eats 6,902 subject pixels.
Stair-stepped edges
On a still frame the outline is a staircase rather than a curve. That is the binary matte: with edge at 0, alpha has no intermediate value, so there is nothing to anti-alias with.
A positive edge opens a ramp instead — 46 units of chroma distance wide at edge +100 for a colour key, 60 for a neutral one, with pixels inside it partly transparent. Soft edges composite more forgivingly, but a ramp laid over a spill rim leaves a translucent green band, so use it after the tint is handled, not instead.
Edges that boil on video
The still frame looks fine and the outline crawls once it plays. Every frame is keyed with the same parameters and no knowledge of the previous one, so noise and grain push pixels back and forth across a single threshold. Thin detail suffers most, because hair, smoke and motion blur already sit near that threshold.
No measurement backs this one: the fixtures are single frames, so there is nothing to score across time. This section is the mechanism, not a number. What helps is widening the ramp, so a flipping pixel changes opacity instead of vanishing, and cleaner footage.
Telling them apart in one pass
- Rim dark, worst in shadow — tolerance too low. The tolerance page shows the units.
- Rim green or blue, subject intact — spill. Use the spill slider, not tolerance: see the spill page.
- Holes inside the subject — tolerance too high or too much trim.
- Staircase edge on a still frame — edge is 0. Add a small positive edge.
- Edge crawls only in playback — noise crossing a hard threshold. Soften it, clean the source.
- Only one side is bad — lighting, not parameters: the lighting guide.
Where the numbers come from
Every figure here is measured against ground truth. The reference set is six deterministic synthetic frames at 320×240 with hand-built masks, regenerated by a script so the tables can be diffed when the algorithm changes. A pixel counts as backdrop when the keyer gave it alpha 0; precision is the share of kept pixels that really are subject, recall is the share of backdrop actually removed, and IoU combines both.
They are synthetic fixtures, not real footage — read them as the shape of the problem, not a score your clip will get.
When this does not work
- The artifact is baked into footage you were given. A key subtracts a colour; it cannot reconstruct what a bad matte already destroyed.
- Motion blur. Those pixels are a genuine mix of subject and screen, and one global threshold cannot split them.
- Hair, fur, smoke and glass. They are semi-transparent by nature, and there is no matte refinement or per-pixel maturity here — just a distance and a threshold.
- Boiling edges. No temporal filter runs in this tool. Softening hides it; only an editor with frame-to-frame analysis removes it.
- A wrinkled or unevenly lit screen. The shading varies across the frame, which needs a threshold that varies too. One slider is global.
- The subject contains the key colour. A green shirt on a green screen is indistinguishable from the backdrop by construction.
- Heavily compressed source. Macroblocking sits exactly where the edge is, and no threshold lands cleanly on it.
- You want per-frame hand work. There is no brush and no rotoscope here. Colour is the only lever.
Find out which one you have
Open the tool, drop in the clip or the photo, and change one control at a time — tolerance, then edge, then spill — watching the same patch of outline. The video walkthrough covers the controls in order. The fixtures and benchmark behind every number on this site are in the reference repository, with the write-up archived at doi.org/10.5281/zenodo.22916767.