Alpha matting: How software cuts out hair and glass without hard edges

Alpha matting is a way of cutting a subject out of a picture that keeps the soft parts. Instead of deciding that each pixel is either “person” or “background,” it gives every pixel a number for how much of it belongs to the subject. The result, called an alpha matte, lets wisps of hair or a glass edge blend into a new background.

The idea sits behind background removal in photo apps, video calls and film effects.

Why a hard outline makes hair look pasted on

Picture a portrait of someone with flyaway hair against a bright window. A simple cutout tool draws an outline and keeps everything inside. Place that person on a dark background and a pale halo appears around the head, or the fine strands vanish.

The reason is that pixels along a hair edge are mixed. One pixel might be 30% hair and 70% window. A yes-or-no outline must call it all hair or all window, and either answer is wrong.

That yes-or-no map is called a segmentation mask. Segmentation answers “which pixels belong to the object?” Matting answers a finer question: “how much of each pixel belongs to the object?”

One number per pixel, and a simple blending formula

The number is called alpha, and it runs from 0 to 1. A value of 0 means the pixel is fully background. A value of 1 means fully subject. A value of 0.3 means the pixel is a blend. Drawn as a grayscale image, the alpha matte looks like a white silhouette with soft gray edges.

Compositing, the step that places the subject on a new background, uses alpha as a mixing dial. The standard “over” formula, described by Thomas Porter and Tom Duff in a 1984 SIGGRAPH paper, blends the new pixel as roughly alpha times the subject plus (1 minus alpha) times the new background. Wikipedia credits the alpha channel idea to Alvy Ray Smith and Ed Catmull in the late 1970s. Smith has said the name comes from this blending formula.

The hard part is going backward. From a finished photo, a program must estimate alpha, the subject’s true color and the original background color at each edge pixel. That is more unknowns than the one measured color, so the problem is under-determined. Smith and Blinn showed in 1996 that an ideal solution for arbitrary footage is impossible, according to Wikipedia’s summary.

From green screens to hints called trimaps

Film used matting long before computers. Traveling mattes, dated by Wikipedia to about 1925, let actors move in front of a different background. Bluescreen and greenscreen work is the best-known modern form: shoot against one flat color and remove it. With a known background color, the unknowns shrink and the problem becomes manageable.

Without a screen, classic software needed help. A common aid is a trimap, a rough three-region map marking sure background, sure subject and an unknown band in between. The algorithm only has to resolve the unknown band, which is usually the hair or fur edge.

Drawing a trimap by hand is slow, and that held matting back for casual users.

Neural networks learn the edge from examples

In 2017, Ning Xu, Brian Price, Scott Cohen and Thomas Huang published “Deep Image Matting” at CVPR (a computer vision conference). Their network took an image plus a trimap and predicted the alpha matte. A second small network then sharpened edges. The authors argued that earlier methods relied on low-level color cues and lacked high-level context, which hurt when subject and background looked alike.

They also built a larger matting dataset, with 49,300 training images and 1,000 test images, according to the arXiv abstract. Later work reduced or removed the need for a trimap, so that a box, a click or a sentence could serve as the hint.

Matting from a text prompt: The Tencent example

Tencent’s WeChat Vision team released WeVisOne-VL-4B on Hugging Face, a vision-language model that handles detection, segmentation and matting in one system. We covered the release in our news story on Tencent’s 4B vision model. The model card lists “promptable alpha matting” as a pixel-level task, and the quick-start example calls task="matting" with the query "person".

According to the card, a separate file, mask_decoder.pt (about 9.6 million parameters), handles the pixel work. It includes a box-guided query decoder with six deformable cross-attention blocks and a head that refines predictions in steps to produce full-resolution alpha mattes. In plain terms, the language model finds the target, and a small add-on turns that location into a soft matte. This means one system can both understand “the person on the left” and cut them out.

Tencent’s README reports matting error as MAD (it does not define the abbreviation in the part we read), scaled by 1,000, where lower is better. For the 4B model it gives 7.3 on P3M-500-NP with box prompts against 15.4 for the best baseline it lists, and 9.8 on WeRefMat in the referring setting against 22.4. These are the vendor’s own figures and should be checked against independent tests.

Where matting still fails, and what to watch

Matting remains hard in the cases that make it useful. The WeVisOne-VL model card itself says fine alpha boundaries are sensitive to image resolution, severe occlusion, complex transparency and prompt phrasing. Smoke, wine glasses and motion blur are classic trouble spots, because the “background” shows through the subject.

Training data is another limit. Real matte labels are costly to make, so many datasets rely on composited images, and a model can learn the look of fake composites rather than real photos.

Licensing is the practical question for builders. Tencent states its own code and weights are under Apache License 2.0 unless otherwise noted, while third-party components keep their own licenses, so each piece needs checking before commercial use.

Things to watch: whether prompt-driven matting extends cleanly to video (the model card says the pixel decoder rejects video grids), whether independent benchmarks confirm vendor numbers, and whether small unified models like this one replace the separate detect-then-segment-then-matte pipelines used today.