High-quality image segmentation has long been a challenge, especially when dealing with high-resolution images where details matter most. Enter BiRefNet (Bilateral Reference Network), a powerful new open-source project that is pushing the boundaries of what's possible in dichotomous image segmentation (DIS).
What is BiRefNet?
BiRefNet is the official implementation of the paper "Bilateral Reference for High-Resolution Dichotomous Image Segmentation" (CAAI AIR 2024). It is designed to tackle the specific challenge of segmenting objects in high-resolution images (2K, 4K, and beyond) with extreme precision.
Unlike traditional models that often struggle to balance global context with fine local details, BiRefNet employs a "Bilateral Reference" strategy. This allows the model to simultaneously maintain a holistic understanding of the image while accurately delineating intricate boundaries, such as hair strands or complex geometric shapes.
Key Capabilities
The project is highly versatile and comes with pre-trained weights for various tasks:
- High-Resolution Segmentation: Capable of processing 2K and 4K images without losing detail.
- Matting: Excellent performance on general matting tasks, providing high-quality alpha mattes.
- Dichotomous Image Segmentation (DIS): The core task it was designed for, separating foreground from background with high accuracy.
- Camouflaged Object Detection (COD): identifying objects that blend into their surroundings.
- Salient Object Detection (SOD): Finding the most important objects in a scene.
Practical Use Cases
BiRefNet's high precision makes it suitable for a wide range of professional applications:
- Automated Background Removal: Perfect for e-commerce platforms requiring clean product shots or graphic design tools.
- Photo Editing & Creative Suites: empowering "select subject" tools with professional-grade accuracy.
- Video Production: Generating high-quality mattes for compositing and visual effects.
- Medical Analysis: assisting in the precise segmentation of anatomical structures in high-resolution medical scans.
- Industrial Inspection: Detecting defects or specific components in high-resolution manufacturing imagery.
Getting Started
If you're interested in trying out BiRefNet, the project is open-source and available on GitHub: https://github.com/ZhengPeng7/BiRefNet
Quick Inference
The authors have made it easy to use via Hugging Face. You can load the model with just a few lines of code.
from transformers import AutoModelForImageSegmentation
import torch
from torchvision import transforms
from PIL import Image
# Load Model
model = AutoModelForImageSegmentation.from_pretrained("ZhengPeng7/BiRefNet", trust_remote_code=True)
torch.set_float32_matmul_precision(['high', 'highest'][0])
model.to("cuda")
model.eval()For more detailed examples, inference scripts, and access to the model zoo, be sure to check out the official repository.
Credits
This project is the work of Peng Zheng, Dehong Gao, Deng-Ping Fan, Li Liu, Jorma Laaksonen, Wanli Ouyang, and Nicu Sebe.
If you use BiRefNet in your research, please consider citing their paper: "Bilateral Reference for High-Resolution Dichotomous Image Segmentation" (CAAI AIR 2024).