What is Instance Segmentation?

There are a wide variety of instance segmentation models available. A popular choice is to use a YOLO model and fine-tune it for your use case. YOLO models such as YOLOv8 and YOLO11 are common choices because they are fast and, when fine-tuned with a high-quality dataset, accurate.

Fine-tuned models are most appropriate for edge deployment. In production, for example, you could run a YOLO model on a Jetson to monitor feeds from a camera positioned over an assembly line in real time.

Here is an example of a YOLO segmentation model that identifies debris in an image of coffee beans:

The purple highlighted regions correspond with stones that were found in the image, a defect that needs to be detected before coffee is packaged and sent to customers.

There is another category of models used for instance segmentation that are slower but allow for use without any prior training: zero-shot models.

Zero-shot segmentation models let you calculate segmentation masks around an object. An example of such a model is Meta's Segment Anything model series. With that said, zero-shot models often return only masks, not classes associated with an object. Thus, zero-shot models are commonly used in annotation as an assistant, where the zero-shot model can be used to draw a mask given a point or a bounding box.

Here is an example showing Meta's latest SAM-2.1 model segmenting regions in an image:

‍

All of the segmentation masks above, indicated by the different colours with which objects are filled in the images, were calculated by SAM-2.1 without specific training on those images.

Deploying Image Segmentation Models

You may be wondering: how practical is it to run image segmentation models in production? How do you start deploying a model? Great questions!

Image segmentation models can run at several FPS on modern hardware. For image segmentation model deployment on the edge, NVIDIA Jetsons and NVIDIA GPUs are common choice for hardware. You can also deploy your model on a Raspberry Pi, in the browser, and more.

For deploying a model to the edge, you can use Roboflow Inference, an open source computer vision deployment solution. With Inference, you can deploy your model to a wide range of devices without having to worry about dependency management and environment setup.

Inference lets you run your model through a Python SDK. You can also use the Python SDK with a Docker Inference container, ideal for projects where you want to containerize your model deployments.

Use the search engine below to explore image segmentation models across a range of use cases. This search engine is powered by Roboflow Universe, a community of over 200,000 computer vision datasets and 50,000 trained models.