Skip to content

Segment Anything Model

Back to MIB | User interface | Panels | Segmentation

Overview

SAM Tool

Uses Segment-anything model (SAM1 or SAM2) for object segmentation with one or few mouse clicks.

General information and references

Developed by Meta AI Research, FAIR, SAM segments objects or entire images with minimal clicks.

Implemented in MIB via an external Python interpreter. See Requirements and installation.

Warning

GPU is recommended (30–60x faster than CPU).

SAM-1 Models in MIB

Original SAM-1 models:

  • vit_b (0.4Gb): fastest (x1), less precise.
  • vit_l (1.2Gb): moderate speed (~x1.4 slower), better predictions.
  • vit_h (2.5Gb): slowest (x2.0 slower), best predictions.

Reference

Reference: Kirillov et al., Segment Anything, arXiv:2304.02643.

Microscopy extensions:

  • vit_b Electron Microscopy (0.4Gb): fast, for EM datasets.
  • vit_b Light Microscopy (0.4Gb): fast, for LM datasets.
  • vit_h Electron Microscopy (2.5Gb): best results, for EM.
  • vit_h Light Microscopy (2.5Gb): best results, for LM.
  • vit_b for EM Generalist (0.4Gb): pretrained for mitochondria in EM.
  • vit_b for EM Boundaries (0.4Gb): pretrained for membrane organelles in EM.

Reference

Archit et al., Segment Anything for Microscopy, bioRxiv.

SAM-2 Models in MIB

Original SAM-2.1 models:
- sam2.1_hiera_tiny (0.15Gb)
- sam2.1_hiera_small (0.18Gb)
- sam2.1_hiera_base_plus (0.32Gb)
- sam2.1_hiera_large (0.90Gb)

Original SAM-2.0 models:
- sam2_hiera_tiny (0.15Gb)
- sam2_hiera_small (0.18Gb)
- sam2_hiera_base_plus (0.32Gb)
- sam2_hiera_large (0.90Gb)

Reference

Ravi et al., SAM 2: Segment Anything in Images and Videos, arXiv:2408.00714.

Requirements and installation instructions

SAM requires installation:

Info

SAM networks are automatically downloaded, connected, and initialized in MIB.
Customize via sam_links.json (SAM-1) or sam2_links.json (SAM-2) in the MIB/Resources subfolder, or specify a new location in SAM settings available within the SAM panel. SAM Tool

How to use

SAM segmentation in MIB has 4 options

Interactive

SAM-2, Interactive

Default mode, segments the visible image area:

  • Destination: choose selection, mask, or model for results.
  • Mode:
    • Replace: replace destination layer objects with new results.
    • Add: add results to the destination layer.
    • Subtract: subtract results from the destination layer.
    • Add, +next material: add to the selected material and create a new one (for 65535+ models, model or selection only).
  • : specify a point on an object to segment.
    • Shift + : expand the object (positive seed).
    • Ctrl + : constrain the object from the clicked area (negative seed).
  • Dataset: 3D, Stack: perform 3D segmentation by clicking on one slice, scrolling, and using Shift + . Interpolates seeds (positive/negative clicks) across slices.

Warning

  • Note 1: Not available for SAM1.
  • Note 2: It is recommended to use the Interactive 3D mode instead as it is faster and more intelligent

Interactive 3D

SAM-2, Interactive 3D

Best for 3D segmentation:

  • Mode: same as above.
  • : start an object.
  • Shift + : expand the object (positive seed).
  • Ctrl + : constrain the object from the clicked area (negative seed).
  • Scroll to another slice, use Shift + to segment between slices.

Warning

  • Note: Only the image area visible in the Image View panel is processed.

Landmarks

SAM-2, Landmarks

Process full images using annotations:

  • Dataset: select stack portion.
  • Destination and Mode: define layer and mode (see Interactive).

  • Use to add annotations:

  • Value=1 for object, positive seed;

  • Value=0 for background, negative seed).
Tips
  • Use Annotations -> Focus on Value to prioritize value.
  • Uncheck Show prompt to skip the dialog.
  • Ctrl + : remove the closest annotation.
  • Access the list with .
  • Press Segment to segment.
  • Use Clear annotations to remove landmarks after successive segmentation

Automatic everything

SAM-2, Automatic everything

Automatically segments all objects into a new model:

  • Dataset: select stack portion.
  • Press Segment to segment.

SAM settings

Press to open settings.

Segment anything model settings

Key parameters:
  • Backbone: choose a pretrained backbone (see General information).
  • Execution environment: select cuda (GPU) or CPU (slower).
  • Show the progress bar in the interactive mode: toggle progress bar.
  • PATH to segment-anything installation: set the unzipped package location
    (use Check to select path for a dialog).

Presets

Use the following key shortcuts to define and restore presets

  • Shift+1, Shift+2, Shift+3 - store preset 1, 2, or 3 correspondingly
  • 1, 2, 3 - restore preset 1, 2, or 3 correspondingly

Back to MIB | User interface | Panels | Segmentation