AI segmentation¶
SIV's segmentation models run in your browser, on your graphics card when possible. Images never leave your machine. See AI segmentation for every option and message.
Outline specific objects with box prompts (SAM)¶
- Zoom so the objects are clearly visible and fully on screen.
- Press R and draw a rectangle around each object, a little larger than the object.
- Click
Segment with SAM (box prompt).
Each rectangle that produced a mask is replaced by the object's outline (the largest connected piece). The others stay, so you can adjust them and run again. New regions are in the class sam.
Outline an object with a click (SAM point prompts)¶
- Click
SAM point prompts. The model loads.
- Click on the object. A preview region appears.
- Refine it: Shift-click (or Alt-click) on parts that should be excluded.
- Press Enter to keep it, or Esc to discard it.
- Click the next object; every plain click starts a new object.
Click the button again, or press S, to stop.
Segment every cell in an area with Cellpose¶
- Draw one or more rectangles (R) over the areas to segment.
- Click
Segment cells (cellpose-SAM).
Each rectangle is replaced by one region per detected cell, in the class cell. Rectangles with no cells found stay in place.
Tip
Cellpose processes the rectangle in tiles. Several medium rectangles are easier to review than one huge one, and a run can be undone in one step with Ctrl+Z.
Pick the right SAM model¶
- Click the next to Segment.
- Hover each to read the model's size, speed and training domain.
- Click a model to make it active (a check marks it).
| Your images | Choose |
|---|---|
| Fluorescence or brightfield microscopy, speed matters | micro-sam ViT-T (default) |
| Microscopy, accuracy matters | micro-sam ViT-B |
| H&E or other stained histology | patho-sam ViT-B |
| Histology on a slow connection | patho-sam ViT-B (int8) |
The choice applies to both SAM tools. It is not remembered after a reload.
Speed up repeated segmentation¶
- SAM encodes the image once and then answers each prompt quickly. Keep the view still between prompts: panning or zooming makes SAM encode again.
- The first use of a model downloads it, with a progress bar in the toast. It is then kept in the browser's cache, so later sessions start quickly.
- The larger models (ViT-B) use WebGPU when available; without it they are much slower.
Prepare for working offline or on a slow network¶
Run each model you need once while you have a good connection: open any image, draw a rectangle and run Segment with each model, and Cellpose once. The models stay in the browser cache. Clearing the browser's site data removes them.
Get better results¶
- Adjust the display first. The tools read the image as displayed, so improve contrast in Channels & Histogram before segmenting faint objects.
- Zoom in on small objects: SAM sees the on-screen pixels.
- Keep rectangles on screen. Only the visible part of a rectangle is segmented.
- Clean up with the vertex eraser or the brush, and remove false detections with Del.
- Simplify dense outlines: right-click → Simplify → Light (1 px).
Give segmented regions a meaningful class¶
Results are in the classes sam and cell. Select them (Ctrl-click or Shift-click their rows in the Region Editor, or Select all regions if the image has no other regions) and follow Relabel many regions at once.
Run a contributed tool¶
Applications can add their own tools (for example detection models in JAX Image Tools). They appear in the segmentation group with a run button, a model picker and Parameters. Unlike SAM and Cellpose, many of them process the whole current view, so there is nothing to draw first. Their documentation is part of the host application's guide.