QA: AI segmentation
Test cases for the in-browser segmentation tools: SAM box prompts, SAM point prompts, the model picker,
Cellpose, progress toasts and model caching. See How to use the plan. Reference:
AI segmentation.
Downloads
The first run of each model downloads it: micro-sam ViT-T about 14 MB, ViT-B about 172 MB, patho-sam about
180 MB (int8 about 100 MB), Cellpose-SAM about 588 MB. Use a fast connection, and clear the site's data (or
use a private window) for the cases that test the first download.
Unless a case says otherwise, use img04.png (fluorescent cells) in the Image view.
Availability
| ID |
Title |
Pri |
Preconditions / data |
Steps |
Expected result |
| SEG-001 |
Segmentation buttons |
P1 |
img04.png |
1. Hover each button after the region tools. |
Segment "Segment the drawn rectangles with SAM (box prompt)", SAM point prompts (tooltip about clicking an object, Shift/Alt-click to exclude, Esc undoes), chevron "Pick the SAM model (used by both segment tools)", Cellpose "Auto-segment cells inside each drawn rectangle (cellpose-SAM)". |
| SEG-002 |
Point-prompt tooltip wording |
P3 |
img04.png |
1. Hover SAM point prompts. |
Record the wording (it says "click a fiber"). ⚠ see Known issues (KI-34). |
| SEG-003 |
Shown with the full region tools |
P2 |
micro-ct z-stack |
1. Image view. 2. Volume view. 3. Open the slice editor. |
Shown in Image; hidden in Volume with the slice editor closed; shown again with it open. |
| SEG-004 |
Shown in Heatmap |
P3 |
img04.png, Heatmap |
1. Look at the toolbar. |
The segmentation group is shown. |
| SEG-005 |
Contributed tools (JIT) |
P2 |
JIT with YOLO / retinal-layer tools |
1. Look at the toolbar. |
Each contributed tool has a run button, a model chevron ("Pick the model") and a Parameters button (sliders icon). Not present in the demo. |
Model picker
| ID |
Title |
Pri |
Preconditions / data |
Steps |
Expected result |
| SEG-020 |
Models listed |
P1 |
|
1. Click the chevron. |
micro-sam ViT-T (checked, default), micro-sam ViT-B, patho-sam ViT-B, patho-sam ViT-B (int8), each with an ⓘ button. |
| SEG-021 |
Model info |
P2 |
Picker open |
1. Hover (or focus) each ⓘ. |
Each shows the model's size, speed, training domain and accuracy (ViT-T ~14 MB light microscopy; ViT-B ~172 MB "Highest accuracy"; patho-sam ~180 MB H&E; int8 ~100 MB "Pick it on slow connections"). |
| SEG-022 |
Switch model |
P2 |
Picker open |
1. Pick micro-sam ViT-B. 2. Reopen the picker. |
ViT-B is checked; both the box and point tools use it (next run downloads and encodes once for both). |
| SEG-023 |
Model choice not persisted |
P3 |
ViT-B picked |
1. Reload the page. |
ViT-T is the default again (unless the host configured another default). |
| SEG-024 |
Picker keyboard access |
P3 |
|
1. Tab to the chevron, press Enter. 2. Tab to an ⓘ. |
The menu opens; ⓘ buttons are focusable and announce their model. |
SAM box prompt
| ID |
Title |
Pri |
Preconditions / data |
Steps |
Expected result |
| SEG-040 |
Segment three cells |
P1 |
img04.png |
1. Draw three rectangles (R) around three cells. 2. Click Segment. |
A "SAM" toast shows "Encoding image…", "Segmenting 1/3…" … then "Added 3 region(s).". Each rectangle is replaced by a mask region of class "sam". |
| SEG-041 |
Colour of SAM regions |
P2 |
After SEG-040, rectangles of different colours |
1. Compare colours. |
Record the colours. The help promised the rectangle's colour; the code applies the class colour for "sam". ⚠ see Known issues (KI-32). |
| SEG-042 |
No rectangles |
P2 |
No regions |
1. Click Segment. |
Message "Draw one or more rectangles, then press Segment." |
| SEG-043 |
Box with nothing in it |
P3 |
A rectangle on empty background |
1. Segment. |
"No masks found."; the rectangle stays so you can retry. |
| SEG-044 |
Failed prompts stay |
P3 |
One good and one empty rectangle |
1. Segment. |
The good rectangle is replaced; the empty one stays. |
| SEG-045 |
Encoder reused |
P3 |
After SEG-040 |
1. Draw another rectangle without moving the view. 2. Segment. |
No "Encoding image…" step this time (the encoding is reused for the same image and view). |
| SEG-046 |
Re-encode after pan/zoom |
P3 |
After SEG-040 |
1. Zoom in. 2. Draw a rectangle. 3. Segment. |
"Encoding image…" shows again. |
| SEG-047 |
Only the visible part is segmented |
P3 |
A rectangle partly outside the view |
1. Pan so half the rectangle is off-screen. 2. Segment. |
Record behaviour: only the on-screen part is used. ⚠ see Known issues (KI-33). |
| SEG-048 |
Press while busy |
P3 |
During a segmentation |
1. Click Segment again. |
Ignored; no second run. |
| SEG-049 |
Undo a segmentation |
P2 |
After SEG-040 |
1. Press Ctrl+Z. |
The segmentation is undone (the rectangles are back). |
| SEG-050 |
patho-sam on H&E |
P2 |
WSI CMU-1; patho-sam ViT-B picked |
1. Draw rectangles around nuclei or glands. 2. Segment. |
Masks fit the stained structures. |
| SEG-051 |
Works in Heatmap |
P3 |
img04.png, Heatmap |
1. Draw a rectangle. 2. Segment. |
A mask region is added. |
SAM point prompts
| ID |
Title |
Pri |
Preconditions / data |
Steps |
Expected result |
| SEG-060 |
Arm the point tool |
P1 |
img04.png |
1. Click SAM point prompts. |
The button is highlighted; "Loading SAM model…" shows while the model preloads. |
| SEG-061 |
Click an object |
P1 |
Point tool armed |
1. Click a cell. |
"Encoding image…", "Segmenting…", then a preview region over the cell and the message "Segmented — click another fiber for a new region, Shift-click to refine, Esc to undo." |
| SEG-062 |
Each click starts a new object |
P1 |
After SEG-061 |
1. Click another cell. |
A second, separate region is created; the first is not grown. |
| SEG-063 |
Exclude point |
P2 |
A preview region |
1. Shift-click (or Alt-click) part of it you want out. |
The current object is refined, excluding that area. |
| SEG-064 |
Commit |
P2 |
A preview region |
1. Press Enter. |
The object is committed (class "sam"). |
| SEG-065 |
Undo the current object |
P2 |
A preview region |
1. Press Esc. |
The current preview region is removed. |
| SEG-066 |
No mask |
P3 |
Point tool armed |
1. Click empty background. |
"No mask for these points." |
| SEG-067 |
Clicks while busy |
P3 |
During "Segmenting…" |
1. Click again quickly. |
The extra click is ignored. |
| SEG-068 |
Shares the model with the box tool |
P3 |
After SEG-061 |
1. Draw a rectangle without moving. 2. Segment. |
No new download or encoding. |
Cellpose
| ID |
Title |
Pri |
Preconditions / data |
Steps |
Expected result |
| SEG-080 |
Segment every cell in a box |
P1 |
002_img.png |
1. Draw one large rectangle. 2. Click Cellpose. |
A "Cellpose" toast steps through "Box 1/1: Cropping…", "Downloading Cellpose-SAM model…" (first time), "Preprocessing image…", "Running inference (tile d/t)…", "Computing flow dynamics…", "Tracing cells…", then "Added N cell region(s).". One region per cell, class "cell"; the rectangle is removed. |
| SEG-081 |
Several boxes |
P2 |
002_img.png |
1. Draw two rectangles. 2. Cellpose. |
Both are processed ("Box 1/2", "Box 2/2"). |
| SEG-082 |
No rectangles |
P2 |
No regions |
1. Click Cellpose. |
"Draw one or more rectangles, then run Cellpose." |
| SEG-083 |
No cells |
P3 |
A rectangle on background |
1. Cellpose. |
"No cells found."; the rectangle stays. |
| SEG-084 |
Cell colours |
P3 |
After SEG-080 |
1. Compare cell colours with the rectangle. |
Record the colour. ⚠ see Known issues (KI-32). |
| SEG-085 |
Retry after a failed load |
P3 |
Block the model URL in DevTools |
1. Run Cellpose (fails: "Cellpose segmentation failed."). 2. Unblock. 3. Run again. |
The second run downloads and works. |
Toasts and caching
| ID |
Title |
Pri |
Preconditions / data |
Steps |
Expected result |
| SEG-100 |
Progress toast during download |
P1 |
Site data cleared |
1. Draw a rectangle, click Segment. |
A sticky bottom-right "SAM" toast with the live status and a determinate download bar, then an indeterminate busy bar. |
| SEG-101 |
Result toast |
P2 |
After a run |
1. Watch the toast. |
Success style when regions were added; warning style when nothing was found, with the last status as detail. |
| SEG-102 |
Failure toast |
P3 |
Block the model URL |
1. Segment. |
Summary "SAM failed" with the error (for example "fetch URL failed: HTTP 404" or "SAM model unavailable."). |
| SEG-103 |
Model cached |
P2 |
After SEG-100 |
1. Reload the page. 2. Segment again. |
No download step; the model loads from the browser cache (Cache Storage sam-onnx-v2 in DevTools → Application). |
| SEG-104 |
Point segmentation toast title |
P3 |
Point tool |
1. Click a cell. |
The toast summary is "SAM point segmentation". |
| SEG-105 |
Works without WebGPU |
P2 |
WebGPU off |
1. With ViT-T, segment a rectangle. 2. With ViT-B, segment again. |
ViT-T (WASM) works. Record ViT-B's behaviour (WebGPU model; expected to fall back or report an error rather than hang). |