Skip to content

QA: AI segmentation

Test cases for the in-browser segmentation tools: SAM box prompts, SAM point prompts, the model picker, Cellpose, progress toasts and model caching. See How to use the plan. Reference: AI segmentation.

Downloads

The first run of each model downloads it: micro-sam ViT-T about 14 MB, ViT-B about 172 MB, patho-sam about 180 MB (int8 about 100 MB), Cellpose-SAM about 588 MB. Use a fast connection, and clear the site's data (or use a private window) for the cases that test the first download.

Unless a case says otherwise, use img04.png (fluorescent cells) in the Image view.

Availability

ID Title Pri Preconditions / data Steps Expected result
SEG-001 Segmentation buttons P1 img04.png 1. Hover each button after the region tools. Segment "Segment the drawn rectangles with SAM (box prompt)", SAM point prompts (tooltip about clicking an object, Shift/Alt-click to exclude, Esc undoes), chevron "Pick the SAM model (used by both segment tools)", Cellpose "Auto-segment cells inside each drawn rectangle (cellpose-SAM)".
SEG-002 Point-prompt tooltip wording P3 img04.png 1. Hover SAM point prompts. Record the wording (it says "click a fiber"). ⚠ see Known issues (KI-34).
SEG-003 Shown with the full region tools P2 micro-ct z-stack 1. Image view. 2. Volume view. 3. Open the slice editor. Shown in Image; hidden in Volume with the slice editor closed; shown again with it open.
SEG-004 Shown in Heatmap P3 img04.png, Heatmap 1. Look at the toolbar. The segmentation group is shown.
SEG-005 Contributed tools (JIT) P2 JIT with YOLO / retinal-layer tools 1. Look at the toolbar. Each contributed tool has a run button, a model chevron ("Pick the model") and a Parameters button (sliders icon). Not present in the demo.

Model picker

ID Title Pri Preconditions / data Steps Expected result
SEG-020 Models listed P1 1. Click the chevron. micro-sam ViT-T (checked, default), micro-sam ViT-B, patho-sam ViT-B, patho-sam ViT-B (int8), each with an ⓘ button.
SEG-021 Model info P2 Picker open 1. Hover (or focus) each ⓘ. Each shows the model's size, speed, training domain and accuracy (ViT-T ~14 MB light microscopy; ViT-B ~172 MB "Highest accuracy"; patho-sam ~180 MB H&E; int8 ~100 MB "Pick it on slow connections").
SEG-022 Switch model P2 Picker open 1. Pick micro-sam ViT-B. 2. Reopen the picker. ViT-B is checked; both the box and point tools use it (next run downloads and encodes once for both).
SEG-023 Model choice not persisted P3 ViT-B picked 1. Reload the page. ViT-T is the default again (unless the host configured another default).
SEG-024 Picker keyboard access P3 1. Tab to the chevron, press Enter. 2. Tab to an ⓘ. The menu opens; ⓘ buttons are focusable and announce their model.

SAM box prompt

ID Title Pri Preconditions / data Steps Expected result
SEG-040 Segment three cells P1 img04.png 1. Draw three rectangles (R) around three cells. 2. Click Segment. A "SAM" toast shows "Encoding image…", "Segmenting 1/3…" … then "Added 3 region(s).". Each rectangle is replaced by a mask region of class "sam".
SEG-041 Colour of SAM regions P2 After SEG-040, rectangles of different colours 1. Compare colours. Record the colours. The help promised the rectangle's colour; the code applies the class colour for "sam". ⚠ see Known issues (KI-32).
SEG-042 No rectangles P2 No regions 1. Click Segment. Message "Draw one or more rectangles, then press Segment."
SEG-043 Box with nothing in it P3 A rectangle on empty background 1. Segment. "No masks found."; the rectangle stays so you can retry.
SEG-044 Failed prompts stay P3 One good and one empty rectangle 1. Segment. The good rectangle is replaced; the empty one stays.
SEG-045 Encoder reused P3 After SEG-040 1. Draw another rectangle without moving the view. 2. Segment. No "Encoding image…" step this time (the encoding is reused for the same image and view).
SEG-046 Re-encode after pan/zoom P3 After SEG-040 1. Zoom in. 2. Draw a rectangle. 3. Segment. "Encoding image…" shows again.
SEG-047 Only the visible part is segmented P3 A rectangle partly outside the view 1. Pan so half the rectangle is off-screen. 2. Segment. Record behaviour: only the on-screen part is used. ⚠ see Known issues (KI-33).
SEG-048 Press while busy P3 During a segmentation 1. Click Segment again. Ignored; no second run.
SEG-049 Undo a segmentation P2 After SEG-040 1. Press Ctrl+Z. The segmentation is undone (the rectangles are back).
SEG-050 patho-sam on H&E P2 WSI CMU-1; patho-sam ViT-B picked 1. Draw rectangles around nuclei or glands. 2. Segment. Masks fit the stained structures.
SEG-051 Works in Heatmap P3 img04.png, Heatmap 1. Draw a rectangle. 2. Segment. A mask region is added.

SAM point prompts

ID Title Pri Preconditions / data Steps Expected result
SEG-060 Arm the point tool P1 img04.png 1. Click SAM point prompts. The button is highlighted; "Loading SAM model…" shows while the model preloads.
SEG-061 Click an object P1 Point tool armed 1. Click a cell. "Encoding image…", "Segmenting…", then a preview region over the cell and the message "Segmented — click another fiber for a new region, Shift-click to refine, Esc to undo."
SEG-062 Each click starts a new object P1 After SEG-061 1. Click another cell. A second, separate region is created; the first is not grown.
SEG-063 Exclude point P2 A preview region 1. Shift-click (or Alt-click) part of it you want out. The current object is refined, excluding that area.
SEG-064 Commit P2 A preview region 1. Press Enter. The object is committed (class "sam").
SEG-065 Undo the current object P2 A preview region 1. Press Esc. The current preview region is removed.
SEG-066 No mask P3 Point tool armed 1. Click empty background. "No mask for these points."
SEG-067 Clicks while busy P3 During "Segmenting…" 1. Click again quickly. The extra click is ignored.
SEG-068 Shares the model with the box tool P3 After SEG-061 1. Draw a rectangle without moving. 2. Segment. No new download or encoding.

Cellpose

ID Title Pri Preconditions / data Steps Expected result
SEG-080 Segment every cell in a box P1 002_img.png 1. Draw one large rectangle. 2. Click Cellpose. A "Cellpose" toast steps through "Box 1/1: Cropping…", "Downloading Cellpose-SAM model…" (first time), "Preprocessing image…", "Running inference (tile d/t)…", "Computing flow dynamics…", "Tracing cells…", then "Added N cell region(s).". One region per cell, class "cell"; the rectangle is removed.
SEG-081 Several boxes P2 002_img.png 1. Draw two rectangles. 2. Cellpose. Both are processed ("Box 1/2", "Box 2/2").
SEG-082 No rectangles P2 No regions 1. Click Cellpose. "Draw one or more rectangles, then run Cellpose."
SEG-083 No cells P3 A rectangle on background 1. Cellpose. "No cells found."; the rectangle stays.
SEG-084 Cell colours P3 After SEG-080 1. Compare cell colours with the rectangle. Record the colour. ⚠ see Known issues (KI-32).
SEG-085 Retry after a failed load P3 Block the model URL in DevTools 1. Run Cellpose (fails: "Cellpose segmentation failed."). 2. Unblock. 3. Run again. The second run downloads and works.

Toasts and caching

ID Title Pri Preconditions / data Steps Expected result
SEG-100 Progress toast during download P1 Site data cleared 1. Draw a rectangle, click Segment. A sticky bottom-right "SAM" toast with the live status and a determinate download bar, then an indeterminate busy bar.
SEG-101 Result toast P2 After a run 1. Watch the toast. Success style when regions were added; warning style when nothing was found, with the last status as detail.
SEG-102 Failure toast P3 Block the model URL 1. Segment. Summary "SAM failed" with the error (for example "fetch URL failed: HTTP 404" or "SAM model unavailable.").
SEG-103 Model cached P2 After SEG-100 1. Reload the page. 2. Segment again. No download step; the model loads from the browser cache (Cache Storage sam-onnx-v2 in DevTools → Application).
SEG-104 Point segmentation toast title P3 Point tool 1. Click a cell. The toast summary is "SAM point segmentation".
SEG-105 Works without WebGPU P2 WebGPU off 1. With ViT-T, segment a rectangle. 2. With ViT-B, segment again. ViT-T (WASM) works. Record ViT-B's behaviour (WebGPU model; expected to fall back or report an error rather than hang).