instances - Instance label operations

The +utils/+instances package holds the algorithms that operate on instance label arrays - volumes in which every object carries its own integer index. They are plain array operations with no dependency on a network or a datastore, shared by DeepMIB prediction, the Instance editor and the Model ribbon tab: cross-slice stitching of 2D instances into 3D objects, splitting an index that covers several separate objects, removing noise objects, and building a per-object index for interactive editing.

utils.instances.cleanup(labelVol, options, wb)

CLEANUP - Remove noise objects from a 3D instance label volume.

Syntax:
labelVol = utils.instances.cleanup(labelVol)
[labelVol, stats] = utils.instances.cleanup(labelVol, options)
[labelVol, stats, cancelled] = utils.instances.cleanup(labelVol, options, wb)

The three post-processing filters of utils.instances.stitch2Dto3D, applied to a finished instance volume. They are here rather than inside the stitcher because the same cleanup is wanted after manual editing, and because the thresholds are dataset-specific enough that the user normally wants to try a few values without re-running the linking pass, which is far more expensive.

The order is fixed and matters:

  1. Absorb fragments - dust is given back to the object around it, so that a speck inside a real object fills a hole rather than punching one.

  2. Remove by voxel count and by occupied slices - what is left of the dust, plus large-in-plane false detections that do not propagate.

  3. Renumber the survivors, optionally.

Fragment absorption runs first precisely so that a speck lying inside a real object is returned to it, while a speck floating in the background - which has no neighbour to join - is still there for the size filters to delete.

Input Arguments:

  • labelVol - [height, width, depth] instance label volume (0 = background). Label values need not be contiguous.

  • options - (optional) structure of parameters:

    • .absorbFragmentVoxels - give the voxels of objects this size or smaller to the object surrounding them in-plane (default: 5, 0 = off). The only filter that is on by default, because unlike the two below it cannot remove anything - it moves voxels between objects. The default is deliberately equal to the stitcher’s minOverlapPixels: an object of fewer voxels than that can never produce a large enough intersection to be linked on any slice pair, so it reaches exactly the objects the linker is structurally unable to reach and nothing else

    • .minObjectVoxels - delete objects smaller than this (default: 0)

    • .minObjectSlices - delete objects occupying this many z-slices or fewer (default: 0; 1 drops single-slice objects). Complements minObjectVoxels: a 2D false positive can be large in-plane yet absent from the neighbouring slices, which an area threshold cannot see

    • .compact - renumber the surviving objects to a contiguous 1..K (default: true). false keeps every survivor’s label value, which is what interactive editing needs - a user who has been working with object 1299 should still find it under that number afterwards

    • .objectVoxelCounts - precomputed voxel count per label value, to skip one pass over the volume (default: [] = compute here)

  • wb - (optional) handle of a caller-owned uiprogressdlg created with 'Cancelable', 'on'. Message is written and CancelRequested polled; Value is left to the caller. Pass [] for none.

Output Arguments:

  • labelVol - the cleaned volume, demoted back to uint16 when the highest surviving label fits. Empty when cancelled.

  • stats - structure:

    • .objectVoxelCounts - voxels per object. Length K and aligned with the new labels when compact is true, otherwise full length and index-aligned with the original label values, zero where an object was removed

    • .objectSliceCounts - same alignment; empty unless minObjectSlices was used, since it is not otherwise computed

    • .numAbsorbedFragments / .numAbsorbedVoxels - what absorption moved

    • .numObjects - surviving object count

    • .numRemoved - objects deleted by the two size filters

    • .remap - [maxIndex x 1] old label value to new label value, 0 for a removed object. Empty when nothing was removed or renumbered

  • cancelled - logical. On cancel no partial volume is returned: a half-cleaned stack looks like a valid result and would be written over the user’s model.

Usage:

Example 1 - the recommended post-stitch cleanup

options.minObjectSlices = 1;      % drop single-slice detections
[labelVol, stats] = utils.instances.cleanup(labelVol, options);

Example 2 - cleanup of a hand-edited model, keeping the label values

options.compact = false;
options.minObjectVoxels = 50;
[labelVol, stats] = utils.instances.cleanup(labelVol, options, wb);

See also: utils.instances.stitch2Dto3D, utils.instances.objectIndex

utils.instances.objectIndex(labelVol, options, wb)

OBJECTINDEX - Build a per-object index (bounding box, size, centroid) of an instance label volume.

Syntax:
index = utils.instances.objectIndex(labelVol)
index = utils.instances.objectIndex(labelVol, options)
[index, cancelled] = utils.instances.objectIndex(labelVol, options, wb)

Instance models produced by utils.instances.stitch2Dto3D hold thousands of objects in a volume of several GB. Every question an interactive editor asks about a single object - where is it, how big is it, how many slices does it span, which index is free next - is a full-volume scan unless the answers are cached. This function builds that cache, and can refresh a part of it after an edit without rescanning the volume.

The returned arrays are index-aligned: entry k describes the object whose label value is k, so index.voxels(objId) is a direct lookup and no id-to-row mapping is needed. Entries for label values that are not present carry exists(k) == false and zeros elsewhere.

PixelIdxList is deliberately not stored - it is the one per-object quantity whose size grows with the data, and it is cheap to derive from the bounding box when a single object is actually operated on:

bb  = index.bbox(objId, :);
sub = labelVol(bb(1):bb(2), bb(3):bb(4), bb(5):bb(6));
idx = dataset.convertPixelIdxListCrop2Full(find(sub == objId), ...
        struct('y', bb(1:2), 'x', bb(3:4), 'z', bb(5:6)));
Input Arguments:

  • labelVol - [height, width, depth] numeric array of instance labels (0 = background). Any integer class; depth may be 1.

  • options - (optional) structure of parameters:

    • .index - an index returned by an earlier call, to be refreshed in place rather than rebuilt (default: [] = full build). Requires .objectIds. Fields of the passed struct that this function does not own (for example the caller’s .timePoint) are carried over untouched

    • .objectIds - vector of label values whose extent may have changed. Only these entries are recomputed. The caller must list every object the edit could have touched, before and after: an object left out keeps its stale bounding box, and acting on a stale box writes the wrong voxels

    • .bbox - [yMin yMax xMin xMax zMin zMax] region in which voxels changed (default: [] = the objects’ own previous boxes only). The rescan covers the union of this box and the previous boxes of objectIds, which is exactly where those objects can now have voxels: their old voxels were inside their old boxes, and new voxels can only be where the volume changed

    • .previousSlices - [height, width, zMax-zMin+1] labels of the whole slices bbox(5):bbox(6) as they were before the edit (default: []). Turns the refresh into a difference over those slices: what each object had on them before is taken out of its entry and what it has now is put in, and nothing outside them is read. For an edit confined to a few slices of objects that span the stack - every 2-D edit of an unstitched model - the union rule above would otherwise rescan nearly the whole volume. Voxel count, centroid and slice count stay exact; the bounding box only grows, see the note below. Requires .bbox

    • .computeSliceCount - fill .sliceCount (default: true). The only part of a full build that needs its own pass over the volume; set false when the count is not going to be read

  • wb - (optional) handle of a caller-owned uiprogressdlg created with 'Cancelable', 'on'. Its Message is updated and CancelRequested polled; Value is left to the caller, which may be scaling one bar over several volumes. Pass [] for no progress dialog.

Output Arguments:

  • index - structure with index-aligned arrays of length maxIndex:

    • .exists - [maxIndex x 1] logical, object k has at least one voxel

    • .voxels - [maxIndex x 1] uint32, voxel count of object k

    • .bbox - [maxIndex x 6] int32 [yMin yMax xMin xMax zMin zMax], inclusive and 1-based, ready to be used as options.y/.x/.z of getData3D / backup. All zero for an absent object

    • .centroid - [maxIndex x 3] single [x y z] - the regionprops ordering, so it can be passed straight to MibDataset.moveView(x, y)

    • .sliceCount - [maxIndex x 1] uint32, number of z-slices the object actually occupies (not its first-to-last span, so an object with a gap in the middle is judged on the slices it is really on). All zero when computeSliceCount was false

    • .maxIndex - highest label value the arrays are sized for

    • .numObjects - nnz(exists)

    Empty ([]) when the run was cancelled.

  • cancelled - logical, true when the user pressed Cancel. No partial index is returned: a half-filled index looks valid and its stale boxes would silently corrupt the next edit.

Note

The next free label value is find(~index.exists, 1), falling back to index.maxIndex + 1 when every value below the maximum is in use.

Note

A refresh never shrinks the index space. If an edit removes the highest label value, maxIndex stays where it was and the vacated entry becomes a free index; a full rebuild of the same volume would size its arrays to the new maximum instead and has no way of knowing the higher value ever existed. So a refreshed index equals a rebuilt one over the range they share, with the refreshed tail marked absent - not element for element.

Note

A refresh from previousSlices never shrinks a bounding box of an object that still has voxels outside those slices. That part of the object is known only by the box it already had, so when the edit removes the voxels that set a box edge, the box stays larger than the object until the next full build. This is safe - every reader uses the box as the bound of a search - but such an entry is not equal to a rebuilt one. An object that has nothing left outside the slices gets its exact box.

Usage:

Example 1 - full build with a cancelable progress dialog

wb = uiprogressdlg(parentFigure, 'Title', 'Indexing objects', ...
    'Indeterminate', 'on', 'Cancelable', 'on');
[index, cancelled] = utils.instances.objectIndex(labelVol, struct(), wb);
delete(wb);
if cancelled; return; end

Example 2 - refresh after merging object 12 into object 7

refresh.index = index;
refresh.objectIds = [7, 12];
index = utils.instances.objectIndex(labelVol, refresh);

Example 3 - bounding box of one object, as getData3D options

bb = double(index.bbox(objId, :));
getDataOptions.y = bb(1:2);  getDataOptions.x = bb(3:4);  getDataOptions.z = bb(5:6);
utils.instances.splitDisconnected(labelVol, options)

SPLITDISCONNECTED - give every connected component of an instance label its own index.

Syntax:
labelVol = utils.instances.splitDisconnected(labelVol)
[labelVol, stats] = utils.instances.splitDisconnected(labelVol, options)

An instance index must name exactly one object. Tiled prediction can break that: segmentObjects occasionally returns a single detection whose mask covers two neighbouring objects, and cross-tile stitching can group detections that do not belong together, so one index ends up painted over two spatially separate blobs. Splitting each label into its connected components repairs the count without touching anything else - the pixels keep their object, only the numbering changes.

The tiny leftovers this exposes (a few pixels shed by mask thresholding) are speckle rather than objects, so minObjectPixels discards them. This is the only step that removes pixels; with the default 0 nothing is lost.

Input Arguments:

  • labelVol - [height, width] or [height, width, depth] instance label array (0 = background). Label values need not be contiguous.

  • options - (optional) structure of parameters:

    • .connectivity - 8 (default) splits within each z-slice, which is what a stack of per-slice 2D predictions needs; 26 treats the volume as one 3D object space, for a model whose indices are already consistent across slices. Splitting a per-slice-numbered stack in 3D would wrongly fuse the unrelated objects that happen to share an index on neighbouring slices

    • .minObjectPixels - drop components smaller than this many pixels (default: 0, keep everything)

    • .perSliceNumbering - restart the numbering at 1 on every z-slice (default: false = one contiguous 1..N for the whole array). Only meaningful with connectivity 8; matching the convention of MibDeep.startPredictionInstances, which keeps instance indices per slice so the model type stays 65535 on deep stacks

Output Arguments:

  • labelVol - the relabelled array, uint16 when the highest index fits and uint32 otherwise. Indices are contiguous from 1 (per slice with perSliceNumbering)

  • stats - structure:

    • .numObjectsBefore / .numObjectsAfter - distinct indices in and out. With perSliceNumbering these are summed over the slices, so they count objects rather than index values

    • .numSplitLabels - indices that covered more than one component

    • .numExtraObjects - objects recovered, i.e. components beyond the first of a split index (before minObjectPixels is applied)

    • .numDroppedComponents / .numDroppedPixels - what minObjectPixels removed

Usage:

Example 1 - repair a stack of per-slice instance predictions

options.minObjectPixels = 100;
options.perSliceNumbering = true;
[labelVol, stats] = utils.instances.splitDisconnected(labelVol, options);

Example 2 - split a 3D instance model whose indices span the volume

[labelVol, stats] = utils.instances.splitDisconnected(labelVol, struct('connectivity', 26));

See also: utils.instances.cleanup, utils.instances.stitch2Dto3D, deepmib.segmentImageInstancesIoUMerge

utils.instances.stitch2Dto3D(inputVol, options, wb)

STITCH2DTO3D - Stitch per-slice 2D instance labels into a 3D instance volume.

Syntax:
labelVol = utils.instances.stitch2Dto3D(inputVol)
[labelVol, stats] = utils.instances.stitch2Dto3D(inputVol, options)
[labelVol, stats] = utils.instances.stitch2Dto3D(inputVol, options, wb)

Given a stack of independently generated 2D instance segmentations (one label map per z-slice, object IDs not consistent across slices), link objects that overlap between neighbouring slices into single 3D instances with one consistent ID through the whole stack. The input ID values are ignored - every slice is internally relabelled to globally-unique nodes, so the routine is safe on genuinely independent per-slice segmentations.

Two linking strategies are provided:

  • 'graph' (default) - build an undirected overlap graph over all slices (an edge whenever a pair of objects on adjacent slices passes the IoU or IoA test) and take connected components (union-find) as 3D instances. Splits and merges are handled natively; no separate reverse pass is needed because the graph is undirected.

  • 'hungarian' - the empanada-style pipeline: 1-to-1 IoU matching per slice pair via matchpairs, then IoA merge-in of the unmatched objects, optionally run forward and backward and reconciled.

Input Arguments:

  • inputVol - [height, width, depth] numeric array of per-slice instance labels (0 = background). Any integer class.

  • options - (optional) structure of parameters:

    • .method - 'graph' (default) or 'hungarian'

    • .splitDisconnected2D - treat each connected component of a per-slice label as its own 2D object, rather than the whole label index (default: true). 2D instance predictors regularly emit a single instance index covering several spatially separate blobs (a SOLOv2 mask head firing at more than one location, or a tile-merge that fused two detections). Such a label welds all of those blobs’ 3D chains into one object, and because the welds chain transitively across slices, a handful of them can fuse most of the stack into a single giant instance. Set to false only when the input indices are trusted and a genuinely disconnected 2D mask must stay one object

    • .iouThreshold - link objects whose IoU exceeds this (default: 0.25)

    • .ioaThreshold - link when intersection-over-smaller-area exceeds this, catching splits/thin bridges (default: 0.50)

    • .minOverlapPixels - absolute minimum intersection to consider a link, guards against 1-2 px spurious overlaps (default: 5)

    • .absOverlapPixels - link a pair whose intersection reaches this many pixels, whatever its IoU and IoA (default: 0 = disabled). Both ratio tests are relative to object area, so a large cross-section meeting a much smaller one scores low on each even when the shared area is substantial in absolute terms: 3795 px against 1689 px sharing 716 px is IoU 0.15 and IoA 0.42, below both defaults. This is a sufficient condition added to the IoU/IoA tests, unlike minOverlapPixels which is a necessary guard applied to all of them. It is an in-plane pixel count, so a sensible value depends on the objects’ size in this dataset - inspect a few genuine links before setting it, and keep the maxCentroidShift gate in mind, which still applies

    • .zLookback - also test slices up to this many planes apart, to bridge single-slice dropouts (default: 1 = adjacent only)

    • .minObjectVoxels - remove 3D objects smaller than this after stitching (default: 0 = keep all)

    • .minObjectSlices - remove 3D objects that appear on this many Z-slices or fewer (default: 0 = keep all; 1 drops single-slice objects, 2 also drops those seen on two slices). Complements minObjectVoxels: a false detection can be large in-plane yet not propagate through the stack, so an area threshold cannot see it while a depth threshold can. Counts the slices an object actually occupies, not its first-to-last span, so a zLookback bridge over a gap does not inflate the count

    • .absorbFragmentVoxels - after stitching, hand the voxels of any object of this size or smaller to the object that surrounds it in-plane, rather than leaving it as a separate speck (default: 5; 0 = off, and the default matches minOverlapPixels because that is exactly the size below which an object can never be linked at all). 2D instance predictors emit stray pixels - a couple of pixels of one index sitting inside another index’s mask, or shaved off its rim - and splitDisconnected2D correctly gives each its own object. Being smaller than minOverlapPixels they can never satisfy the link guard, so they survive stitching as unlinkable dust, typically a hole punched in an otherwise solid object. Lowering minOverlapPixels is not the cure: a speck lying over object A on one slice and under object B on the next then links both and welds two unrelated objects together. Reassigning voxels here runs after the linking is finished, so it cannot weld anything. A fragment with no labelled neighbour is left alone - use minObjectVoxels to delete those

    • .anisotropyZ - voxel aspect ratio pixSize.z / pixSize.x (>= 1). For anisotropic stacks (thick sections) a true continuation is displaced more between slices, so its IoU legitimately drops; the effective IoU threshold is lowered to max(iouThreshold / anisotropyZ, iouFloor). IoA (containment) is left unchanged - it is scale-robust - and the maxCentroidShift gate below guards against the relaxed IoU fusing distant objects (default: 1 = isotropic, no relaxation)

    • .iouFloor - lower clamp for the anisotropy-relaxed IoU threshold, so it never falls below a meaningful value (default: 0.05)

    • .maxCentroidShift - reject a link when the two objects’ centroids are more than this many pixels apart (scaled by the slice gap for zLookback > 1). Lets IoU be relaxed for anisotropy without letting far-apart objects merge (default: Inf = gate disabled)

    • .centroidLinkRadius - enable centroid-nearest-neighbour gap bridging ('graph' only). For objects that have no overlap partner on a slice pair, add a link to the mutually-nearest such orphan on the other slice when their centroids are within this many pixels (scaled by the slice gap). Reconnects a continuation that is laterally displaced or briefly absent - the residual split the overlap graph cannot see (default: 0 = disabled)

    • .centroidSizeRatio - a centroid-NN link additionally requires min(areaA,areaB)/max(areaA,areaB) to be at least this, so only comparably-sized objects are bridged (default: 0.5)

    • .bidirectional - for 'hungarian', also run a reverse pass and reconcile (default: true; ignored by 'graph')

    • .verbose - logical, print a short summary (default: false)

  • wb - (optional) handle of a caller-owned uiprogressdlg, created with 'Cancelable', 'on'. Its Message is updated with the current phase and slice counter, and CancelRequested is polled in every loop, so a stitch over a large stack can be interrupted. Pass [] for none. Value is left alone - the caller owns it, because it may be stitching several volumes and scaling the bar over all of them. This replaces the former options.showWaitbar, which could never work: it built a core.PoolWaitbar with an empty parent, which that class rejects

Output Arguments:

  • labelVol - [height, width, depth] relabelled 3D instance volume, IDs 1..K compacted, class uint16 (or uint32 if K > 65535). Empty when the user cancelled

  • stats - structure with .numInput2DObjects, .numOutput3DObjects, .objectVoxelCounts (K×1), .objectSliceCounts (K×1, empty unless minObjectSlices was used), .numAbsorbedFragments and .numAbsorbedVoxels (both 0 unless absorbFragmentVoxels was used), .cancelled (logical; when true nothing else in the structure is meaningful and labelVol is empty), .method, .options

Notes

  • Over-merging (one output object swallowing most of the stack) is almost always caused by per-slice labels whose pixels form several separate blobs - check with bwconncomp on a slice and compare the component count against numel(unique(slice(slice>0))). splitDisconnected2D (on by default) removes that failure mode.

  • Memory: the output volume is materialised in RAM (same footprint order as the input). The graph itself needs only a union-find array over the total 2D-object count plus two slices at a time.

  • matchpairs (used by 'hungarian') is a core MATLAB function and needs no toolbox.

Example 1 - stitch a folder of 2D label tiffs read into a volume:

files = dir(fullfile(folder, '*.tif'));
V = zeros(h, w, numel(files), 'uint16');
for z = 1:numel(files); V(:,:,z) = imread(fullfile(folder, files(z).name)); end
opt.iouThreshold = 0.25;
L = utils.instances.stitch2Dto3D(V, opt);

Example 2 - default one-liner on an in-memory stack, then inspect stats:

[L, stats] = utils.instances.stitch2Dto3D(V);
fprintf('%d 2D objects -> %d 3D instances\n', ...
    stats.numInput2DObjects, stats.numOutput3DObjects);
histogram(stats.objectVoxelCounts);   % 3D object size distribution

Example 3 - drop noise fragments (typical for a large, noisy stack such as an EM mitochondria volume):

opt = struct('minObjectVoxels', 200, 'verbose', true);
L = utils.instances.stitch2Dto3D(V, opt);   % objects < 200 voxels removed

A false detection can be large in-plane yet live on one slice only, which no voxel count will catch. Add a depth threshold for those:

opt.minObjectSlices = 2;   % also drop anything seen on 1 or 2 slices
L = utils.instances.stitch2Dto3D(V, opt);

Specks that sit inside a real object should be given back to it rather than deleted, which would leave a hole:

opt.absorbFragmentVoxels = 4;   % <=4-voxel objects join their neighbour
[L, stats] = utils.instances.stitch2Dto3D(V, opt);
fprintf('%d fragments absorbed (%d voxels)\n', ...
    stats.numAbsorbedFragments, stats.numAbsorbedVoxels);

Example 4 - bridge single-slice dropouts (an object that vanishes for one plane and reappears) by matching across a 2-slice gap:

opt.zLookback = 2;          % test slices z-1 and z-2 against z
opt.ioaThreshold = 0.4;     % looser containment test for thin bridges
L = utils.instances.stitch2Dto3D(V, opt);

Example 5 - faithful empanada-style pipeline (1-to-1 Hungarian matching + IoA merge-in, forward and reverse passes) for comparison against 'graph':

opt = struct('method', 'hungarian', 'bidirectional', true, ...
             'iouThreshold', 0.25, 'ioaThreshold', 0.5);
Lh = utils.instances.stitch2Dto3D(V, opt);

Example 6 - apply to the active MIB dataset’s labels layer (once wired into MIB, this is the intended call site):

id = obj.mibModel.getActiveId();
V  = obj.mibModel.getData3D('labels', [], 3, NaN, struct('id', id));
L  = utils.instances.stitch2Dto3D(V{1});
obj.mibModel.setData3D({L}, 'labels', [], 3, NaN, struct('id', id));
notify(obj.mibModel, 'ShowImage');

Example 7 - conservative linking (require a strong IoU, ignore weak containment) to keep touching-but-distinct objects separate:

opt = struct('iouThreshold', 0.5, 'ioaThreshold', 1.01, ...
             'minOverlapPixels', 20);   % ioaThreshold>1 disables IoA links
L = utils.instances.stitch2Dto3D(V, opt);

Example 8 - report progress and let the user stop a long stitch:

wb = uiprogressdlg(parentFigure, 'Title', 'Stitch 2D instances to 3D', ...
    'Indeterminate', 'on', 'Cancelable', 'on');
[L, stats] = utils.instances.stitch2Dto3D(V, opt, wb);
delete(wb);
if stats.cancelled; return; end   % L is empty, nothing was produced