MibBigDataLabels

class core.MibBigDataLabels

Bases: core.@MibLabels63.MibLabels63

MIBBIGDATALABELS - disk-backed, packed 63-class segmentation labels for BigData datasets.

Subclass of core.MibLabels63. Keeps the identical bit-packing scheme (bits 1-6 = material 0-63, bit 7 = mask, bit 8 = selection) but stores the packed uint8 data in a writable multi-resolution zarr pyramid on disk (one level per image pyramid level), so models for datasets too large for memory can be segmented and persisted.

How it integrates. MibImage.getData/setData route non-image layers of a MibLabels63 to getData63/setData63; this class overrides those to read/modify/write the on-disk pyramid. Because the seam is exactly getData63/setData63, all of MibDataset.getData2D/3D/4D, setData2D/... and every segmentation tool work unchanged.

Pyramid. The model mirrors the image pyramid: same number of levels, same per-level sizes (levelImageSizes) and scale factors, chunk-aligned to the image. getData63 reads the level matching options.magFactor and resizes to the displayed resolution exactly like the image reader (MibVirtualImage.getDataZarr), so image and model always line up. setData63 writes the edited region only at the working level and all COARSER levels (cheap downsample, preserving packed bytes), and records the working level in the per-tile level map (matLevel). Finer levels are left dirty and reconstructed on demand: getData63 recomputes a finer tile from its matLevel source (no coarse-echo halo), writes it into the level (materialize) and caches it. saveLevelMap/loadLevelMap persist the map in a side-file; materializeAll (Save) finalizes every level. See development/bigdata/bigdata_logic.md.

Axis order. Levels are created with ZarrArray → they round-trip in native MATLAB [y, x, z] order, no permutation. That holds in both zarr formats: v3 gets a transpose codec, v2 gets Fortran chunk order, which is the v2 spelling of the same thing. obj.data stays empty; dimensions come from level 0.

Zarr format. The store is v3 by default and v2 when the path ends in .zarr2 (createStore / zarrFormatFromPath); both are fully editable, and everything above applies unchanged to either. A store MIB created carries the mibModelStore marker - a v2 store WITHOUT it was written by another tool and is handled read-only by core.MibBigDataLabelsZarr2 instead.

Constructor Summary
MibBigDataLabels(img, meta)

MIBBIGDATALABELS - Construct an empty disk-backed label container.

Syntax:
obj = core.MibBigDataLabels([], meta)

Creates the in-memory object with no pyramid store attached. The on-disk store is allocated separately by createStore (new dataset) or openStore (existing model). Until one of those is called, obj.exists == false and all reads/writes are no-ops.

Input Arguments:

  • img (optional) - [empty] pass []; pixel data is never stored in memory for BigData labels.

  • meta (optional) - [dictionary] metadata dictionary used by the parent core.MibLabels63 constructor to set dimensions. Default: empty MibImage info.

Example - create a fresh labels object and attach a pyramid store:

meta = core.MibImage.initializeImgInfo();
lb   = core.MibBigDataLabels([], meta);
lb.createStore([4096 4096 100], 'C:\data\model.zarr3', pyramid);
Property Summary
Compressors

[nLevels x 3] per-level [yScale, xScale, zScale] relative to level 0.

levelMapPath

[uint8 [coarseY x coarseX x coarseZ]] per coarsest-grid tile, the FINEST pyramid level index holding materialized data (0 = empty). Finer = smaller index; level 1 = full resolution; level N = coarsest. An edit writes the working level + coarser and sets matLevel = working level; finer levels are implicitly dirty and recomputed on read (getData63) or at Save (materializeAll).

matLevel

— multi-resolution “level map” (see setData63 / getData63 / materializeAll) —

modelArrays

[string] zarr group root path of the packed model pyramid.

modelLevelNames

{1 x nLevels} io.zarr.Array handles, one per resolution level (finest first).

modelLevelSizes

{1 x nLevels} relative level paths within the group (‘0’,’1’,…).

modelScaleFactors

[nLevels x 3] per-level [y, x, z] size.

modelStorePath
selectionBBoxFull

[char] side-file path (‘<storePath>.levelmap’) persisting matLevel.

Method Summary
static bboxUnion(a, b)

BBOXUNION - union of two [y0 y1 x0 x1 z0 z1] boxes ([] acts as empty).

static clampRange(r, n)

CLAMPRANGE - clamp a [lo hi] index range to [1, n] and keep it ascending. Guards against invalid zarr bboxes (e.g. when a loaded model’s level is smaller than the requested region).

closeStore()

CLOSESTORE - release level handles (data remains on disk). Marks the labels as non-existent so reads/writes become no-ops until a store is (re)opened. Persists the level map first so a reopened model knows which finer levels are still virtual.

createStore(dims, storePath, pyramid, zarrFormat)

CREATESTORE - Allocate a zero-filled packed label pyramid on disk.

Syntax:
obj.createStore(dims)
obj.createStore(dims, storePath)
obj.createStore(dims, storePath, pyramid)
obj.createStore(dims, storePath, pyramid, zarrFormat)

Creates a new zarr group at storePath with one uint8 array per pyramid level (bits 1-6 = material 0-63, bit 7 = mask, bit 8 = selection). The level count, sizes, scale factors, and chunk shapes are copied from pyramid so the model mirrors the image pyramid exactly. An empty level map (matLevel) is initialised and persisted as a side-file.

The store is written in zarr v3 by default and in v2 when the path ends in .zarr2; both are fully editable and behave identically, since the packed bytes round-trip in [y, x, z] either way (a transpose codec in v3, Fortran chunk order in v2). Either way the group is marked with the mibModelStore attribute, which is how models.MibModel.loadModel later tells a MIB-written store from a foreign one (see core.MibBigDataLabelsZarr2).

Input Arguments:

  • dims - [1x3 numeric] [height, width, depth] in pixels; used as the fallback level-0 size when pyramid is empty.

  • storePath (optional) - [char|string] zarr group root directory. Default: a temporary path ([tempname '_bigdata_model.zarr3']).

  • pyramid (optional) - [struct] image pyramid struct with fields:

    • .levelImageSizes - [nLevels x 3] [height, width, depth] per level

    • .levelScaleFactors - [nLevels x 3] [yScale, xScale, zScale]

    • .chunkSizes - {1 x nLevels} per-level chunk vectors in axis order

    • .axisOrder - [char] axis order string (default 'tczyx')

When pyramid is empty, a single full-resolution level is created.

  • zarrFormat (optional) - [numeric] 2 or 3; overrides the format implied by the extension. Default: derived from storePath.

Example - create a 3-level model matching a loaded BigData image:

img = mib.mibModel.I{1};   % core.MibDataset handle
lb  = core.MibBigDataLabels([], core.MibImage.initializeImgInfo());
storePath = fullfile(fileparts(img.image.Virtual.filenames{1}), ...
                     'Labels.zarr3');
lb.createStore([img.image.height, img.image.width, img.image.depth], ...
               storePath, img.image.pyramid);

Example - the same model as a zarr v2 store:

lb.createStore(dims, 'C:\data\Labels.zarr2', img.image.pyramid);
delete(~)

DELETE - destructor: nothing to release beyond the handles (data on disk is left as-is; the level map is persisted by closeStore).

findClosestLevelForImport(sourceDims)

FINDCLOSESTLEVELFORIMPORT - Pick the pyramid level whose resolution best matches an imported model.

Syntax:
[levelIdx, levelSize] = obj.findClosestLevelForImport(sourceDims)

When a model saved at an arbitrary pyramid resolution is imported into this disk-backed pyramid, the source must be written at the level whose resolution is closest to the source’s own resolution. The source’s downsampling relative to full resolution is estimated from the size ratio along Y (modelLevelSizes(1,1) / sourceDims(1)) and fed to pickLevel, which selects the nearest level on modelScaleFactors(:,1). This mirrors how the image reader (MibVirtualImage.getDataZarr) chooses a level by magFactor, so the imported model lands on the same level the image would display at that resolution.

Input Arguments:

  • sourceDims - [1x2 | 1x3 numeric] size of the source model as [Y X] or [Y X Z] (the full-slice extent at the source’s own resolution). Only the Y extent is used for the ratio (X is downsampled by the same factor in MIB pyramids).

Output Arguments:

  • levelIdx - [numeric scalar] 1-based pyramid level index (1 = finest / full-resolution; nLevels = coarsest).

  • levelSize - [1x3 numeric] modelLevelSizes(levelIdx, :) = the chosen level’s [Y X Z] size.

See also pickLevel, core.MibBigDataLabels/getData63, core.MibBigDataLabels/setData63.

getData63(type, orient, materialIndex, options)

GETDATA63 - Read a label/mask/selection layer from the disk-backed BigData pyramid.

Syntax:
dataset = obj.getData63(type, orient, materialIndex, options)

Override of core.MibLabels63.getData63. Selects the pyramid level that matches options.magFactor (or options.pyramidLevel), materializes any dirty finer-level tiles on demand (materializeForRead), reads the requested region from that level, permutes it to the screen orientation, and resizes it to the display resolution exactly like the image reader (MibVirtualImage.getDataZarr) so the model overlay lines up pixel-for-pixel with the image in every orientation.

Bit packing (packed uint8, same as core.MibLabels63):

  • bits 1-6 - material index 0-63 (type='labels')

  • bit 7 - mask flag (type='mask')

  • bit 8 - selection flag (type='selection')

  • all bits - returned as-is (type='everything')

Input Arguments:

  • type (optional) - [char] layer to unpack:

    • 'labels' - material indices 0-63 (or a binary map when materialIndex set)

    • 'mask' - binary mask (bit 7)

    • 'selection' - binary selection (bit 8)

    • 'everything'- raw packed uint8 (all 3 layers)

    Default: 'labels'.

  • orient (optional) - [numeric] viewing orientation:

    • 1 - ZX (vertical = Z, horizontal = X, slice = Y)

    • 2 - YZ (vertical = Y, horizontal = Z, slice = X)

    • 3 - YX (standard XY; vertical = Y, horizontal = X, slice = Z)

    Default: 3.

  • materialIndex (optional) - [numeric scalar | empty] when non-empty and type='labels', returns a binary uint8 mask that is 1 where the label equals materialIndex. Pass [] to return all material indices (0-63).

  • options (optional) - [struct] with fields:

    • .magFactor - [numeric] current display magnification factor (dataset.magFactor); the nearest pyramid level is chosen. Default: 1.

    • .pyramidLevel - [numeric] explicit 1-based level index (1 = finest). Overrides magFactor entirely when set.

    • .x - [1x2 numeric] horizontal screen coordinate range [x1 x2]. Default: full width of the selected level.

    • .y - [1x2 numeric] vertical screen coordinate range [y1 y2]. Default: full height of the selected level.

    • .z - [1x2 numeric] depth (slice) range [z1 z2] in the selected level. Default: full depth of the selected level.

Output Arguments:

  • dataset - [uint8] unpacked layer at display resolution. Shape is [ny, nx, nz] for orientation 3 (YX), with ny/nx/nz determined by the options.y/x/z ranges after level-scale division and display resize. Returns [] when the store is closed (obj.exists == false).

Example 1 - read the label map for the current view at the display zoom level:

opts.magFactor = dataset.magFactor;   % e.g. 4 at 25% zoom
opts.x = dataset.slices{2};
opts.y = dataset.slices{1};
opts.z = dataset.slices{3};
labels = obj.mibModel.I{1}.labels.getData63('labels', 3, [], opts);

Example 2 - read the selection at a specific pyramid level:

opts.pyramidLevel = 2;   % second finest level
sel = obj.mibModel.I{1}.labels.getData63('selection', 3, [], opts);
initLevelMapEmpty()

INITLEVELMAPEMPTY - allocate an empty level map over the coarsest grid.

initLevelMapFallback()

INITLEVELMAPFALLBACK - no side-file: assume the on-disk pyramid is already PRECISE at every level (matLevel = 1, i.e. fully materialized).

This is the correct assumption for an imported / externally-created model (all levels were properly downsampled when written) and for any model MIB saved fully - MIB always persists the sidecar on closeStore, so a MISSING sidecar means “not a deferred interactive session”, i.e. nothing virtual.

It must NOT assume coarsest-only: doing so would make the first zoom-in past the coarsest level trigger materializeForRead, which upsamples the coarsest data and OVERWRITES the precise finer levels - silently degrading an imported model. With matLevel = 1, reads go straight to the requested level; later edits degrade only the touched tiles (markTiles), which are then recomputed lazily on read or rewritten on Save.

static isMibModelStore(storePath)

ISMIBMODELSTORE - was this store written by MIB as an editable model?

Reads the group’s mibModelStore marker (see storeMarker). This is what separates an editable MIB model store from a foreign label store that happens to be a multiscales group of uint8 arrays: the two are indistinguishable from their pixels alone, and writing MIB’s packed bytes into a foreign store would corrupt it.

Never throws - an unreadable or absent store simply answers false, which routes to the read-only path.

Older MIB stores predate the marker. That costs nothing in practice: zarr v3 stores are always opened by this class anyway, and a v2 model store could not be created before this marker existed, so there are none in the wild to misclassify.

Input Arguments:

  • storePath - [char|string] store path or URL.

Output Arguments:

  • tf - [logical] true when the marker is present.

static levelMapPathFor(storePath)

LEVELMAPPATHFOR - side-file path for the level map of a model store. Replaces the store’s extension (e.g. ‘.zarr3’) with ‘.levelmap’ so the sidecar sits next to the store as ‘<name>.levelmap’ (a MAT-format file saved/loaded with the ‘-mat’ key; NOT ‘<name>.zarr3.levelmap.mat’).

loadLevelMap()

LOADLEVELMAP - restore matLevel from the side-file; false if missing/mismatched. ‘-mat’ forces MAT parsing since the file has no ‘.mat’ extension.

markTiles(fullY, fullX, fullZ, levelIdx)

MARKTILES - set matLevel = levelIdx for every coarsest tile covering the full-res region (latest-edit-wins: invalidates any finer materialization).

materializeAll(pwbFcn)

MATERIALIZEALL - materialize every non-empty tile down to level 1 (Save). Walks the coarsest grid one tile-row per step (bounded), recomputing each finer level for that row; sets matLevel = 1 for materialized tiles.

materializeForRead(L, Yl, Xl, Zl)

MATERIALIZEFORREAD - ensure level L holds materialized data over the level-L index window [Yl Xl Zl] for every non-empty tile. Tiles whose matLevel is FINER-than-materialized for L (matLevel > L) are recomputed by upsampling from their matLevel source, written to L, and marked matLevel=L. Bounded to this window. Clean tiles (matLevel <= L) are left untouched.

Upsampling uses label-aware signed-distance smoothing (resizeBlockSmooth) when the smoothing preference is on, so coarse-level block patterns do not appear when zooming into a region drawn at a coarser zoom. Falls back to nearest-neighbour when smoothing is off or when the target is not finer (resizeBlockSmooth already handles this internally).

openStore(storePath)

OPENSTORE - attach to an EXISTING packed model pyramid on disk.

Reattaches this labels object to a model store previously written by createStore (or a compatible OME-NGFF multiscales group of packed uint8 levels). Level sizes and scale factors are restored from the stored multiscales metadata and array shapes; no pixel data is read into memory. Use this to reopen a saved model so segmentation can continue across sessions.

Input Arguments:

  • storePath - [char|string] path to the model zarr group.

pickLevel(options)

PICKLEVEL - Choose the pyramid level index for the given read/write options.

Syntax:
levelIdx = obj.pickLevel(options)

Returns options.pyramidLevel when it is present and non-empty (explicit override), otherwise finds the level whose modelScaleFactors(:,1) is closest to options.magFactor (nearest-neighbour on the scale axis). The result is clamped to [1, nLevels].

Input Arguments:

  • options - [struct] with fields:

    • .pyramidLevel (optional) - [numeric] explicit level (1 = finest).

    • .magFactor (optional) - [numeric] current display magnification factor (dataset.magFactor). Default: 1 (full resolution).

Output Arguments:

  • levelIdx - [numeric scalar] 1-based pyramid level index (1 = finest / full-resolution; nLevels = coarsest).

readPackedLevel(levelIdx, Ylim, Xlim, Zlim)

READPACKEDLEVEL - read a packed [ny x nx x nz] block from one level.

static resizeBlockNearest(block, targetSize)

RESIZEBLOCKNEAREST - nearest-neighbour resize of a packed [y x z] uint8 block to targetSize=[ty tx tz], preserving exact packed bytes.

static resizeBlockSmooth(block, targetSize)

RESIZEBLOCKSMOOTH - label-aware, boundary-smoothing UP-sample of a packed [y x z] uint8 block to targetSize=[ty tx tz].

Packed integers cannot be interpolated directly, so the three layers (material bits 1-6, mask bit 7, selection bit 8) are unpacked and each is upsampled with smoothing in YX, then repacked. Smoothing uses a signed distance transform: each region’s signed distance field (positive inside, negative outside) is a smooth function that is bicubic upsampled and Gaussian-smoothed (sigma = half the up-sampling factor); thresholding the result at 0 reconstructs a smooth boundary at sub-pixel accuracy (a coarse circle becomes a smooth circle, not a faceted one - far better than bilinear-on-binary, which only rounds a one-pixel ramp). The Gaussian erases the working-level stair-steps while the modest sigma keeps thin structures from being eroded.

  • materials: per-label signed-distance upsample + arg-max;

  • mask / selection: signed-distance upsample of the binary indicator.

Z is resized first with nearest (the pyramid keeps Z, so usually a no-op). Falls back to nearest when not actually up-sampling in YX.

static resizeLayerSmooth(layer, targetSize, isLabelMap)

RESIZELAYERSMOOTH - smooth UP-sample of a raw (unpacked) layer to targetSize=[ty tx tz], used by setData63 to bring a brush stroke captured at display resolution onto a finer working level without blocky steps.

  • isLabelMap=true -> multi-material map (values 0-63): per-label SDF arg-max (smoothLabelUpsampleYX);

  • isLabelMap=false -> binary layer (selection / mask / single-material indicator): signed-distance upsample.

Z is matched first with nearest. Falls back to nearest when not up-sampling in YX (down-sample / equal size).

saveLevelMap()

SAVELEVELMAP - persist matLevel to the side-file (<storePath>.levelmap). The file has no ‘.mat’ extension, so ‘-mat’ forces MAT format on save.

setData63(dataset, type, orient, materialIndex, options)

SETDATA63 - Write a label/mask/selection layer to the disk-backed BigData pyramid.

Syntax:
result = obj.setData63(dataset, type, orient, materialIndex, options)

Override of core.MibLabels63.setData63. The incoming data is at display resolution and screen orientation; it is:

  1. Converted back to native [y, x, z] physical order.

  2. Resized to the working level (level matching options.magFactor) using nearest-neighbour; optionally upgraded to a smooth, label-aware signed-distance upsample (resizeLayerSmooth) when the display is zoomed out and io.zarr.Config.smoothing() is on.

  3. Merged into the working level using the packed bit scheme (same logic as the parent MibLabels63).

  4. Propagated downward to all coarser levels (cheap nearest-neighbour downsample of the changed bounding box only). Finer levels are marked dirty in the level map (matLevel) and recomputed lazily on the next getData63 call, so a brush stroke at 2% zoom does not trigger a full-resolution write.

Bit packing - same scheme as getData63: bits 1-6 = material, bit 7 = mask, bit 8 = selection.

Input Arguments:

  • dataset - [uint8] layer data at display resolution, in screen orientation. Shape [ny, nx, nz] matching what getData63 would return for the same orient/options.

  • type (optional) - [char] layer to write:

    • 'labels' - write material index; materialIndex controls single-material vs full-map mode (see below)

    • 'mask' - write binary mask (bit 7)

    • 'selection' - write binary selection (bit 8)

    • 'everything'- overwrite raw packed uint8 (undo / restore; no smoothing)

    Default: 'labels'.

  • orient (optional) - [numeric] screen orientation (1/2/3; see getData63). Default: 3.

  • materialIndex (optional) - [numeric scalar | empty] when non-empty and type='labels', treats dataset as a binary indicator and writes only the voxels where dataset == 1 to material materialIndex (other materials unchanged). Pass [] to replace the full material map.

  • options (optional) - [struct] with the same fields as getData63: .magFactor, .pyramidLevel, .x, .y, .z.

Output Arguments:

  • result - [logical] true when at least one voxel changed and was written to disk; false if the store is closed, the input is empty, or the merge produced no change (early-exit, no disk I/O).

Example 1 - paint a brush mask onto the selection layer:

opts.magFactor = dataset.magFactor;
opts.x = dataset.slices{2};
opts.y = dataset.slices{1};
opts.z = dataset.slices{3};
brushMask = uint8(createBrushMask(...));   % [ny nx 1] binary
obj.mibModel.I{1}.labels.setData63(brushMask, 'selection', 3, [], opts);

Example 2 - accept selection into material 2 (programmatic undo step):

packed = obj.mibModel.I{1}.labels.getData63('everything', 3, [], opts);
packed = bitset(packed, 8, 0);   % clear selection bit
obj.mibModel.I{1}.labels.setData63(packed, 'everything', 3, [], opts);
static signedDistUpsample(maskSlice, targetYX)

SIGNEDDISTUPSAMPLE - smooth boundary up-sample of a 2-D binary mask via a signed distance transform: D = bwdist(~M) - bwdist(M) (>0 inside), upsampled + Gaussian-smoothed (smoothDistField), then thresholded at 0. Reconstructs a smooth boundary at sub-pixel accuracy. Returns a logical [ty tx]. Empty/full masks short-circuit (bwdist would be Inf and break the interpolation).

static smoothDistField(D, targetYX)

SMOOTHDISTFIELD - bicubic-upsample a signed distance field D to targetYX and Gaussian-smooth it so the zero level set is a smooth curve (not a working-level facet polygon). Sigma is half the up-sampling factor: large enough to erase the coarse grid stair-steps, small enough to keep thin structures from being eroded by curvature flow.

static smoothLabelUpsampleYX(labSlice, targetYX)

SMOOTHLABELUPSAMPLEYX - smooth up-sample of a 2-D label slice. For every label (incl. background 0) the signed distance field of its region is upsampled + Gaussian-smoothed (smoothDistField); each output pixel takes the arg-max label. Because the distance field is smooth across the whole region (not just a 1-px ramp), boundaries reconstruct as smooth curves rather than working-level facets.

static storeMarker()

STOREMARKER - the mibModelStore attribute stamped on a MIB model store.

layout records the two things a reader has to assume to use the arrays directly: values are MIB’s packed byte, and the axes are [y, x, z]. version is there so a future layout change can be detected rather than silently misread.

tilesForFullRegion(fullY, fullX, fullZ)

TILESFORFULLREGION - full-res region -> inclusive coarsest-grid tile ranges.

updateSelectionBBoxFromWrite(fullBlock, Yl, Xl, Zl, levelIdx)

UPDATESELECTIONBBOXFROMWRITE - keep selectionBBoxFull in sync after a setData63 write that may have changed selection bits (type ‘selection’/’everything’).

Parameters:

  • nearest-merged (**fullBlock** - the WHOLE processed working block (the)

  • bit (result over the incoming region [Yl Xl Zl]); its selection)

  • region. (is authoritative for the resulting selection over that)

  • fullBlock. (**Yl/Xl/Zl** - working-level absolute index ranges of)

  • at. (**levelIdx** - working level fullBlock lives)

The processed region [Yl Xl Zl] is the area we have authoritative info over. When it fully covers the previous bbox we REPLACE (so a clear or a selection→material consume shrinks/empties the box); otherwise we only have partial info → never shrink, only UNION the new footprint in. Using the exact selection bits (not the display block) catches a 1-pixel stroke.

writeMaterialMetadata()

WRITEMATERIALMETADATA - persist material names/colours to the store.

Writes a mibMaterials attribute (alongside multiscales) so material names and colours survive a close/reopen of the model. Call after material names/colours change (creation, rename, etc.). No-op when the store is not open; best-effort (never throws).

writePackedLevel(levelIdx, block, Ylim, Xlim, Zlim)

WRITEPACKEDLEVEL - write a packed [ny x nx x nz] block to one level.

static zarrFormatFromPath(storePath)

ZARRFORMATFROMPATH - zarr format implied by a store path’s extension.

.zarr2 means v2; anything else (.zarr3, .zarr, no extension) means v3, which keeps every existing caller unchanged.

Input Arguments:

  • storePath - [char|string] store path.

Output Arguments:

  • format - [numeric] 2 or 3.