SA-1B

SA-1B (Segment Anything 1-Billion) is a segmentation dataset of 11 million photographs and 1.1 billion masks, released by Meta AI with the Segment Anything Model (SAM) and described in Kirillov et al.‘s ICCV 2023 paper “Segment Anything”; the dataset page is dated 5 April 2023. Its masks carry no class labels: each marks a region, from a whole building to a door handle, in the dataset page’s examples, that SAM found in the image. Every released mask was generated fully automatically by SAM in the last stage of the authors’ data engine, and the images were licensed from a photo company, with faces and licence plates blurred. As of November 2026 it is distributed for research under its own licence.

This article covers the dataset as an object: its files and format, the automatic-generation recipe that produced every mask, how the released code’s defaults differ from that recipe, the authors’ quality audit, and the licence regime. The model, its training and the three stages of the data engine belong to the SAM article, which this one links rather than repeats.

Purpose

SA-1B was built to train a foundation model for image segmentation. The paper’s starting point is that segmentation masks “are not abundant on the internet” (§4), so the authors built a data engine around SAM: annotators first corrected SAM’s masks interactively, then labelled what a detector had missed, and finally SAM annotated images alone. The released dataset is the output of that final stage. The paper calls SA-1B “our dataset of over 1 billion masks” (Figure 1) and releases it “to aid future development of foundation models for computer vision” (§5). The dataset page lists its intended uses as research purposes only, training and evaluating generic object segmentation models, and access to “a privacy protecting and copyright friendly” image collection.

Dataset Size

The paper’s §4 and §5, and the dataset page, give 11M images and 1.1B masks. Figure 1 of the paper rounds differently: its panel reads “1+ billion masks” and its caption “over 1 billion masks”; the image segmentation article uses “over 1 billion”, and the foundation models and SAM articles use 1.1 billion. Other works count the images more precisely: the Depth Anything article lists SA-1B as 11.1M images in Depth Anything’s unlabeled set.

• Images. The licensed originals average 3300×4950 pixels; the released copies are downsampled so that the shorter side is 1,500 pixels (§5). The dataset page gives an average released resolution of 1500×2250 pixels. For comparison, the paper puts COCO images at about 480×640.
• Masks. About 100 per image on average (dataset page and data card). The paper reports that 99.1% of the 1.1B masks the data engine produced were generated fully automatically, and SA-1B includes only those (§5); the 4.3M masks of the assisted-manual stage and the 5.9M of the semi-automatic stage (§4) are not part of it, and the data card states that the manually annotated masks will not be released.
• Scale against other datasets. SA-1B has 11× more images and 400× more masks than the second largest, Open Images, and on average 36× more masks per image; ADE20K, the closest in this respect, has 3.5× fewer masks per image (§5, Figure 6).

Classes

None. The dataset page says that there are no class labels for the images or the masks, and the data card says that no categories or text are associated with the masks. Each image is accompanied by a short free-form caption, but the agreement with the photo provider does not allow the captions to be released; the authors used them only to estimate the geographic distribution (data card, Composition 6).

Annotation Types

Every annotation is a class-agnostic binary mask with the metadata the automatic generator produced. The repository’s README describes one JSON file per image:

• image: image_id, width, height and file_name.
• annotations, one per mask: id; segmentation, the mask in COCO run-length encoding (RLE); bbox, the box around the mask as [x, y, w, h]; area in pixels; predicted_iou, “the model’s own prediction of the mask’s quality”; stability_score; crop_box, the image crop in which the mask was generated, in XYWH; and point_coords, the point prompt that produced it.

The masks are stored in COCO RLE format (README and dataset page), which the README decodes with pycocotools’ mask_utils.decode; COCO’s uncompressed form, column-major runs of zeros and ones starting with zeros, is described in the COCO article. A list of image ids, sa_images_ids.txt, is distributed with the files.

Two of these fields are worth reading with the code. predicted_iou and stability_score are the two scores the automatic stage filtered on (below). And in the released code, batched_mask_to_box returns the largest occupied row and column themselves as the box’s lower-right corner, so the width and height it writes are one less than the mask’s extent in pixels; the example below prints this. The paper and README do not say whether the released JSON files were written by this code, so whether the dataset’s bbox fields follow the same convention is not established here.

Splits

SA-1B is released as a single collection without train, validation or test splits; the dataset page lists its data function as “Training, testing”. The data card says that about 2,000 randomly selected images were withheld for testing purposes and that the rest of the licensed images are included.

Tasks

SA-1B was made to train promptable segmentation models such as SAM, which return a valid mask for a point, box or rough mask (see interactive segmentation). Because its masks have no categories and cover parts and subparts as well as whole objects, it serves class-agnostic segmentation and mask proposal generation rather than semantic or instance segmentation with a label set. The data card invites the community to add further annotations, which those who make them host and distribute themselves.

Evaluation

The paper and the dataset page describe no benchmark or evaluation server for SA-1B: SAM was evaluated zero-shot on 23 other datasets, as the SAM article describes. What SA-1B has instead is a generation recipe, with filters that decide which masks enter the dataset, and an audit of the result.

The automatic-generation recipe

The supplement’s §C describes the fully automatic stage that produced the released masks:

• Prompts and crops. A regular $32 \times 32$ grid of points on the full image, plus 20 zoomed-in crops from $2 \times 2$ and $4 \times 4$ partially overlapping windows with $16 \times 16$ and $8 \times 8$ grids. The crops were taken from the original high-resolution images, the only time they were used. Masks touching the inner boundaries of a crop were removed.
• Filters. A predicted-IoU filter “at a threshold of 88.0”; a stability filter, keeping a mask only if the IoU between its versions thresholded at logits $-1$ and $+1$ is at least 95.0; and the removal of masks covering 95% or more of the image, which the authors call “generally uninteresting”.
• Duplicates. Greedy box-based NMS at 0.7 (see non-maximum suppression) in two phases: within each crop, ranked by predicted IoU, then across crops, ranked from the most zoomed-in crop to the full image.
• Postprocessing. Connected components under 100 pixels removed, “including removing entire masks if the largest component is below this threshold”, and holes under 100 pixels filled; the authors estimate that each defect affected about 4% of masks.
• Model. A special version of SAM, trained on the manual and semi-automatic data only, for longer (177,656 iterations instead of 90k) and with large-scale jitter.

The stability score is defined in two ways in the paper. The main text (§4) calls a mask stable if thresholding “the probability map at $0.5 - \delta$ and $0.5 + \delta$” gives similar masks, with no value of $\delta$; the supplement and the code threshold the logits at $-1$ and $+1$. In our reading the two agree if $\delta \approx 0.231$, since the logistic function maps $\pm 1$ to about 0.269 and 0.731.

What the released code computes

The repository at commit dca509f (18 September 2024) implements the stage in SamAutomaticMaskGenerator (automatic_mask_generator.py) and utils/amg.py, read line by line here. For each batch of point prompts, _process_batch keeps masks whose predicted IoU is strictly greater than pred_iou_thresh (0.88), computes the stability score, keeps masks with score at least stability_score_thresh (0.95), thresholds the logits at the model’s mask_threshold (0.0, in sam.py), computes boxes and removes masks near a crop edge. _process_crop then runs box NMS within the crop ranked by predicted IoU, and _generate_masks runs NMS across crops with score $1/\text{area of the crop}$, so smaller crops win, as in the recipe. calculate_stability_score divides the number of pixels above $0 + 1$ by the number above $0 - 1$: the intersection over union of the two masks, because one is always inside the other, as its comment notes. Where the code and the paper differ:

• Coverage filter. The code has no filter for masks covering 95% of the image.
• Crops. crop_n_layers defaults to 0, so the generator processes only the full image; the recipe used two layers. With two layers the recipe’s $16 \times 16$ and $8 \times 8$ grids also need crop_n_points_downscale_factor=2; the default of 1 gives $32 \times 32$ points in every crop.
• Crop edges. is_box_near_crop_edge removes a mask whose box lies within 20 pixels (atol=20.0) of an inner crop edge, not only one that touches it.
• Postprocessing. min_mask_region_area defaults to 0, which skips it; when set, it needs OpenCV. It runs after both NMS phases, fills holes before removing islands, uses 8-connectivity, and re-runs NMS at 0.7, preferring masks it did not change. remove_small_regions treats a mask whose every component is under the threshold differently from the paper: it keeps the largest component instead of removing the mask. The paper lists components before holes but gives no order.
• Predicted IoU. The code’s test is strict (>), where the paper says “at a threshold of 88.0”, which does not say whether 0.88 itself passes. The stability test is >=, as in the paper.

The repository’s scripts/amg.py command line passes none of these settings unless they are given, so it uses the same defaults.

The mask-quality audit

To estimate mask quality, the authors sampled 500 images (about 50k masks) and asked professional annotators to improve every mask with SAM and pixel-precise brush and eraser tools. Between each automatic mask and its corrected version, 94% of pairs had IoU greater than 90% and 97% greater than 75% (§5). The paper compares this with inter-annotator consistency of 85–91% IoU estimated in prior work, and its §6 human ratings and training ablation, described in the SAM article, also support the automatic masks.

The paper also compares mask properties with other datasets (§5, Figures 5 and 6): SA-1B covers image corners better than LVIS v1 and ADE20K, while COCO and Open Images V5 have a more prominent centre bias; it has a larger share of small and medium masks relative to image size; and its distribution of mask concavity, after stratifying by mask size, is broadly similar to the other datasets’.

Example: the generator’s filters on synthetic soft masks

The NumPy and SciPy script below ports the filtering functions of amg.py and the postprocessing logic of SamAutomaticMaskGenerator at dca509f, with OpenCV’s 8-connected component labelling replaced by SciPy’s, and runs them on synthetic logit maps; the official code itself needs PyTorch, torchvision and, for postprocessing, OpenCV, and no SA-1B file or model was used. Greedy NMS, the box IoU and the cascade are our own code; torchvision’s batched_nms was not read, so our NMS drops boxes with IoU above 0.7 and computes box area as $(x_1 - x_0)(y_1 - y_0)$ on the code’s boxes.

There is no model, so the “SAM outputs” are our analogue: for a target shape, the logit map is $s \cdot d$, where $d$ is the signed distance in pixels to the target’s boundary (positive inside, computed from a $5\times$ supersampled rendering) and $s$ the slope in logits per pixel, plus noise. The noise is a white Gaussian field smoothed with $\sigma = 2$ pixels and rescaled to a standard deviation of 1 logit; we chose smoothed noise because SAM’s logits are bilinearly upsampled from a low-resolution map (sam.py). Predicted IoU is not simulated, so the predicted-IoU filter is left out and NMS needs another ranking: the script prints two, by stability score and at random, the latter being the paper’s own choice for a single-output SAM that has no IoU score (supplement §E.3).

The $256 \times 256$ scene has six ground-truth objects: a large ellipse A (4,231 pixels), a medium ellipse B (669), a 6-pixel-wide bar C and a 14-pixel-wide bar D (both 100 pixels long), a small disk E of radius 5 (81 pixels) and a disk F of radius 9 (253 pixels). From each object come three targets, the correct shape, the shape shifted 6 pixels to the right and its left half; three more are the merged pairs A+B, C+D and E+F, and one is a near-whole-image rectangle covering 96.9% of the image. Each target is rendered at five slopes, 0.5 to 8, giving 110 candidate masks per condition, without noise and with noise seeds 0–4. The masks are thresholded at logit 0, as the code does. Our analogue of the audit scores each kept mask by its best IoU with any ground-truth object and counts the masks above 0.9 and above 0.75, strictly, as in the paper; unlike the paper’s audit, about half the candidates here are wrong by construction, so the shares are not comparable with its 94%. Three cascades are run: the code’s defaults (stability ≥ 0.95, then NMS); the recipe with the code’s postprocessing (adding the coverage filter before NMS, which is our placement, and postprocessing at 100 pixels); and the recipe with the paper’s wording, which also drops a mask whose largest component is under 100 pixels.

We wrote the hypotheses before running anything. From the construction: a shifted logit map is a translation of the correct one, so without noise its stability should equal the correct mask’s exactly (H1). The thresholds $\pm 1$ fall at signed distance $\pm\delta$, $\delta = 1/s$, so by the Steiner formula for parallel sets a convex shape of area $A$ and perimeter $P$ should have stability near $(A - P\delta + \pi\delta^2)/(A + P\delta + \pi\delta^2)$ (H2a; for a rectangle, the inner set is $(w - 2\delta)(L - 2\delta)$). For a disk of radius $r$ this is $((r - \delta)/(r + \delta))^2$, at least 0.95 only if $r \ge 78\delta$, and a long bar of width $w$ needs $w \ge 78\delta$ likewise, so small and thin objects should fail at slopes where large ones pass (H2b). Under noise, IoU should rise with slope and the stability filter should keep the steeper masks (H3a), while shifted, partial and merged masks pass when steep, so the audit share stays below 1 (H3b). The whole-image mask should pass stability and be removed only by the coverage filter (H4). After NMS no two kept boxes should overlap above 0.7, only one correct copy of an object should survive, and a shifted copy should survive beside the correct one only where the shift leaves box IoU $(w-6)/(w+6) \le 0.7$ for a box of width $w$: B (0.600), E (0.250) and F (0.500), not A (0.818), C or D (0.886) (H5). Section 4 of the output checks the code-against-recipe differences above on constructed cases (H6–H8). The pass bars and the 0.05 tolerance for H2a were set after two planning scripts had printed the values they test on the same set-up, so the example is an illustration of the recipe’s behaviour rather than a test of open questions. It runs in about 4 seconds with NumPy 2.3.5 and SciPy 1.16.3.

# The fully automatic stage's mask filters on synthetic soft masks.
#
# calculate_stability_score, batched_mask_to_box, remove_small_regions, generate_crop_boxes,
# build_point_grid, build_all_layer_point_grids, is_box_near_crop_edge and the logic of
# SamAutomaticMaskGenerator.postprocess_small_regions are ported to NumPy/SciPy from
# facebookresearch/segment-anything (segment_anything/utils/amg.py and
# automatic_mask_generator.py) at commit dca509fe793f601edb92606367a655c15ac00fdf.
# Copyright (c) Meta Platforms, Inc. and affiliates. All rights reserved.
# Modified (ported from PyTorch/OpenCV to NumPy/SciPy) for this article.
# Licensed under the Apache License, Version 2.0:
# https://github.com/facebookresearch/segment-anything/blob/dca509fe793f601edb92606367a655c15ac00fdf/LICENSE
import os
for v in ("OMP_NUM_THREADS", "OPENBLAS_NUM_THREADS", "MKL_NUM_THREADS"):
    os.environ[v] = "1"
import math
from itertools import product
import numpy as np
import scipy
from scipy import ndimage

print(f"NumPy {np.__version__}, SciPy {scipy.__version__}")
H = W = 256
F = 5                                    # supersampling for the signed distance
fy, fx = np.mgrid[:H * F, :W * F]
Y, X = (fy - (F - 1) / 2) / F, (fx - (F - 1) / 2) / F    # pixel coordinates of the fine centres
SLOPES = (0.5, 1.0, 2.0, 4.0, 8.0)       # logits per pixel across the boundary
SIGMA, SHIFT, SEEDS = 1.0, 6, (0, 1, 2, 3, 4)


def ellipse(cy, cx, ay, ax):
    return ((Y - cy) / ay) ** 2 + ((X - cx) / ax) ** 2 <= 1


def rect(y0, y1, x0, x1):                # pixels y0..y1-1, x0..x1-1
    return (Y >= y0 - 0.5) & (Y < y1 - 0.5) & (X >= x0 - 0.5) & (X < x1 - 0.5)


def sdf(fine):
    """Signed distance to the shape's boundary in pixels (positive inside), at pixel centres."""
    din = ndimage.distance_transform_edt(fine)
    dout = ndimage.distance_transform_edt(~fine)
    d = np.where(fine, din - 0.5, -(dout - 0.5)) / F
    return d[(F - 1) // 2::F, (F - 1) // 2::F]


# name: (shape on the fine grid, x of the cut for the partial mask); geometry for Steiner's formula
OBJ = {"A": (ellipse(70, 80, 45, 30), 80), "B": (ellipse(70, 190, 18, 12), 190),
       "C": (rect(150, 156, 30, 130), 80), "D": (rect(180, 194, 30, 130), 80),
       "E": (ellipse(220, 200, 5, 5), 200), "F": (ellipse(170, 200, 9, 9), 200)}
GEOM = {"A": ("ell", 45, 30), "B": ("ell", 18, 12), "C": ("rect", 6, 100), "D": ("rect", 14, 100),
        "E": ("ell", 5, 5), "F": ("ell", 9, 9)}
PAIRS = (("A", "B"), ("C", "D"), ("E", "F"))
mid = (F - 1) // 2
GT = {k: v[0][mid::F, mid::F] for k, v in OBJ.items()}
gt_stack = np.stack([GT[k] for k in OBJ])

targets = []                             # (label, kind, signed distance)
for k, (fine, cx) in OBJ.items():
    d = sdf(fine)
    targets += [(k, "correct", d), (k + ">", "shifted", np.roll(d, SHIFT, axis=1)),
                (k + "/2", "partial", sdf(fine & (X < cx)))]
for a, b in PAIRS:
    targets.append((a + "+" + b, "merged", sdf(OBJ[a][0] | OBJ[b][0])))
targets.append(("whole", "whole", sdf(rect(2, 254, 2, 254))))


def steiner(k, s):
    g, p, q = GEOM[k]
    dl = 1 / s
    if g == "rect":                      # w x L rectangle: inner and outer parallel sets
        return max(p - 2 * dl, 0) * max(q - 2 * dl, 0) / (p * q + 2 * dl * (p + q) + math.pi * dl ** 2)
    if dl >= min(p, q):
        return 0.0
    A = math.pi * p * q
    h = ((p - q) / (p + q)) ** 2
    P = math.pi * (p + q) * (1 + 3 * h / (10 + math.sqrt(4 - 3 * h)))   # Ramanujan's perimeter
    return (A - P * dl + math.pi * dl ** 2) / (A + P * dl + math.pi * dl ** 2)


# ---- ports from segment_anything at dca509f (see notice above) ----
def calculate_stability_score(masks, mask_threshold, threshold_offset):
    # One mask is always contained inside the other.
    intersections = ((masks > (mask_threshold + threshold_offset))
                     .sum(-1, dtype=np.int16).sum(-1, dtype=np.int32))
    unions = ((masks > (mask_threshold - threshold_offset))
              .sum(-1, dtype=np.int16).sum(-1, dtype=np.int32))
    return intersections / unions


def batched_mask_to_box(masks):          # (B, H, W) bool -> (B, 4) XYXY, inclusive max
    b, h, w = masks.shape
    in_height = masks.max(-1)
    in_height_coords = in_height * np.arange(h)[None, :]
    bottom_edges = in_height_coords.max(-1)
    in_height_coords = in_height_coords + h * (~in_height)
    top_edges = in_height_coords.min(-1)
    in_width = masks.max(-2)
    in_width_coords = in_width * np.arange(w)[None, :]
    right_edges = in_width_coords.max(-1)
    in_width_coords = in_width_coords + w * (~in_width)
    left_edges = in_width_coords.min(-1)
    empty_filter = (right_edges < left_edges) | (bottom_edges < top_edges)
    out = np.stack([left_edges, top_edges, right_edges, bottom_edges], -1)
    return out * (~empty_filter)[:, None]


def remove_small_regions(mask, area_thresh, mode):
    # cv2.connectedComponentsWithStats(..., 8) replaced by ndimage.label with a 3x3 structure
    assert mode in ["holes", "islands"]
    correct_holes = mode == "holes"
    working_mask = (correct_holes ^ mask).astype(np.uint8)
    regions, n = ndimage.label(working_mask, structure=np.ones((3, 3), int))
    n_labels = n + 1
    sizes = np.bincount(regions.ravel(), minlength=n_labels)[1:]   # row 0 is background
    small_regions = [i + 1 for i, s in enumerate(sizes) if s < area_thresh]
    if len(small_regions) == 0:
        return mask, False
    fill_labels = [0] + small_regions
    if not correct_holes:
        fill_labels = [i for i in range(n_labels) if i not in fill_labels]
        if len(fill_labels) == 0:        # if every region is below threshold, keep largest
            fill_labels = [int(np.argmax(sizes)) + 1]
    mask = np.isin(regions, fill_labels)
    return mask, True


def generate_crop_boxes(im_size, n_layers, overlap_ratio):
    crop_boxes, layer_idxs = [], []
    im_h, im_w = im_size
    short_side = min(im_h, im_w)
    crop_boxes.append([0, 0, im_w, im_h])
    layer_idxs.append(0)

    def crop_len(orig_len, n_crops, overlap):
        return int(math.ceil((overlap * (n_crops - 1) + orig_len) / n_crops))
    for i_layer in range(n_layers):
        n_crops_per_side = 2 ** (i_layer + 1)
        overlap = int(overlap_ratio * short_side * (2 / n_crops_per_side))
        crop_w = crop_len(im_w, n_crops_per_side, overlap)
        crop_h = crop_len(im_h, n_crops_per_side, overlap)
        crop_box_x0 = [int((crop_w - overlap) * i) for i in range(n_crops_per_side)]
        crop_box_y0 = [int((crop_h - overlap) * i) for i in range(n_crops_per_side)]
        for x0, y0 in product(crop_box_x0, crop_box_y0):
            crop_boxes.append([x0, y0, min(x0 + crop_w, im_w), min(y0 + crop_h, im_h)])
            layer_idxs.append(i_layer + 1)
    return crop_boxes, layer_idxs


def build_point_grid(n_per_side):
    offset = 1 / (2 * n_per_side)
    points_one_side = np.linspace(offset, 1 - offset, n_per_side)
    points_x = np.tile(points_one_side[None, :], (n_per_side, 1))
    points_y = np.tile(points_one_side[:, None], (1, n_per_side))
    return np.stack([points_x, points_y], axis=-1).reshape(-1, 2)


def build_all_layer_point_grids(n_per_side, n_layers, scale_per_layer):
    return [build_point_grid(int(n_per_side / (scale_per_layer ** i))) for i in range(n_layers + 1)]


def is_box_near_crop_edge(boxes, crop_box, orig_box, atol=20.0):
    crop_box_np = np.asarray(crop_box, float)
    orig_box_np = np.asarray(orig_box, float)
    x0, y0, _, _ = crop_box
    boxes = (boxes + np.array([[x0, y0, x0, y0]])).astype(float)       # uncrop_boxes_xyxy
    near_crop_edge = np.isclose(boxes, crop_box_np[None, :], atol=atol, rtol=0)
    near_image_edge = np.isclose(boxes, orig_box_np[None, :], atol=atol, rtol=0)
    return np.any(near_crop_edge & ~near_image_edge, axis=1)


# ---- our parts: box IoU, greedy NMS, the cascade, the audit analogue ----
def box_iou(a, b):
    x0 = np.maximum(a[:, None, 0], b[None, :, 0]); y0 = np.maximum(a[:, None, 1], b[None, :, 1])
    x1 = np.minimum(a[:, None, 2], b[None, :, 2]); y1 = np.minimum(a[:, None, 3], b[None, :, 3])
    inter = np.clip(x1 - x0, 0, None) * np.clip(y1 - y0, 0, None)
    area = lambda c: (c[:, 2] - c[:, 0]) * (c[:, 3] - c[:, 1])
    return inter / (area(a)[:, None] + area(b)[None, :] - inter)


def nms(boxes, scores, thr):
    """Greedy NMS: highest score first, drop boxes with IoU > thr; returns kept indices sorted."""
    order = np.argsort(-scores, kind="stable")
    iou = box_iou(boxes.astype(float), boxes.astype(float))
    keep, dead = [], np.zeros(len(boxes), bool)
    for i in order:
        if not dead[i]:
            keep.append(i)
            dead |= iou[i] > thr
    return np.array(sorted(keep), int)


def largest_component(mask):
    lab, n = ndimage.label(mask, structure=np.ones((3, 3), int))
    return 0 if n == 0 else np.bincount(lab.ravel())[1:].max()


def postprocess(masks, min_area, nms_thresh, paper_rule):
    """postprocess_small_regions: holes, then islands; NMS prefers unchanged masks.
    paper_rule: also drop a mask whose largest component is under min_area."""
    new, scores, alive = [], [], []
    for m in masks:
        m2, changed1 = remove_small_regions(m, min_area, "holes")
        m2, changed2 = remove_small_regions(m2, min_area, "islands")
        alive.append(not (paper_rule and largest_component(m2) < min_area))
        new.append(m2)
        scores.append(float(not (changed1 or changed2)))
    new, alive = np.array(new), np.array(alive)
    keep = nms(batched_mask_to_box(new), np.array(scores), nms_thresh)
    return new, keep[alive[keep]]


def audit_iou(masks):
    """Our analogue of the paper's audit: best IoU with any ground-truth object."""
    inter = (masks[:, None] & gt_stack[None]).sum((-1, -2))
    union = (masks[:, None] | gt_stack[None]).sum((-1, -2))
    return (inter / union).max(1)


def make_candidates(seed):
    rng = None if seed is None else np.random.default_rng(seed)
    labels, logits = [], []
    for lab, kind, d in targets:
        for s in SLOPES:
            z = 0.0
            if rng is not None:
                z = ndimage.gaussian_filter(rng.standard_normal((H, W)), 2.0)
                z = SIGMA * z / z.std()
            labels.append((lab, kind, s))
            logits.append(s * d + z)
    return labels, np.array(logits)


def cascade(logits, stab, rank, coverage, min_area, paper_rule):
    masks = logits > 0.0
    stages = [("all", np.arange(len(masks)), masks)]
    idx = np.nonzero(stab >= 0.95)[0]
    stages.append(("stability", idx, masks))
    if coverage:
        idx = idx[masks[idx].mean((-1, -2)) < 0.95]
        stages.append(("coverage", idx, masks))
    idx = idx[nms(batched_mask_to_box(masks[idx]), rank[idx], 0.7)]
    stages.append(("nms", idx, masks))
    after_nms = idx
    if min_area > 0:
        new, k = postprocess(masks[idx], min_area, 0.7, paper_rule)
        m2 = masks.copy(); m2[idx] = new
        idx = idx[k]
        stages.append(("postproc", idx, m2))
    counts = []
    for name, ix, mm in stages:
        a = audit_iou(mm[ix]) if len(ix) else np.zeros(0)
        counts.append((name, len(ix), int((a > 0.9).sum()), int((a > 0.75).sum())))
    return counts, after_nms, idx, stages[-1][2]


# ---- data checks ----
print("object areas (px):", {k: int(v.sum()) for k, v in GT.items()})
gaps = [ndimage.distance_transform_edt(~np.any(np.delete(gt_stack, i, 0), 0))[GT[k]].min()
        for i, k in enumerate(OBJ)]
print(f"min distance between objects: {min(gaps):.2f} px; correct targets reproduce GT at 0:",
      all(np.array_equal(d > 0, GT[lab]) for lab, kind, d in targets if kind == "correct"))
cov = (targets[-1][2] > 0).mean()
print(f"whole-image target covers {cov:.4f} of the image")

# ---- 1. stability without noise: Steiner prediction, shift ----
labels, logits = make_candidates(None)
stab0 = calculate_stability_score(logits, 0.0, 1.0)
nested = all(np.all((lg > 1) <= (lg > -1)) for lg in logits)
print(f"\n1. noise-free stability, measured / Steiner (delta = 1/s); high mask inside low mask: {nested}")
print("   slope:   " + "  ".join(f"{s:>13}" for s in SLOPES))
bad = []
for k in OBJ:
    row = []
    for s in SLOPES:
        m, p = stab0[labels.index((k, "correct", s))], steiner(k, s)
        row.append(f"{m:.4f}/{p:.4f}{'*' if m >= 0.95 else ' '}")
        if abs(m - p) > 0.05:
            bad.append(f"{k}@{s}:{abs(m - p):.4f}")
    print(f"   {k}: " + " ".join(row))
print("   (* passes 0.95)  |measured - Steiner| > 0.05 at:", ", ".join(bad) or "none")
d_shift = max(abs(stab0[labels.index((k + ">", "shifted", s))] - stab0[labels.index((k, "correct", s))])
              for k in OBJ for s in SLOPES)
iou0 = audit_iou(logits > 0)
box0 = batched_mask_to_box(logits > 0)
print(f"   shifted vs correct: max |stability difference| {d_shift:.1e}")
print("   shifted: IoU with its object / box IoU with the correct copy:",
      ", ".join(f"{k} {iou0[labels.index((k + '>', 'shifted', 1.0))]:.4f}/"
                f"{box_iou(box0[[labels.index((k, 'correct', 1.0))]], box0[[labels.index((k + '>', 'shifted', 1.0))]])[0, 0]:.3f}"
                for k in OBJ))

# ---- 2. with noise ----
print("\n2. smoothed noise (std 1 logit): objects passing (correct/shifted) and mean IoU of correct masks")
print("   seed  " + "  ".join(f"s={s:<4} pass  IoU " for s in SLOPES) + "  IoU>0.9 before/after stability")
results = {}
min_margin = np.abs(stab0 - 0.95).min()
iou_margin = min(np.abs(iou0 - 0.9).min(), np.abs(iou0 - 0.75).min())
cond = [("none", None)] + [(str(s), s) for s in SEEDS]
for name, seed in cond:
    labels, logits = make_candidates(seed)
    st = stab0 if seed is None else calculate_stability_score(logits, 0.0, 1.0)
    io = audit_iou(logits > 0)
    min_margin = min(min_margin, np.abs(st - 0.95).min())
    iou_margin = min(iou_margin, np.abs(io - 0.9).min(), np.abs(io - 0.75).min())
    results[name] = (labels, logits, st)
    nested = nested and all(np.all((lg > 1) <= (lg > -1)) for lg in logits)
    if seed is None:
        continue
    cells, means = [], []
    for s in SLOPES:
        ic = [labels.index((k, "correct", s)) for k in OBJ]
        ish = [labels.index((k + ">", "shifted", s)) for k in OBJ]
        means.append(io[ic].mean())
        cells.append(f"{int((st[ic] >= .95).sum())}/{int((st[ish] >= .95).sum())}   {io[ic].mean():.4f}")
    corr = np.array([i for i, l in enumerate(labels) if l[1] == "correct"])
    kept = corr[st[corr] >= 0.95]
    print(f"   {seed}     " + "   ".join(cells) +
          f"   {(io[corr] > .9).sum()}/{len(corr)} -> {(io[kept] > .9).sum()}/{len(kept)}")
print(f"   high mask inside low mask in every condition: {nested}")
lab0, lg0 = make_candidates(0)
print(f"   noise std check (seed 0, first candidate): {np.std(lg0[0] - SLOPES[0] * targets[0][2]):.4f}")
for name, seed in cond:
    labels, logits, st = results[name]
    passA = [s for s in SLOPES if st[labels.index(("A", "correct", s))] >= .95]
    also = [f"{k}@{s}" for s in passA for k in ("C", "E") if st[labels.index((k, "correct", s))] >= .95]
    print(f"   noise {name:>4}: A passes at s = {passA}; C or E also passing there: {also or 'none'}")
    print("              whole-image mask stability by slope:",
          " ".join(f"{st[labels.index(('whole', 'whole', s))]:.4f}" for s in SLOPES))

# ---- 3. the cascades ----
print("\n3. kept masks with audit IoU > 0.9 / > 0.75 / kept   (whole-image mask kept?)")
print("   noise  ranking   code defaults        recipe, code postproc    recipe, paper wording")
nms_ok, copies_ok, pair_notes, partial_ok = 0.0, True, [], True
for name, seed in cond:
    labels, logits, st = results[name]
    for rk in ("stability", "random"):
        rank = st if rk == "stability" else np.random.default_rng(1000 + (99 if seed is None else seed)).random(len(st))
        cells = []
        for cname, cov_f, min_a, paper in (("code", False, 0, False), ("recipe-code", True, 100, False),
                                           ("recipe-paper", True, 100, True)):
            counts, after_nms, idx, mm = cascade(logits, st, rank, cov_f, min_a, paper)
            n, a9, a75 = counts[-1][1:]
            whole = any(labels[i][0] == "whole" for i in idx)
            cells.append(f"{a9:>2}/{a75:>2}/{n:>2} ({'yes' if whole else 'no '})")
            b = batched_mask_to_box(logits[after_nms] > 0).astype(float)
            iou_b = box_iou(b, b); np.fill_diagonal(iou_b, 0)
            nms_ok = max(nms_ok, iou_b.max() if len(b) > 1 else 0)
            surv = [labels[i] for i in after_nms]
            passed = [labels[i] for i in np.nonzero(st >= 0.95)[0]]
            for k in OBJ:
                if sum(l[0] == k for l in surv) > 1:
                    copies_ok = False
                both = [l for l in surv if l[0] in (k, k + ">")]
                if k in "ACD" and len(both) > 1:
                    pair_notes.append(f"{k} {name} {rk} {cname}: both survive")
                if k in "BEF" and any(l[0] == k for l in passed) and any(l[0] == k + ">" for l in passed):
                    pair_notes.append(f"{k} {name} {rk} {cname}: {'both survive' if len(both) == 2 else 'not both'}")
            if not cov_f:
                for l in passed:
                    if l[1] == "partial" and l not in surv:
                        partial_ok = False
        print(f"   {name:>5}  {rk:<9} " + "   ".join(cells))
    if name in ("none", "0"):
        counts, _, _, _ = cascade(logits, st, st, True, 100, False)
        print("          stages (recipe, code postproc, stability ranking):",
              ", ".join(f"{c[0]} {c[2]}/{c[3]}/{c[1]}" for c in counts))
print(f"   max box IoU between masks kept by NMS: {nms_ok:.4f}; at most one correct copy per object: {copies_ok}")
print(f"   every partial mask passing stability survives NMS (code defaults): {partial_ok}")
print("   correct and shifted copies of one object, both passing stability (A, C, D: listed if both survive):")
for i in range(0, len(pair_notes), 3):
    print("     " + "; ".join(pair_notes[i:i + 3]))
if not pair_notes:
    print("     no case")
print(f"   min |stability - 0.95| {min_margin:.2e}; min |audit IoU - 0.9 or 0.75| {iou_margin:.2e}")

# ---- 4. the released code against the recipe: exact cases ----
print("\n4. code against recipe")
for size in ((1500, 2250), (3300, 4950)):
    boxes, layers = generate_crop_boxes(size, 2, 512 / 1500)
    print(f"   crop_n_layers=2 on {size[0]}x{size[1]}: boxes per layer {[layers.count(i) for i in range(3)]},"
          f" first of each {[boxes[layers.index(i)] for i in range(3)]}")
for f in (2, 1):
    g = build_all_layer_point_grids(32, 2, f)
    print(f"   crop_n_points_downscale_factor={f}: grids {[math.isqrt(len(x)) for x in g]}, "
          f"prompts on 1+4+16 crops {len(g[0]) + 4 * len(g[1]) + 16 * len(g[2])}")
boxes, layers = generate_crop_boxes((1500, 2250), 2, 512 / 1500)
cb = boxes[1]
w = cb[2] - cb[0]
test = np.array([[100, 100, w - 15, 300], [100, 100, w - 25, 300], [0, 100, 200, 300]])
print(f"   crop {cb}: right edge 15 px, 25 px from the inner edge, box on the image edge -> removed",
      is_box_near_crop_edge(test, cb, [0, 0, 2250, 1500]).tolist())
m = np.zeros((1, 32, 32), bool); m[0, 5:15, 3:23] = True
bx = batched_mask_to_box(m)[0]
print(f"   10x20 mask at rows 5-14, cols 3-22: box {bx.tolist()}, XYWH {[int(bx[0]), int(bx[1]), int(bx[2] - bx[0]), int(bx[3] - bx[1])]}")
a = np.zeros((64, 64), bool); a[20:26, 20:30] = True
r, ch = remove_small_regions(a, 100, "islands")
print(f"   single 60-px component: code keeps {int(r.sum())} px (changed={ch}); paper wording keeps "
      f"{0 if largest_component(r) < 100 else int(r.sum())} px")
b = np.zeros((64, 64), bool); b[10:40, 10:40] = True; b[20:30, 20:30] = False; b[23:27, 23:27] = True
h1 = remove_small_regions(remove_small_regions(b, 100, "holes")[0], 100, "islands")[0]
i1 = remove_small_regions(remove_small_regions(b, 100, "islands")[0], 100, "holes")[0]
print(f"   ring with a 4x4 island in a 10x10 hole ({int(b.sum())} px): holes first {int(h1.sum())}, islands first {int(i1.sum())}")
for hh, ww in ((10, 10), (9, 11)):
    c = np.zeros((64, 64), bool); c[5:35, 5:35] = True; c[45:45 + hh, 45:45 + ww] = True
    print(f"   30x30 square plus a {hh * ww}-px island: {int(c.sum())} -> {int(remove_small_regions(c, 100, 'islands')[0].sum())} px")

NumPy 2.3.5, SciPy 1.16.3
object areas (px): {'A': 4231, 'B': 669, 'C': 600, 'D': 1400, 'E': 81, 'F': 253}
min distance between objects: 25.00 px; correct targets reproduce GT at 0: True
whole-image target covers 0.9690 of the image

1. noise-free stability, measured / Steiner (delta = 1/s); high mask inside low mask: True
   slope:             0.5            1.0            2.0            4.0            8.0
   A: 0.7944/0.7987  0.8885/0.8938  0.9422/0.9454  0.9685/0.9723* 0.9758/0.9861*
   B: 0.5551/0.5681  0.7426/0.7549  0.8527/0.8691  0.9319/0.9323  0.9484/0.9655 
   C: 0.1853/0.1852  0.4804/0.4809  0.6533/0.7004  1.0000/0.8378* 1.0000/0.9154*
   D: 0.5139/0.5138  0.7206/0.7210  0.8400/0.8496  1.0000/0.9218* 1.0000/0.9601*
   E: 0.1678/0.1837  0.3982/0.4444  0.6289/0.6694  0.7753/0.8186  0.7753/0.9048 
   F: 0.3846/0.4050  0.6088/0.6400  0.7543/0.8006  0.8700/0.8948  0.9234/0.9460 
   (* passes 0.95)  |measured - Steiner| > 0.05 at: C@4.0:0.1622, C@8.0:0.0846, D@4.0:0.0782, E@8.0:0.1295
   shifted vs correct: max |stability difference| 0.0e+00
   shifted: IoU with its object / box IoU with the correct copy: A 0.7751/0.818, B 0.5187/0.600, C 0.8868/0.886, D 0.8868/0.886, E 0.1825/0.250, F 0.4174/0.500

2. smoothed noise (std 1 logit): objects passing (correct/shifted) and mean IoU of correct masks
   seed  s=0.5  pass  IoU   s=1.0  pass  IoU   s=2.0  pass  IoU   s=4.0  pass  IoU   s=8.0  pass  IoU   IoU>0.9 before/after stability
   0     0/0   0.6965   0/0   0.8341   0/0   0.9049   2/1   0.9597   4/4   0.9870   17/30 -> 6/6
   1     0/0   0.7376   0/0   0.8495   0/0   0.9129   1/2   0.9642   4/4   0.9767   18/30 -> 5/5
   2     0/0   0.7217   0/0   0.8470   0/0   0.9148   2/1   0.9676   5/4   0.9825   20/30 -> 7/7
   3     0/0   0.7350   0/0   0.8536   0/0   0.9088   2/1   0.9582   4/4   0.9821   18/30 -> 6/6
   4     0/0   0.7230   0/0   0.8489   0/0   0.9295   1/1   0.9635   4/5   0.9826   19/30 -> 5/5
   high mask inside low mask in every condition: True
   noise std check (seed 0, first candidate): 1.0000
   noise none: A passes at s = [4.0, 8.0]; C or E also passing there: ['C@4.0', 'C@8.0']
              whole-image mask stability by slope: 0.9385 0.9688 0.9842 1.0000 1.0000
   noise    0: A passes at s = [4.0, 8.0]; C or E also passing there: ['C@8.0']
              whole-image mask stability by slope: 0.9512 0.9713 0.9846 0.9941 0.9998
   noise    1: A passes at s = [4.0, 8.0]; C or E also passing there: ['C@8.0']
              whole-image mask stability by slope: 0.9556 0.9702 0.9842 0.9943 0.9998
   noise    2: A passes at s = [4.0, 8.0]; C or E also passing there: ['C@8.0']
              whole-image mask stability by slope: 0.9548 0.9726 0.9843 0.9940 0.9997
   noise    3: A passes at s = [4.0, 8.0]; C or E also passing there: ['C@8.0']
              whole-image mask stability by slope: 0.9528 0.9712 0.9843 0.9936 0.9998
   noise    4: A passes at s = [4.0, 8.0]; C or E also passing there: ['C@8.0']
              whole-image mask stability by slope: 0.9519 0.9711 0.9837 0.9942 0.9997

3. kept masks with audit IoU > 0.9 / > 0.75 / kept   (whole-image mask kept?)
   noise  ranking   code defaults        recipe, code postproc    recipe, paper wording
    none  stability  3/ 4/ 8 (yes)    3/ 4/ 7 (no )    3/ 4/ 7 (no )
    none  random     3/ 4/ 8 (yes)    3/ 4/ 7 (no )    3/ 4/ 7 (no )
          stages (recipe, code postproc, stability ranking): all 30/55/110, stability 6/14/24, coverage 6/14/20, nms 3/4/7, postproc 3/4/7
       0  stability  3/ 5/11 (yes)    3/ 5/10 (no )    3/ 5/10 (no )
       0  random     2/ 5/11 (yes)    2/ 5/10 (no )    2/ 5/10 (no )
          stages (recipe, code postproc, stability ranking): all 17/42/110, stability 6/12/22, coverage 6/12/17, nms 3/5/10, postproc 3/5/10
       1  stability  4/ 5/11 (yes)    4/ 5/10 (no )    4/ 5/10 (no )
       1  random     3/ 5/11 (yes)    3/ 5/10 (no )    3/ 5/10 (no )
       2  stability  4/ 6/12 (yes)    4/ 6/11 (no )    4/ 6/11 (no )
       2  random     3/ 6/12 (yes)    3/ 6/11 (no )    3/ 6/11 (no )
       3  stability  3/ 5/11 (yes)    3/ 5/10 (no )    3/ 5/10 (no )
       3  random     2/ 5/11 (yes)    2/ 5/10 (no )    2/ 5/10 (no )
       4  stability  3/ 5/12 (yes)    3/ 5/11 (no )    3/ 5/11 (no )
       4  random     2/ 5/12 (yes)    2/ 5/11 (no )    2/ 5/11 (no )
   max box IoU between masks kept by NMS: 0.6069; at most one correct copy per object: True
   every partial mask passing stability survives NMS (code defaults): False
   correct and shifted copies of one object, both passing stability (A, C, D: listed if both survive):
     B 0 stability code: both survive; B 0 stability recipe-code: both survive; B 0 stability recipe-paper: both survive
     B 0 random code: both survive; B 0 random recipe-code: both survive; B 0 random recipe-paper: both survive
     B 1 stability code: both survive; B 1 stability recipe-code: both survive; B 1 stability recipe-paper: both survive
     B 1 random code: both survive; B 1 random recipe-code: both survive; B 1 random recipe-paper: both survive
     B 2 stability code: both survive; B 2 stability recipe-code: both survive; B 2 stability recipe-paper: both survive
     B 2 random code: both survive; B 2 random recipe-code: both survive; B 2 random recipe-paper: both survive
     B 3 stability code: both survive; B 3 stability recipe-code: both survive; B 3 stability recipe-paper: both survive
     B 3 random code: both survive; B 3 random recipe-code: both survive; B 3 random recipe-paper: both survive
     B 4 stability code: both survive; B 4 stability recipe-code: both survive; B 4 stability recipe-paper: both survive
     B 4 random code: both survive; B 4 random recipe-code: both survive; B 4 random recipe-paper: both survive
   min |stability - 0.95| 9.80e-04; min |audit IoU - 0.9 or 0.75| 0.00e+00

4. code against recipe
   crop_n_layers=2 on 1500x2250: boxes per layer [1, 4, 16], first of each [[0, 0, 2250, 1500], [0, 0, 1381, 1006], [0, 0, 755, 567]]
   crop_n_layers=2 on 3300x4950: boxes per layer [1, 4, 16], first of each [[0, 0, 4950, 3300], [0, 0, 3038, 2213], [0, 0, 1660, 1248]]
   crop_n_points_downscale_factor=2: grids [32, 16, 8], prompts on 1+4+16 crops 3072
   crop_n_points_downscale_factor=1: grids [32, 32, 32], prompts on 1+4+16 crops 21504
   crop [0, 0, 1381, 1006]: right edge 15 px, 25 px from the inner edge, box on the image edge -> removed [True, False, False]
   10x20 mask at rows 5-14, cols 3-22: box [3, 5, 22, 14], XYWH [3, 5, 19, 9]
   single 60-px component: code keeps 60 px (changed=True); paper wording keeps 0 px
   ring with a 4x4 island in a 10x10 hole (816 px): holes first 900, islands first 800
   30x30 square plus a 100-px island: 1000 -> 1000 px
   30x30 square plus a 99-px island: 999 -> 900 px

The data checks hold: the objects lie at least 25 pixels apart, each correct target reproduces its object at logit 0, the $+1$ mask lies inside the $-1$ mask in every condition, and the noise has unit standard deviation.

Stability measures steepness, not correctness. Without noise, the shifted masks have exactly the stability of the correct ones (largest difference 0.0e+00) while their IoU with the object is 0.7751 for A, 0.8868 for C and D and only 0.1825 for E: H1 held. With noise, the number of objects whose correct and shifted masks pass differs by at most one at every slope and seed. A wrong mask with a steep boundary is as stable as a right one; in our reading, this is why the recipe pairs the stability filter with the model’s predicted IoU, and it is why our audit share stays low below.

Size, thinness and the pixel grid. At slopes 0.5 to 2 the measured stabilities follow the Steiner prediction within 0.05 for all six objects, and at slope 4 for A, B, E and F and slope 8 for A, B, D and F: H2a held there. It failed at C and D at slope 4 (differences 0.1622 and 0.0782) and at C and E at slope 8 (0.0846 and 0.1295). Our plan’s list of expected failures, written after a planning script had run, also named E at slope 4 and D at slope 8; both held (0.7753 against 0.8186, and 1.0000 against 0.9601). The failures have a reason in the construction: the bars’ edges are aligned with the pixel grid, so every pixel centre lies at least half a pixel from them, and once $\delta < 0.5$ both thresholded masks equal the bar and the stability is 1.0000. The small disk E is held at 0.7753 at both steep slopes for the same reason, in our reading: once $\delta$ is smaller than the distance from its boundary to the nearest pixel centres, steeper logits no longer change which pixels are counted. H2b held for the size part: E never reaches 0.95, while A passes at slopes 4 and 8 in every condition. It failed for the thinness part: the 6-pixel bar C passes wherever A passes at slope 8, in every condition, and at slope 4 without noise. With noise at slope 4, C’s stability falls below 0.95 while A’s stays above, the effect H2b predicted, but the printed check counts the failures at slope 8. In our reading, the continuous prediction is the closer guide for boundaries that do not follow the pixel grid: at a slope of 8 logits per pixel, an object needs a radius or width of about 10 pixels to pass.

The filters and the audit. With noise, the mean IoU of the correct masks rises with slope in every seed, from about 0.70–0.74 at slope 0.5 to 0.977–0.987 at slope 8, and among correct masks the share above 0.9 IoU rises from 17–20 of 30 to every mask the stability filter keeps (5–7): H3a held. The kept sets still contain wrong masks: B’s shifted copy survives in every noisy seed, and in every run 3 to 7 kept masks have audit IoU of 0.75 or less. After every cascade, in every condition and under both rankings, at most 4 of the 7 to 12 kept masks exceed 0.9 IoU, so H3b held. Random ranking keeps one mask fewer above 0.9 than stability ranking in each noise seed, and the same number without noise.

Coverage. The whole-image mask’s stability is 0.9385 at slope 0.5 without noise and at least 0.9512 everywhere else; it is kept by the code’s defaults in all twelve runs and removed by both recipe cascades, so H4 held. The recipe’s coverage filter removes a mask that the stability filter cannot.

NMS. No two masks kept by NMS have box IoU above 0.6069, and at most one correct copy of each object survives. The printed box IoUs of correct and shifted copies equal the values derived by hand; A, C and D never keep both copies, and B keeps both wherever both pass, in every noisy seed, ranking and cascade: those parts of H5 held. For E and F the case never arose, since the correct and shifted copies never both pass stability, so that part is untested. The check that every partial mask passing stability survives NMS failed: the printed value is False. In our reading, copies of the same half-shape at different slopes share one box, so NMS keeps one of them, and the check did not allow for that.

Postprocessing. In the two conditions whose stages are printed, the postprocessing stage removed no mask, and the two recipe variants give the same counts in every condition; the only object under 100 pixels, E, never passes stability. Section 4 shows the differences on constructed cases, and each held as derived: a mask whose only component has 60 pixels is kept by the code (and flagged as changed, so the second NMS ranks it below unchanged masks) and removed under the paper’s wording; a ring whose 100-pixel hole holds a 16-pixel island becomes solid (900 pixels) when holes are filled first, as the code does, and keeps its hole (800) when islands are removed first; and a 100-pixel island survives while a 99-pixel one is removed, since the test is strict.

Crops and boxes. Two crop layers give 1 + 4 + 16 boxes for both the released and the original image size, the recipe’s 20 extra crops; the grids are 32, 16 and 8 points per side only with a downscale factor of 2, and the default factor would run 21,504 prompts instead of 3,072; a box 15 pixels from an inner crop edge is removed, one 25 pixels away is not, and one on the image’s own edge is not. The 10 × 20 mask gets the box [3, 5, 22, 14], so its XYWH width and height are 19 and 9. These checks held.

Robustness of the printed decisions. No stability value lies within 9.80e-04 of 0.95, so the pass decisions do not depend on rounding or on PyTorch’s float precision; that check held. The check that no audit IoU lies exactly on 0.9 or 0.75 failed: the printed margin is 0.00e+00, so at least one candidate’s audit IoU sits exactly on a threshold, where the strict “greater than” counts it as not above. The output does not show which candidate. Two runs gave identical output.

The script was planned before it ran. A pilot of the planned script ran before the run shown here (twice, with identical output). It did not print one planned measure, the whole-image mask’s stability at each slope (for H4), whose values had already appeared in a diagnostic run; that print was added before this run, with the editor’s approval, together with a change to the layout of the correct-and-shifted list, which now names the cascade on each entry instead of repeating each entry once per cascade without naming it. No other change was made, and the output above is from the first run of this script; a repeat gave identical output.

License

As of November 2026 the full SA-1B licence text could not be read for this article: the dataset page links it through a download page that renders with JavaScript, and nothing behind that gate was opened or accepted. The public statements are these:

• The repository’s README names the licence the “SA-1B Dataset Research License” and states that by downloading the datasets “you agree that you have read and accepted” its terms.
• The dataset page gives the licence as “Limited; see full license language for use”, the intended use as “Research purposes only” and the access cost as “Open access”, and says the dataset is intended for computer vision research “for the purposes permitted under our Data License”.
• The paper (§5) says SA-1B is released under “a favorable license agreement for certain research uses”; the data card says users must agree to the terms of use before downloading or using the dataset, and that under the licence “it is prohibited to attempt to identify or associate an image with a particular individual”.

What the licence permits beyond these words, including any commercial use of models trained on SA-1B, is not established here. The images were licensed by Meta from a third-party photo provider (data card). SAM’s code is under the Apache License 2.0 (the repository’s LICENSE file), and the README states that the model is licensed under it; the dataset has its own licence, named separately in the README.

Limitations and Bias

• Masks made by a model. Every released mask was produced by SAM, and the data card notes that “there may be errors or inconsistencies in the masks”. The audit covers 500 images; the supplement names small spurious components and holes as two error types, each estimated to affect about 4% of masks and addressed by postprocessing. The example above shows the stability filter passing shifted masks exactly as often as correct ones without noise; in our reading, this is why the recipe also relies on predicted IoU.
• What gets segmented. The masks reflect what the automatic generator found and kept: no categories, parts mixed with wholes, and, in our reading, fewer small masks than an annotator might draw, since the stability and size rules disfavour them (as the example shows for its small disk). The paper reports that SA-1B nevertheless has a larger share of small and medium relative-size masks than the datasets it compares.
• Geography and income. The authors’ responsible-AI analysis, summarized in the SAM article, estimated from the captions that Africa and low-income countries are underrepresented, as in COCO and Open Images. The data card asks users to be mindful that the dataset does not have parity across all groups; its pointer to that analysis is printed as “§??”, a broken reference in the supplement.
• Content. The data card reports that a small portion of images shows protests or other gatherings that may suggest religious beliefs or political opinions, with faces blurred, and that the authors could not filter all such images.
• Privacy processing. Faces were detected with RetinaFace and blurred, and licence plates blurred with a model that has not been made public (data card). The released images are downsampled and the unaltered photos are not released.
• Updates. The only planned updates are removals of reported images, and older versions are not kept (data card); as of November 2026 the dataset page lists a single version, V1.0.

Common Uses

• Training SAM, the use the data card records, with the SAM article describing the training run and ablations.
• Training SAM’s successors. The SAM article reports that SAM 2 (Hiera-B+), trained on SA-1B alone, reached 58.9 one-click mIoU on SAM’s 23 datasets; SAM 2’s own video dataset, SA-V, is a separate dataset.
• Unlabeled images. Depth Anything used SA-1B’s images, without masks, among about 62 million unlabeled images for pseudo-labelled depth estimation.

Historical Milestones

• 2023. SA-1B V1.0 released with SAM (dataset page dated 5 April); the paper appeared at ICCV 2023. With 1.1 billion masks it had 400× more masks than Open Images, the largest segmentation dataset before it (Figure 6 caption; §5 calls Open Images the second largest, counting SA-1B).
• 2024–2025. SAM 2 (ICLR 2025) extended the model to video with its own SA-V dataset, and reported results for a model trained on SA-1B alone, as the SAM article describes.

Related Datasets

• Open Images V5 and later: about 2.8 million instance masks on 350 classes, the largest segmentation dataset SA-1B was compared with.
• COCO and LVIS: instance masks with category labels on COCO’s images, and the RLE format SA-1B’s files use.
• ADE20K: scene parsing with dense labels; the paper’s closest comparison in masks per image.
• SA-V: the video dataset released with SAM 2.

Resources

• SA-1B dataset page (overview, data card and the gated download page).
• facebookresearch/segment-anything: SamAutomaticMaskGenerator, utils/amg.py, the dataset format in the README, and scripts/amg.py.
• The paper and supplement on CVF Open Access, with the automatic-generation details in §C and the dataset card in §G.1.