Zamboni

Zamboni cleans hockey game videos by removing ads from the hockey rink boards.

• https://spacecruft.org/deepcrayon/zamboni

The current pipeline (v3, Phase 6) is zero-shot end to end — no manual labeling, no custom model training:

zamboni clean game.mp4 -o clean.mp4
  1. Decode + scene-cut detection
  2. Per shot: classical gameplay gate (kickplate + ice + persistence) —
     non-gameplay shots (commercials, close-ups) pass through untouched
  3. Grounded-SAM-2 (Grounding DINO + SAM 2.1 tracking) produces per-frame
     ad masks plus a tracked "person." occluder layer, stored as
     packed-bitmask .masks sidecars (format v2) — never colored pixels
  4. Per shot: register frames against adaptive anchors (SIFT+RANSAC+ECC),
     edit only the anchor KEYFRAMES with Qwen-Image-2.1-Turbo (base
     checkpoint as per-clip quality fallback), build one clean boards
     texture per anchor island, and reproject it onto every frame by
     homography — composited under a per-frame write mask (tracked boards
     footprint, minus dilated occluders and scoreboard exclusions)
  5. Frames that fail registration fall back to per-frame diffusion under
     the per-frame ad mask; then ffmpeg audio remux

--no-reproject restores the exact v2 behavior (per-frame diffusion under the static union band). Reprojection settings live under reproject: in configs/default.yaml (matcher, confidence thresholds, occluder detector, exposure anchoring, fallback).

v3 upgrade notes: the mask stage now also tracks a person. occluder layer (v2 .masks sidecars; disable with zamboni-mask --no-person-layer or masking.person_layer: false). Cleaning a v1 sidecar uses clean-time FCOS person detection, which downloads ~100 MB of torchvision weights from pytorch.org on first use (cached thereafter; reproject.occluder.detector: "none" opts out of occluder gating). Old zamboni versions reject v2 sidecars — regenerate masks with --no-person-layer if you need to share a sidecar with an older checkout.

Write-region semantic (unchanged from the Gate 5 capstone): reprojected frames write the tracked boards plane's footprint only. Masked content off that plane — in-ice logos, or the near-side boards in corner shots — passes through untouched on reprojected frames (v2's static union band covered it). Multi-plane coverage is a deliberate follow-up (the plane-splitting contingency in docs/NEXTGEN_PLAN.md).

Phase 7 mask-stage hardening (docs/PHASE7_PLAN.md):

• Drift-cut refinement: a shot whose mask-bbox track collapses mid-shot (a wipe the histogram cutter missed — e.g. a close-up wedged inside a wide shot) is split at the collapse; every sub-shot is re-gated on its own sampled frames and a sub-shot that fails passes through unedited instead of receiving the wide shot's band geometry at clean time. Disable with --no-drift-cut / masking.drift_cut_iou: null.
• In-ice ads policy (RC5): mask components that do not overlap the shot's gate-validated band region are dropped at mask time — in-ice ads pass through unedited, consistently. masking.in_ice_ads: exclude (default) | remove (a no-op placeholder pending the next-gen pipeline's deliberate removal decision).
• Gate vetoes (RC4b): broadcast split-screens (a tall central vertical seam with both luminance and chroma steps) and promo bumpers (a white band over a dark-and-saturated graphic) fail the gameplay gate.
• Known limitation (RC4c): close-ups with defocused boards (a skater filling the frame, boards blurred behind) fail the gate and pass through unedited — the ad reads as the arena's generic white boards. This is the designed safe behavior (a passthrough, not a failed edit); a blur-robust gate variant would only recover single-digit ad pixels, and the synthetic-blur trials could not distinguish blur from uniform frames reliably enough to risk it.

Output is frame-exact by construction and asserted numerically on every run: same frame count, fps, and timestamps as the input; audio remuxed without drift.

Installation

Requires Python >= 3.12, an NVIDIA GPU with >= ~40 GB VRAM for the default configuration (Qwen-Image-2.1-Turbo peaks at 35-37 GiB), and the ffmpeg binary.

With uv (recommended):

git clone https://spacecruft.org/deepcrayon/zamboni
cd zamboni/
uv venv
uv pip install -e .

For development (pytest + ruff):

uv pip install -e . --group dev

For the optional LPIPS quality metric in zamboni-bench:

uv pip install -e . --group bench

Install the ffmpeg binary separately, such as:

apt install ffmpeg

Install a CUDA build of torch matching your driver, e.g.:

uv pip install torch --index-url https://download.pytorch.org/whl/cu130

Model weights (Grounding DINO, SAM 2.1, Qwen-Image-2.1) download automatically from Hugging Face on first use; all are ungated.

Usage

zamboni-clean

Clean a game video in one pass (masks computed in memory):

zamboni-clean game.mp4 -o clean.mp4

Mask sidecars enable reproducible reruns (inpainting can be retuned without re-masking; per-frame seeds are seed_base + frame_index):

zamboni-clean game.mp4 -o clean.mp4 --masks-out game.masks   # also save masks
zamboni-clean game.mp4 -o clean.mp4 --masks-in  game.masks   # reuse masks

Useful flags: --checkpoint base (40-step quality fallback), --max-frames N (debug), --exclude-rect r0,r1,c0,c1 (band-exclusion rect, repeatable), --debug-strips DIR, -c configs/default.yaml (every key is also a CLI flag; CLI wins). Full list: zamboni-clean --help.

Long videos: chunking with checkpoint/resume (Phase 4)

A 20-minute period runs for hours at diffusion speeds. --chunk-frames N cleans in shot-aligned chunks (period-length convention: 36000 = 20 min @ 30 fps) with a checkpoint manifest under <output>.chunks/:

zamboni-clean game.mp4 -o clean.mp4 --chunk-frames 36000
# interrupted? re-run the identical command — completed chunks are skipped
zamboni-clean game.mp4 -o clean.mp4 --chunk-frames 36000 --no-resume  # start over

Masks are computed once and cached in the workspace, so a resumed run never re-masks. Chunks are concatenated (stream copy) and the assembled output goes through the same frame-exactness validation as a monolithic run; the workspace is deleted after success (--keep-chunks retains it). Per-frame seeds are absolute-frame based, so chunked and monolithic outputs match.

Multi-GPU

With several GPUs, both GPU phases parallelize — editing across frames (one Qwen instance per card) and masking across shots (one GSAM-2 instance per card; shots are independent, tracking resets at cuts). Output content is identical to single-GPU (per-frame seeds are absolute):

zamboni-clean game.mp4 -o clean.mp4 --devices cuda:0,cuda:1
zamboni-mask  game.mp4 --devices cuda:0,cuda:1

Per-arena calibration cache (Phase 4)

Broadcast arenas repeat. --arena KEY records gate-validated band geometry and kickplate hue under KEY (~/.cache/zamboni/arena_calibration.json, override with --arena-cache PATH) and, on later games from the same arena, validates the cached band rect directly with the same strict band criteria (no relaxation — the commercial negative control still yields zero masks):

zamboni-clean game.mp4 -o clean.mp4 --arena enterprise-center

zamboni-mask

Standalone mask producer (inspection/debugging); writes the same .masks sidecar format (docs/MASK_FORMAT.md); accepts --arena KEY like zamboni-clean:

zamboni-mask game.mp4                      # writes game.masks
zamboni-mask game.mp4 --debug-overlay DIR  # red-tint overlay PNGs

zamboni-bench

Benchmarks for the clean pipeline (Phase 4): throughput vs strip resolution with peak-VRAM curves, and quality vs synthetic-clean ground truth (PSNR, optional LPIPS via the bench dependency group, temporal warping error vs the natural flicker floor):

zamboni-bench throughput                          # size matrix x s/frame + VRAM
zamboni-bench quality --lpips -o quality.json
zamboni-bench throughput --fake                   # CPU harness smoke test

Legacy migration: Era-1/2 green-screened videos can be converted to .masks sidecars without re-masking the source:

zamboni-mask --convert-green-screen hockey-mask.mp4 -o hockey.masks

(Prefer re-running zamboni-mask on the original source when available.)

zamboni-rec

Record screen and save to mp4 (from the ffrec package):

$ zamboni-rec --help
usage: zamboni-rec [-h] [-a AUDIO_CODEC] [-A AUDIO_INPUT] [-c VIDEO_CODEC] [-D] [-C AUDIO_CHANNELS] [-d DIR] [-l] [-m] [-o] [-p PREFIX] [-r FRAMERATE] [-s SIZE] [-t TIME] [-T | -N] [-v] [-V]

options:
  -h, --help            show this help message and exit
  -a AUDIO_CODEC, --audio-codec AUDIO_CODEC
                        Audio codec to use (default: aac)
  -A AUDIO_INPUT, --audio-input AUDIO_INPUT
                        Audio input source (default: pulse)
  -c VIDEO_CODEC, --video-codec VIDEO_CODEC
                        Video codec to use (default: hevc_nvenc)
  -D, --debug           Debugging
  -C AUDIO_CHANNELS, --audio-channels AUDIO_CHANNELS
                        Number of audio channels (default: 2)
  -d DIR, --dir DIR     Directory to save the output video file
  -l, --loop            Loop input until stream is terminated manually
  -m, --mouse           Record the mouse cursor (default false)
  -o, --overwrite       Overwrite existing files (default: true)
  -p PREFIX, --prefix PREFIX
                        Prefix for the output video file name
  -r FRAMERATE, --framerate FRAMERATE
                        Framerate for recording (default: 30)
  -s SIZE, --size SIZE  Video size (default: 1920x1080)
  -t TIME, --time TIME  Length of recording in HH:MM:SS format (default: 00:01:00)
  -T, --timestamp       Add timestamp to filename (default)
  -N, --no-timestamp    Do not add timestamp to filename
  -v, --verbose         Increase output verbosity
  -V, --version         Show version

Model licenses

All models on the critical path are ungated Hugging Face downloads.

Model	Role	License
Grounding DINO 1.0 base	Open-vocab ad detection ("advertising banner.")	Apache-2.0
SAM 2.1 hiera-large	Per-shot mask tracking	Apache-2.0
Qwen-Image-2.1-Turbo	Per-frame strip edit (default)	Qwen Research License Agreement
Qwen-Image-2.1	Per-clip quality fallback	Qwen Research License Agreement

Migrating from v0.6 (Era 1/2)

The supervised label->train->mask leg and the Era-2 per-frame inpainter are gone (Phase 3):

• Removed commands: zamboni-label, zamboni-train, zamboni-vid2img, zamboni-random-images, zamboni-download (its default target, the custom zamboni.pt, was removed; v2 models download from Hugging Face).
• Removed dependencies: ailabeler, ultralytics, supervision, bitsandbytes, peft, python-ffmpeg.
• zamboni-clean v2 does not consume green-screen videos. Convert legacy intermediates with zamboni-mask --convert-green-screen, or better, re-run zamboni-mask on the original source.
• Environment management switched to uv + ruff; Python >= 3.12.

Design and gate evidence: docs/SOTA_REFACTOR_PLAN.md, docs/DECISION_RECORD_PHASE0.md, docs/GATE1_RESULTS.md, docs/GATE2_RESULTS.md. Superseded 2025-era plans are archived under docs/legacy/.

Testing

python -m unittest discover -s test -v    # or: pytest test/

The suite is CPU-only and covers the mask sidecar format, scene-cut detection, the gameplay gate, strip geometry/compositing, the frame-exactness invariant, and an end-to-end integration run on a synthetic 10 s fixture clip. See docs/TESTING.md.

Status

Alpha.

Under development.

License

Apache 2.0 or Creative Commons CC by SA 4.0 International. You may use this code, files, and text under either license.

Unofficial project, not related to upstream projects.

Upstream sources under their respective copyrights; model weights under the licenses listed above.

Copyright © 2025 Jeff Moe.