RF-DETR on NVIDIA DGX Spark

First public RF-DETR latency numbers on NVIDIA GB10 (DGX Spark), plus a live UniFi Protect RTSP loop that stays real-time on the same box.

Roboflow’s own Spark write-up shows a Waymo counter and no FPS/ms table. Their official benchmark page is NVIDIA T4 + TensorRT. This repo fills the Spark gap with a script you can re-run.

Stack: rfdetr + supervision ByteTrack. Not the Inference server — same models, local Python.

Protocol copied from Roboflow’s inference-latency cookbook: torch.cuda.Event, 20 warmup + 100 timed runs, batch 1, synthetic RGB (seed 42), official per-size resolution. PyTorch FP32 and FP16+JIT — not TensorRT. Official T4 column is a different runtime; it is there so the gap is honest.

Host: NVIDIA GB10, sm_121, PyTorch 2.13.0+cu130, driver 580.159.03, aarch64. Unedited script output: benchmarks/gb10-2026-08-14.json.

python benchmark.py --models nano,small,medium

Other published Roboflow figures, for context only:

• T4 TensorRT FP16, 200 ms thermal gap (docs): Nano 2.3 ms, Small 3.5 ms, Medium 4.4 ms
• Jetson Orin Nano (edge post): Nano “around 25 FPS” (runtime not specified)

Same box, different measurement: Protect medium stream (H.264 1280×720 @ 30 fps) → decode → RF-DETR Nano FP16 → ByteTrack → annotated MP4.

~36 FPS after warmup, 7–10 tracks/frame. Keeps up with the camera. Isolated-model FPS above is higher because it skips decode, tracking, and draw.

Do not commit clips. Indoor / family footage stays local.

python run_live.py --model nano --seconds 20 --classes person

Python 3.12, CUDA 13 GPU, ffmpeg, uv.

uv venv --python 3.12
source .venv/bin/activate
uv pip install torch torchvision --index-url https://download.pytorch.org/whl/cu130
uv pip install -r requirements.txt
cp .env.example .env

Protect gives rtsps://<nvr>:7441/<token>?enableSrtp. Rewrite to unencrypted RTSP before use:

RTSP_URL=rtsp://<nvr-host>:7447/<token>

Confirm with:

ffprobe -rtsp_transport tcp -v error -show_entries stream=codec_name,width,height,avg_frame_rate \
  "$RTSP_URL"

• Not TensorRT / SAB. Those T4 milliseconds will not appear here until someone exports and times an engine on GB10.
• Not Roboflow Inference Server or Workflows. That is the obvious next packaging step.
• Not a Jetson port. Xavier / Orin is a later deploy of the same script.