Visitar URL original
Skip tracklet stitching when no animals were detected by C-Achard · Pull Request #3504 · DeepLabCut/DeepLabCut · GitHub
Skip to content

Skip tracklet stitching when no animals were detected - #3504

Draft
C-Achard wants to merge 21 commits into
devfrom
cy/fix-empty-tracklet-stitch
Draft

C-Achard wants to merge 21 commits into
devfrom
cy/fix-empty-tracklet-stitch

Conversation

@C-Achard

@C-Achard C-Achard commented Sep 9, 2026

Copy link
Copy Markdown
Member

Problem

For multi-animal analysis with auto_track=True, a video with no detections causes the entire operation to abort.

Undetected frames are represented by -1 padded predictions. These entries are removed when assemblies are generated, so a fully undetected video produces an empty assemblies file. Tracking then proceeds anyway, eventually causing TrackletStitcher to raise OSError("Tracklets are empty.").

Changes

  • Return the retained assembly count from _generate_assemblies_file
  • Skip automatic tracking when a video contains no assemblies
  • Warn that no tracking output was created while keeping the _full.pickle predictions available
  • Introduce EmptyTrackletsError as a ValueError subclass for empty tracklet input
  • Make stitch_tracklets skip videos that raise EmptyTrackletsError, allowing batch processing to continue
  • Add regression tests for empty assemblies, valid assembly counts, header-only tracklets, and the exceptions

Compatibility

Changes the exception raised for empty tracklets from OSError to ValueError. It also changes stitch_tracklets so that empty-tracklet videos are skipped rather than propagated as failures, including for GUI and standalone callers.
No callers in the repository currently catch either exception.

Follow-up to #3488, where handling an OverflowError in _generate_output_data revealed this failure.

deruyter92 and others added 21 commits September 8, 2026 09:30
Handle the valid case where PyTorch video inference returns zero predictions.
videos with zero
detections would fail after serialization on `np.stack([])`

Return an
empty DataFrame with the expected DLC column schema instead and add a
regression test for the empty-output case.
- add informative warning
- add guard in dataframe export: raise ValueError if no predictions
Ensure inference emits results for images whenever a prediction exists, even when no detections are produced and thus no queue item is generated. This avoids dropping frames with valid predictions but empty detection outputs.
Fixes a mismatch in `analyze_videos` where the DataFrame export step still used `predictions` instead of `ctd_predictions`. This ensures saved video analysis results use the corrected prediction set from the current processing path.
When a video produces no pose predictions, analysis now writes all-NaN placeholder predictions so downstream HDF5 export and filtering can continue instead of failing on an empty result set. The DataFrame export path was adjusted to handle these fallback frames correctly and raises clearer errors only for genuinely invalid empty input. FMPOSE 3D inference now also raises a clear RuntimeError when no 2D poses are detected for a video.
Move frame key zero-padding logic into `runners.shelving.frame_key_width` and use it from both video output generation and shelf writing. This keeps frame key formatting consistent across full-data exports and on-the-fly shelving, while preserving the default width when the frame count is unknown.
Refines `video_inference` logging so empty results distinguish unreadable videos from detector misses, and keeps re-encoding guidance for partial reads. Expands tests to cover zero-prediction, partial-prediction, and shelf-writer cases.
Simplifies `analyze_videos` by always passing the inference `predictions` directly to `create_df_from_prediction` when `save_as_df` is enabled, and removes the special-case branch that generated all-NaN placeholder rows when no predictions were produced. The unused `_nan_predictions` helper and its warning/error handling were also deleted.
Use the CTD prediction list when deciding whether to write the dataframe, so `.h5` export still happens based on the converted CTD results instead of an unrelated variable. The warning now also explains that the CTD pickle had no frames when export is skipped.
Add an explanatory comment in `analyze_videos` documenting why `ctd_predictions` can be empty, and why the code warns and skips dataframe creation instead of raising. This captures edge cases like zero-frame pickle output and `save_as_df=False`.
Refines the test docstring in `test_runners_inference.py` to describe the async behavior more clearly by comparing it to the sequential path without implying a regression pin.
ShelfWriter now always derives its frame key width through `frame_key_width()` instead of keeping a separate default. This removes duplicated fallback logic and keeps frame index formatting consistent with the shared helper.
Avoid running stitch/tracklet conversion when no animals were detected in a video. `analyze_videos` now detects empty assemblies, warns, and leaves the pose predictions available without creating tracking files. `TrackletStitcher` raises a dedicated `EmptyTrackletsError`, and the stitcher test now expects that case.
Cover the no-detection path in pose-estimation assembly generation and the header-only tracklet case in the stitcher. These tests pin the expected empty-input behavior and the error type for invalid tracklet files.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Identity-padded predictions are miscounted, and skipped stitching can incorrectly be reported as successful.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Prevents empty multi-animal detections from aborting tracking and introduces a dedicated empty-tracklet exception.

Changes:

  • Counts generated assemblies and skips stitching when none exist.
  • Adds EmptyTrackletsError and continues batch stitching after empty input.
  • Adds regression tests for assembly counts and empty tracklets.
File summaries
File Description
deeplabcut/pose_estimation_pytorch/apis/videos.py Detects empty assemblies and skips automatic tracking.
deeplabcut/refine_training_dataset/stitch.py Defines and handles EmptyTrackletsError.
tests/pose_estimation_pytorch/apis/test_videos.py Tests assembly counting.
tests/test_stitcher.py Tests empty-tracklet exceptions.
Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

snapshot_index=snapshot_index,
detector_snapshot_index=detector_snapshot_index,
)
h5_files_created = True # .h5 file was created by stitch_tracklets
mask = ~np.all(preds < 0, axis=(1, 2))
preds = preds[mask]

num_assemblies += len(preds)
@C-Achard C-Achard added the 3.1 label Sep 17, 2026
Base automatically changed from jaap/fix-empty-video-predictions to dev October 23, 2026 15:09

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants