Visitar URL original
RTMpose_x superanimal_humanbody by MMathisLab · Pull Request #3043 · DeepLabCut/DeepLabCut · GitHub
Skip to content

RTMpose_x superanimal_humanbody - #3043

Merged
AlexEMG merged 35 commits into
mainfrom
mwm/humanbody
Jul 6, 2025
Merged

RTMpose_x superanimal_humanbody#3043
AlexEMG merged 35 commits into
mainfrom
mwm/humanbody

Conversation

@MMathisLab

@MMathisLab MMathisLab commented Jul 5, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

Major Changes:

  • This integrates the RTMPose model that we demo with the low-level API in this notebook fully into the model zoo.

  • For ease, this is a new function called deeplabcut/pose_estimation_pytorch/modelzoo/superanimal_humanbody_video_inference.py

  • it also works in the api call

  • It also updates the model-zoo tab for better handling of thresholds and more elegant layout:

Screenshot 2025-07-05 at 13 12 26
  • It also now tells the user if not all frames have a detected animal:
Screenshot 2025-07-05 at 13 13 18
  • Right now it does not support explicitly support video adaptation in the gui.

Minor:

  • fixes to some missing info on superbird

@MMathisLab
MMathisLab requested a review from Copilot July 5, 2025 09:09

This comment was marked as outdated.

Sign in to view

@MMathisLab
Update video_inference.py
277fb22 
- adding missing docs for superbird
@MMathisLab
MMathisLab requested a review from Copilot July 5, 2025 09:18

This comment was marked as outdated.

Sign in to view

@MMathisLab
MMathisLab requested a review from Copilot July 5, 2025 11:24

This comment was marked as outdated.

Sign in to view

@MMathisLab
MMathisLab requested review from AlexEMG and Copilot July 5, 2025 14:35
Copilot AI reviewed Jul 5, 2025
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Integrate the RTMPose_x human‐body model into the DeepLabCut model zoo, add a dedicated video inference pipeline using torchvision detectors, update configs, docs, tests, and GUI layout.

  • Added superanimal_humanbody_video_inference.py for a clean, Colab-style human‐body demo
  • Extended load_super_animal_config and utils to download configs from HuggingFace and support torchvision detectors
  • Updated tests, documentation, and the GUI model-zoo tab for thresholds, layout, and human‐body support

Reviewed Changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/pose_estimation_pytorch/modelzoo/test_webapp.py Include superanimal_humanbody in webapp tests
tests/pose_estimation_pytorch/modelzoo/test_modelzoo_utils.py Include superanimal_humanbody in utils tests
test_superanimal_humanbody.py New top-level script for human-body detector integration test
deeplabcut/pose_estimation_pytorch/modelzoo/utils.py Add HF download and special handling for human-body configs
deeplabcut/pose_estimation_pytorch/runners/inference.py Introduce TorchvisionDetectorInferenceRunner
deeplabcut/gui/tabs/modelzoo.py Revamp GUI: thresholds, media selector, adaptation options
Comments suppressed due to low confidence (2)

deeplabcut/pose_estimation_pytorch/runners/inference.py:866

  • BaseDetector is not imported in this module, causing a NameError. Add from deeplabcut.pose_estimation_pytorch.models.detectors.base import BaseDetector at the top of the file.
class TorchvisionDetectorInferenceRunner(InferenceRunner[BaseDetector]):

deeplabcut/gui/tabs/modelzoo.py:365

  • deeplabcut.superanimal_analyze_images is not exposed on the deeplabcut module at top level, leading to an AttributeError. Import or fully qualify this function before use.
            help_text = deeplabcut.superanimal_analyze_images.__doc__

Comment thread test_superanimal_humanbody.py Resolved
AlexEMG
AlexEMG reviewed Jul 5, 2025
View reviewed changes
Comment thread deeplabcut/pose_estimation_tensorflow/modelzoo/api/superanimal_inference.py Resolved
AlexEMG
AlexEMG approved these changes Jul 5, 2025
View reviewed changes

@AlexEMG AlexEMG left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent

@AlexEMG
AlexEMG merged commit ecc06f8 into main Jul 6, 2025
@maximpavliv maximpavliv mentioned this pull request Jul 8, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@AlexEMG AlexEMG AlexEMG approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@MMathisLab @AlexEMG