Visitar URL original
[Sync] merge to main for 3.0.2 release - bug fixes; knowledge indexing; future compatibility & dependencies by deruyter92 · Pull Request #3495 · DeepLabCut/DeepLabCut · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
246 commits
Select commit Hold shift + click to select a range
9cc1d18
Add Matplotlib colormap compatibility shim
C-Achard Jul 13, 2026
9518b9d
Allow custom warning stacklevel in deprecated
C-Achard Jul 13, 2026
bd7fdbb
Use shared get_colormap helper everywhere
C-Achard Jul 13, 2026
4bf4921
Merge branch 'main' into cy/init-mpl-version-change
C-Achard Jul 16, 2026
aafaa2e
Guard CoW read-only .to_numpy()/.values mutations (#3362)
AxelNoun Jul 19, 2026
8529146
linting
deruyter92 Jul 22, 2026
18bc897
Guard remaining CoW read-only .to_numpy()/.values mutations
AxelNoun Jul 25, 2026
b953416
fix pandas future mode tool: enable instead of "warn"
deruyter92 Jul 27, 2026
7101f3c
add tests for pd future mode tool
deruyter92 Jul 27, 2026
db296a8
Merge branch 'main' into fix/pandas3-cow-readonly-tonumpy
AxelNoun Jul 27, 2026
22ffb1b
fix test parametrization in test_pandas_cow
deruyter92 Jul 28, 2026
7c99777
Add copy=True to to_numpy in evaluate_multianimal peaks_gt
deruyter92 Jul 28, 2026
a3977ff
fix test_pandas_cow leaking global `infer_string` and `copy_on_write`…
deruyter92 Jul 28, 2026
fcdfc44
Add detector_eval_interval to train_from_coco to enable validation ou…
juan-cobos Aug 5, 2026
a5d5221
Merge remote-tracking branch 'upstream/main' into cy/init-mpl-version…
deruyter92 Aug 9, 2026
d5945cc
Fix get_colormap_names() legacy fallback to call _legacy_get_colormap…
deruyter92 Aug 9, 2026
d77d4f8
update version for "deprecated since .." warning
deruyter92 Aug 9, 2026
5e739e2
Add name override to deprecated() for legacy shim warnings
deruyter92 Aug 9, 2026
e6cfac6
Rename matplotlib_future_mode.py → matplotlib_compat.py
deruyter92 Aug 9, 2026
d9288fc
add tests for matplotlib compat layer
deruyter92 Aug 9, 2026
d359d83
add top-level None-type normalization in ProjectConfig
deruyter92 Aug 9, 2026
5471c9f
remove `ignore_empty` parameter in DLCBaseConfig.from_yaml
deruyter92 Aug 9, 2026
d4d73b3
add tests for None-type/empty ProjectConfig fields
deruyter92 Aug 9, 2026
eb63bda
fix docstring clarity and deprecation warning message
deruyter92 Aug 9, 2026
432cd25
fix config editor crash: QLabel does not accept a Path
deruyter92 Aug 12, 2026
75b317b
move config editor opening into DefaultTab
deruyter92 Aug 12, 2026
a1c901e
add tests for the DefaultTab config editor helper
deruyter92 Aug 12, 2026
f34a8c6
Fix incorrect scorer filtering in viz
C-Achard Aug 13, 2026
2f309c2
Add visualization evaluation regression tests
C-Achard Aug 13, 2026
13521a6
Validate plotting mode and clarify warnings
C-Achard Aug 13, 2026
3d15036
Fix unique bodypart plotting per row
C-Achard Aug 13, 2026
bed40e0
Harden evaluation plotting and typing
C-Achard Aug 13, 2026
fb4b16d
Normalize visualization file headers
C-Achard Aug 13, 2026
32fd7e7
Fix unique bodypart handling in plotting
C-Achard Aug 13, 2026
c1dfdc4
Clean up visualization shape and type naming
C-Achard Aug 13, 2026
947f28b
Don’t skip plots on individual label mismatch
C-Achard Aug 13, 2026
8eb9998
Guard unique-row plotting when data is missing
C-Achard Aug 13, 2026
c88f69c
Fix invalid fixture arg
C-Achard Aug 13, 2026
6bce484
Fix color indexing in eval visualization
C-Achard Aug 13, 2026
c422601
Fix evaluation plot color offset handling
C-Achard Aug 13, 2026
5bc3260
Add tests for unique bodypart evaluation plotting
C-Achard Aug 13, 2026
ba536eb
Normalize video path matching in extraction
C-Achard Aug 14, 2026
819fecc
Add frame extraction path handling tests
C-Achard Aug 14, 2026
012bf98
Fix missing closing parenthesis
C-Achard Aug 14, 2026
23b601e
Fix io import in frame extraction tests
C-Achard Aug 14, 2026
06dba4a
Merge branch 'cy/fix-frame-extraction-path-matching' of https://githu…
C-Achard Aug 14, 2026
74d25a3
Fix snapshot download rename mapping arg
C-Achard Aug 14, 2026
d59eb24
Warn when model zoo inference returns no poses
C-Achard Aug 14, 2026
d8dd415
Fix model download filename remapping
C-Achard Aug 17, 2026
b629813
Raise error when video pose predictions are empty
C-Achard Aug 17, 2026
39b883e
Use logging for eval warnings and update tests
C-Achard Aug 18, 2026
9be111c
Refactor labeled image rendering utilities
C-Achard Aug 18, 2026
974cb52
Simplify frame size handling in visualization
C-Achard Aug 18, 2026
2e01e34
Robust png replacement
C-Achard Aug 18, 2026
9fc4de5
Fix eval plot y-axis orientation
C-Achard Aug 18, 2026
1008670
Fix snapshot download rename mapping
C-Achard Aug 18, 2026
d0d4a9e
Fix super animal snapshot rename check
C-Achard Aug 19, 2026
fdfd856
Merge branch 'main' into cy/viz-fix-and-tests
C-Achard Aug 25, 2026
635bd3a
Empty - Trigger CI
C-Achard Aug 26, 2026
d63ba79
Tighten video path typing in frame extraction
C-Achard Aug 26, 2026
cf79a7d
Merge branch 'main' into cy/fix-modelzoo-humanbody-download
C-Achard Aug 26, 2026
6fc0ca3
Fix missing func usage
C-Achard Aug 28, 2026
c202539
fix WandbLoggerConfig include additional kwargs in pydantic schema
deruyter92 Aug 28, 2026
fcfb6eb
Treat a null box_score_thresh as the default detector threshold
rysk-t Aug 30, 2026
4746150
fix misaligned error log for GUI create videos tab
deruyter92 Aug 28, 2026
d7dec0a
Silence misleading video-extension warning for non-video collections
deruyter92 Aug 28, 2026
a18ede6
Fix h5->csv/nwb conversion silently skipping non-mp4 projects
deruyter92 Aug 28, 2026
6c33529
fix missing str conversion in failed_videos report for GUI CreateVide…
deruyter92 Aug 28, 2026
400454c
fix str missing conversion in plot_trajectories
deruyter92 Aug 28, 2026
ec58481
Fix outdated str function call to correct Path suffix swap for CSV to…
C-Achard Aug 28, 2026
f95cf13
fix `export_model` dumping objects in pickle: add `normalize=True` in…
deruyter92 Aug 28, 2026
b447933
add test to ensure export model writes only primitive types
deruyter92 Aug 31, 2026
6db842d
Normalize detector box score threshold
C-Achard Aug 31, 2026
21a0eb9
Refactor detector threshold tests to config level
C-Achard Aug 31, 2026
c61ddcc
Fix incorrect import
C-Achard Aug 31, 2026
c19bc07
Skip TensorFlow-backed tests cleanly when the tf extra is absent
natsukium Jul 20, 2026
1a729f6
Raise numpy lower bound to 1.22.4 for np.percentile's method= keyword
natsukium Jul 20, 2026
1107360
Replace NumPy 2.0-removed array APIs with supported equivalents
natsukium Jul 20, 2026
252183c
Estimate tracklet rank with a dense SVD instead of interpolative SVD
natsukium Jul 20, 2026
73f20e8
Compare montblanc assemblies order-independently in the assembler test
natsukium Jul 20, 2026
1f034ae
`Tracklet`: restore divsion-by-zero guard
deruyter92 Jul 27, 2026
1734b8e
add basic test for `Tracklet.estimate_rank`
deruyter92 Jul 27, 2026
4e0983d
deduplicate crossvalutils._calc_separability into a shared utility
deruyter92 Jul 27, 2026
b070e52
Hoist `np.trapz` compat check to module-level constants
deruyter92 Jul 27, 2026
9e611d5
Update uv.lock
C-Achard Aug 26, 2026
f304fba
fix COCOLoader max_individuals: make PoseConfig authorative, not json
deruyter92 Aug 5, 2026
488af3f
add basic tests for COCOLoader
deruyter92 Aug 5, 2026
4b733ba
refactor COCOLoader get_project_parameters: non-authorative bootstrap…
deruyter92 Aug 5, 2026
85fe2c0
Merge pull request #3436 from juan-cobos/feat/detector-eval-interval
deruyter92 Aug 31, 2026
f0e1c21
Merge pull request #3439 from deruyter92/jaap/project-config-null-nor…
deruyter92 Aug 31, 2026
43a47d9
Merge pull request #3441 from DeepLabCut/jaap/gui-config-editor-helper
deruyter92 Aug 31, 2026
c6ad634
Merge pull request #3445 from AxelNoun/fix/pandas3-cow-guards
deruyter92 Aug 31, 2026
085efe5
Merge pull request #3447 from DeepLabCut/cy/viz-fix-and-tests
deruyter92 Aug 31, 2026
b4ae22f
Merge pull request #3452 from DeepLabCut/cy/fix-modelzoo-humanbody-do…
deruyter92 Aug 31, 2026
73fca7b
update `test_apis_export`: strict type checking in asserting primitiv…
deruyter92 Aug 31, 2026
1c8da4e
Broaden video selection types and error message
C-Achard Aug 31, 2026
92f9413
Pass selected videos as Path objects in GUI
C-Achard Aug 31, 2026
8556ccf
Clarify extract_frames docstring and examples
C-Achard Aug 31, 2026
2df5f30
Raise on unmatched selected videos
C-Achard Aug 31, 2026
df59a9f
Update test_frame_extraction.py
C-Achard Aug 31, 2026
1150d78
Merge pull request #3473 from DeepLabCut/jaap/fix-export-model
deruyter92 Sep 1, 2026
aac861c
Fix model config test defaults for TD
C-Achard Sep 1, 2026
85072d9
Use config cutoff for detector boxes
C-Achard Sep 1, 2026
7893a35
Fix detector box_score_thresh handling
C-Achard Sep 1, 2026
5e58815
Clarify detector threshold docs
C-Achard Sep 1, 2026
3e2611e
Fix `create_video_with_all_detections` -> read default `bbox_pcutoff`…
deruyter92 Sep 1, 2026
baff692
revert shared box score threshold default -> inline config default
deruyter92 Sep 1, 2026
0f5330a
update docstring for `DetectorModelConfig`
deruyter92 Sep 1, 2026
256dfb9
add test for building config with null threshold
deruyter92 Sep 1, 2026
6e0d695
add `DefaultIfNone` normalizer to validation.py
deruyter92 Sep 1, 2026
aa4bedc
add tests for `DefaultIfNone` validator
deruyter92 Sep 1, 2026
95068f1
add tests for other config validators
deruyter92 Sep 1, 2026
83ab775
apply `DefaultIfNone` to `DetectorModelConfig.box_score_thresh`
deruyter92 Sep 1, 2026
2a00c40
Make `ProjectConfig.bboxes_pcutoff`` a property rather than a field
deruyter92 Sep 1, 2026
2689d30
add xfail for `ProjectConfig.bboxes_pcutoff`
deruyter92 Sep 1, 2026
27eb40d
Bump tornado from 6.5.7 to 6.5.8
dependabot[bot] Sep 1, 2026
b1dd957
Merge pull request #3470 from rysk-t/detector-bugfixes
deruyter92 Sep 3, 2026
685bb08
Merge pull request #3477 from DeepLabCut/main
deruyter92 Sep 3, 2026
390c5da
fix typo in docstring
deruyter92 Sep 3, 2026
61ea91a
partly revert c202539956f6f1da90ca2f32a93485643cabdc87: remove top-le…
deruyter92 Sep 3, 2026
55dd210
WandbLoggerConfig: add before validator that accepts top-level WandB …
deruyter92 Sep 3, 2026
dea2622
WandbLoggerConfig add field validator for wandb_kwargs (do not allow …
deruyter92 Sep 3, 2026
7f9a86c
add tests for WandbLoggerConfig
deruyter92 Sep 3, 2026
1af2081
Bump mkdocs-material from 9.7.6 to 9.7.7
dependabot[bot] Sep 3, 2026
e20e03a
fix gui tests: use `config_path` in `MainWindow` config tests
deruyter92 Sep 4, 2026
aaa86db
Merge pull request #3480 from DeepLabCut/jaap/fix-gui-test-config-path
deruyter92 Sep 4, 2026
b698519
Improve WandB logger error handling
C-Achard Sep 4, 2026
13ebd15
Add error handling tests for wandb logger
C-Achard Sep 4, 2026
0473dcd
Potential fix for pull request finding
deruyter92 Sep 4, 2026
e3d0bb8
Potential fix for pull request finding
deruyter92 Sep 4, 2026
11fe5c7
Update deeplabcut/pose_estimation_pytorch/config/logger.py
deruyter92 Sep 4, 2026
17080e8
copy dict `data` in `WandbLoggerConfig`
deruyter92 Sep 4, 2026
12c7be9
Merge branch 'dev' into cy/fix-frame-extraction-path-matching
C-Achard Sep 4, 2026
6bce3a2
fix WandbLoggerConfig missing return value
deruyter92 Sep 4, 2026
350e209
Import matplotlib.cm explicitly in the compat layer
C-Achard Sep 4, 2026
54ac2ac
Prevent duplicate legacy colormap registration
C-Achard Sep 4, 2026
b1c97fb
Route all plt.get_cmap calls throught the compat layer
C-Achard Sep 4, 2026
c1cf140
Use compat helper to clear matplotlib artists
C-Achard Sep 4, 2026
0e4a691
Patch matplotlib's Axes logger by name
C-Achard Sep 4, 2026
d3472c3
Test the name and stacklevel options of @deprecated
C-Achard Sep 4, 2026
0be8837
Expand matplotlib compat test coverage
C-Achard Sep 4, 2026
fe9d488
Remove unused compat helpers
C-Achard Sep 4, 2026
304bcc6
Remove unecessary tests
C-Achard Sep 4, 2026
4a82207
Merge pull request #3449 from DeepLabCut/cy/fix-frame-extraction-path…
deruyter92 Sep 4, 2026
aa007f6
Rename matplotlib compat module to utils
C-Achard Sep 4, 2026
0584d53
Replace matplotlib_compat colormap helpers
C-Achard Sep 4, 2026
1c15225
Simplify Matplotlib axes logger helper
C-Achard Sep 4, 2026
8a0ea71
Fix Matplotlib artist iteration
C-Achard Sep 4, 2026
1535b51
Merge pull request #3467 from DeepLabCut/jaap/pathlib-fixes
deruyter92 Sep 8, 2026
cc37cd0
Fix `tools` imports under a plain `pytest` invocation
deruyter92 Sep 7, 2026
44b47ca
Merge pull request #3475 from DeepLabCut/jaap/add-config-null-default…
deruyter92 Sep 8, 2026
3e8ce1a
Merge pull request #3469 from DeepLabCut/jaap/fix-wandb-logger-config
deruyter92 Sep 8, 2026
6646f58
Merge pull request #3481 from DeepLabCut/cy/harden-wandb-logger-errors
deruyter92 Sep 8, 2026
0e87bb2
docs: fix dead Model Zoo link in examples/README
wilmund Sep 7, 2026
91bff49
Merge pull request #3486 from wilmund/fix-dead-modelzoo-link
deruyter92 Sep 8, 2026
4cf9f35
Merge branch 'dev' into jaap/fix-create-video
deruyter92 Sep 8, 2026
fe1a73f
Merge pull request #3468 from DeepLabCut/jaap/fix-create-video
deruyter92 Sep 8, 2026
426943f
Merge branch 'dev' into cy/init-mpl-version-change
deruyter92 Sep 8, 2026
4c0907a
Merge pull request #3492 from DeepLabCut/main
deruyter92 Sep 8, 2026
7e8dcd2
Merge pull request #3484 from DeepLabCut/jaap/fix-tool-imports-pytest
deruyter92 Sep 8, 2026
2b7bce5
Tools: add knowledge indexing generator for an LLM-friendly docs index
deruyter92 Aug 12, 2026
c37bd0d
fix default output dir: `_build`
deruyter92 Aug 26, 2026
a2a5e0b
use full commit hash of repo
deruyter92 Aug 26, 2026
fe0fbfb
add duplicate check
deruyter92 Aug 26, 2026
65cc178
move knowledge index to JSONL + llms.txt
deruyter92 Aug 26, 2026
75ef519
add content hash and distinct `package_version` (in addition to docs …
deruyter92 Aug 26, 2026
9baf364
update pyproject.toml: fix missing `griffelib` dependency for knowled…
deruyter92 Aug 26, 2026
3ca5c90
allow separate adjustment of doc index and dev-docs api index
deruyter92 Aug 26, 2026
e309fcd
add CI workflows for knowlede-index deployment
deruyter92 Aug 26, 2026
cc46137
update docstrings and readme
deruyter92 Aug 26, 2026
6dc52fb
separate dependencies for leaner CI build&deployment of knowledge-index
deruyter92 Aug 26, 2026
6237e28
simplify llms.txt no per-page summary of doc sections. Prominent ref…
deruyter92 Aug 26, 2026
72f5494
update uv.lock
deruyter92 Aug 26, 2026
21d68b9
validate version labels + tighten manifest reading
deruyter92 Aug 27, 2026
c2db3dd
CI: harden reusable docs workflows against script injection
deruyter92 Aug 27, 2026
cf1f87d
fix delete version + refuse delete "main".
deruyter92 Aug 27, 2026
bf00732
Fix bug in checkout from tag merging into pre-existing checkout
deruyter92 Aug 27, 2026
06967d3
knowledge-index Fix bug in checkout from tag merging into pre-existin…
deruyter92 Aug 27, 2026
8b23cf9
check if dev-docs version is deployed before attempting a delete
deruyter92 Aug 27, 2026
908c6dc
mark dev-docs and knowledge-index dependencies as dependecy-group (ra…
deruyter92 Aug 27, 2026
67111b8
update uv.lock
deruyter92 Sep 4, 2026
e7ab837
add tests for tools/knowledge-indexing
deruyter92 Sep 4, 2026
ff047dd
set `ignore_errors=False` on `delete_version` -> raise if directory i…
deruyter92 Sep 7, 2026
0630f83
fix docs_index frontmatter matching via regex
deruyter92 Sep 7, 2026
6b618fb
update test_write: use real API dataclasses instead of `NameSpace` + …
deruyter92 Sep 7, 2026
e10d279
add tests for frontmatter matching
deruyter92 Sep 7, 2026
76cb4ae
conditional guard dependency install knowledge-indexing: deployment-o…
deruyter92 Sep 7, 2026
58461ed
Move intermediate nodes to schemas (importable without dependencies)
deruyter92 Sep 7, 2026
8ce5334
remove tests/tools/knowledge_indexing/conftest.py
deruyter92 Sep 7, 2026
fcad299
update pyproject.toml and uv lock - add markdow-it-py as dev dependency
deruyter92 Sep 7, 2026
e5bcc48
lazy import docutils (so it is not required at test-time)
deruyter92 Sep 7, 2026
57a9f93
Only the first H1 titles a docs page
deruyter92 Sep 7, 2026
2c884cb
Harden docs frontmatter and record ids
deruyter92 Sep 7, 2026
7864a65
Read docs section anchors from the built site instead of markdown
deruyter92 Sep 7, 2026
b5fb2a0
Resolve api.latest through the latest-release alias
deruyter92 Sep 7, 2026
4453c30
Separate symbol ids from submodule ids with `#`
deruyter92 Sep 7, 2026
02b4f95
Index class methods as their own api records
deruyter92 Sep 7, 2026
3eaa5e1
install knowlegde-index dependency group for testing
deruyter92 Sep 8, 2026
c96b3ff
add agents.md
deruyter92 Sep 7, 2026
a993fac
Update AGENTS.md
deruyter92 Sep 8, 2026
5a6061a
Update AGENTS.md
deruyter92 Sep 8, 2026
242174c
Update AGENTS.md
deruyter92 Sep 8, 2026
7a27dcc
Update AGENTS.md (incorporate review feedback)
deruyter92 Sep 8, 2026
1ff346f
Merge pull request #3453 from DeepLabCut/jaap/add-llm-knowledge-index
deruyter92 Sep 8, 2026
8af305a
Merge pull request #3483 from DeepLabCut/jaap/add-agents-md
deruyter92 Sep 8, 2026
31380eb
Merge branch 'dev' into cy/init-mpl-version-change
deruyter92 Sep 8, 2026
42d1b62
Merge pull request #3478 from DeepLabCut/dependabot/uv/mkdocs-materia…
deruyter92 Sep 8, 2026
c1f7aaa
Merge pull request #3476 from DeepLabCut/dependabot/uv/tornado-6.5.8
deruyter92 Sep 8, 2026
db6639e
bound dependencies for dependencies for knowledge-index dependency group
deruyter92 Sep 8, 2026
bbaa99f
discriminate between stable release and rolling main reference in llm…
deruyter92 Sep 8, 2026
de29151
Merge branch 'dev' into cy/init-mpl-version-change
deruyter92 Sep 9, 2026
8ebe974
Reject --skip-api on non-main labels as a no-op
deruyter92 Sep 9, 2026
0b594d9
Write empty JSONL as an empty file
deruyter92 Sep 9, 2026
c47f846
Avoid publishing empty knowledge version directories
deruyter92 Sep 9, 2026
9f87d15
Keep published-anchor occurrence counts aligned
deruyter92 Sep 9, 2026
c1664eb
update tests: docs link resolution and root API re-exports
deruyter92 Sep 9, 2026
29b1594
update uv.lock
deruyter92 Sep 9, 2026
795d2b0
Merge pull request #3394 from DeepLabCut/cy/init-mpl-version-change
deruyter92 Sep 9, 2026
7f504c0
Fix llms.txt API reference links 404s
C-Achard Sep 9, 2026
569b7fa
Fix API docs URLs for aliased symbols
C-Achard Sep 9, 2026
f8d9bb4
Clarify empty knowledge indexing error
C-Achard Sep 9, 2026
8480fc4
Fix docs URLs for re-exported class methods
C-Achard Sep 9, 2026
2a6e3a1
Clarify llms.txt API link test
C-Achard Sep 9, 2026
8c6db5c
fix numpy compat in crossvalutils and predict_multianimal
deruyter92 Sep 9, 2026
2b0ac75
Add test: cannot swap main/stable api links
C-Achard Sep 9, 2026
676f0f2
Update README.md
C-Achard Sep 9, 2026
8b0536b
Assert the api-autonav constants match dev-docs/mkdocs.yml
C-Achard Sep 9, 2026
a325912
Bump gitpython from 3.1.50 to 3.1.59
dependabot[bot] Sep 9, 2026
f2f16c2
Merge pull request #3501 from DeepLabCut/dependabot/uv/gitpython-3.1.59
deruyter92 Sep 9, 2026
053d105
Merge pull request #3498 from DeepLabCut/jaap/numpy-2-compatibility
deruyter92 Sep 9, 2026
1df0fc1
fix(trackingutils): take the real part of eig eigenvectors in Ellipse…
ender10 Sep 9, 2026
fce1e4f
Merge pull request #3502 from michael-denyer/fix/ellipse-fit-complex-eig
deruyter92 Sep 9, 2026
3ea6e61
Merge branch 'dev' into jaap/knowledge-index-improvements
C-Achard Sep 11, 2026
175e10f
Remove hard-coded overwrite
C-Achard Sep 14, 2026
3461180
Add proper overwrite kwarg to analyze_videos
C-Achard Sep 14, 2026
da969d4
Pass overwrite to tracklets generation as well
C-Achard Sep 14, 2026
c0a6b11
Avoid shuffling existing positional calls
C-Achard Sep 14, 2026
7be7ca3
Fix overwrite forwarding in analyze_videos
C-Achard Sep 14, 2026
96f0c44
Follow-up fixes for numpy2 compatibility (#3508)
C-Achard Sep 14, 2026
5599af5
Tighten strict handling for canary CI cells (#3509)
C-Achard Sep 14, 2026
0417ae2
remove dependency upper bounds for numpy and matplotlib (#3499)
deruyter92 Sep 14, 2026
1c38232
Merge pull request #3494 from DeepLabCut/jaap/knowledge-index-improve…
deruyter92 Sep 14, 2026
a897636
Harden regression test for issue #3513
deruyter92 Sep 14, 2026
5ea382c
improve docstring in analyze_videos
deruyter92 Sep 14, 2026
fd4a89f
Add regression check for overwrite in auto-track path
deruyter92 Sep 14, 2026
60998ef
Merge pull request #3514 from DeepLabCut/cy/fix-overwrite-torch-kwarg
deruyter92 Sep 14, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
331 changes: 202 additions & 129 deletions deeplabcut/generate_training_dataset/frame_extraction.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,38 @@
# Licensed under GNU Lesser General Public License v3.0
#

from collections.abc import Collection, Iterable
from pathlib import Path


def _filter_config_videos(
configured_videos: Iterable[str | Path],
selected_videos: Collection[str | Path] | None,
) -> list[str | Path]:
"""Return configured video keys matching the selected paths.

The original configuration keys are preserved so they remain valid for
subsequent dictionary lookups. If no selection is provided, all configured
videos are returned.

Raises:
ValueError: If a selection is provided but none of its paths match a
configured video.
"""
configured_videos = list(configured_videos)

if selected_videos is None:
return configured_videos

selected = {Path(video) for video in selected_videos}
videos = [video for video in configured_videos if Path(video) in selected]

if not videos:
raise ValueError("None of the selected videos matched the video paths stored in the project configuration.")

return videos


def select_cropping_area(config: str | Path, videos=None):
"""Interactively select the cropping area of all videos in the config. A user
interface pops up with a frame to select the cropping parameters. Use the left click
Expand Down Expand Up @@ -71,154 +100,194 @@ def extract_frames(
slider_width=25,
config3d=None,
extracted_cam=0,
videos_list=None,
videos_list: list[str | Path] | None = None,
):
"""Extracts frames from the project videos.

Frames will be extracted from videos listed in the config.yaml file.
"""Extract frames from videos in a DeepLabCut project.

The frames are selected from the videos in a randomly and temporally uniformly
distributed way (``uniform``), by clustering based on visual appearance
(``k-means``), or by manual selection.
Videos are read from the project's ``config.yaml`` file. When
``videos_list`` is provided, only matching configured videos are processed.

After frames have been extracted from all videos from one camera, matched frames
from other cameras can be extracted using ``mode = "match"``. This is necessary if
you plan to use epipolar lines to improve labeling across multiple camera angles.
It will overwrite previously extracted images from the second camera angle if
necessary.
In ``"automatic"`` mode, frames are selected either at approximately
uniform temporal intervals with ``algo="uniform"`` or by clustering
downsampled frames by visual appearance with ``algo="kmeans"``. In
``"manual"`` mode, the first selected video is opened in the napari plugin for
interactive frame selection and cropping.

Please refer to the user guide for more details on methods and parameters
https://www.nature.com/articles/s41596-019-0176-0 or the preprint:
https://www.biorxiv.org/content/biorxiv/early/2018/11/24/476531.full.pdf
In ``"match"`` mode, frame numbers already extracted for one camera are
used to extract corresponding frames from the other cameras in a 3D
project. Existing PNG frames in the output directories for those cameras
may be removed and replaced.

Args:
config (string): Full path of the config.yaml file as a string.
mode (string): Either ``"automatic"``, ``"manual"`` or ``"match"``.
String containing the mode of extraction. It must be either ``"automatic"`` or
``"manual"`` to extract the initial set of frames. It can also be ``"match"``
to match frames between the cameras in preparation for the use of epipolar line
during labeling; namely, extract from camera_1 first, then run this to extract
the matched frames in camera_2.

WARNING: if you use ``"match"``, and you previously extracted and labeled
frames from the second camera, this will overwrite your data. This will require
you to delete the ``collectdata(.h5/.csv)`` files before labeling. Use with
caution!
algo (string): Either ``"kmeans"`` or ``"uniform"``. Defaults to ``"kmeans"``.
String specifying the algorithm to use for selecting the frames. Currently,
deeplabcut supports either ``kmeans`` or ``uniform`` based selection. This flag
is only required for ``automatic`` mode and the default is ``kmeans``. For
``"uniform"``, frames are picked in temporally uniform way, ``"kmeans"``
performs clustering on downsampled frames (see user guide for details).

NOTE: Color information is discarded for ``"kmeans"``, thus e.g. for
camouflaged octopus clustering one might want to change this.
crop (bool or str, optional): If ``True``, video frames are cropped according to the corresponding
coordinates stored in the project configuration file. Alternatively, if
cropping coordinates are not known yet, crop=``"GUI"`` triggers a user
interface where the cropping area can be manually drawn and saved.
userfeedback (bool, optional): If this is set to ``False`` during ``"automatic"`` mode then frames for all
videos are extracted. The user can set this to ``"True"``, which will result in
a dialog, where the user is asked for each video if (additional/any) frames
from this video should be extracted. Use this, e.g. if you have already labeled
some folders and want to extract data for new videos. Defaults to True.
cluster_resizewidth (int): For ``"k-means"`` one can change the width to which the images are downsampled
(aspect ratio is fixed). Defaults to 30.
cluster_step (int): By default each frame is used for clustering, but for long videos one could
only use every nth frame (set using this parameter). This saves memory before
clustering can start, however, reading the individual frames takes longer due
to the skipping. Defaults to 1.
cluster_color (bool): If ``"False"`` then each downsampled image is treated as a grayscale vector
(discarding color information). If ``"True"``, then the color channels are
considered. This increases the computational complexity. Defaults to False.
opencv (bool): Uses openCV for loading and extracting (otherwise moviepy (legacy)). Defaults to True.
slider_width (int): Width of the video frames slider, in percent of window. Defaults to 25.
config3d (string, optional): Path to the project configuration file in the 3D project. This will be used to
match frames extracted from all cameras present in the field 'camera_names' to
the frames extracted from the camera given by the parameter 'extracted_cam'.
extracted_cam (int): The index of the camera that already has extracted frames. This will match
frame numbers to extract for all other cameras. This parameter is necessary if
you wish to use epipolar lines in the labeling toolbox. Only use if
``mode='match'`` and ``config3d`` is provided. Defaults to 0.
videos_list (list[str]): A list of the string containing full paths to videos to extract frames for. If
this is left as ``None`` all videos specified in the config file will have
frames extracted. Otherwise one can select a subset by passing those paths. Defaults to None.
config: Path to the project ``config.yaml`` file.
mode: Frame-extraction mode. Supported values are:

* ``"automatic"``: Select and extract frames automatically.
* ``"manual"``: Open the interactive frame-selection interface.
* ``"match"``: Extract corresponding frames from the other
cameras in a 3D project.

Defaults to ``"automatic"``.
algo: Selection algorithm. ``"uniform"`` selects
frames at approximately uniform temporal intervals, while
``"kmeans"`` clusters downsampled frames by visual appearance.
This parameter is only used in ``"automatic"`` mode. Defaults to
``"kmeans"``.
crop: Cropping behavior. If ``True``, frames are cropped using the
coordinates stored in the project configuration. If ``"GUI"``,
an interface is opened to select and save cropping coordinates
before extraction. If ``False``, frames are not cropped. Defaults
to ``False``.
userfeedback: Whether to ask before extracting frames from each video
in ``"automatic"`` mode. If ``False``, all selected videos are
processed without this prompt. Defaults to ``True``.
cluster_step: Use every nth frame as input to k-means clustering.
Increasing this value can reduce the number of frames held for
clustering. This parameter is only used when
``mode="automatic"`` and ``algo="kmeans"``. Defaults to ``1``.
cluster_resizewidth: Width, in pixels, to which frames are resized
before k-means clustering. The aspect ratio is preserved. This
parameter is only used when ``mode="automatic"`` and
``algo="kmeans"``. Defaults to ``30``.
cluster_color: Whether k-means clustering uses color information. If
``False``, each downsampled frame is treated as a grayscale
vector. If ``True``, its color channels are retained, increasing
the computational cost. This parameter is only used when
``mode="automatic"`` and ``algo="kmeans"``. Defaults to ``False``.
opencv: Whether to use OpenCV-based video loading and frame extraction.
If ``False``, the legacy MoviePy implementation is used for
automatic extraction. Defaults to ``True``.
slider_width: Width of the frame-selection slider as a percentage of
the window width. This parameter is used in ``"manual"`` mode.
Defaults to ``25``.
config3d: Path to the configuration file of the associated 3D project.
Required in ``"match"`` mode to identify the project cameras.
Defaults to ``None``.
extracted_cam: Index in the 3D project's ``camera_names`` list of the
camera for which frames have already been extracted. The
corresponding frame numbers are extracted for the remaining
cameras in ``"match"`` mode. Defaults to ``0``.
videos_list: Full paths of the configured videos to process. Entries
may be strings or `pathlib.Path` objects. The original
configuration keys are retained after matching. If ``None``, all
configured videos applicable to the selected mode are processed.
Defaults to ``None``.

Returns:
None
In ``"automatic"`` mode, a list of booleans with one entry for each
video considered by the extraction loop. ``True`` indicates that no
valid selected frames were extracted from that video, and ``False``
indicates success or that extraction was skipped by the user. An
empty list may be returned if frame selection produces no frames.

In ``"manual"`` and ``"match"`` modes, returns ``None``.

Raises:
ValueError: If ``videos_list`` is provided but none of its paths match
the video paths in the project configuration, or if a required
video cannot be found while matching cameras.
RuntimeError: If no videos are processed.
Exception: If automatic extraction settings in ``config.yaml`` are
invalid, or if ``"match"`` mode cannot load a valid 3D project
configuration.

Warning:
``mode="match"`` may remove and replace previously extracted PNG
frames for cameras other than ``extracted_cam``. If those frames have
already been labeled, their associated annotation data may no longer
correspond to the extracted images.

Note:
Use the function ``add_new_videos`` at any stage of the project to add new videos
to the config file and extract their frames.

The following parameters for automatic extraction are used from the config file
Automatic extraction reads ``numframes2pick``, ``start``, and ``stop``
from the project configuration.

* ``numframes2pick``
* ``start`` and ``stop``
Use `deeplabcut.add_new_videos` to add videos to the project
configuration before extracting frames from them.

While selecting the frames manually, you do not need to specify the ``crop``
parameter in the command. Rather, you will get a prompt in the graphic user
interface to choose if you need to crop or not.
In ``"manual"`` mode, cropping is selected through the interactive
interface rather than through the ``crop`` argument.

Examples:
To extract frames automatically with 'kmeans' and then crop the frames
Extract frames automatically using k-means clustering:

deeplabcut.extract_frames(
config='/analysis/project/reaching-task/config.yaml',
mode='automatic',
algo='kmeans',
crop=True,
)
```python
deeplabcut.extract_frames(
"/analysis/project/reaching-task/config.yaml",
mode="automatic",
algo="kmeans",
)
```

To extract frames automatically with 'kmeans' and then defining the cropping area
using a GUI
Select cropping coordinates interactively before automatic
extraction:

deeplabcut.extract_frames(
'/analysis/project/reaching-task/config.yaml',
'automatic',
'kmeans',
'GUI',
)
```python
deeplabcut.extract_frames(
"/analysis/project/reaching-task/config.yaml",
mode="automatic",
algo="kmeans",
crop="GUI",
)
```

To consider the color information when extracting frames automatically with
'kmeans':
Include color information during k-means clustering:

deeplabcut.extract_frames(
'/analysis/project/reaching-task/config.yaml',
'automatic',
'kmeans',
cluster_color=True,
)
```python
deeplabcut.extract_frames(
"/analysis/project/reaching-task/config.yaml",
mode="automatic",
algo="kmeans",
cluster_color=True,
)
```

To extract frames automatically with 'uniform' and then crop the frames
Extract uniformly selected, cropped frames:

deeplabcut.extract_frames(
'/analysis/project/reaching-task/config.yaml',
'automatic',
'uniform',
crop=True,
)
```python
deeplabcut.extract_frames(
"/analysis/project/reaching-task/config.yaml",
mode="automatic",
algo="uniform",
crop=True,
)
```

To extract frames manually
Extract frames only from selected configured videos:

deeplabcut.extract_frames(
'/analysis/project/reaching-task/config.yaml', 'manual'
)
```python
from pathlib import Path

deeplabcut.extract_frames(
"/analysis/project/reaching-task/config.yaml",
mode="automatic",
videos_list=[
Path("/analysis/project/reaching-task/videos/reaching1.mp4"),
Path("/analysis/project/reaching-task/videos/reaching2.mp4"),
],
)
```

To extract frames manually, with a 60% wide frames slider
Open the manual frame-selection interface:

deeplabcut.extract_frames(
'/analysis/project/reaching-task/config.yaml', 'manual', slider_width=60,
)
```python
deeplabcut.extract_frames(
"/analysis/project/reaching-task/config.yaml",
mode="manual",
slider_width=60,
)
```

To extract frames from a second camera that match the frames extracted from the
first
Extract frames from the other cameras that correspond to frames
extracted from the first camera:

deeplabcut.extract_frames(
'/analysis/project/reaching-task/config.yaml',
mode='match',
extracted_cam=0,
)
```python
deeplabcut.extract_frames(
"/analysis/project/reaching-task/config.yaml",
mode="match",
config3d="/analysis/project/reaching-3d/config.yaml",
extracted_cam=0,
)
```
"""
import re
import sys
Expand All @@ -230,14 +299,14 @@ def extract_frames(

from deeplabcut.utils import auxiliaryfunctions, frameselectiontools

videos_list = None if videos_list is None else [Path(video) for video in videos_list]

config_file = Path(config)
cfg = auxiliaryfunctions.read_config(config_file)
print("Config file read successfully.")

if videos_list is None:
videos = list(cfg.get("video_sets_original") or cfg["video_sets"])
else: # filter video_list by the ones in the config file
videos = [v for v in cfg["video_sets"] if v in videos_list]
configured_videos = list(cfg.get("video_sets_original") or cfg["video_sets"])
videos = _filter_config_videos(configured_videos, videos_list)

if mode == "manual":
from deeplabcut.gui.widgets import launch_napari
Expand Down Expand Up @@ -407,7 +476,11 @@ def extract_frames(
else: # NO!
has_failed.append(False)

if all(has_failed):
if not has_failed:
raise RuntimeError(
"No frames were extracted. The project configuration lists no videos, or none could be opened."
)
elif all(has_failed):
print("Frame extraction failed. Video files must be corrupted.")
return has_failed
elif any(has_failed):
Expand All @@ -427,9 +500,9 @@ def extract_frames(
config_file = Path(config)
cfg = auxiliaryfunctions.read_config(config_file)
print("Config file read successfully.")
videos = sorted(cfg["video_sets"].keys())
if videos_list is not None: # filter video_list by the ones in the config file
videos = [v for v in videos if v in videos_list]

videos = _filter_config_videos(sorted(cfg["video_sets"]), videos_list)

project_path = Path(config).parents[0]
labels_path = project_path / "labeled-data"
try:
Expand Down
Loading
Loading