Is there an existing issue for this?

• I have searched the existing issues

Operating System

Linux

DeepLabCut version

3.0.2

What engine are you using?

pytorch

DeepLabCut mode

multi animal

Device type

intel+ H100

Bug description 🐛

I start my video analysis as
deeplabcut.analyze_videos(config = config_path, videos = [video_path], video_extensions = ".mkv", shuffle = 7, save_as_csv = True)

but I get this error that I did not have before (I only changed "videotype" option to "video_extensions" to fit with the new API interface, -- nothing else changed);

Traceback (most recent call last):
  File "/scratch/user/u.mu172596/Projects/MiceTesting/HPC_Scripts/./miceNIR_video_analysis_code.py", line 10, in <module>
    deeplabcut.analyze_videos(config = config_path, videos = [video_path], video_extensions = ".mkv", shuffle = 7, save_as_csv = True, auto_track = True)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/deprecation.py", line 198, in wrapper
    return fn(*args, **kwargs)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/deprecation.py", line 198, in wrapper
    return fn(*args, **kwargs)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/compat.py", line 835, in analyze_videos
    return analyze_videos(
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/deprecation.py", line 198, in wrapper
    return fn(*args, **kwargs)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/pose_estimation_pytorch/apis/videos.py", line 421, in analyze_videos
    loader = DLCLoader(
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/pose_estimation_pytorch/data/dlcloader.py", line 76, in __init__
    super().__init__(self._project_root, self._project_root, model_config_path)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/deprecation.py", line 198, in wrapper
    return fn(*args, **kwargs)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/pose_estimation_pytorch/data/base.py", line 67, in __init__
    normalized_config: PoseConfig = PoseConfig.from_any(model_config)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/config/base_config.py", line 92, in from_any
    return cls.from_yaml(config)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/config/base_config.py", line 108, in from_yaml
    cfg = cls.from_dict(yaml_dict)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/config/base_config.py", line 346, in from_dict
    return super().from_dict(cfg_dict)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/config/base_config.py", line 82, in from_dict
    return cls.model_validate(cfg_dict)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/pydantic/main.py", line 732, in model_validate
    return cls.__pydantic_validator__.validate_python(
pydantic_core._pydantic_core.ValidationError: 1 validation error for PoseConfig
runner.load_scheduler_state_dict
  Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool]
    For further information visit https://errors.pydantic.dev/2.13/v/extra_forbidden

Steps To Reproduce

Other than running the command above, not sure

Relevant log output

Traceback (most recent call last):
  File "/scratch/user/u.mu172596/Projects/MiceTesting/HPC_Scripts/./miceNIR_video_analysis_code.py", line 10, in <module>
    deeplabcut.analyze_videos(config = config_path, videos = [video_path], video_extensions = ".mkv", shuffle = 7, save_as_csv = True, auto_track = True)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/deprecation.py", line 198, in wrapper
    return fn(*args, **kwargs)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/deprecation.py", line 198, in wrapper
    return fn(*args, **kwargs)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/compat.py", line 835, in analyze_videos
    return analyze_videos(
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/deprecation.py", line 198, in wrapper
    return fn(*args, **kwargs)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/pose_estimation_pytorch/apis/videos.py", line 421, in analyze_videos
    loader = DLCLoader(
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/pose_estimation_pytorch/data/dlcloader.py", line 76, in __init__
    super().__init__(self._project_root, self._project_root, model_config_path)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/deprecation.py", line 198, in wrapper
    return fn(*args, **kwargs)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/pose_estimation_pytorch/data/base.py", line 67, in __init__
    normalized_config: PoseConfig = PoseConfig.from_any(model_config)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/config/base_config.py", line 92, in from_any
    return cls.from_yaml(config)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/config/base_config.py", line 108, in from_yaml
    cfg = cls.from_dict(yaml_dict)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/config/base_config.py", line 346, in from_dict
    return super().from_dict(cfg_dict)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/deeplabcut/core/config/base_config.py", line 82, in from_dict
    return cls.model_validate(cfg_dict)
  File "/scratch/user/u.mu172596/.conda/envs/dlc3/lib/python3.10/site-packages/pydantic/main.py", line 732, in model_validate
    return cls.__pydantic_validator__.validate_python(
pydantic_core._pydantic_core.ValidationError: 1 validation error for PoseConfig
runner.load_scheduler_state_dict
  Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool]
    For further information visit https://errors.pydantic.dev/2.13/v/extra_forbidden

Anything else?

*** the config file:

Project definitions (do not edit)

Task: MiceTesting
scorer: Umorin
date: May31
multianimalproject: true
identity: false

Project path (change when moving around)

project_path: /scratch/user/u.mu172596/Projects/MiceTesting

Default DeepLabCut engine to use for shuffle creation (either pytorch or tensorflow)

engine: pytorch

Annotation data set configuration (and individual video cropping parameters)

video_sets:
/scratch/user/u.mu172596/Projects/MiceTesting/videos/test_2_mice_run_2_part_000.mkv:
crop: 0, 1280, 0, 1024
/scratch/user/u.mu172596/Projects/MiceTesting/videos/test_2_mice_run_2_part_001.mkv:
crop: 0, 1280, 0, 1024
/scratch/user/u.mu172596/Projects/MiceTesting/videos/test_2_mice_run_2_part_002.mkv:
crop: 0, 1280, 0, 1024
/scratch/user/u.mu172596/Projects/MiceTesting/videos/test_2_mice_run_2_part_003.mkv:
crop: 0, 1280, 0, 1024
/scratch/user/u.mu172596/Projects/MiceTesting/videos/test_2_mice_run_2_part_004.mkv:
crop: 0, 1280, 0, 1024
/scratch/user/u.mu172596/Projects/MiceTesting/videos/test_2_mice_run_2_part_005.mkv:
crop: 0, 1280, 0, 1024
individuals:

• mouse_1
• mouse_2
  uniquebodyparts: []
  multianimalbodyparts:
• nose
• lower_jaw
• left_eye
• right_eye
• left_ear
• right_ear
• head_top
• head_back
• neck_lower
• back_mid
• back_lower
• tail_base
• tail_prox_third
• tail_mid
• tail_dist_third
• tail_tip
• left_elbow
• left_front_paw
• right_elbow
• right_front_paw
• left_knee
• left_heel
• left_hind_paw
• right_knee
• right_heel
• right_hind_paw
  bodyparts: MULTI!

Fraction of video to start/stop when extracting frames for labeling/refinement

start: 0
stop: 1
numframes2pick: 200

Plotting configuration

skeleton:

• [nose, head_top]
• [lower_jaw, head_top]
• [left_eye, head_top]
• [right_eye, head_top]
• [left_ear, head_top]
• [right_ear, head_top]
• [head_top, head_back]
• [head_back, neck_lower]
• [neck_lower, back_mid]
• [back_mid, back_lower]
• [back_lower, tail_base]
• [tail_base, tail_prox_third]
• [tail_prox_third, tail_mid]
• [tail_mid, tail_dist_third]
• [tail_dist_third, tail_tip]
• [left_elbow, neck_lower]
• [left_front_paw, left_elbow]
• [right_elbow, neck_lower]
• [right_front_paw, right_elbow]
• [left_knee, back_lower]
• [left_heel, left_knee]
• [left_hind_paw, left_heel]
• [right_knee, back_lower]
• [right_heel, right_knee]
• [right_hind_paw, right_heel]
  skeleton_color: gray
  pcutoff: 0.6
  dotsize: 8
  alphavalue: 0.7
  colormap: rainbow

Training,Evaluation and Analysis configuration

TrainingFraction:

• 0.95
  iteration: 0
  default_net_type: resnet_50
  default_augmenter: albumentations
  default_track_method: ellipse
  snapshotindex: -1
  detector_snapshotindex: -1
  batch_size: 8

Cropping Parameters (for analysis and outlier frame detection)

cropping: false
#if cropping is true for analysis, then set the values here:
x1: 0
x2: 640
y1: 277
y2: 624

Refinement configuration (parameters from annotation dataset configuration also relevant in this stage)

corner2move2:

• 50
• 50
  move2corner: true

Conversion tables to fine-tune SuperAnimal weights

SuperAnimalConversionTables:
detector_batch_size: 8

*** and the pytorch_config.yaml

data:
bbox_margin: 40
colormode: RGB
inference:
normalize_images: true
auto_padding:
pad_width_divisor: 32
pad_height_divisor: 32
train:
affine:
p: 0.5
rotation: 30
scaling:
- 0.5
- 1.25
translation: 0
crop_sampling:
width: 448
height: 448
max_shift: 0.1
method: hybrid
gaussian_noise: 12.75
motion_blur: true
normalize_images: true
device: auto
inference:
multithreading:
enabled: true
queue_length: 4
timeout: 30.0
compile:
enabled: false
backend: inductor
autocast:
enabled: false
metadata:
project_path: /scratch/user/u.mu172596/Projects/MiceTesting
pose_config_path:
/scratch/user/u.mu172596/Projects/MiceTesting/dlc-models-pytorch/iteration-0/MiceTestingMay31-trainset95shuffle7/train/pytorch_config.yaml
bodyparts:

• nose
• lower_jaw
• left_eye
• right_eye
• left_ear
• right_ear
• head_top
• head_back
• neck_lower
• back_mid
• back_lower
• tail_base
• tail_prox_third
• tail_mid
• tail_dist_third
• tail_tip
• left_elbow
• left_front_paw
• right_elbow
• right_front_paw
• left_knee
• left_heel
• left_hind_paw
• right_knee
• right_heel
• right_hind_paw
  unique_bodyparts: []
  individuals:
• mouse_1
• mouse_2
  with_identity: false
  method: bu
  model:
  backbone:
  type: HRNet
  model_name: hrnet_w48
  freeze_bn_stats: false
  freeze_bn_weights: false
  interpolate_branches: true
  increased_channel_count: false
  backbone_output_channels: 720
  heads:
  bodypart:
  type: DEKRHead
  weight_init: dekr
  target_generator:
  type: DEKRGenerator
  num_joints: 26
  pos_dist_thresh: 17
  bg_weight: 0.1
  criterion:
  heatmap:
  type: DEKRHeatmapLoss
  weight: 1
  offset:
  type: DEKROffsetLoss
  weight: 0.03
  predictor:
  type: DEKRPredictor
  apply_sigmoid: false
  use_heatmap: false
  clip_scores: true
  num_animals: 2
  keypoint_score_type: combined
  max_absorb_distance: 75
  nms_threshold: 0.05
  apply_pose_nms: true
  heatmap_config:
  channels:
  - 720
  - 48
  - 27
  num_blocks: 1
  dilation_rate: 1
  final_conv_kernel: 1
  offset_config:
  channels:
  - 720
  - 390
  - 26
  num_offset_per_kpt: 15
  num_blocks: 2
  dilation_rate: 1
  final_conv_kernel: 1
  net_type: dekr_w48
  runner:
  type: PoseTrainingRunner
  gpus:
  key_metric: test.mAP
  key_metric_asc: true
  eval_interval: 10
  optimizer:
  type: AdamW
  params:
  lr: 0.0001
  scheduler:
  type: LRListScheduler
  params:
  lr_list:
  • • 1e-05
  • • 1e-06
      milestones:
  • 90
  • 120
    load_scheduler_state_dict: false
    snapshots:
    max_snapshots: 3
    save_epochs: 10
    save_optimizer_state: false
    train_settings:
    batch_size: 64
    dataloader_workers: 4
    dataloader_pin_memory: true
    display_iters: 1000
    epochs: 50
    seed: 42
    with_center_keypoints: true

Code of Conduct

• I agree to follow this project's Code of Conduct