Visitar URL original
Update TensorFlow backend for Python 3.12 compatibility (Keras 2 via `tf-keras`) by maximpavliv · Pull Request #3087 · DeepLabCut/DeepLabCut · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
aa1cafe
Relax pyside6
maximpavliv Sep 4, 2025
ae989b1
Relax tensorflow non-Windows
maximpavliv Sep 4, 2025
a7a8a86
Add higher python releases to CI/CD
maximpavliv Sep 4, 2025
05cf09a
Relax tensorflow in CI/CD
maximpavliv Sep 4, 2025
17c4e13
Relax pytables in CI/CD
maximpavliv Sep 4, 2025
0f7c454
Remove pytables from CI/CD installation
maximpavliv Sep 4, 2025
79e54a9
Install tf-keras for tensorflow>=2.16 support
maximpavliv Sep 4, 2025
40d159a
Add TF_USE_LEGACY_KERAS env var to CI/CD
maximpavliv Sep 6, 2025
2a1de93
TMP Add tables to requirements?
maximpavliv Sep 6, 2025
bf51dbc
Add _tf_legacy.py
maximpavliv Sep 8, 2025
57ca3f6
TMP try install specific wrapt version?
maximpavliv Sep 8, 2025
b4aca6e
Set WRAPT_DISABLE_EXTENSIONS env var
maximpavliv Sep 8, 2025
74b48a0
Exclude Windows & python=3.13 from CI/CD
maximpavliv Sep 8, 2025
01c3008
TMP tf macos pin
maximpavliv Sep 8, 2025
d597152
Remove python 3.12 from cicd
maximpavliv Sep 8, 2025
d08878e
Try pin tensorflow-macos==2.15.0
maximpavliv Sep 8, 2025
e6655f7
TMP try unpin tensorflow-macos and add tf-keras
maximpavliv Sep 8, 2025
a6fd9a9
Replace macos-14 by macos-latest
maximpavliv Sep 9, 2025
8b8ec8c
Set dataloader_workers to 0
maximpavliv Sep 9, 2025
f1ccf01
Add tables to setup.py
maximpavliv Sep 9, 2025
db39c94
Relax tensorflow-macos
maximpavliv Sep 9, 2025
fb8bd4e
Update 3mice colab installation
maximpavliv Sep 9, 2025
23d4bc3
Update modelzoo reid colab installations
maximpavliv Sep 9, 2025
6fe5238
Merge branch 'main' into maxim/relax_dependencies
maximpavliv Sep 12, 2025
4cc497d
Merge branch 'main' into maxim/relax_dependencies
maximpavliv Sep 16, 2025
1f3806e
Merge branch 'main' into maxim/relax_dependencies
maximpavliv Sep 22, 2025
8c8dbf6
Merge branch 'main' into maxim/relax_dependencies
maximpavliv Oct 1, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 5 additions & 10 deletions .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-14, windows-latest]
python-version: ["3.10"]
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ["3.10", "3.11", "3.12"]
include:
- os: ubuntu-latest
path: ~/.cache/pip
- os: macos-14
- os: macos-latest
path: ~/Library/Caches/pip
- os: windows-latest
path: ~\AppData\Local\pip\Cache
Expand All @@ -38,18 +38,13 @@ jobs:
shell: bash -el {0} # Important: activates the conda environment
if: runner.os != 'macOS'
run: |
pip install tensorflow==2.12.0 tensorpack==0.11 tf_slim==1.1.0
pip install tensorflow tensorpack==0.11 tf_slim==1.1.0 tf-keras

- name: Install TensorFlow on macOS
shell: bash -el {0} # Important: activates the conda environment
if: runner.os == 'macOS'
run: |
pip install tensorflow-macos==2.12.0 tensorpack==0.11 tf_slim==1.1.0

- name: Install PyTables through Conda
shell: bash -el {0} # Important: activates the conda environment
run: |
conda install pytables==3.8.0 "numpy<2"
pip install tensorflow-macos tensorpack==0.11 tf_slim==1.1.0 tf-keras

- name: Install dependencies
shell: bash -el {0} # Important: activates the conda environment
Expand Down
2 changes: 2 additions & 0 deletions deeplabcut/pose_estimation_tensorflow/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@
# Licensed under GNU Lesser General Public License v3.0
#

from . import _tf_legacy

# Suppress tensorflow warning messages
import tensorflow as tf
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)
Expand Down
12 changes: 12 additions & 0 deletions deeplabcut/pose_estimation_tensorflow/_tf_legacy.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import os
import sys

os.environ.setdefault("TF_USE_LEGACY_KERAS", "1")
os.environ.setdefault("WRAPT_DISABLE_EXTENSIONS", "1")

try:
import tf_keras.src.legacy_tf_layers as legacy_tf_layers
sys.modules["tf_keras.legacy_tf_layers"] = legacy_tf_layers
except ImportError:
# Older tf-keras didn’t use src/, so nothing to do
pass
40 changes: 2 additions & 38 deletions examples/COLAB/COLAB_3miceDemo.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -36,32 +36,7 @@
"- a quick guide to maDLC: https://deeplabcut.github.io/DeepLabCut/docs/tutorial.html\n",
"- a demo COLAB for how to use maDLC on your own data: https://github.com/DeepLabCut/DeepLabCut/blob/main/examples/COLAB/COLAB_maDLC_TrainNetwork_VideoAnalysis.ipynb\n",
"\n",
"### To get started, please go to \"Runtime\" ->\"change runtime type\"->select \"Python3\", and then select \"GPU\"\n",
"\n",
"As the COLAB environments were updated to CUDA 12.X and Python 3.11, we need to install DeepLabCut and TensorFlow in a distinct way to get TensorFlow to connect to the GPU."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "HoNN2_0Z9rr_"
},
"outputs": [],
"source": [
"# Install TensorFlow, tensorpack and tf_slim versions compatible with DeepLabCut\n",
"!pip install \"tensorflow==2.12.1\" \"tensorpack>=0.11\" \"tf_slim>=1.1.0\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Downgrade PyTorch to a version using CUDA 11.8 and cudnn 8\n",
"# This will also install the required CUDA libraries, for both PyTorch and TensorFlow\n",
"!pip install torch==2.3.1 torchvision --index-url https://download.pytorch.org/whl/cu118"
"### To get started, please go to \"Runtime\" ->\"change runtime type\"->select \"Python3\", and then select \"GPU\""
]
},
{
Expand All @@ -71,18 +46,7 @@
"outputs": [],
"source": [
"# Install the correct (older) version of DeepLabCut\n",
"!pip install deeplabcut==2.3.11"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# As described in https://www.tensorflow.org/install/pip#step-by-step_instructions, \n",
"# create symbolic links to NVIDIA shared libraries:\n",
"!ln -svf /usr/local/lib/python3.11/dist-packages/nvidia/*/lib/*.so* /usr/local/lib/python3.11/dist-packages/tensorflow"
"!pip install \"deeplabcut[tf]\""
]
},
{
Expand Down
38 changes: 2 additions & 36 deletions examples/COLAB/COLAB_DLC_ModelZoo.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -45,30 +45,7 @@
"\n",
"## **Let's get going: install DeepLabCut into COLAB:**\n",
"\n",
"*Also, be sure you are connected to a GPU: go to menu, click Runtime > Change Runtime Type > select \"GPU\"*\n",
"\n",
"As the COLAB environments were updated to CUDA 12.X and Python 3.11, we need to install DeepLabCut and TensorFlow in a distinct way to get TensorFlow to connect to the GPU."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Install TensorFlow, tensorpack and tf_slim versions compatible with DeepLabCut\n",
"!pip install \"tensorflow==2.12.1\" \"tensorpack>=0.11\" \"tf_slim>=1.1.0\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Downgrade PyTorch to a version using CUDA 11.8 and cudnn 8\n",
"# This will also install the required CUDA libraries, for both PyTorch and TensorFlow\n",
"!pip install torch==2.3.1 torchvision --index-url https://download.pytorch.org/whl/cu118"
"*Also, be sure you are connected to a GPU: go to menu, click Runtime > Change Runtime Type > select \"GPU\"*\n"
]
},
{
Expand All @@ -78,18 +55,7 @@
"outputs": [],
"source": [
"# Install the latest version of DeepLabCut\n",
"!pip install \"git+https://github.com/DeepLabCut/DeepLabCut.git#egg=deeplabcut[modelzoo]\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# As described in https://www.tensorflow.org/install/pip#step-by-step_instructions, \n",
"# create symbolic links to NVIDIA shared libraries:\n",
"!ln -svf /usr/local/lib/python3.11/dist-packages/nvidia/*/lib/*.so* /usr/local/lib/python3.11/dist-packages/tensorflow"
"!pip install --pre \"deeplabcut[tf,modelzoo]\""
]
},
{
Expand Down
108 changes: 3 additions & 105 deletions examples/COLAB/COLAB_transformer_reID.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -32,91 +32,18 @@
"- a quick guide to maDLC: https://deeplabcut.github.io/DeepLabCut/docs/tutorial.html\n",
"- a demo COLAB for how to use maDLC on your own data: https://github.com/DeepLabCut/DeepLabCut/blob/main/examples/COLAB/COLAB_maDLC_TrainNetwork_VideoAnalysis.ipynb\n",
"\n",
"### To get started, please go to \"Runtime\" ->\"change runtime type\"->select \"Python3\", and then select \"GPU\"\n",
"\n",
"As the COLAB environments were updated to CUDA 12.X and Python 3.11, we need to install DeepLabCut and TensorFlow in a distinct way to get TensorFlow to connect to the GPU."
"### To get started, please go to \"Runtime\" ->\"change runtime type\"->select \"Python3\", and then select \"GPU\"\n"
]
},
{
"cell_type": "markdown",
"source": [
"‼️ **Attention: this demo is for maDLC, which is version 2.2**\n",
"- the installation is very slow on Colab due to the several steps needed to use older versions of `torch` and `dlc`."
"‼️ **Attention: this demo is for maDLC, which is version 2.2**\n"
],
"metadata": {
"id": "xOe2hvy85EVP"
}
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "6_Q4_GKM45Oe"
},
"outputs": [],
"source": [
"# Downgrade PyTorch to a version using CUDA 11.8 and cudnn 8\n",
"# This will also install the required CUDA libraries, for both PyTorch and TensorFlow\n",
"!pip install torch==2.3.1 torchvision --index-url https://download.pytorch.org/whl/cu118"
]
},
{
"cell_type": "code",
"source": [
"import torch\n",
"print(torch.cuda.is_available())\n",
"print(torch.cuda.get_device_name(0))\n",
"!nvcc --version # sometimes not available"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "3HLeBRGL9jYJ",
"outputId": "b8e49307-f51c-4f12-8dec-d2c2929928e2"
},
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"True\n",
"NVIDIA A100-SXM4-40GB\n",
"nvcc: NVIDIA (R) Cuda compiler driver\n",
"Copyright (c) 2005-2024 NVIDIA Corporation\n",
"Built on Thu_Jun__6_02:18:23_PDT_2024\n",
"Cuda compilation tools, release 12.5, V12.5.82\n",
"Build cuda_12.5.r12.5/compiler.34385749_0\n"
]
}
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Vz3B6Zyf45Od"
},
"outputs": [],
"source": [
"# Install TensorFlow, tensorpack and tf_slim versions compatible with DeepLabCut\n",
"!pip install \"tensorflow==2.12.1\" \"tensorpack>=0.11\" \"tf_slim>=1.1.0\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "UUpac75T45Of"
},
"outputs": [],
"source": [
"# As described in https://www.tensorflow.org/install/pip#step-by-step_instructions,\n",
"# create symbolic links to NVIDIA shared libraries:\n",
"!ln -svf /usr/local/lib/python3.11/dist-packages/nvidia/*/lib/*.so* /usr/local/lib/python3.11/dist-packages/tensorflow"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -126,15 +53,13 @@
"outputs": [],
"source": [
"# Install DLC version 2.2-2.3 (pre DLC3):\n",
"!pip install deeplabcut==2.3.11"
"!pip install \"deeplabcut[tf]\""
]
},
{
"cell_type": "code",
"source": [
"import tensorflow\n",
"import deeplabcut\n",
"import torch\n",
"import os"
],
"metadata": {
Expand All @@ -143,33 +68,6 @@
"execution_count": 3,
"outputs": []
},
{
"cell_type": "code",
"source": [
"print(\"DLC version: \", deeplabcut.__version__)\n",
"print(\"torch version: \",torch.__version__)\n",
"print(\"tensorflow version: \",tensorflow.__version__)\n"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "wRojzQO98X2H",
"outputId": "fad889d6-15da-4888-8138-283f1f13cf99"
},
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"DLC version: 2.3.11\n",
"torch version: 2.3.1+cu118\n",
"tensorflow version: 2.12.1\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
Expand Down
2 changes: 2 additions & 0 deletions examples/testscript_pytorch_multi_animal.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,12 +64,14 @@ def main(
"train_settings.display_iters": 50,
"train_settings.epochs": epochs_,
"train_settings.batch_size": batch_size,
"train_settings.dataloader_workers": 0,
"runner.device": device,
"runner.snapshots.save_epochs": save_epochs,
"runner.snapshots.max_snapshots": max_snapshots_to_keep,
"detector.train_settings.display_iters": 1,
"detector.train_settings.epochs": detector_epochs,
"detector.train_settings.batch_size": detector_batch_size,
"detector.train_settings.dataloader_workers": 0,
"detector.runner.snapshots.save_epochs": save_epochs,
"detector.runner.snapshots.max_snapshots": max_snapshots_to_keep,
"logger": logger,
Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ scikit-image>=0.17
scikit-learn>=1.0
scipy>=1.9
statsmodels>=0.11
tables==3.8.0
tables
torch==2.7.1
torchvision
tqdm
9 changes: 6 additions & 3 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,23 +78,26 @@ def pytorch_config_paths() -> list[str]:
"pycocotools",
"pyyaml",
"Pillow>=7.1",
"tables",
],
extras_require={
"gui": [
"pyside6==6.4.2",
"pyside6",
"qdarkstyle==3.1",
"napari-deeplabcut>=0.2.1.6",
],
"openvino": ["openvino-dev==2022.1.0"],
"docs": ["numpydoc"],
"tf": [
"tensorflow>=2.0,<=2.10;platform_system=='Windows'",
"tensorflow>=2.0,<=2.12;platform_system!='Windows'",
"tensorflow>=2.0;platform_system!='Windows'",
"tf-keras;platform_system!='Windows'",
"tensorpack>=0.11",
"tf_slim>=1.1.0",
], # Last supported TF version on Windows Native is 2.10
"apple_mchips": [
"tensorflow-macos<2.13.0",
"tensorflow-macos",
"tf-keras",
"tensorflow-metal",
"tensorpack>=0.11",
"tf_slim>=1.1.0",
Expand Down