Visitar URL original
Release/3.0 by C-Achard · Pull Request #42 · conda-forge/deeplabcut-feedstock · GitHub
Skip to content

Release/3.0 - #42

Draft
C-Achard wants to merge 14 commits into
conda-forge:mainfrom
C-Achard:release/3.0
Draft

C-Achard wants to merge 14 commits into
conda-forge:mainfrom
C-Achard:release/3.0

Conversation

@C-Achard

@C-Achard C-Achard commented Jun 4, 2026 •

Copy link
Copy Markdown

Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

@hmaarrfk Hello, myself and @deruyter92 are part of the DLC core dev team. We just released 3.0 recently and were wondering if we could help maintain this more directly.

I updated the recipe in this PR, will continue work on this until the build succeeds, happy to have any feedback.

Note

I commented out all patches, mostly so we can re-evaluate whether they are needed, fix what we can upstream, and add new ones or re-enable them only if needed.
Happy to have your input (or happy to answer on your issue: DeepLabCut/DeepLabCut#2887)

Thanks for your work on this and best regards,

Cyril

C-Achard added 6 commits June 4, 2026 13:48
Update conda recipe context: bump version from 2.3.11 to 3.0.0 and add python_min: "3.10" to indicate the package requires Python 3.10 or newer.
In recipe/recipe.yaml: pass --no-build-isolation to pip to use host build dependencies, replace separate CLI/GUI entry_points with a single dlc -> deeplabcut.__main__:main entry point, and add a host requirement for setuptools >=61 to ensure a compatible build toolchain.
Revise run dependencies in recipe.yaml to modernize the environment and align with newer ML tooling. Bump dlclibrary to >=0.0.12 and replace the previous legacy scientific stack with packages such as albumentations, einops, filelock (>=3.12,<3.16), huggingface_hub, packaging>=26, pandas>=2.2,<3, pycocotools, pydantic>=2,<3, pytorch>=2, timm, and torchvision. Keep tensorflow as a runtime dependency and add run_constraints requiring tensorflow>=2.12. Also update napari-deeplabcut constraint to >=0.3.1. These changes simplify and modernize the dependency set for compatibility with newer libraries and toolchains.
Let's see which of these are problematic and either fix upstream or re-enable/update here :)
@C-Achard
C-Achard requested a review from hmaarrfk as a code owner June 4, 2026 11:56
@C-Achard

C-Achard commented Jun 4, 2026

Copy link
Copy Markdown
Author

@conda-forge-admin, please rerender

@conda-forge-admin

Copy link
Copy Markdown
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/recipe.yaml) and found it was in an excellent condition.

@conda-forge-admin

Copy link
Copy Markdown
Contributor

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you but ran into some issues. Please check the output logs of the GitHub Actions workflow below for more details. You can also ping conda-forge/core (using the @ notation) for further assistance or you can try rerendering locally.

The following suggestions might help debug any issues:

  • Is the recipe/{meta.yaml,recipe.yaml} file valid?
  • If there is a recipe/conda-build-config.yaml file in the feedstock make sure that it is compatible with the current global pinnings.
  • Is the fork used for this PR on an organization or user GitHub account? Automated rerendering via the webservices admin bot only works for user GitHub accounts.

This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/26950210571. Examine the logs at this URL for more detail.

@C-Achard
C-Achard marked this pull request as draft June 4, 2026 12:18
@hmaarrfk

hmaarrfk commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Congratulations!!!

Long ago, DLC’s stance was « just use our dockers ». Has that idea changed?

I would love to see the upper bounds removed on nearly all packages.

I’ve worked over the years to fix many things that were surrounding larger compatibility with the community.

So with that. I would appreciate an in earnest code review from some of the (which you have indicated you were willing to do in your comment)

we package the tensorflow and PyTorch differently than upstream. So things like « protobuf » are managed automatically and don’t need pins.

@C-Achard

C-Achard commented Jun 4, 2026

Copy link
Copy Markdown
Author

cc @deruyter92

@hmaarrfk

hmaarrfk commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Looking back at this:
DeepLabCut/DeepLabCut#2887

I see that my main two points to make DLC easier to package were:

  1. Make things agnostic to PySide6 vs PyQt6
  2. Vendor tf_slim

Comment thread recipe/recipe.yaml
- numba >=0.54
- matplotlib-base >=3.3,!=3.7.0,!=3.7.1
- imgaug >=0.4
- matplotlib-base >=3.3,<3.9,!=3.7,!=3.7.1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i would really like to see this pin released.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is likely the most doable

Comment thread recipe/recipe.yaml
- numpy >=1.18.5
- pandas >=1.0.1,!=1.5.0
- numba >=0.54
- numpy >=1.18.5,<2

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is virtually no reason in 2026 to pin numpy to less than 2. this would be "anti" conda-forge in my opinion.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For numpy the main blocker is the tf-cu11 optional dependency. We could drop it without dropping tensorflow altogether, which should allow us to finally release this

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason: as @C-Achard mentioned, the academic nature of DeepLabCut warrants a strong focus on backward compatibility with older TensorFlow projects. Since TensorFlow does not specify upper-bounds in their own metadata, we have to take care of that ourselves. Besides TF, also imgaug (unmaintained since 2020) and possibly other packages rely on numpy 1.x still.

@hmaarrfk for the 3.1 DeepLabCut release, we will make sure to move all TF-related packages as optional dependency. And probably be able to release this numpy upper-bound in the main dependency list. The question now is, would you be ok with keeping the upperbound for the 3.0 release on conda-forge?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If numpy 2 has real incompatibilities then it is fine to pin. But I would really want to see a reference to an upstream issue.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentionned in my issues I posted in 2887. I’m pretty sure I unblocked things at conda-forge by patching tf-life to modernize it. So I think numpy2 works well at conda forge.

@C-Achard C-Achard Jun 10, 2026 •

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread recipe/recipe.yaml
- numpy >=1.18.5,<2
- opencv
- packaging >=26
- pandas >=2.2,<3

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

likewise for pandas version 3, this upper bound should be removed in 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is a bit trickier, we are trying to move away from pandas, but it will take quite some time to get to a proper agnostic state. Not sure if our timeline could include fixing all deprecated pandas usage in the code, but that would be easier than replacing it entirely

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pandas v3 was released beginning of this year and introduced breaking changes for DeepLabCut. We can likely fix these break points for DeepLabCut 3.1 release, before moving away from Pandas altogether. But since pandas lives at the core of DeepLabCut, this upperbound is really needed for now until migration is fully taken care of.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add the reference to the upstream issue that discusses the breakage.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the reference just above the pandas 3 pin in the recipe itself.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread recipe/recipe.yaml Resolved
Co-authored-by: Mark Harfouche <mark.harfouche@gmail.com>
@C-Achard

C-Achard commented Jun 5, 2026

Copy link
Copy Markdown
Author
  1. Make things agnostic to PySide6 vs PyQt6

Definitely agree there, this would be medium effort I think and we can still explicitly test only on PySide. It would also protect us from future transitions like PySide 2 to 6.
However, we would rather avoid having users have to specify or take extra steps to install either PyQt or PySide, this tends to be confusing (e.g. napari) so we'd rather specify one in deps while importing from the qtpy compat in the code.

  1. Vendor tf_slim

We are planning to drop tensorflow, ideally before the end of the year. There are still a lot of improvements we'd like to bring to the code before doing this though. Not sure how much effort that would be, if it creates an extra maintenance burden it would not be ideal.

@C-Achard

C-Achard commented Jun 5, 2026

Copy link
Copy Markdown
Author

@hmaarrfk Looking at this, I think most of these are concerns we should definitely address moving forward; back to this PR, would you like us to review the patches and re-enable the relevant ones like qtpy ? Or what would you recommend we do before merging this ?

It would be great to have 3.0 up on this channel soon, if possible, but definitely want to fix what we can even if it's through patches for now

@hmaarrfk

hmaarrfk commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

I understand that there is a concern of "backward" compatibility.

But I would like to ask that you take a second to listn to this great talk from one of the creators of conda
https://www.youtube.com/watch?v=Kamld5Z-xx0

I think that if the conda-froge package has become popular, it is because I too care about packaging and maintainance and backward compatibility.

Truthfully, I am concerned with "upstream developers" coming in and adding alot of pins because "that is what they do for pypi". Truthfully, we are a little "looser" with the definition of packages and will patch to enable "compatibility with newer software".

This is why, you should be able to run DLC2 with very modern ternsoflow (without an upper bound) on conda-forge.

You do get cuda constraints, and ultimately, you have yoru docker releases for this.

Having it conda-forge for me is the ability to have it "remixed" with other software, as such, I would really insist on releasing the numpy and pandas pins so we can run on Numpy 2 and Pandas 3, or at least, really understand why this is "100% impossible".

Again, we have "patched" tf-slim so that it maintains compatibility with modern tensorflow at conda-forge, so i think the "upper bound" you fear from pypi isn't really applicable here.

https://github.com/conda-forge/tf-slim-feedstock/blob/main/recipe/meta.yaml

If there is a bug that surfaces from removing the upper bounds today, i would love to pointed to it, so we can fix it.

@conda-forge-admin

Copy link
Copy Markdown
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I was trying to look for recipes to lint for you, but it appears we have a merge conflict. Please try to merge or rebase with the base branch to resolve this conflict.

Please ping the 'conda-forge/core' team (using the @ notation in a comment) if you believe this is a bug.

@conda-forge-admin

Copy link
Copy Markdown
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/recipe.yaml) and found it was in an excellent condition.

Comment thread recipe/recipe.yaml
# on e.g. python 3.12 with numpy 2.4.6.
# See example of failing tests:
# https://github.com/C-Achard/DeepLabCut/actions/runs/27232381795/job/80415647150?pr=9
- numpy >=1.18.5,<2

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- numpy >=1.18.5,<2
- numpy >=1.18.5,<2.2.6

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still incompatible with the tf-cu11 case, and will not be solved simply by patching imgaug as it is due to the tensorflow binaries being compiled with numpy<2.
See https://github.com/C-Achard/DeepLabCut/actions/runs/33783747433/job/100743544489?pr=11, which here was run against the latest main commit for convenience, but the same failure would happen on 3.0 as it has the same tf-cu11 pin (using tensorflow==2.14).

See C-Achard/DeepLabCut@b800d92#diff-50c86b7ed8ac2cf95bd48334961bf0530cdc77b5a56f852c5c61b89d735fd711L107 for the exact dependencies this ran with (note these optional extras were introduced in DeepLabCut/DeepLabCut#3292).

@hmaarrfk

Copy link
Copy Markdown
Contributor

i really want to warn against the numpy 2 upper bound. i would really want to "fix" imgaug and patch it so that it works with numpy 2. there seems to be a submitted patch to the upstream repo that we can backport.

The problem is that conda-forge has mostly moved to numpy2. so by having <2 you are releasing "a dead version".

@roaldarbol

roaldarbol commented Sep 3, 2026 •

Copy link
Copy Markdown

@C-Achard Is it the plan to resume this? I'd be happy to try and jump in to help get the feedstock back up to date if any help is needed. Would love to see v3 on conda-forge!

@C-Achard

C-Achard commented Sep 3, 2026 •

Copy link
Copy Markdown
Author

Hi @roaldarbol, thanks for your message, yes, we'd very much like to get 3.0 shipped, and any help is welcome.

cc @hmaarrfk @deruyter92

We've been working on the points above for the upcoming 3.1 version, keeping in mind the points that were re-raised here about compatibility.

A small update on our side:

We'd genuinely like the dependency stack to be simplified, it's a burden for us as much as for anyone else. It's just difficult for us to retroactively change what 3.0 is built against once it's out.

On PySide6/PyQt6 via qtpy: that one's still open on our side, as mentioned. If it matters enough to you to help reintroduce the patch here or an upstream PR on DLC, or just scoping what it'd take, we'd be very thankful, it'd free up bandwidth we need to ship 3.1, which has a lot of important changes and fixes.

For the numpy pin because of imgaug the break appears is some np.sctypes calls, e.g. in aleju/imgaug#868. If that can be patched, perhaps the numpy upper bound can be dropped entirely rather than just narrowed, happy to have anyone with bandwidth test it, otherwise we'll get to it as we can.

With that, I think this recipe was otherwise in good shape to move forward, aside from what we discussed here about pinned versions.

If we can at least get a clear list all together of what truly is and isn't blocking on this side it would help us a lot already. 3.1 will solve or improve a lot of the friction around those points hopefully but it'd be nice to have 3.0 up and running still.

Thanks again and best,
Cyril

@hmaarrfk

hmaarrfk commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Great to see the progress.

For me, it is really about introducing a new package today that is simply less compatible (numpy 2 compatibility being the critical one) than the one i had "lightly patched" before.

I'm very much for the "rolling release model"

You all have done so much great work. Just tag what you have today as 3.1?

@C-Achard

C-Achard commented Sep 3, 2026

Copy link
Copy Markdown
Author

i really want to warn against the numpy 2 upper bound. i would really want to "fix" imgaug and patch it so that it works with numpy 2. there seems to be a submitted patch to the upstream repo that we can backport.

Note that as mentioned in #42 (comment), merely patching imgaug will not be sufficient, as the problem stems from tensorflow binaries rather than imgaug.

@C-Achard

C-Achard commented Sep 3, 2026

Copy link
Copy Markdown
Author

Just tag what you have today as 3.1?

3.1 is still being developed. We would like to finalize the recipe for 3.0 specifically here.

@C-Achard

C-Achard commented Sep 3, 2026

Copy link
Copy Markdown
Author

For me, it is really about introducing a new package today that is simply less compatible (numpy 2 compatibility being the critical one) than the one i had "lightly patched" before.

Could you link the validation showing that the previously patched package installed and functioned correctly with NumPy 2? In particular, pytest or CI results would help establish whether removing the pin is supported by evidence comparable to the failures linked above.

@hmaarrfk

hmaarrfk commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

just install the package, i have been using it with numpy 2..... for a while.

hmm you might be right....

@C-Achard

C-Achard commented Sep 4, 2026

Copy link
Copy Markdown
Author

Thanks for checking. Shall we keep the pin for now and proceed with the review?

@hmaarrfk

hmaarrfk commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Please try to take it from my perspective i've been maintaining this conda-forge pacakge for years, and tried to work with your team to ensure that there is bilateral communication

DeepLabCut/DeepLabCut#1549

What really matters to me is broader compatibility.

However, as it stands, this PR mostly feels like:

  • "We are upstream"
  • "I now care about this"
  • "I don't really care about the broad compatibility that makes conda-forge great"
  • "Please take my PR that deletes all of your patches"
  • "We promise to do something in the future".

as of today, numpy 2 is 2 years old. You can probably fix your compatibility issues as an olive branch, and release before removing tensorflow compatibility.

Historically, the response to conda-forge has been:

We belive we should release a docker image, so do what you want with conda-forge.

If conda-forge has become popular, it is because I've emphasized maintenance (when I was maintaining this regularly) with updated versions of scientific packages. For example i've worked to release upstream's development version of tf-slim to help with modern tensorflow compatibility https://github.com/conda-forge/tf-slim-feedstock/blob/main/recipe/meta.yaml to give DLC 2 an extra 3-4 years of life.

My ask:

  1. Get numpy2 to work. I really think this is a "low lift".
  2. Please honestly review my patches, those were hand written before AI, and I believe are strong contenders for upstreaming. You may ask AI to simply port them over.
  3. Please work to remove upper bounds on packages.
    • Especially those that are not "today's latest version"

@hmaarrfk

hmaarrfk commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Pandas 3 might be the only exception i see today, but you should cross reference your upstream issue, where you are actively working toward it.

Again, this is what I think is important for a good "software distribution at conda-forge".

@hmaarrfk

hmaarrfk commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

At conda forge it often ends up being helping other packages get maintained too when one hits roadblocks.

conda-forge/albucore-feedstock#51

@hmaarrfk

hmaarrfk commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

so basically, albumentations, which chagned business models, good for them for trying something new, can't really be used in any 2026 release.

I threw claude at things, I think you can too, and this is in part why I think some "problems" should be addressed head on

#46

@hmaarrfk

hmaarrfk commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

In my efffort i asked claude to look at your patches to backport and to use as many of them as it could, you seem to have already done a tremendous amount of effort playing nice with dependencies.

Albumentations was holding you all back, i just ripped it out.

@deruyter92

deruyter92 commented Sep 7, 2026 •

Copy link
Copy Markdown

@hmaarrfk

What really matters to me is broader compatibility.
However, as it stands, this PR mostly feels like:

  • "We are upstream"
  • "I now care about this"
  • "I don't really care about the broad compatibility that makes conda-forge great"
  • "Please take my PR that deletes all of your patches"
  • "We promise to do something in the future".

I'm really sorry but I think we've left the wrong impression on you here. We do appreciate your dedication to broader compatibility, and share the same ambition. Maybe just to be clear, @C-Achard opened this PR with all good intentions to ask your expertise on how to settle for an acceptable 3.0 distribution.

We acknowledge that there are many improvements to be made with respect to the dependencies and are working hard on improving the broader compatibility. Thanks for all your suggestions in this regard, they have been helpful.

Since we cannot change our own release retroactively, we'd rather focus on the future, by further improving the codebase for next releases. We are more than willing to help out with small patches in this PR, but honestly we really have our hands quite full already. Unfortunately we really don't have the resources to implement radical patches (e.g. fully replacing albumentations). We appreciate all your efforts here and trust your expertise on this regard.

I'd propose this: let's move try to close this PR in an acceptable form (maybe the broad compatibility is less ideal than you have in mind for this version, but will be better for next) and let us focus on improving the upstream codebase directly (see our recent efforts here).

Happy to receive further feedback. And thanks again for everything!

@hmaarrfk

hmaarrfk commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Sorry I came hard, I guess I've really tried in the past to both share the value of conda-forge, and to upstream some suggestions, they were just met with lackluster enthusiasm and it feels backward to me "not to attempt to rebase them".

For example, the PySide6->pyqt took a few minutes with claude, and then I just decided to keep pulling at the thread. I'm obviously not going to ship the whole tensorflow thing.

I'm a little torn because:

  1. I think many of my patches from Aug 2026 and before (i.e. before I just threw AI at DLC) were very good. I did hand write these, thinking through the downstream consequence of each line.
  2. Honestly I'm a little over the daily maintenance of this deeplabcut feedstock.
  3. I truely believe, with every bone of my OSS hat, that any code that requires numpy<2 in 2026 is doing a disservice to its end users.
    • The reason is that downgrading packages with conda(-forge) really messes with the solver, so typing in conda install "deeplabcut>=3" would probably throw the solver in a 30 min spin, even with mamba.
  4. I really think you all should consider delaying dropping tensorflow, and releasing the version you have now on main/master that plays nicer with numpy/pandas.
  5. Really patch DLC3 to make it modern #46 can be left as a reference for what you can do with AI and a computer with a GPU to port over your code.
    • I definitely didn't give explicit instructions to test "loading old projects" which I know is likely a huge usecase for you all.
    • I think the albumentations stuff is very likely safe to merge in your own code. Anybody can flip an image, and I feel like with the whole "left hand" and "right hand" problem you all struggled with using stock augmentations.
    • I honestly haven't read more than 10 lines of code from Really patch DLC3 to make it modern #46.

So how to move forward from here:

  1. Please feel free to update this to the best version you can get to.
  2. We don't like orphan files at conda-forge, so if you are going to drop my patches, just drop them.
  3. Really consider backporting fixes, or just releasing a new version so you can have broader numpy compatibility
  4. Please try to maintain my paches (you can likely pull them from Really patch DLC3 to make it modern #46)
  5. Please don't block my future patches to enhance compatibility, even if it is "against you wishes" as an upstream project.
    • If I try to rewrite the whole tensorflow blackend, I'll probably just fork at that stage and not really release on this feedstock.

@hmaarrfk

hmaarrfk commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

I would strongly recommend listening to https://www.youtube.com/watch?v=Kamld5Z-xx0&t=8s

which was foundational for me to learn how to "distribute" software.

Deeplabcut, as you all market it upstream, is definitely a "fixed application". However, if you want to integrate it in other libraries, you want to give downstream users a little more flexibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants