Repository navigation
MNT Add nightly workflow that tests the latest released version - #1809
Open
SpiliosDimakopoulos wants to merge 2 commits into
Open
SpiliosDimakopoulos wants to merge 2 commits into
SpiliosDimakopoulos wants to merge 2 commits into
Conversation
2 of 7 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #1188.
This adds a
test-latest-release.ymlworkflow and calls it fromnightly.yml. It looks up the latest Fairlearn version on PyPI, checks out the matchingv<version>tag, installs it with the current versions of the dependencies and runs the same pytest command that the release itself used intest-all-deps.yml(without coverage). This way a dependency release that breaks the version users actually install shows up in the nightly run, not only through bug reports.Following the discussion in the issue, it installs from the checked-out release tag (
pip install .) rather than from PyPI, so that the tests and the package come from the same commit. A step checks that the installedfairlearn.__version__matches the PyPI version. The workflow also hasworkflow_dispatch, so it can be started by hand for a quick check after a dependency release.I could not run the workflow on GitHub Actions from my fork (no runs were created for it), so I ran the same steps on an Ubuntu 24.04 machine with Python 3.12 instead. Against today's dependencies (pandas 3.1.1, numpy 2.5.4, scikit-learn 1.9.1, scipy 1.18.1), the v0.15.0 tests give
2408 failed, 4523 passed. The failures I looked at come frompandas.errors.Pandas4Warning: Setting a value on a Series via .at with a key that does not exist in the index is deprecated, which is turned into an error byfilterwarnings = ["error"]. This is already fixed onmainby #1801 but not released yet, so the new job would currently be red. That is the kind of signal this job is meant to give.Tests
Documentation