Visitar URL original
Speed up the AllVersionsCrossVersion flaky quarantine by cobexer · Pull Request #39494 · gradle/gradle · GitHub
Skip to content

Speed up the AllVersionsCrossVersion flaky quarantine - #39494

Merged
cobexer merged 3 commits into
releasefrom
cobexer/release-flaky-quarantine-speedup
Oct 19, 2026
Merged

cobexer merged 3 commits into
releasefrom
cobexer/release-flaky-quarantine-speedup

Conversation

@cobexer

@cobexer cobexer commented Oct 19, 2026

Copy link
Copy Markdown
Member

Backports the flaky quarantine speedups from master:

The coverage carries roughly 146 minutes of quarantined test execution
on a single agent, which puts Windows right at the timeout. Use the
TeamCity parallel tests feature to split it into 4 batches, as the
regular functional test builds already do.

(cherry picked from commit 8a24685)
Under -PflakyTests=ONLY the filter is a JUnit Platform tag evaluated
during discovery inside the test JVM, so Gradle cannot know ahead of
time that a task selects nothing and forks a JVM for it anyway. For
AllVersionsCrossVersion that is one task per tested Gradle version per
subproject: a quarantine bucket runs 1121 test tasks, 1074 of which
discover nothing, and those empty forks alone account for 511 of its
768 minutes.

Each project now registers a cacheable DetectFlakyTests task per test
source set that scans its sources, never its classpath, for the Flaky
annotation. A Test task whose source set declares none is skipped
instead of forking a JVM to find that out.

The quarantine also runs quickFeedbackCrossVersionTest rather than
allVersionsCrossVersionTest. Every surviving task is now a real one and
costs minutes to unpack the distribution under test, and whether a test
still flakes does not depend on which patch release it runs against, so
one target per major carries the same signal.

With the empty tasks gone the coverage no longer needs tcParallelTests
and runs on a single agent again. Windows drops from 214 minutes over
four agents to 21 on one, Linux from 99 to 10.

AI tools were used to implement and verify this change.

(cherry picked from commit 8ce2b90)
PR #39110 pointed the AllVersionsCrossVersion quarantine at
quickFeedbackCrossVersionTest to cut its runtime. That makes the
coverage a duplicate: QuickFeedbackCrossVersion quarantines already
exist for Linux and Windows on the same MIN_VERSION JVM, so the two
pairs of builds run an identical task set. It also drops the target
list from one per minor release to one per major, in the only build
that runs @flaky cross-version tests at all -- and a cross-version
test is quarantined precisely because its behaviour is target
dependent in some way we do not understand yet.

The cost problem is now handled by the other half of #39110: with
DetectFlakyTests skipping the empty test tasks, the full
allVersionsCrossVersionTest quarantine fits on a single agent inside
the 240 minute timeout.

(cherry picked from commit 2878591)
@cobexer cobexer self-assigned this Oct 19, 2026
@cobexer
cobexer marked this pull request as ready for review October 19, 2026 10:08
@cobexer
cobexer requested review from a team as code owners October 19, 2026 10:08
@github-actions github-actions Bot added the @cortex Issues/PRs to add to Cortex team's Project Board label Oct 19, 2026
@cobexer cobexer added a:chore Minor issue without significant impact in:building-gradle gradle/gradle build labels Oct 19, 2026
@cobexer
cobexer added this pull request to the merge queue Oct 19, 2026
@bot-gradle bot-gradle added this to the 9.8.2 milestone Oct 19, 2026
Merged via the queue into release with commit 8de7bcf Oct 19, 2026
17 checks passed
@cobexer
cobexer deleted the cobexer/release-flaky-quarantine-speedup branch October 19, 2026 11:11
@mlopatkin
mlopatkin removed request for a team October 20, 2026 11:20
@github-actions github-actions Bot removed the @cortex Issues/PRs to add to Cortex team's Project Board label Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

a:chore Minor issue without significant impact in:building-gradle gradle/gradle build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants