Visitar URL original
CI perf: Gradle patch compatibility — PR runs repeat ci.yml's Gradle e2e tier and run nightly-only extras (~17,000 Linux job-min/day) · Issue #1177 · SocketDev/socket-patch · GitHub
Skip to content

CI perf: Gradle patch compatibility — PR runs repeat ci.yml's Gradle e2e tier and run nightly-only extras (~17,000 Linux job-min/day) #1177

Description

Measurement

Window: 2026-10-01 → 2026-10-08. Runs: 453 Gradle patch compatibility pull_request runs in 7d (431 that weren't skipped, ~62/day; 185 in the last 24h across 50 PRs). Jobs sampled: 67 PR runs (40 successful).

  • Successful PR run: 536 job-min p50 (e.g. https://github.com/SocketDev/socket-patch/actions/runs/37770694851), of which
    • 12 ubuntu cells (4 Gradle lines × agent/hosted/vendor): 144 job-min/run
    • 13 ubuntu extras (JDK ceilings, configuration-cache, isolated-projects, real-central): 160 job-min/run
    • 12 windows cells: 204 job-min/run; build ubuntu+windows: the rest.
  • Mean over all sampled PR runs, cancelled included: 488 job-min/run.
  • Estimated 7-day average: Gradle compat = 15,400 Linux + 10,400 Windows job-min/day. Its "Run the hosted suites" step is the Remove tsc from npm prepare script #1 step in the repo by total minutes (~20,000 job-min/day).
  • Linux queue wait last 24h: p50 1.2 min, p90 12.8 min, worst hour (2026-10-07T16) p50 15.7 min. This hits ci.yml's Gradle e2e legs, which were the merge_group critical path in 16 of 21 sampled successful merge_group runs.
  • Failure rate on PRs: 10 failures in 431 runs (2.3%); 3 of them on feat/gradle-support.

Where the time goes

job (PR run) n p50 p90
gradle 8.14.3 / jdk 21 / hosted / windows-latest 62 40.6 41.6
gradle 7.6.6 / jdk 17 / hosted / windows-latest 62 39.1 42.4
gradle 8.14.3 / jdk 21 / hosted / ubuntu-latest 62 30.1 31.9
gradle 7.6.6 / jdk 17 / hosted / ubuntu-latest 62 28.7 31.2
gradle 8.14.3 / jdk 24 / hosted / jdk-ceiling 62 26.8 30.4
gradle 9.8.0 / jdk 21 / hosted / configuration-cache 62 26.2 27.8
gradle 9.8.0 / jdk 21 / hosted / isolated-projects 62 25.8 28.1
Almost all of it is the real-Gradle test step (hosted suite: 43 serial real-Gradle builds, 1544 s in https://github.com/SocketDev/socket-patch/actions/runs/37721395948/job/113131196521).

Root cause

  1. Exact duplication on PRs. ci.yml's e2e job already runs, on every PR and merge_group, the ubuntu PR tier: 4 Gradle lines (6.9.4/jdk11, 7.6.6/jdk17, 8.14.3/jdk21, 9.8.0/jdk21) × {agent suites + all gradle_hosted_ (split across 3 legs since Cut merge-group CI from ~46 to ~20 min: shard Gradle e2e and test legs, skip test-release in queue, cancel orphaned runs #1133), gradle_vendor_ + gradle_multi_project} (ci.yml:1240-1255). The ubuntu cells of gradle-compatibility.yml run the same suites, filters, Gradle versions and JDKs on the same OS.
  2. Grid extras on every PR. The JDK-ceiling / configuration-cache / isolated-projects (record-only) / real-Central rows are the "rows the PR tier never runs" per the workflow header, but they still trigger on every PR touching the broad paths: list (Cargo.toml, Cargo.lock, hosted/**, vex/**, commands/apply.rs, commands/scan/**, tests/common/**, …).

Proposed fix

In .github/workflows/gradle-compatibility.yml:

  1. Exclude ubuntu-latest from cells on pull_request, the same way macOS is already excluded:
    exclude: - os: ${{ github.event_name == 'pull_request' && 'ubuntu-latest' || '' }} (keep the macOS exclusion as a second entry). The build ubuntu-latest job is then needed on PRs only when extras run.
  2. Gate extras to schedule / workflow_dispatch, plus PRs that touch the Gradle core (crates/socket-patch-core/src/gradle/**, patch/redirect/*gradle*, vendor/jvm/**, this workflow). E.g. a small changes job with dorny/paths-filter (pinned SHA) or a git diff --name-only step, and if: github.event_name != 'pull_request' || needs.changes.outputs.gradle_core == 'true' on extras and build (ubuntu).
  3. Windows cells stay on PRs: they're the only PR coverage of Gradle on Windows besides ci.yml's single e2e_vendor_jvm_build windows leg.

Expected saving

  • Ubuntu cells + extras = 304 of 536 job-min per successful run (57%). At ~62 PR runs/day × 488 job-min mean × 0.57 ≈ ~17,000 Linux job-min/day (7d average; the last 24h ran ~3× the 7d rate, so the 24h saving is higher). Minus extras on Gradle-core PRs (a minority).
  • macOS: 0. Windows: 0.
  • Critical path: none directly; indirectly it lowers Linux runner contention (p90 queue 12.8 min) that delays ci.yml's e2e Gradle legs on PRs and merge_group.

Coverage and risk

  • Ubuntu cells: the same suites/versions/JDKs still run on every PR and merge_group via ci.yml e2e, and in this workflow nightly (17 4 * * *). One small difference: the cells' probe-report artifact names (gradle-probe-ubuntu-…) won't be produced on PRs; ci.yml uploads gradle-probe-pr-ubuntu-latest-* from the same tests.
  • Extras: still run nightly, on dispatch, and on PRs that change Gradle core code. A regression that only shows on a JDK ceiling and comes from non-Gradle code would surface in the nightly instead of on the PR (≤24h later).
  • Not a required check; ci-ok / clippy unaffected.

Effort

S (an exclude entry + a paths gate on one job).

ROI

17,000 Linux job-min/day × 0.5 (Linux weight) × confidence 0.7 / effort 1 ≈ 6000.

Activity

  1. mikolalysenko commented on Oct 8, 2026

    @mikolalysenko
    CollaboratorAuthor

    [agent] Triaged as priority:p3 (CI-only). No open PR references this yet; it is not a duplicate of the other CI-perf reports (#1170–#1178 each target a different workflow cost).


    Generated by Claude Code

  2. mikolalysenko commented on Oct 9, 2026

    @mikolalysenko
    CollaboratorAuthor

    Profiler refresh, 2026-10-09 00:17 UTC (window: last 24h)

    The cost has grown since this issue was filed. Gradle patch compatibility on pull_request ran ~213 PR runs/day, sampled at 272 Linux + 178 Windows job-min per run. That is about 58,000 Linux + 38,000 Windows job-min/day, the largest single consumer after CI itself (CI on PRs is ~89,000 Linux job-min/day).


    Generated by Claude Code

  3. mikolalysenko commented on Oct 9, 2026

    @mikolalysenko
    CollaboratorAuthor

    Working on this in the CI perf implementer.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions