Visitar URL original
fix(agent): keep incremental builds when CI is set by patwrall · Pull Request #15 · patwrall/ferry · GitHub
Skip to content

fix(agent): keep incremental builds when CI is set - #15

Merged
patwrall merged 8 commits into
mainfrom
fix/ci-incremental
Oct 26, 2026
Merged

patwrall merged 8 commits into
mainfrom
fix/ci-incremental

Conversation

@patwrall

Copy link
Copy Markdown
Owner

Why

Since Cargo 1.99 (rust-lang/cargo#17220), setting CI or TF_BUILD to any value turns incremental compilation off. That setting beats an explicit [profile.*] incremental = true, and it is part of every unit's hash. So when an agent tool exports CI, a Ferry agent loses its incremental first edit and recompiles every workspace crate of a base warmed without CI. Closes #2.

Fixing it exposed an older hole. unit_inputs ignored the settings that change unit hashes, so a seeded refresh across a CARGO_INCREMENTAL or CI change kept the earlier base's units unpruned. An agent that selected them then ran old code under mtime freshness. That is reproduced on main with CARGO_INCREMENTAL=0.

Scope

  • base::unit_inputs hashes CARGO_INCREMENTAL and whether CI or TF_BUILD is set. A version tag in the hash makes bases recorded by an older Ferry count as different inputs once, which closes the same hole across an upgrade.

  • shim::keep_incremental sets CARGO_INCREMENTAL=1 for the cargo Ferry runs, both in agents (agent::exec_cargo) and for base warm commands, when all of these hold:

    • Cargo is 1.99 or newer.
    • CI or TF_BUILD is set.
    • The user has set neither CARGO_INCREMENTAL nor build.incremental, read from the environment, --config, and the Cargo configs in the cwd's ancestors and in Cargo home.

    CI and TF_BUILD reach build scripts, tests and cargo run unchanged.

  • tests/ci.rs, tests/base.rs and unit tests in src/shim.rs cover it. troubleshooting.md, architecture.md and configuration.md document the rule and its costs.

Tradeoffs

The first version removed CI from Cargo's own environment and passed it back to child processes with --config env.CI=.... That kept every profile setting exactly, but rewriting Cargo's argv broke user aliases named b, c, d, t or r, served stale code on Cargo before 1.84, and missed external subcommands such as nextest. Setting an environment variable reaches every subcommand and alias. It costs this, as documented:

  • Under CI, release and bench builds become incremental too.
  • An explicit [profile.*] incremental = false is overridden. build.incremental = false is respected.
  • Processes Cargo starts inherit the value.

Blast radius

The change only affects builds Ferry runs while CI or TF_BUILD is set on Cargo 1.99 or newer. Every existing base counts as having different unit inputs once after the upgrade, which costs one prune, or one cold build where the warm build was not observed.

Verification

A refresh seeded from a base with the same unit inputs is not pruned,
on the premise that both builds select the same units. But Cargo hashes
each local unit's incremental setting, which CARGO_INCREMENTAL decides,
and which Cargo 1.99 (cargo#17220) turns off when CI or TF_BUILD is set.
Unit inputs hashed neither. So a base warmed with CARGO_INCREMENTAL=0
and refreshed without it kept the seed's units, built from older
sources, and an agent building with CARGO_INCREMENTAL=0 found them
fresh: under mtime freshness it ran the seed's code.

Unit inputs now include CARGO_INCREMENTAL's value and whether CI or
TF_BUILD is set to a UTF-8 value, which is how Cargo tests it. A refresh
across either change is then seeded across inputs and pruned, or built
from scratch when its warm build was not observed. Neither is hashed
when unset, so bases built without them keep their unit inputs.

This predates the CI work for #2 and that work needs it: a base warmed
with CI set gets CARGO_INCREMENTAL=1 there, so its release units differ
from those of a base warmed without CI.
Cargo 1.99 (cargo#17220) turns incremental compilation off whenever CI
or TF_BUILD is set. It applies that switch after merging profiles, so it
beats [profile.*] incremental = true, and a unit's hash includes the
result. An agent tool that exports CI therefore made an agent rebuild
every workspace member of a base warmed without it, and its first edit
compiled without incremental state, which is most of what Ferry gains on
Linux. A base warmed with CI set had the same problem the other way.

Only CARGO_INCREMENTAL and build.incremental rank above the switch, so
the cargo Ferry runs for an agent (ferry run, the shim, hooks) and for a
base's warm commands now gets CARGO_INCREMENTAL=1 when all hold: the
toolchain is Cargo 1.99 or newer (the base's recorded one for agents,
the one it detected for warming), CI or TF_BUILD is set, and neither
CARGO_INCREMENTAL, CARGO_BUILD_INCREMENTAL nor build.incremental in a
--config value or a .cargo/config{,.toml} the call reads is. CI and
TF_BUILD stay as they are for everything cargo runs. Cargo's arguments
are left alone, so aliases keep working, and an external subcommand
such as cargo nextest hands the variable on to the cargo it runs.

The costs, documented: under CI, release and bench builds become
incremental too, and [profile.*] incremental = false is overridden.
CARGO_INCREMENTAL=0 opts out.

Closes #2.
A base warmed with CI set builds its release units incrementally, under
Ferry's CARGO_INCREMENTAL=1, and one warmed without it does not. A
refresh with CI set, seeded from a base warmed without it, therefore
used to keep the seed's release units as if both selected the same
units, and a plain agent's cargo run --release found them fresh with the
seed's code. Counting CI among the unit inputs fixed that; this test
keeps it fixed, and checks that dev units still match across the
difference.
Each nested workspace builds into a target dir of its own, under
ferry-ws, and is a case of its own (Workspace::Nested) where the shim
and ferry run decide to set CARGO_INCREMENTAL=1. Nothing checked that it
gets the variable under CI: if the warm commands or an agent left it
out, the nested units would differ and every agent build of the nested
workspace would rebuild it. The test warms a base with CI set and builds
the nested workspace in an agent with and without CI; neither may
rebuild anything.
A build.incremental in $CARGO_HOME/config.toml ranks above Cargo's CI
switch, so Ferry must leave CARGO_INCREMENTAL unset then, but no test
covered Cargo home: cargo_configs read it from the process environment,
which a unit test cannot set safely. cargo_configs now takes Cargo home
as an argument, keep_incremental passes the real one, and a unit test
passes one with build.incremental set.
The CI notes left out three consequences readers can trip over. Release
and bench units of a base and an agent match only when both have CI or
TF_BUILD set or both lack it, since only CI makes them incremental. A
refresh across a change in CARGO_INCREMENTAL or in whether CI or TF_BUILD
is set is now seeded across unit inputs, so it is pruned, or built from
scratch when its warm build is not observed. And processes cargo starts
inherit Ferry's CARGO_INCREMENTAL=1, so a cargo call a build script or
test makes on a workspace outside the agent builds incrementally too.
The lists of what makes a refresh prune now name the incremental
settings as well.
A base warmed by an older Ferry with CARGO_INCREMENTAL or CI set has the
unit inputs the new code computes when they are unset. A refresh warmed
without them was then seeded as same-input, kept the old base's units
unpruned, and an agent selecting them under mtime freshness ran the old
code. Salting the hash with a version makes every base recorded under
the old definition a cross-input seed once: pruned if its warm build was
observed, rebuilt from scratch otherwise.

unit_inputs also takes the CI variables from shim::CI_VARS now.
An explicit [profile.*] incremental = false is overridden under CI, so
agents with CI set rebuild that profile's crates of a base warmed
without it; build.incremental = false keeps both. Whether CI is set is
a unit input on every Cargo version. includes, cargo -C and a nested
workspace's own toolchain file are not followed.
@patwrall
patwrall merged commit b33cdc7 into main Oct 26, 2026
21 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agents lose incremental reuse when CI is set (Cargo 1.99 default)

1 participant