Ferry
Ferry gives every parallel AI coding agent its own git worktree with an instantly forked, already-warm Cargo build. Agents stop waiting on cold compiles, and they stop corrupting each other’s builds.
The problem
Running several coding agents on one Rust repository needs one git worktree per agent, so their edits do not collide. That leaves two bad options for the build:
- A target dir per worktree. Every new worktree starts cold. On a real
workspace the first
cargo buildorcargo testtakes minutes, and every agent pays it. - One shared
CARGO_TARGET_DIR. Builds queue behind each other: Cargo holds an exclusive lock on the build directory for the whole build. Worse, the result can be silently wrong. Cargo decides whether a crate is fresh by comparing source modification times with its own records. If agent B edits a file, then agent A edits the same file differently and builds, B’s next build sees sources older than A’s artifacts. Cargo rebuilds nothing, and B runs A’s code.
The second case is not hypothetical. Ferry’s test suite reproduces it with
plain cargo (tests/contamination.rs):
B’s source says ONLY_IN_B, B’s build rebuilds 0 crates, and B’s binary
prints ONLY_IN_A. The same file checks that two Ferry agents given the same
edits each build their own code.
What Ferry does
ferry basebuilds one immutable, warm copy of your workspace for a commit.ferry spawn <name>creates a git worktree and branch for an agent and forks the base’s target dir for it: a copy-on-write clone, or where the filesystem has none an overlayfs layer, or a plain copy. A copy-on-write clone costs time per file, not per byte; an overlay fork takes the same time whatever the size of the target dir.ferry run <name> -- cargo test, or plaincargowith Ferry’s shim onPATH, builds in that agent’s own target dir. Cargo sees the fork as fresh for every file that is identical to the base, so only what the agent changed is rebuilt.- On Linux, each build runs in a private mount namespace where every agent’s worktree and target dir appear at the same two paths. rustc’s incremental state, which records absolute paths, then stays valid, and the agent’s first edit compiles incrementally.
No agent ever writes to the base or to another agent’s build state.
What Ferry is not
Ferry isolates build state, not processes. Agents run with your user’s full
privileges and can read and write anything you can. Use a VM, a container or
your agent tool’s own sandbox to contain code you do not trust. See
SECURITY.md for
the threat model.
Status
Ferry is pre-1.0. It runs on Linux and macOS; Windows is not supported. The full benefit (incremental reuse on an agent’s first edit) needs unprivileged user namespaces on Linux. Everywhere else Ferry still gives every agent a warm, private target dir with its dependencies already built. See Platform support.
Where to go next
- Getting started: install Ferry and run two agents.
- Agent integrations: wire Ferry into Claude Code, Codex or any other agent.
- How it works: bases, forks, stamping and the sandbox.
- Troubleshooting: namespaces, AppArmor, filesystems and disk usage.
Getting started
This page installs Ferry, builds a warm base, and runs two agents’ test suites in parallel. Most of the time goes into your first warm build.
The sample output on this page comes from a one-crate demo repository on Linux (btrfs), with paths shortened.
Requirements
- Linux or macOS. Windows is not supported.
git2.20 or newer.- Rust installed with rustup. Ferry runs whatever
cargoandrustcyour repository selects, includingrust-toolchain.toml. - For incremental reuse on an agent’s first edit: Linux with unprivileged user
namespaces.
ferry doctortells you whether you have them. Without them Ferry still works, reusing unchanged crates but recompiling an edited one from scratch (see Platform support).
Install
# From crates.io (the crate is ferry-rs; the binary is ferry)
cargo install ferry-rs --locked
# Prebuilt binary, Linux and macOS
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/patwrall/ferry/releases/latest/download/ferry-rs-installer.sh | sh
# Homebrew
brew install patwrall/tap/ferry
Or build from a clone of the repository:
git clone https://github.com/patwrall/ferry
cargo install --path ferry --locked
On Ubuntu 24.04 and later, also run ferry setup apparmor (it shows each step
and asks before one sudo call), or read
AppArmor on Ubuntu
before relying on the namespace sandbox.
1. Check the machine
Run ferry doctor inside the repository you want to use:
$ ferry doctor
✓ git git version 2.55.0
✓ repository /home/you/src/myproject
· config no ferry.toml; using defaults (`ferry init` writes one)
✓ toolchain rustc 1.99.0 (stable), cargo 1.99.0 (5f94df478 2026-08-27)
✓ freshness mtime stamping of files proven identical to the base
✓ store /home/you/.cache/ferry/myproject-d5c4fd3a48754db4 (btrfs, 30.3 GiB free)
✓ reflink copy-on-write clones work on btrfs
✓ namespaces unprivileged user + mount namespaces work
✓ overlayfs mounts inside user namespaces
→ agents fork with reflink (btrfs supports copy-on-write clones) and build in a namespace sandbox at canonical paths (incremental reuse on the first edit)
The last line is the plan: how agents will fork their target dir and whether
their first edit can reuse rustc’s incremental state. A check marked ! comes
with a fix: line. ferry doctor exits non-zero only when a check fails
outright (no git, no toolchain, not in a repository).
2. Write a config (optional)
ferry init
This writes ferry.toml to the root of your main checkout and prints the same
plan. Ferry works without it; the defaults warm the base with
cargo build --workspace and cargo test --no-run --workspace. Edit it to
warm other commands or profiles: see Configuration.
Ferry reads ferry.toml from the main checkout’s working tree, so you do not
have to commit it for agents to use it.
3. Build the base
$ ferry base
ferry: running `cargo build --workspace`
...
ferry: running `cargo test --no-run --workspace`
...
✓ base 9ae1f5d4b0aa-20509c3c ready in 0.3s (namespace sandbox, mtime freshness)
A base is an immutable warm build of one commit (HEAD unless you pass
--rev). It is built once and never modified. When your main branch moves
on, run ferry base again: the new base is seeded from the previous one, so
only what changed is rebuilt. Pass --cold to build from scratch. To have
this happen on its own, keep ferry base --watch running in a spare
terminal: it builds a new base whenever HEAD moves.
4. Spawn agents
$ ferry spawn a1
✓ spawned a1 in 0.09s
worktree /home/you/.local/share/ferry/worktrees/myproject-d5c4fd3a48754db4/a1 (branch ferry/a1)
base 9ae1f5d4b0aa-20509c3c
fork reflink — btrfs supports copy-on-write clones
sandbox namespace
stamped 3/3 tracked files
run ferry run a1 -- cargo build
$ ferry spawn a2
Each agent gets a git worktree on its own branch (ferry/<name>, starting at
HEAD unless you pass --from) and a private fork of the base’s target dir.
Spawning never builds anything.
5. Build in parallel
Point each agent (or each terminal) at its worktree and run cargo through
Ferry. Either use ferry run:
# terminal 1
ferry run a1 -- cargo test
# terminal 2
ferry run a2 -- cargo test
or put Ferry’s cargo shim first on PATH and use plain cargo inside each
worktree:
export PATH="$(ferry shim):$PATH" # run this inside the repository
cd ~/.local/share/ferry/worktrees/myproject-d5c4fd3a48754db4/a1
cargo test
The shim resolves the agent from the directory cargo runs in, on every call.
Inside an agent worktree it builds in that agent’s sandbox with that agent’s
target dir. Anywhere else it runs the real cargo unchanged, so it is safe to
leave on PATH.
Both builds start warm. Each agent only rebuilds the crates its own edits touch, and neither agent’s build can change the other’s.
6. See what exists
$ ferry list
Bases
ID BUILT DISK USED BY
* 9ae1f5d4b0aa-20509c3c 30s ago 7.5 MiB 2 agents
Agents
NAME BASE FORK SANDBOX DISK STATUS WORKTREE
a1 9ae1f5d4b0aa-20509c3c reflink namespace 136.0 KiB idle /home/you/.local/share/ferry/worktrees/myproject-d5c4fd3a48754db4/a1
a2 9ae1f5d4b0aa-20509c3c reflink namespace 0 B idle /home/you/.local/share/ferry/worktrees/myproject-d5c4fd3a48754db4/a2
* marks the current base, which new agents fork from. DISK for an agent is
the build state it holds beyond the base.
7. Clean up
ferry rm a1
ferry gc
ferry rm refuses while the worktree has uncommitted changes (untracked files
count) or while processes still run in the agent’s
sandbox; --force overrides both. It deletes the
agent’s branch only if the branch has no new commits, and otherwise tells you
how many it kept. ferry gc deletes bases that are neither current nor used
by an agent.
Running what you built
Inside the namespace sandbox, the paths cargo prints exist only there, and with the overlay backend so does the whole target dir. Run binaries and tests through Ferry:
ferry run a1 -- cargo run --bin myapp
ferry shell a1 # an interactive shell in a1's build context
Inside ferry run and ferry shell, the target dir is at the path that
ferry --json env a1 prints as build_target_dir. Reflink and copy targets
are also on the host, at host_target_dir.
Next steps
- Agent integrations: Claude Code hooks, Codex, and any other agent.
- Configuration: warm
clippy, release builds or custom commands. - Troubleshooting if
ferry doctorreports a problem.
How it works
This chapter explains what Ferry does and why, at the level you need to use it well. The Architecture chapter is the precise contract the code is written against: invariants, on-disk layout, locks and lifecycle.
ferry base bases/<commit>-<fingerprint>/
(once per commit) src/ immutable checkout
target/ immutable warm build
|
+--------------------+--------------------+
| | |
ferry spawn a1 ferry spawn a2 ferry spawn a3
| | |
worktree a1 worktree a2 worktree a3 your branches, at host paths
fork of target fork of target fork of target reflink, overlay layer or copy
|
cargo for a1 (via `ferry run a1` or the shim) runs in a1's private
mount namespace, where
<store>/mnt/src shows a1's worktree
<store>/mnt/target shows a1's fork
Every agent builds at these same two paths, so the base's incremental
state stays valid, and no agent can see another agent's build state.
Bases
A base is a warm build of one commit, made by ferry base. Ferry checks the
commit out into its store (a locked git worktree), runs the warm commands
(by default cargo build --workspace and cargo test --no-run --workspace),
and records the result in base.json. From then on nothing writes to it.
A base is identified by its commit plus a fingerprint of everything that
would make its artifacts unusable: the rustc -vV and cargo -V output, the
commit’s rust-toolchain.toml, the warm commands, the freshness mode, and
whether it was built in the namespace sandbox. Changing any of these makes
ferry base build a new base instead of reusing one. Agents keep the base
they were forked from until you remove them.
Refreshing is cheap: a new base for a later commit is seeded from the newest
ready base with the same fingerprint, so only what changed between the two
commits is rebuilt. That includes changes to Cargo.tomls, Cargo.locks,
Cargo configs and toolchain files, with one more step: the seed’s target
holds units the new build may not select any more, and a feature switched off
and on again, or a dependency bump that was reverted, would select one of
those as if it were fresh. So Ferry watches the warm build through the cargo
shim and deletes every unit it neither compiled nor found fresh before the
base is published. When it cannot see every cargo call (see
Refreshing a base), such a refresh is a
cold build. ferry base --cold skips the seeding.
Forks
ferry spawn gives an agent its own copy of the base’s target dir without
copying it, where possible. There are three backends:
| Backend | How | Cost per agent |
|---|---|---|
reflink | copy-on-write clone of every file (FICLONE on Linux, clonefile(2) on macOS) | time per file; disk only for files the agent rewrites |
overlay | overlayfs: the base’s target is a read-only lower layer, the agent writes to its own upper layer | constant time to fork; a file is copied into the upper layer on its first write (on btrfs and XFS that copy is itself a reflink) |
copy | a full copy | time and disk for the whole target dir |
auto (the default) picks reflink when the store’s filesystem supports
copy-on-write clones, else overlay when the agent runs in the namespace
sandbox and overlayfs works there, else copy. A reflink fork is a plain
directory, so a build needs no overlay mount. On the per-call path (shim,
hooks), where each cargo call gets its own sandbox, an overlay build also
waits for that sandbox’s teardown, most likely the kernel syncing the whole
store filesystem: about 0.2 s per build on ripgrep. Builds inside one
long-lived ferry run session share its sandbox and do not pay this. Ferry
never uses hard links: Cargo rewrites some files in place, so a hard-linked
fork would let an agent’s build modify the base.
Stamping: making the fork look fresh
A fresh git checkout gives every file a new modification time. Cargo compares source mtimes with the time it last built each crate, so a forked target next to a fresh checkout looks stale and Cargo rebuilds every workspace crate for nothing. In the prototype that was 9 spurious crate rebuilds on ripgrep.
At spawn, Ferry sets the mtime of every tracked file to a fixed old time, but only if the file is provably identical to the base’s copy: same git blob, same mode, same bytes on disk. Directories are stamped only if they list the same entries as the base. Anything that differs keeps its real mtime and is rebuilt. Stamping happens once, when the agent is created; Ferry never re-stamps a live worktree, because that could hide an edit or a deletion.
On nightly toolchains Ferry uses Cargo’s checksum freshness instead (see Configuration), and still stamps, because build script inputs are always compared by mtime. A symlink retargeted within the checkout (or into or out of it) needs one more step there. Ferry leaves the files it reaches unstamped, but when the file the base read through the link is inside the crate’s package, Cargo compares that file’s content, which the agent has too. So Ferry also empties Cargo’s record of what each crate that read through the link depends on, and those crates rebuild.
Canonical paths: keeping incremental state
Crate-level reuse is not the whole story. rustc’s incremental cache records absolute paths: the working directory and the output directory. When an agent builds at a different path from the base, rustc throws that cache away and the first edit recompiles each changed crate from scratch.
On Linux, Ferry runs each build in a private user and mount namespace. Inside
it, the agent’s worktree is bind-mounted at <store>/mnt/src and its target
dir at <store>/mnt/target. The base was built at the same two paths, so
rustc finds its incremental state valid. In the prototype the first edit
after spawning took 1.75 s at the canonical path against 5.3 s at a different
path (ripgrep, 2 vCPU).
The namespace changes only those two mounts. Host paths keep working inside
it, your uid maps to itself (not root), and the mounts vanish when the last
process in the namespace exits. On the host, <store>/mnt/src and
<store>/mnt/target are empty directories.
Without namespaces (macOS, or Linux where they are blocked or sandbox = "off"),
agents build at their own paths. A crate can capture the base’s absolute
paths in ways Cargo never checks again: a build script that writes its
CARGO_MANIFEST_DIR into generated code, a proc macro that reads it from
the environment (the rust-embed pattern), a test that bakes in the path of
the binary it runs. Reusing such a crate would make the agent read the base’s
files. So spawn looks for those captures in what Cargo and rustc recorded in
the target dir (dep-info, build-script output and OUT_DIRs, proc-macro
libraries) and makes only the crates that have one rebuild, with the build
scripts that would rerun because of them; every other crate, the workspace’s
included, stays reused. The first edit still loses rustc’s incremental state.
The experimental remap-paths mode recovers most of it. The
rules, and what they cannot see, are in the
architecture chapter.
Routing cargo: ferry run and the shim
Ferry has to intercept cargo to give it the right target dir and sandbox. There are two ways in:
ferry run <name> -- <command>enters the agent’s sandbox and runs the command with Ferry’s shim first onPATH. The working directory stays a host path. Use it for a single command or to start a whole agent.- The shim.
ferry shimprints a directory holding acargolink to the ferry binary. With it first onPATH, everycargocall looks at its working directory. Inside an agent worktree it enters that agent’s sandbox, moves to the canonical path, setsCARGO_TARGET_DIRandCARGO_BUILD_BUILD_DIRto the agent’s fork, and runs the real cargo (the nextcargoonPATHthat is not Ferry). Anywhere else it runs the real cargo unchanged.
The agent is always derived from the working directory, never from the
environment. Agent tools copy their session environment into sub-agents that
work in other worktrees, so an exported CARGO_TARGET_DIR would point a
sub-agent at another agent’s build state. That is the shared-target bug again,
which is why Ferry never exports it to anything but the cargo process it
starts.
What each agent owns
Everything an agent’s build writes lands in its own fork: overlay upper layer,
reflink clone or copy. Both the target dir and Cargo’s build dir point at it,
so a build.build-dir in your Cargo config cannot redirect intermediates to a
shared place. The base is read-only to every backend. Removing an agent
removes its worktree and fork; the base stays until ferry gc finds it unused
and not current.
Each Cargo workspace gets its own target dir, because Cargo names a path
crate’s artifacts after its path relative to the workspace root: two
workspaces sharing a target dir overwrite each other’s crates of the same
name. The workspace at the worktree root uses the fork itself; one nested at
<path> (a fuzz/ or xtask/ workspace, say) uses <fork>/ferry-ws/<path>;
and a workspace outside the worktree, such as
cargo test --manifest-path ../other-agent/Cargo.toml, builds into its own
target dir, as it would without Ferry. Base warm commands get the same
mapping, so a nested workspace the base warmed is warm in agents too. Without
the namespace sandbox, a path package outside the nested workspace’s root,
and what depends on it, rebuilds once: Cargo hashes such a package by its
absolute path.
Where things live
| What | Default (Linux) | Default (macOS) | Override |
|---|---|---|---|
| Build state: bases, forks, the shim | $XDG_CACHE_HOME/ferry or ~/.cache/ferry | ~/Library/Caches/ferry | FERRY_HOME, [store] dir |
| Agent worktrees (your uncommitted work) | $XDG_DATA_HOME/ferry/worktrees or ~/.local/share/ferry/worktrees | ~/Library/Application Support/ferry/worktrees | FERRY_WORKTREES, [agent] worktree-dir, ferry spawn --path |
Worktrees live outside the cache directory on purpose: they hold work you have not committed, and caches get cleared. All build state Ferry writes is under the one store directory, so an agent tool’s sandbox needs a single writable entry for it.
Agent integrations
Ferry works with any agent that can be started in a directory. This page has
the rules every integration follows and a generic recipe. The next pages cover
Claude Code and Codex in detail.
Working example files are in
examples/.
The rules
- One Ferry agent per coding agent. Each gets its own worktree, branch and
fork. Create it with
ferry spawn <name>or, for Claude Code, let itsWorktreeCreatehook do it. - Route cargo through Ferry. Either run the whole agent under
ferry run <name> -- <agent>, or put the shim first onPATH(export PATH="$(ferry shim):$PATH"). Both make everycargothe agent runs build in its own sandbox and target dir. - Never export
CARGO_TARGET_DIRinto an agent session. Agent tools copy their environment into sub-agents, including sub-agents that work in other worktrees. An exported target dir sends all of them to one shared target: the contamination bug Ferry exists to prevent. The same goes forCARGO_BUILD_BUILD_DIR, atarget-dirin a settingsenvblock, and symlinkingtarget/between worktrees. The shim exists so that nothing needs to be exported: it works out the agent from the directory cargo runs in, on every call. - Start the agent at the host worktree path, not at Ferry’s canonical
mount path. Agent tools key transcripts and memory on the working
directory; the canonical path is the same for every agent, so they would
all share one history.
ferry runkeeps host paths and only moves cargo to the canonical path.
Generic recipe
For any agent that takes a working directory and inherits the environment:
ferry spawn a1
ferry run a1 -- my-agent # the whole agent in a1's build context
ferry run starts the command in the agent’s worktree when your current
directory is outside it. Inside the session, PATH has the shim first, and
FERRY_AGENT and FERRY_WORKTREE name the agent and its worktree for your
scripts.
If the agent cannot be started through ferry run, start it however you
normally would in the worktree, with the shim first on PATH.
ferry env a1 prints the two lines that set this up:
eval "$(ferry env a1)" # exports FERRY_WORKTREE, puts the shim first on PATH
cd "$FERRY_WORKTREE"
my-agent
ferry env deliberately does not print CARGO_TARGET_DIR. For scripts,
ferry --json env a1 prints the worktree, branch, shim directory and target
paths as JSON.
examples/generic/parallel-agents.sh
spawns N agents, runs a command in each concurrently and reports the results.
Agents with their own sandbox
Agent tools often run commands in a sandbox that limits writes to the working
directory. Ferry’s build state lives elsewhere, in the store, so the sandbox
must also allow writes to the store’s parent directory (by default
~/.cache/ferry on Linux). It is the parent of the directory ferry shim
prints. All of Ferry’s writable state, including overlay upper layers and the
canonical mount points, is under it.
On Linux, the shim also has to create a user namespace. Some agent sandboxes
block that; then either run the agent under ferry run, which creates the
namespace before the agent’s sandbox starts, or let cargo bypass the agent’s
sandbox. The Claude Code and Codex pages give the exact settings.
Other agents
Cursor, Gemini CLI, OpenCode, Aider and Amp all start in a working directory
and inherit the environment, so the generic recipe applies to them. These
combinations have not been tested with Ferry. Do not use an agent’s own
worktree feature to create the worktree unless it lets Ferry create it (as
Claude Code’s WorktreeCreate hook does): a worktree Ferry did not create has
no fork, and the shim runs plain cargo there.
Claude Code
There are two ways to use Ferry with Claude Code:
- R1: run the whole session in an agent’s sandbox. One command, nothing to configure. Best for interactive sessions you start yourself.
- R2: let Claude create Ferry agents through hooks. Covers
claude --worktree,EnterWorktree, background sessions and subagents withisolation: worktree, so every worktree Claude creates is a Ferry agent with a warm build.
Both need a base: run ferry base in the repository first, and again when
your main branch moves on.
Example files are in
examples/claude-code/.
R1: ferry run a1 -- claude
ferry spawn a1
ferry run a1 -- claude
Claude starts in a1’s worktree at its host path, so its transcripts and
memory stay separate per agent. Every cargo it runs goes through the shim
and builds in a1’s target dir. Because the namespace exists before Claude
starts, this works whatever Claude’s own sandbox allows (see
Claude Code’s sandbox).
Map your own uid, not root: Ferry already does, and Claude refuses
--dangerously-skip-permissions as root.
R2: WorktreeCreate, WorktreeRemove and SessionStart hooks
Add this to ~/.claude/settings.json (it is
examples/claude-code/settings.json):
{
"hooks": {
"WorktreeCreate": [
{ "hooks": [ { "type": "command", "command": "ferry hook claude-worktree-create", "timeout": 120 } ] }
],
"WorktreeRemove": [
{ "hooks": [ { "type": "command", "command": "ferry hook claude-worktree-remove", "timeout": 120 } ] }
],
"SessionStart": [
{ "hooks": [ { "type": "command", "command": "ferry hook claude-session-start", "timeout": 30 } ] }
]
}
}
If you already have hooks, merge these entries into your hooks object.
ferry must be on the PATH Claude Code runs hooks with.
You can put the hooks in a project’s .claude/settings.json instead, but only
a committed one works: Claude Code loads project settings for a
hook-created worktree from that worktree’s checkout, and an uncommitted file in
your main checkout is not in it.
Then start Claude in a worktree as usual:
cd ~/src/myproject
claude --worktree feature-x
This recipe was verified with Claude Code 2.1.280: WorktreeCreate created a
Ferry agent, the session’s working directory was the agent’s worktree,
SessionStart put the shim on PATH, and cargo build in the session was
warm (0.04 s).
What each hook does
ferry hook claude-worktree-create reads Claude’s JSON on stdin (cwd
and the worktree name), creates a Ferry agent of that name, and prints the
worktree path as the last line of stdout, which is what Claude Code expects.
Progress goes to stderr.
- The agent name is Claude’s worktree name with anything other than letters,
digits,
-,_and.replaced by-(feat/login pagebecomesfeat-login-page). If a fully created agent of that name exists, the hook reuses it. - The branch is
ferry/<name>and starts atHEADof the checkout Claude was started from. Claude’sworktree.baseRefsetting does not apply, because the hook replaces Claude’s own worktree creation. - The hook never builds a base. Without one it fails with
no warm base exists for this repository, and Claude reports that the worktree could not be created. - Claude Code does not process
.worktreeincludewhen aWorktreeCreatehook is configured, and Ferry does not copy those files either. The only file Ferry adds to a new worktree is the base’sCargo.lock, when the repository does not track one.
ferry hook claude-worktree-remove runs when Claude removes a worktree it
created through the hook (for example when you exit an interactive session
and choose to remove it). It removes the agent’s worktree and fork. It refuses,
exiting non-zero and leaving the worktree in place, when the worktree has
uncommitted changes or processes still run in the agent’s sandbox. It deletes
the branch only if the branch has no new commits.
ferry hook claude-session-start appends one line to $CLAUDE_ENV_FILE:
export PATH=/home/you/.cache/ferry/bin:"$PATH"
Claude Code runs that file before every Bash command, so every cargo goes
through the shim. It writes only PATH. A CARGO_TARGET_DIR written there
would also reach subagents working in other worktrees (this was measured),
recreating the shared-target bug. Outside a git repository the hook does
nothing. Outside an agent worktree the shim runs the real cargo unchanged, so
the hook is safe for every session.
Subagents with isolation: worktree
A subagent whose frontmatter has isolation: worktree gets its own worktree.
With the hooks installed, that is a Ferry agent with a warm build:
---
name: rust-implementer
description: Implements a focused change in this Rust workspace in its own git worktree.
tools: Read, Edit, Write, Bash, Grep, Glob
isolation: worktree
---
You implement one focused change ...
The full file is
examples/claude-code/agents/rust-implementer.md.
Put it in .claude/agents/ (project) or ~/.claude/agents/ (user).
Claude names subagent worktrees agent-<hex>, so their Ferry agents are named
that way too. Claude Code keeps a hook-created subagent worktree when the
subagent finishes and does not run WorktreeRemove for it; nothing cleans up
after claude -p runs either. ferry gc removes these agents for you: it
reaps agents created by the hook that are clean, have no new commits, have
nothing running and are not locked by Claude (git worktree lock, which Claude
holds while an agent runs). Agents you created with ferry spawn are never
reaped. Run ferry gc --dry-run to see what it would remove.
Claude Code’s sandbox
Claude Code’s optional sandbox restricts what Bash commands can do. Two things matter for Ferry:
- Writes. cargo writes to the agent’s fork and to the canonical mount
paths, which live under Ferry’s store, outside the worktree. Allow writes to
the store’s parent directory (the parent of the directory
ferry shimprints; by default~/.cache/ferryon Linux). - Nested user namespaces. Claude Code 2.1.305 and later block them inside the sandbox by default, so the shim cannot create the agent’s namespace. Claude Code 2.1.280 still allowed them.
- Ubuntu 24.04 and later with Ferry’s AppArmor profile. Everything
ferry runstarts runs under theunpriv_ferryprofile, without capabilities in user namespaces, so Claude’s ownbwrapsandbox is expected to fail insideferry run a1 -- claudethere (not yet measured). Recipes 2 and 3 below start Claude outsideferry run.
Pick one of these:
-
Recipe R1.
ferry run a1 -- claudecreates the namespace before Claude’s sandbox starts. Claude’s sandbox still needs the store inallowWrite:{ "sandbox": { "enabled": true, "filesystem": { "allowWrite": ["/home/you/.cache/ferry"] } } } -
Allow nested namespaces in
~/.claude/settings.json(Claude Code honors this only from user settings,--settingsor managed settings). It turns off one of the sandbox’s security boundaries for every command:{ "sandbox": { "allowNestedUserNamespaces": true, "filesystem": { "allowWrite": ["/home/you/.cache/ferry"] } } } -
Run cargo outside Claude’s sandbox:
{ "sandbox": { "excludedCommands": ["cargo *"] } }Claude Code matches this against the command text. A compound command such as
cd crates/x && cargo teststays sandboxed, and then fails.
Use absolute paths in allowWrite. These settings follow Claude Code’s
documentation and measurements with @anthropic-ai/sandbox-runtime 0.0.81;
they have not been tested end to end with Claude Code 2.1.305 or later.
Things not to do
- Do not set
worktree.symlinkDirectories: ["target"]. It makes every worktree share one target dir. - Do not put
CARGO_TARGET_DIR(orCARGO_BUILD_BUILD_DIR) in the settingsenvblock or in$CLAUDE_ENV_FILE. - Do not start Claude at Ferry’s canonical path (
<store>/mnt/src). Every agent has the same canonical path, so they would share one transcript directory and memory.
Codex
Codex CLI has no hook that lets another tool create its worktrees, so Ferry
creates the worktree and Codex is started in it. Do not use
codex --worktree: it creates its own worktree under $CODEX_HOME/worktrees,
which has no Ferry fork.
Example files are in
examples/codex/.
Start Codex in a Ferry agent
ferry spawn a1
ferry run a1 -- codex --cd ~/.local/share/ferry/worktrees/myproject-d5c4fd3a48754db4/a1
ferry run puts the cargo shim first on PATH for Codex and everything it
starts, so every cargo Codex runs builds in a1’s target dir. Codex’s own
sandbox works inside Ferry’s namespace.
The launcher script
examples/codex/ferry-codex.sh
does this in one step, creating the agent when it does not exist:
ferry-codex.sh a1 # interactive
ferry-codex.sh a1 exec "make the parser tests pass"
Run it inside the repository, after ferry base. It reads the worktree path
from ferry env and ends with
exec ferry run "$name" -- codex --cd "$worktree" "$@".
If you prefer to start Codex yourself, export the shim before starting it:
eval "$(ferry env a1)" # exports FERRY_WORKTREE, puts the shim first on PATH
codex --cd "$FERRY_WORKTREE"
Codex has no equivalent of Claude Code’s environment file, and
shell_environment_policy.set only takes literal values, so set PATH
before Codex starts. Never set CARGO_TARGET_DIR there.
Sandbox settings
In workspace-write mode, Codex lets commands write to the working directory
and temp dirs. cargo running through Ferry also writes to the agent’s fork and
to Ferry’s canonical mount points. Both live under Ferry’s store, outside the
worktree, so without more configuration builds fail with:
error: failed to open: /home/you/.cache/ferry/myproject-d5c4fd3a48754db4/mnt/target/debug/.cargo-build-lock
Caused by:
Read-only file system (os error 30)
Make the store’s parent directory a writable root in ~/.codex/config.toml
(examples/codex/config.toml):
sandbox_mode = "workspace-write"
[sandbox_workspace_write]
writable_roots = ["/home/you/.cache/ferry"]
or pass it per session with codex --add-dir /home/you/.cache/ferry.
To find the path, run dirname "$(ferry shim)" inside the repository. It is
$XDG_CACHE_HOME/ferry (usually ~/.cache/ferry) on Linux and
~/Library/Caches/ferry on macOS, unless you set FERRY_HOME or [store] dir.
ferry --json env a1 shows the related paths: shim_dir is
<store parent>/bin, and build_target_dir is under the store.
Status: the read-only failure above was reproduced with codex sandbox
(Codex CLI 0.162.0) inside ferry run. The writable_roots and --add-dir
settings come from Codex’s configuration reference and were not exercised in a
live Codex session.
Codex also keeps .git inside writable roots read-only, and that includes the
git directory a worktree points to, so git commit from inside Codex’s
sandbox needs approval to run outside it.
Several Codex sessions in parallel
Give each session its own agent:
ferry-codex.sh parser # terminal 1
ferry-codex.sh cli # terminal 2
Each one builds warm, in its own target dir, and neither can change the
other’s build. Remove an agent with ferry rm <name> when you have merged or
dropped its branch.
Troubleshooting
Start with ferry doctor. Every warning it prints comes with a fix: line,
and ferry --json doctor shows the raw probe results and relevant sysctls.
User namespaces
On Linux, the namespace sandbox needs unprivileged user and mount namespaces. A quick manual check (it should print nothing and exit 0):
unshare --user --map-root-user --mount true
When they do not work, Ferry builds at host paths (crate-level reuse only) and
ferry base warns:
warning: namespace sandbox unavailable (...); building without canonical paths, so an agent's first edit rebuilds without incremental reuse. Run `ferry doctor` for fixes.
With sandbox = "namespace" it fails instead:
error: namespace sandboxes are not supported on this platform: ...
hint: run `ferry doctor` for details, or set FERRY_SANDBOX=off to build without the canonical-path sandbox
The common causes, in the order ferry doctor checks them:
| Cause | Typical systems | Fix |
|---|---|---|
kernel.apparmor_restrict_unprivileged_userns = 1 | Ubuntu 24.04 and later | ferry setup apparmor: see below |
kernel.unprivileged_userns_clone = 0 | Debian kernels with the default changed | sudo sysctl kernel.unprivileged_userns_clone=1 |
user.max_user_namespaces = 0 | RHEL hardened to the DISA STIG; NixOS with security.allowUserNamespaces = false | sudo sysctl user.max_user_namespaces=15000 (on NixOS, set security.allowUserNamespaces = true) |
| a seccomp filter | Docker’s default profile; agent sandboxes | see Containers and Agent sandboxes |
sysctl changes made this way last until the next reboot. To keep one, put
it in a file under /etc/sysctl.d/. Check your organisation’s policy first:
these settings exist to reduce the kernel attack surface.
If you cannot or do not want to change the system, set sandbox = "off" (or
FERRY_SANDBOX=off). Everything still works, with crate-level reuse: only
crates that captured the base’s absolute paths rebuild after spawn, and the
first edit gets no incremental reuse.
Containers
Inside a container, Ferry falls back to crate-level reuse unless the container allows the namespace sandbox. Accepting the fallback needs nothing else. Otherwise up to four layers are in the way:
- Seccomp. Docker’s default profile allows
unshare,mount,setns,fsopen,fsconfig,fsmountandmove_mountonly to containers withCAP_SYS_ADMIN, and fails them withEPERMotherwise. Rootless Podman’s default profile allows them. - AppArmor, on AppArmor hosts (Ubuntu, Debian, SUSE). Docker’s
docker-defaultand rootful Podman’scontainers-defaultprofiles containdeny mount, so Ferry’s bind and overlay mounts fail even inside its own user namespace. Rootless Podman applies no AppArmor profile. - Ubuntu 24.04 and later hosts. With
apparmor=unconfined, container processes are AppArmor’sunconfinedprofile, which the host’s restriction moves tounprivileged_userns, so nothing inside the container fixes this. Either lift the restriction on the host, or load a container AppArmor profile withoutdeny mounton the host and name it with--security-opt apparmor=<name>.ferry setup apparmorrefuses to run inside a container, because only the host can load AppArmor policy. - The store’s filesystem. The container’s root filesystem is overlayfs,
which overlayfs cannot use as its upper layer (
ferry doctorwarns when the store is on overlayfs). PutFERRY_HOMEon a volume or bind mount (ext4, XFS or btrfs), or on a tmpfs on Linux 6.6 or newer that allows exec. Without that,autouses reflink or copy, and the sandbox itself still works.FERRY_HOMEmust be owned by the container user and not group or world writable, so a fresh volume needs achown; worktrees then default to$FERRY_HOME/worktrees, which survives container rebuilds.
For seccomp, either use --security-opt seccomp=unconfined, which removes
all syscall filtering, or your Docker version’s default profile with two
rules added: unshare only with exactly the flags Ferry passes
(CLONE_NEWUSER | CLONE_NEWNS = 268566528), and the mount calls:
jq '.syscalls += [
{"names":["unshare"],"action":"SCMP_ACT_ALLOW","args":[{"index":0,"value":268566528,"op":"SCMP_CMP_EQ"}]},
{"names":["mount","setns","fsopen","fsconfig","fsmount","move_mount"],"action":"SCMP_ACT_ALLOW"}
]' default.json > .devcontainer/ferry-seccomp.json
setns joins a running overlay sandbox; the fs* calls are optional, as
Ferry falls back to mount(2). The kernel still requires CAP_SYS_ADMIN
over the target namespace, so container root gains no mount power outside
user namespaces. Do not use --privileged or --cap-add SYS_ADMIN: they
grant real CAP_SYS_ADMIN, far more than Ferry needs.
A devcontainer that does all of this (on a Debian host, or any AppArmor host
without Ubuntu’s restriction, add "apparmor=unconfined" to securityOpt):
{
"image": "mcr.microsoft.com/devcontainers/rust:1",
"securityOpt": ["seccomp=unconfined"], // or "seccomp=${localWorkspaceFolder}/.devcontainer/ferry-seccomp.json"
"mounts": [{ "type": "volume", "source": "ferry-${devcontainerId}", "target": "/ferry" }],
"containerEnv": { "FERRY_HOME": "/ferry" },
"postCreateCommand": "sudo chown \"$(id -u):$(id -g)\" /ferry && chmod 700 /ferry"
}
Docker Desktop runs containers in a Linux VM without the host’s AppArmor, so
seccomp should be the only layer there. On SELinux hosts, nested container
tooling commonly needs --security-opt label=disable.
These container facts come from the profiles’ and the kernel’s sources and from standalone probes; Ferry itself has not been tested in containers.
Agent sandboxes
Agent tools that sandbox their commands may block user namespaces. Claude
Code 2.1.305 and later do by default; see
Claude Code’s sandbox. Codex’s
Linux sandbox does not block them (see Codex). Running the
whole agent with ferry run <name> -- <agent> always works, because the
namespace exists before the agent’s sandbox starts.
AppArmor on Ubuntu 24.04 and later
Ubuntu 24.04 turned on an AppArmor policy that lets unprivileged processes
create user namespaces but gives them no capabilities inside. Ferry’s sandbox
then fails at the uid mapping or the first mount, and ferry doctor shows:
! namespaces setgroups failed: Permission denied ...
fix: Ubuntu's AppArmor policy denies capabilities in unprivileged user namespaces: run `ferry setup apparmor`, ...
The fix is an AppArmor profile that allows the ferry binary, and only it,
to use user namespaces. ferry setup apparmor installs it:
ferry setup apparmor # shows every step, then asks
ferry setup apparmor --dry-run # only shows them
It prints why the setup is needed, each step, the profile, the root script
and the exact sudo command, and asks before running that one command. As
root, it copies the running ferry to /usr/local/bin/ferry (owned by root),
writes /etc/apparmor.d/usr.local.bin.ferry, checks that it parses, removes
older Ferry profiles such as /etc/apparmor.d/ferry, and loads it. Then, as
you, it checks that /usr/local/bin/ferry can create a user namespace and
points the cargo shim at it. Run ferry base again in each repository
afterwards: agents spawned before keep building at host paths until they are
spawned again. See ferry setup apparmor for
its options and the cases where it refuses. Ferry’s CI sets up its Ubuntu
24.04 AppArmor job this way, with the restriction on.
The profile persists across reboots. After upgrading Ferry, run the
upgraded copy: after cargo install, that is ~/.cargo/bin/ferry setup apparmor. It installs the new binary and leaves the profile alone. A bare
ferry may still be the old /usr/local/bin/ferry, which reports that it is
already set up and says when ~/.cargo/bin/ferry is a different build. To
undo it, run ferry setup apparmor --uninstall, which unloads
and removes Ferry’s profile files and leaves /usr/local/bin/ferry for you to
remove.
To do it by hand instead (remove an older /etc/apparmor.d/ferry first, so
that only one file defines the profile):
# 1. Install ferry to a root-owned path.
sudo install -m 755 "$(command -v ferry)" /usr/local/bin/ferry
# 2. Generate the profile for that path and load it.
/usr/local/bin/ferry doctor --apparmor-profile | sudo tee /etc/apparmor.d/usr.local.bin.ferry
sudo apparmor_parser -r /etc/apparmor.d/usr.local.bin.ferry
# 3. Check.
/usr/local/bin/ferry doctor
The same profile for /usr/local/bin/ferry is in the repository as
contrib/apparmor/usr.local.bin.ferry.
Why a root-owned path. AppArmor attaches the profile by path. If the path
is writable by your user (for example ~/.cargo/bin/ferry), any process
running as you could replace the binary and inherit the permission. ferry setup apparmor therefore installs a copy, and ferry doctor --apparmor-profile warns when the binary’s path is not root-owned. If
/usr/local/bin is group-writable on your system, pass --bin-dir /opt/ferry/bin.
Make sure that binary is the one that runs. The profile covers only
/usr/local/bin/ferry. Remove other copies, or put /usr/local/bin first on
PATH: the shim link follows whichever ferry ran last, and Claude Code’s
hooks call ferry through PATH. ferry setup apparmor warns when the first
ferry on PATH is another copy, and ferry doctor says when a Ferry
profile covers a different binary than the one you ran.
What the profile allows. It is the stacked form Ubuntu uses for its own
bwrap-userns-restrict profile: ferry itself may create the namespace and
mount, while everything it runs (cargo, build scripts, tests, your agent) runs
under a second profile, unpriv_ferry, that denies capabilities. Your agent’s
processes therefore cannot create privileged user namespaces themselves. A
simpler flags=(unconfined) profile would exempt the agent’s whole process
tree from the restriction. One exception: ferry re-running its own binary (the
namespace probe does) stays under ferry (an ix rule for its exact path),
because a capability-less probe would report that namespaces do not work. The
remaining trade-off is that any local process that can run
/usr/local/bin/ferry can use it to create user namespaces. On Ubuntu 24.04
and 24.10, where kernel.apparmor_restrict_unprivileged_unconfined is 0,
an unconfined process can also run any program under the ferry profile with
aa-exec -p ferry; that holds for every user-namespace profile Ubuntu ships,
and 25.04 and later set the sysctl to 1. See
SECURITY.md.
Tools that need their own namespaces break under ferry run. Because of
that stacking, a sandbox tool such as bwrap (used by Claude Code’s sandbox)
or a test that calls unshare gets no capabilities inside its namespace and
fails. Ubuntu ships its own bwrap profile disabled for similar breakage.
Last resort: turn the restriction off.
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 # until reboot
This re-opens unprivileged user namespaces for every process on the machine,
including browsers and anything an attacker manages to run. That is the
kernel attack surface the restriction exists to close: AppArmor’s
documentation lists exploits such as CVE-2022-0185 and CVE-2024-1086 that
were reachable through user namespaces. Prefer the profile. If neither is
acceptable, leave the restriction on and use sandbox = "off".
Filesystems
overlayfs fails. The overlay backend needs Linux 5.11 or newer and a store
filesystem that supports user extended attributes: ext4, btrfs and XFS do,
tmpfs only from Linux 6.6. Overlayfs cannot use another overlayfs as its
upper layer, which matters inside containers: ferry doctor warns when the
store is on overlayfs, and the fix is a volume for FERRY_HOME (see
Containers). auto uses overlay only where copy-on-write
clones are unavailable; without either, it uses copy.
Slow or flaky builds on a network filesystem. ferry doctor warns when
the store is on NFS, CIFS or FUSE: builds are slow there and mtimes
unreliable. Point FERRY_HOME at a local disk.
No reflinks. Copy-on-write clones work on btrfs, XFS (created with
reflink=1, the default since xfsprogs 5.1), OpenZFS 2.2.0 and 2.3.0 or newer
(2.2.2 to 2.2.x have them off by default), and APFS. Not on ext4 or tmpfs, and
never across filesystems. Without them, agents use overlay (Linux, with the
namespace sandbox) or full copies.
path ... cannot be used in an overlayfs mount. On Linux kernels before
6.13, Ferry passes overlay paths as mount options, which cannot express ,,
:, \ or newlines. Set FERRY_HOME to a path without them.
Disk usage
ferry list shows each base’s size and each agent’s DISK: the build state
it holds beyond the base.
- Reflink agents share all data with the base until they rewrite a file.
ducounts shared data once per agent, so it reports about a full target dir for each;btrfs filesystem duorcompsizeshow what is exclusive. - Overlay agents (where clones are unavailable, such as ext4) copy a file into their upper layer the first time they write to it. Relinked binaries and rustc’s incremental session directories account for most of it. On ext4 every copy-up is a full copy: in the prototype on ripgrep (ext4) each agent’s upper layer was about 190 MB over a 536 MB base.
- Copy agents cost a full copy of the target dir each.
- Bases stay until no agent uses them and they are not current. Every
ferry basefor a new commit creates a new base seeded from the previous one (a reflink clone where possible, else a full copy); the old one is deleted byferry gconce no agent uses it.
To reclaim space: ferry rm agents you are done with, then ferry gc
(--dry-run first to see what it would delete). Agents forked from an old
base keep that base alive.
Cargo configuration pitfalls
build.target-dirandbuild.build-dirin.cargo/config.tomlare overridden: Ferry setsCARGO_TARGET_DIRandCARGO_BUILD_BUILD_DIRfor every build it runs, and Cargo’s environment variables beat config files. Both are set because abuild-dirtemplate such as{workspace-path-hash}resolves to the same directory for every agent at the canonical path.- A
.cargo/config.tomlabove the repository, or an untracked one in it, does not apply to Ferry’s builds, which run under the store where Cargo never finds it.ferry baseandferry doctorname such files. Move their settings into the repository’s tracked.cargo/config.tomlor into$CARGO_HOME/config.toml. --target-dir,--config build.target-dir=...and--config build.build-dir=...on the command line beat the environment. A build run that way leaves the agent’s fork and can share state with other agents. Do not use them in agents or warm commands.- Differences between the base and the agent rebuild everything. Cargo
hashes
RUSTFLAGS,RUSTC_WORKSPACE_WRAPPER, features and profile settings into its fingerprints. If agents build with different values than the warm commands, nothing is reused.RUSTFLAGS=-Dwarningsis such a difference. cargo clippy -- -D warningsre-lints every workspace member from scratch, because the base warmed clippy without lint flags (dependencies stay fresh). UseCARGO_BUILD_WARNINGS=deny cargo clippyinstead (Cargo 1.97+), which turns warnings into errors without touching any fingerprint.CARGO_INCREMENTAL=0(set by some CI images and agent tools) switches off incremental compilation, and with it Ferry’s incremental reuse. Unset it.CIorTF_BUILDset (some agent tools exportCI). Cargo 1.99 and later then turn incremental compilation off, ahead of every profile setting, and since a unit’s hash includes it, an agent would also rebuild every workspace crate of a base warmed without it. So when either variable is set, Ferry setsCARGO_INCREMENTAL=1for the cargo it runs in agents and warm commands, unlessCARGO_INCREMENTAL,CARGO_BUILD_INCREMENTALorbuild.incremental(in a Cargo config or--config) is set.CIandTF_BUILDreach build scripts, tests andcargo rununchanged. So doesCARGO_INCREMENTAL=1: a cargo call they make on a workspace outside the agent builds incrementally too. This has costs. Release and bench builds become incremental, so they are bigger and slower. Their units match between a base and an agent only when both haveCIorTF_BUILDset, or both lack them.incremental = falsein a[profile.*]is overridden, so an agent withCIset rebuilds that profile’s crates of a base warmed without it.build.incremental = falsekeeps both the setting and the reuse. A refresh across a change in whetherCIorTF_BUILDis set, or inCARGO_INCREMENTAL, is seeded across unit inputs, so it does not keep the earlier base’s units. That holds on every Cargo version, including those before 1.99, whereCIchanges no unit. Ferry does not follow a Cargo config’sinclude,cargo -C, or a nested workspace’s own toolchain file when it looks forbuild.incrementaland Cargo’s version. To opt out, setCARGO_INCREMENTAL=0, which turns incremental compilation off as plain Cargo does underCI.- Build caches as
RUSTC_WRAPPER. kache strips Cargo’s incremental argument, so a base built through it has no incremental state; sccache refuses incremental units. Neither changes Cargo’s fingerprints, so crate-level reuse still works.
Builds that rebuild more than expected
warning: N of M tracked files differ from base ...at spawn: the agent starts from a different commit than its base (--from), so crates using the changed files rebuild on the first build. Runferry base --rev <commit>to have a base at that commit (the warning names it), or, for agents started atHEAD, keepferry base --watchrunning so the base follows it. The warning says when a base for the agent’s commit is already being built.warning: N build units captured the base's absolute paths ...at a spawn without the namespace sandbox. The agent builds at other absolute paths than its base, and those crates recorded the base’s: library code readingenv!("CARGO_MANIFEST_DIR"), a build script writing it into generated code or arustc-envvalue, users of a proc macro that reads it, tests usingenv!("CARGO_BIN_EXE_..."). They rebuild once, with whatever depends on them; everything else stays reused. Fix the namespace sandbox (ferry doctor) to avoid it, or stop baking the path in (read it at run time, or make it relative).- A refresh is a cold build after a manifest change.
ferry baseseeds across a change toCargo.tomls,Cargo.locks, Cargo configs, toolchain files orRUSTFLAGS-style variables only from a base whose warm build it observed ("observed": trueinferry --json list), because it must then delete the units the new build does not use. The same goes for a change toCARGO_INCREMENTAL, or to whetherCIorTF_BUILDis set. Warm commands that runcargo nextest, an alias likecargo xtask, or pass--message-format=shortare not observed; see Refreshing a base.[base] prune = falseturns this off on purpose. warning: rebuilt the base from scratch: could not tell which of the seed's units the warm commands used (...)fromferry base: the seed’s warm build was observed but this one was not, or something in the target dir did not match what Cargo reported. The reason in parentheses says which. The base is correct, just built cold; a cargo call that bypasses the shim and rebuilds something is the usual cause.warning: the warm commands wrote N file(s) into the source tree ...fromferry base: a build script writes outsideOUT_DIR. Agents do not get those files, so whatever reads them may rebuild or behave differently.- Every crate rebuilds after a toolchain update. The base’s fingerprint
includes the toolchain; run
ferry baseagain.
Builds that rebuild less than they should
Cargo’s default freshness check compares modification times. A file whose new
content arrives with an old mtime (copied with cp -p or rsync -t,
extracted from an archive, or set with touch -d) can look unchanged to
Cargo, with or without Ferry. touch the file to force a rebuild. On nightly
toolchains, Ferry’s auto freshness mode uses Cargo’s checksum freshness,
which compares rustc’s inputs by content; build-script inputs stay mtime-based.
Ferry stamps files only once, when the agent is created, and only files that are byte-for-byte identical to the base; it never re-stamps a live worktree.
Errors and odd states
- An untracked
Cargo.lockin a new worktree. If the repository does not trackCargo.lock, Ferry copies the base’s into each new worktree so agents resolve the same dependency versions.ferry rm, the Claude CodeWorktreeRemovehook andferry gcrecognise that unchanged copy and do not treat it as the agent’s work; once the agent changes it (say, by adding a dependency), it counts as uncommitted. agent ... was not fully created, orincompleteinferry list: spawning was interrupted. Remove it withferry rm <name>.base ... is missing or incomplete: runferry baseto rebuild it, thenferry gc.the agent's overlay is still mounted by processes that cannot be joined (pids: ...): processes started inside the agent’s sandbox created their own namespaces (unshare,bwrap, a container) and still have the agent’s overlay mounted after the agent’s own sessions ended. A second mount of the same overlay would corrupt it, so Ferry waits up to 10 seconds and then refuses. Wait for those processes or stop them, then retry.the agent's mount lock is still held by ...: a session that is exiting still holds the agent’s overlay while the kernel unmounts it, which can take seconds under heavy disk writes. Retry; if it persists, the named processes (or ones in another pid namespace) are still using it.- Paths printed by cargo do not exist. Inside the namespace sandbox, cargo
prints canonical paths such as
<store>/mnt/target/debug/deps/app-.... They exist only inside the sandbox; on the host those directories are empty. Run binaries and tests throughferry runorferry shell. ferry (cargo shim): no real `cargo` found on PATH after the shim: the shim’s directory is onPATH, but no othercargois. Install Rust with rustup, or fix thePATHorder.refusing to continue: branch ... already exists: an earlier agent’s branch was kept because it has commits. Pass--branchto use another name, or merge or delete the old branch.
FAQ
Is Ferry a sandbox for untrusted agents?
No. Ferry isolates build state, not processes. Agents run as your user, with
your files and your network. The Linux namespace Ferry creates only changes
two mount points so that every agent builds at the same paths; host paths stay
reachable inside it. Use a VM, a container or your agent tool’s sandbox to
contain code you do not trust. See
SECURITY.md.
Why not point every worktree at one CARGO_TARGET_DIR?
Because it gives wrong builds. Cargo compares source mtimes with the time it
last built each crate. If agent B edits a file, then agent A edits it
differently and builds, B’s next build finds its source older than A’s
artifact and rebuilds nothing: B runs A’s code. Ferry’s test suite reproduces
this with plain cargo in tests/contamination.rs. Cargo’s checksum freshness
(nightly) narrows the problem but does not close it: build-script inputs are
still compared by mtime, so B can still get A’s build-script output. Separately,
Cargo holds an exclusive lock on the build directory for the whole build, so
agents sharing one target dir wait for each other.
How does Ferry compare with sccache, kache or plain worktrees?
| Approach | New worktree | First edit in it | Parallel agents |
|---|---|---|---|
| Plain worktrees, one target dir each | cold build | normal | correct |
Copied target dir (cp --reflink, Worktrunk’s copy-ignored) | dependencies reused; workspace crates rebuild, because a fresh checkout’s sources are newer than the copied artifacts | not incremental: the path changed | correct |
Shared CARGO_TARGET_DIR | warm | depends on the other agents | builds wait on Cargo’s lock, and one agent can silently get another’s code |
| sccache | registry crates hit; workspace crates miss, because the key includes the working directory and absolute CARGO_* paths | not incremental: sccache does not cache incremental units | correct, with a target dir per worktree |
| kache 1.2.2 | warm: a content-addressed RUSTC_WRAPPER with path-normalized keys restores artifacts | not incremental: restored artifacts carry no incremental state | correct, with a target dir per worktree |
| Ferry | warm, nothing rebuilt | incremental on Linux with user namespaces; elsewhere unchanged crates are reused and the edited crates recompile from scratch | correct, each agent has its own fork |
kache and Ferry solve different parts of the problem and can be combined.
kache shares artifacts across machines, commits and CI through remote stores,
which Ferry does not do at all. Ferry gives each local agent a private,
already-incremental build. kache’s main branch (unreleased as of this
writing) adds incremental seeding between worktrees: a crate is seeded once
another worktree has already built an edit of it. One caveat when combining
them: kache strips Cargo’s incremental argument for every crate it handles,
so a Ferry base built with RUSTC_WRAPPER=kache has no incremental state.
Ferry does not ship an integration for this yet.
Cargo itself may eventually share build results between workspaces: a cross-workspace cache is a 2026 Rust project goal, starting with simple crates on nightly. It is not usable yet.
Can I use sccache or kache with Ferry?
As RUSTC_WRAPPER, both leave Cargo’s fingerprints alone, so crate-level
reuse keeps working. But sccache refuses incremental units, and kache strips
the incremental argument, so you lose Ferry’s incremental reuse for whatever
they handle. For now, use them for builds outside Ferry.
Do I have to commit ferry.toml?
No. Ferry reads it from the main checkout’s working tree. Commit it if you
want everyone on the project to warm the same commands. Note that a
ferry.toml can run arbitrary commands when someone runs ferry base.
Where are my agent’s binaries?
In its own target dir, at the path ferry --json env <name> prints as
build_target_dir; in the namespace sandbox that path exists only inside it.
Use ferry run <name> -- cargo run, or work inside ferry shell <name>. With
the reflink and copy backends, host_target_dir is a real directory you can
also read from the host; with the overlay backend the target dir exists only
inside the sandbox.
Which cargo commands go through Ferry?
All of them, including subcommands such as cargo clippy or cargo nextest,
as long as cargo resolves to the shim: inside ferry run and ferry shell,
or with the shim first on your PATH. Tools that run rustc directly, or
call cargo by an absolute path, bypass it. Warm what agents run: the default
base warms build, test --no-run, check and clippy --all-targets for the
dev profile. check shares only build scripts and proc macros with
build, clippy shares everything but workspace members with check, and
profiles share nothing (see
Configuration).
What happens when my main branch moves on?
Run ferry base again, or keep ferry base --watch running to have that
done each time HEAD (or the --rev you give it) moves. The new base is
seeded from the previous one, so only what changed is rebuilt, also after
Cargo.toml or Cargo.lock changes, and new agents fork from it. Existing
agents keep the base they were forked from. If you rebase an agent’s branch,
the files that changed get new mtimes and Cargo rebuilds what they affect;
Ferry never re-stamps a live worktree.
What does Ferry add to my repository?
Git worktrees and branches. Each agent is a worktree on its own branch
(ferry/<name> by default), and each base is a detached worktree in the
store, locked with git worktree lock so git never prunes it. They show up in
git worktree list. Ferry writes nothing into your main checkout, except
ferry.toml when you run ferry init and, in remap-paths mode, one line in
.git/info/exclude.
Does the shim leave anything behind in repositories I never use with Ferry?
No. The shim and the Claude Code SessionStart hook look up every git
repository they run in, but only read: a repository without a store gets plain
cargo, and no store directory is created for it.
Does Ferry handle rust-analyzer?
Not yet. Ferry does not set up rust-analyzer per worktree. An editor whose rust-analyzer runs cargo in an agent’s worktree uses whatever target dir its own configuration gives it.
How do I remove everything?
Remove agents with ferry rm <name> and unused bases with ferry gc. There
is no command that removes the current base. To remove all of Ferry’s state
for a repository, delete its store directory (under ~/.cache/ferry by
default) and its worktree directory, then in the repository run
git worktree list, git worktree unlock <path> for each leftover base
checkout, and git worktree prune.
Does Ferry need root?
No. It uses unprivileged user namespaces where they are available.
ferry doctor may suggest sudo commands (a sysctl, for example) for you to
review and run; it never runs them itself. The one exception is ferry setup apparmor on Ubuntu 24.04 and later, which you run on purpose: it prints the
profile, the root script and the exact sudo command, asks, and then runs
that one command.
Windows?
Not supported. Ferry relies on Unix filesystem and namespace features, and it does not compile on Windows.
What are Ferry’s limitations?
- The incremental win needs Linux user namespaces. Where they are blocked (Ubuntu 24.04+ without the AppArmor profile, Docker’s default seccomp profile, hardened systems), agents reuse unchanged crates but recompile an edited crate from scratch, and the few crates that captured the base’s absolute paths rebuild once after spawn.
- Overlay copy-up costs disk where clones are unavailable. On ext4 and
tmpfs agents fork with overlay, and an agent’s first write to a file copies
all of it into the agent’s upper layer; relinked binaries and incremental
session files make up most of it.
ferry listshows each agent’s share. - Paths that cargo prints in the sandbox do not exist on the host. Run
built binaries through
ferry runorferry shell. Reflink and copy targets are also reachable on the host atferry env’shost_target_dir; an overlay target exists only inside the sandbox. - Build scripts that write outside
OUT_DIRwrite into the base’s checkout, and agents do not get those files.ferry basewarns when it happens. - mtime freshness has blind spots. A file whose new content arrives with an old mtime (for example copied with preserved timestamps) can look unchanged to Cargo. On nightly, checksum freshness covers rustc’s inputs; build-script inputs stay mtime-based.
- macOS has no incremental reuse by default. The remap-paths mode that addresses it is experimental and unverified on a Mac.
- rust-analyzer per worktree is not addressed.
- Claude Code 2.1.305 and later block nested user namespaces in their
sandbox unless configured; run the session with
ferry runor see Claude Code’s sandbox. - Local only. Ferry does not share builds across machines.
Configuration
Ferry works without any configuration. To change what it does, use a
ferry.toml file, FERRY_* environment variables, or both.
Where the file is read from
$FERRY_CONFIG, if set. The file must exist.- Otherwise
ferry.tomlin the root of the repository’s main checkout, if it exists. Ferry reads the file from the working tree, so it does not need to be committed, and agents’ worktrees do not need their own copy. - Otherwise the built-in defaults.
ferry init writes the commented default file; ferry doctor shows which
file was loaded. Unknown keys and invalid values are errors, so a typo cannot
silently fall back to a default.
A ferry.toml in a repository you clone can run arbitrary commands when you
run ferry base, like a Makefile or a build.rs.
Precedence
Lowest to highest:
- built-in defaults;
- the top-level tables of
ferry.toml; - its
[linux]or[macos]table, for the platform Ferry runs on; FERRY_*environment variables.
The default file
This is exactly what ferry init writes. Every value in it is the default.
# Ferry configuration: https://patwrall.github.io/ferry/configuration.html
[base]
# Cargo subcommands that warm the base: "check", "build", "test" (--no-run),
# "clippy" (--all-targets --keep-going; skipped if not installed, failures only warn).
targets = ["build", "test", "check", "clippy"]
# Cargo profiles to warm. "dev" uses Cargo's default profile selection.
profiles = ["dev"]
# Extra arguments for every generated command.
cargo-args = ["--workspace"]
# Replace the generated commands entirely; each entry runs with `sh -c`.
# commands = ["cargo build --workspace --all-targets --locked"]
# Seed refreshes across manifest, lockfile and Cargo config changes, deleting the
# units the warm commands no longer use; false builds such refreshes from scratch.
prune = true
[agent]
# Branch created for each agent: <prefix><name>.
branch-prefix = "ferry/"
# Where agent worktrees (and their uncommitted work) live, one subdirectory per repository.
# Default: $XDG_DATA_HOME/ferry/worktrees or ~/.local/share/ferry/worktrees (Linux),
# ~/Library/Application Support/ferry/worktrees (macOS).
# worktree-dir = "~/agents"
[store]
# Parent directory for Ferry's per-repository build state (bases, forks).
# Default: $XDG_CACHE_HOME/ferry or ~/.cache/ferry (Linux), ~/Library/Caches/ferry (macOS).
# dir = "/fast/disk/ferry"
[fork]
# How an agent's target dir is forked from the base: "auto", "reflink", "overlay", "copy".
backend = "auto"
[sandbox]
# "auto" uses a private mount namespace when available (Linux), "namespace" requires it, "off" never uses it.
mode = "auto"
# Experimental: keep rustc's incremental state reusable when building without the
# namespace sandbox (macOS; Linux without user namespaces). Needs rustc 1.98+.
remap-paths = false
[freshness]
# "auto" uses checksum freshness when the toolchain supports it, otherwise mtime stamping.
mode = "auto"
# Per-platform overrides of backend, sandbox and freshness.
# [linux]
# sandbox = "namespace"
# [macos]
# backend = "reflink"
[base]: what warms a base
ferry base runs the warm commands, in order, at the root of the base’s
checkout (at the canonical path, inside the namespace sandbox when it is in
use). Each runs with sh -c, with CARGO_TARGET_DIR and
CARGO_BUILD_BUILD_DIR set to the base’s target dir, and
CARGO_BUILD_WARNINGS=warn, so a build.warnings = "deny" in your
environment or tracked Cargo config cannot fail a base (Cargo applies that
setting after a unit is built, so the units still match your agents’
builds). Their output goes to stderr. If one fails, the base is discarded,
except for the generated clippy command (see below).
Warm what your agents will run. Cargo reuses a unit only when the agent’s invocation selects the same package, features, profile and target kind.
| Key | Type | Default | Meaning |
|---|---|---|---|
targets | array of "check", "build", "test", "clippy" | ["build", "test", "check", "clippy"] | Cargo subcommands to run. "test" runs cargo test --no-run, which builds the test binaries without running them. "clippy" runs cargo clippy --all-targets --keep-going (each flag left out when cargo-args already selects targets or has it); it is skipped with a warning when clippy is not installed (an error when it is the only target), and its failure only warns. |
profiles | array of profile names | ["dev"] | Profiles to warm. "dev" adds no flag (Cargo’s own selection, so tests keep using the test profile), "release" adds --release, any other name adds --profile <name>. Names must be non-empty and contain no whitespace. |
cargo-args | array of strings | ["--workspace"] | Appended to every generated command, shell-quoted. |
commands | array of strings | unset | Replaces the generated commands. Each entry is a shell command line. Must not be empty when set. |
prune | boolean | true | Seed a refresh from the previous base even when manifests, lockfiles, Cargo configs, toolchain files, Cargo flag variables, CARGO_INCREMENTAL or whether CI or TF_BUILD is set changed, and delete the units its warm commands did not use. false builds such refreshes from scratch. See Refreshing a base. |
Without commands, Ferry generates one command per profile and target, in
that order: for each profile, for each target,
cargo <subcommand> [profile flag] <cargo-args...>. targets and profiles
must not be empty then. With the defaults that is:
cargo build --workspace
cargo test --no-run --workspace
cargo check --workspace
cargo clippy --all-targets --keep-going --workspace
Cargo keys every unit on its compile mode, so cargo check shares only
build scripts and proc macros (and what they depend on) with cargo build:
warming build alone leaves an agent’s first cargo check recompiling
every other dependency. cargo clippy lints only workspace members, so it
shares every other package’s units with cargo check.
--all-targets also checks tests, examples and benches with their
dev-dependencies; --keep-going warms the rest when some target fails, for
example benches that need nightly.
The generated clippy command is optional. When cargo clippy --version
fails in the main checkout, ferry base warns and leaves it out, or fails
when it is the only warm command, since a base must warm something. When it
runs and fails (a deny-level lint, a bench that needs nightly), ferry base
warns, keeps the base and lists the command in incomplete. Commands from
commands or FERRY_BASE_COMMAND are never optional.
The warm commands that run are part of the base’s identity: changing them
makes the next ferry base build a new base, and so does installing clippy
after a base was built without it.
Each warm command costs base time and disk, reported per command in
ferry --json base (command_secs). To trim:
[base]
targets = ["build", "test"] # what Ferry warmed before check and clippy
# targets = ["check", "clippy"] # agents never build: no codegen at all
# targets = ["build", "test", "check"] # no clippy
To keep benches out, or to match agents that pass lint flags, write the
commands yourself. These are required, so append || true to a command
whose lint failures should not fail the base:
[base]
commands = [
"cargo build --workspace",
"cargo test --no-run --workspace",
"cargo check --workspace",
"cargo clippy --workspace --lib --bins --tests --examples",
]
Lint flags after -- (cargo clippy -- -D warnings) are part of what
clippy records for each member, so an agent that passes different ones
re-lints every workspace member from scratch. Prefer
CARGO_BUILD_WARNINGS=deny cargo clippy (Cargo 1.97+), which keeps every
unit the base warmed fresh. RUSTFLAGS=-Dwarnings changes every unit and
rebuilds everything.
Refreshing a base
A refresh is seeded from the newest base with the same fingerprint, so only
what changed is rebuilt. When the commit’s manifests, lockfiles, Cargo
configs or toolchain files differ from the seed’s, the seed’s target holds
units the new build may not select any more, and an agent that selected one
again (by reverting the change) would find it fresh while it holds older
code. So Ferry watches the warm build through the cargo shim and deletes,
before the base is published, every unit the build neither compiled nor
found fresh. It does the same when the seed was warmed with other Cargo
flag variables or another CARGO_INCREMENTAL, or when only one of the two
builds has CI or TF_BUILD set.
For that, every cargo call in the warm commands must be one Ferry can
observe: build, check, test, bench, run, rustc, rustdoc, doc,
fix or clippy, through cargo on PATH, without
--message-format=human or short (JSON formats are fine). The shim adds
--message-format=json-render-diagnostics to calls that do not ask for a
format; diagnostics still look as usual. A warm build that runs anything else
(cargo nextest, an alias such as cargo xtask, cargo install) is not
observed: such a base seeds only refreshes with identical inputs, as with
prune = false. If the warm commands become unobservable in the middle of a
lineage (a script they run changed), the refresh is built again from scratch,
with a warning. A cargo call that bypasses the shim (an absolute path to
cargo, $CARGO from a build tool) cannot be seen: what it finds fresh is
deleted, and what it rebuilds makes Ferry rebuild the base from scratch.
Do not pass --target-dir or --config build.target-dir=... /
--config build.build-dir=... in warm commands. Command-line options beat
the environment variables Ferry sets, so the build would land outside the
base.
[agent]: worktrees and branches
| Key | Type | Default | Meaning |
|---|---|---|---|
branch-prefix | string | "ferry/" | Each agent’s branch is <branch-prefix><name>, unless ferry spawn --branch names it. |
worktree-dir | path | see below | Parent directory for agent worktrees. Each repository gets a subdirectory <repo-name>-<hash>, and each agent a directory inside it. A leading ~ is expanded; a relative path is relative to the main checkout. |
The worktree parent directory is, in order of precedence:
FERRY_WORKTREES;[agent] worktree-dir;<store parent>/worktrees, when the store parent was set explicitly withFERRY_HOMEor[store] dir;$XDG_DATA_HOME/ferry/worktreesor~/.local/share/ferry/worktreeson Linux,~/Library/Application Support/ferry/worktreeson macOS.
ferry spawn --path <dir> overrides all of these for one agent.
[store]: build state
| Key | Type | Default | Meaning |
|---|---|---|---|
dir | path | $XDG_CACHE_HOME/ferry or ~/.cache/ferry (Linux), ~/Library/Caches/ferry (macOS) | Parent directory for Ferry’s build state: one store per repository (bases, forks, canonical mount points) plus bin/, which holds the cargo shim. A leading ~ is expanded; a relative path is relative to the main checkout. |
FERRY_HOME overrides it. The resolved path must be absolute. Put it on a
local disk, ideally a copy-on-write filesystem (btrfs, XFS): forks then become
reflinks. Network and FUSE filesystems make builds slow
and mtimes unreliable; ferry doctor warns about them. On Linux kernels
before 6.13, the overlay backend refuses store paths containing ,, :, \
or a newline, which those kernels’ overlayfs mount options cannot express.
[fork]: how agents fork the base
| Key | Values | Default |
|---|---|---|
backend | "auto", "reflink", "overlay", "copy" | "auto" |
autopicksreflinkwhen the store’s filesystem supports copy-on-write clones, elseoverlaywhen the agent runs in the namespace sandbox and overlayfs works there, elsecopy. An agent keeps the backend it was spawned with.overlay: Linux only, and needs the namespace sandbox.reflink: Linux on btrfs, XFS or OpenZFS 2.2+; macOS on APFS.copy: a full copy, available everywhere.
An explicit backend that is not available is an error, never a silent fallback. See How it works for the trade-offs.
[sandbox]
| Key | Values | Default | Meaning |
|---|---|---|---|
mode | "auto", "namespace", "off" | "auto" | auto uses the Linux namespace sandbox when it works and builds at host paths otherwise. namespace requires it: ferry base and ferry spawn fail without it. off never uses it. |
remap-paths | boolean | false | Experimental. Path-independent incremental builds for agents that run without the namespace sandbox. Needs rustc 1.98 or newer. See Path remapping. |
The isolation a base was built with is part of its identity. Agents of a base
built with the namespace sandbox use it too; if namespaces stop working later,
such agents build at host paths with a warning (or fail, with
mode = "namespace").
[freshness]
| Key | Values | Default |
|---|---|---|
mode | "auto", "checksum", "mtime" | "auto" |
mtime: Cargo’s normal freshness check, made reliable by stamping files that are identical to the base (see How it works).checksum: Cargo’s unstable checksum freshness. rustc inputs are compared by size and hash instead of mtime; build-script inputs, rustdoc and the comparison of a unit’s outputs with its dependencies’ stay mtime-based, so Ferry still stamps. Needs a nightly toolchain; on stable,checksumis an error. Ferry setsCARGO_UNSTABLE_CHECKSUM_FRESHNESS=trueandCARGO_BUILD_FINGERPRINT=contentfor the base and every agent.auto:checksumon nightly toolchains,mtimeotherwise.
Switching the mode rebuilds every unit, so it is part of the base’s identity.
[linux] and [macos]
Per-platform overrides. Only the table for the running platform applies.
| Key | Overrides | Values |
|---|---|---|
backend | [fork] backend | "auto", "reflink", "overlay", "copy" |
sandbox | [sandbox] mode | "auto", "namespace", "off" |
freshness | [freshness] mode | "auto", "checksum", "mtime" |
[linux]
sandbox = "namespace"
[macos]
backend = "reflink"
Environment variables
Variables Ferry reads. They override ferry.toml.
| Variable | Overrides | Values |
|---|---|---|
FERRY_HOME | [store] dir | Absolute path of the store parent. Also moves the default worktree parent to $FERRY_HOME/worktrees. Empty means unset. |
FERRY_CONFIG | config file location | Path of the config file to load instead of <main checkout>/ferry.toml. |
FERRY_WORKTREES | [agent] worktree-dir | Absolute path of the worktree parent. Empty means unset. |
FERRY_BACKEND | backend | auto, reflink, overlay, copy |
FERRY_SANDBOX | sandbox mode | auto, namespace, off |
FERRY_FRESHNESS | freshness mode | auto, checksum, mtime |
FERRY_BASE_COMMAND | [base] warm commands | One shell command that replaces all warm commands. Blank means unset. |
FERRY_REMAP_PATHS | [sandbox] remap-paths | 1 or true; 0, false or empty |
FERRY_BASE_PRUNE | [base] prune | 1 or true; 0, false or empty |
An invalid value is an error that names the variable and the accepted values.
Environment Ferry sets
For commands started with ferry run and ferry shell:
| Variable | Value |
|---|---|
PATH | the shim directory first, then your PATH |
FERRY_AGENT | the agent’s name, for your scripts and prompts (Ferry never reads it) |
FERRY_WORKTREE | the agent’s worktree (host path) |
For cargo started by the shim (and for warm commands):
| Variable | Value |
|---|---|
CARGO_TARGET_DIR, CARGO_BUILD_BUILD_DIR | the agent’s target dir as the build sees it |
CARGO_UNSTABLE_CHECKSUM_FRESHNESS, CARGO_BUILD_FINGERPRINT | true, content (checksum freshness only) |
RUSTC_WORKSPACE_WRAPPER | the ferry-rustc wrapper (remap-paths mode only) |
CARGO_BUILD_WARNINGS | warn (warm commands only) |
Ferry deliberately never exports CARGO_TARGET_DIR to anything else. See
Agent integrations.
What makes a new base
A base is reused only if its commit and fingerprint match. The fingerprint
covers the rustc -vV and cargo -V output, the commit’s
rust-toolchain.toml or rust-toolchain file, the warm commands that run
(so also whether clippy is installed), the freshness mode, the isolation (namespace sandbox or not) and remap-paths.
Changing any of them and running ferry base builds a new base. Existing
agents keep using the base they were forked from.
Examples
Annotated variants are in
examples/config/:
ci-checks.toml: warmcheck,clippyandtestfor all targets with--locked.release-profile.toml: warm the dev and release profiles.custom-commands.toml: your own warm commands.workspace-in-subdirectory.toml: a Cargo workspace below the repository root (cd rust && cargo build --workspace).platform-overrides.toml: store location, branch prefix and explicit per-platform choices.
CLI reference
Usage: ferry [OPTIONS] <COMMAND>
| Command | Summary |
|---|---|
init | Write ferry.toml and report what this machine supports |
base | Build an immutable warm base that new agents fork from |
spawn | Create an agent: a git worktree plus a forked warm target dir |
run | Run a command in an agent’s build context |
shell | Start an interactive shell in an agent’s build context |
env | Print an agent’s paths and environment for wiring into agent tools |
list | List bases and agents with their status and disk usage |
rm | Remove an agent’s worktree and build state |
gc | Delete bases that no agent uses and that are not current |
doctor | Check platform capabilities and print fixes for what is missing |
shim | Print the directory holding Ferry’s cargo shim |
hook | Hook handlers for agent tools (reads the tool’s JSON on stdin) |
setup | Change the system so Ferry’s namespace sandbox works (asks first) |
help | Print this message or the help of the given subcommand(s) |
Global options
These work with every command, before or after it.
| Option | Meaning |
|---|---|
-C <DIR> | Run as if ferry was started in DIR |
--json | Print machine-readable JSON on stdout |
-h, --help | Print help (-h prints a summary) |
-V, --version | Print version |
Commands run against the git repository containing the current directory (or
the -C directory). Outside one, doctor still reports the machine-level
checks plus a failing repository check, and the hook handlers use the
directory named in their JSON input; every other command fails.
Output, JSON and exit codes
Data goes to stdout; progress, warnings and errors go to stderr. Colour is used only on a terminal. Errors look like this:
error: no warm base exists for this repository
hint: run `ferry base` to build one
with caused by: lines in between when there is an underlying error.
--json is accepted by every command. It switches init, base, spawn,
env, list, rm, gc, doctor (also with --apparmor-profile), shim
and setup apparmor to a single JSON document on stdout; the shapes are
listed under each command. Ferry is pre-1.0, so fields may still change between releases.
--json has no effect on run and shell, which replace themselves with
your command, or on the hook handlers, which speak the agent tool’s
protocol.
Exit status is 0 on success and 1 on error. doctor exits 1 when a check
fails. run and shell exit with the status of the command they run.
ferry init
Usage: ferry init [OPTIONS]
Write ferry.toml and report what this machine supports.
| Option | Meaning |
|---|---|
--force | Overwrite an existing ferry.toml |
Writes the default configuration to
ferry.toml in the root of the main checkout, then prints the same plan as
ferry doctor. Without --force, an existing file is left alone with a
warning.
JSON: { "config": <path>, "written": <bool>, "doctor": <doctor report> }.
ferry base
Usage: ferry base [OPTIONS]
Build an immutable warm base that new agents fork from.
| Option | Meaning |
|---|---|
--rev <REV> | Commit to build (any git revision) [default: HEAD] |
--cold | Build from scratch instead of seeding from the newest compatible base |
--watch | Keep running and build a base each time --rev moves to a new commit (not with --cold) |
--interval <SECS> | With --watch: seconds between polls of --rev [default: 2] |
--gc | With --watch: run ferry gc after each new base |
Checks out the commit into the store, runs the warm commands,
and makes the base current, so new agents fork from it. If a base with the
same commit and fingerprint already exists, it is made current without
building. Otherwise, unless --cold is given, the new base is seeded from the
newest ready base with the same fingerprint, so only what changed is rebuilt.
The seed must have the same manifests, lockfiles, Cargo configs, toolchain
files, Cargo flag variables and CARGO_INCREMENTAL, and agree on whether
CI or TF_BUILD is set. Otherwise its warm build must have been observed
([base] prune, the default): the units the new build does not use are then
deleted, and the success line says how many (pruned N units). Build output
goes to stderr.
Warnings are printed when the namespace sandbox is unavailable (builds then
run without canonical paths), when a Cargo config above the repository (or an
untracked one in it) will not apply to Ferry’s builds, when the warm
commands wrote files into the source tree, which agents will not have (build
scripts writing outside OUT_DIR), when clippy is not installed (the
generated clippy command is skipped), when the generated clippy command
failed (the base is kept), and when a refresh seeded across such changes was
rebuilt from scratch because Ferry could not tell which units the warm
commands used.
JSON: { "base": {...}, "reused": <bool>, "warnings": [...] }, where base
has id, commit, created_unix, build_secs, freshness, isolation,
toolchain, commands (the ones that ran), command_secs (seconds per
command), incomplete (clippy commands that failed), ferry_version,
dirtied, remap_paths, unit_inputs (the hash seeding compares),
seeded_from, observed (Ferry saw every cargo call of the warm build) and
pruned ({ "units", "bytes" } deleted from the seed’s units, or null).
ferry base --watch
Runs in the foreground until interrupted (run it in a terminal tab, a
systemd or launchd user service, or a devcontainer’s postStartCommand), so
the current base follows --rev (default HEAD; --rev origin/main follows
fetches; a revision that names no commit at startup is refused). It polls
the revision, and once a new commit has been seen on two polls in a row it
runs ferry base --rev <commit> in a child process. Polls
are skipped while a rebase, merge, cherry-pick, revert or bisect is in
progress. A build for a commit that fails keeps the previous base current and
is not retried until the revision moves again; a build is never cut short
when the revision moves, the newest commit is built next. Only one watcher
runs per repository; a second one is refused. With --gc, unused bases are
deleted after each new base, which matters on filesystems without reflinks,
where every base holds a full copy of the target dir.
JSON: one object per line, with event set to building (commit),
built (commit, report: what ferry --json base prints), failed
(commit, error), collected (report: what ferry --json gc prints)
or gc-failed (error).
ferry spawn
Usage: ferry spawn [OPTIONS] <NAME>
Create an agent: a git worktree plus a forked warm target dir.
| Argument or option | Meaning |
|---|---|
<NAME> | Agent name (letters, digits, ‘-’, ‘_’, ‘.’) |
--from <REF> | Revision the agent’s branch starts at [default: HEAD] |
--path <DIR> | Where to create the worktree (default: the worktree directory, see [agent] worktree-dir) |
--branch <BRANCH> | Branch name (default: the configured branch prefix plus NAME) |
Creates the branch and worktree, stamps the files that are identical to the
base, and forks the base’s target dir. It never builds. The base is the
current one if it is at --from, else any ready base at that commit, else the
current base; files that differ from the chosen base are rebuilt on the first
build. Fails if the agent, the branch or a non-empty worktree directory
already exists, or if there is no base. A failure rolls back what the spawn
created: the worktree, the branch if it still points at --from, and the
fork.
Without --path, the worktree goes in the worktree parent directory (see
Configuration), which is not
inside the store unless you configured it that way. A relative --path is
relative to the current directory. The directory must not exist or must be empty.
JSON: { "agent": {...}, "backend_reason": <string>, "fork": { "files", "bytes", "elapsed_secs" }, "elapsed_secs": <number>, "warnings": [...] },
where agent has name, state, base, from, branch, worktree,
backend, isolation, freshness, created_unix, stamp and origin.
ferry run
Usage: ferry run [OPTIONS] [NAME] -- <COMMAND>...
Run a command in an agent’s build context.
Without NAME, uses the agent whose worktree contains the current directory.
| Argument | Meaning |
|---|---|
[NAME] | The agent |
<COMMAND>... | The command and its arguments, after -- |
Enters the agent’s sandbox and replaces itself with the command. The working
directory is the current directory if it is inside the agent’s worktree, and
the worktree root otherwise; either way it is a host path. PATH starts with
the shim directory, and FERRY_AGENT and FERRY_WORKTREE are set. Every
cargo the command runs, directly or from scripts, goes through the shim and
builds in the agent’s target dir.
ferry run a1 -- cargo test -p parser
ferry run a1 -- claude
ferry run -- cargo build # inside a1's worktree: NAME comes from the directory
ferry shell
Usage: ferry shell [OPTIONS] [NAME]
Start an interactive shell in an agent’s build context.
Like ferry run [NAME] -- $SHELL (or /bin/sh when SHELL is unset).
Without NAME, uses the agent whose worktree contains the current directory.
ferry env
Usage: ferry env [OPTIONS] [NAME]
Print an agent’s paths and environment for wiring into agent tools.
Without NAME, uses the agent whose worktree contains the current directory.
Prints shell lines you can eval:
export FERRY_WORKTREE=/home/you/.local/share/ferry/worktrees/myproject-d5c4fd3a48754db4/a1
export PATH=/home/you/.cache/ferry/bin:"$PATH"
# cargo in /home/you/.local/share/ferry/worktrees/myproject-d5c4fd3a48754db4/a1 now builds in agent a1's sandbox; other commands: ferry run a1 -- <command>
It never prints CARGO_TARGET_DIR: agent tools pass their environment to
sub-agents working in other worktrees, and the shim picks the target dir per
call instead.
JSON: { "name", "worktree", "branch", "base", "backend", "isolation", "build_target_dir", "host_target_dir", "shim_dir", "run_prefix" }.
build_target_dir is the target dir as builds see it (the canonical path in
the namespace sandbox). host_target_dir is the fork’s host path for the
reflink and copy backends and null for overlay, whose target exists only
inside the sandbox. run_prefix is ["ferry", "run", <name>, "--"].
ferry list
Usage: ferry list [OPTIONS]
List bases and agents with their status and disk usage.
Bases: * marks the current base; BUILT is its age (or incomplete);
DISK its size; USED BY the number of agents forked from it. Agents:
BASE (with (old) when it is not the current base), FORK (backend),
SANDBOX (namespace or none), DISK, STATUS and WORKTREE.
DISK for an agent is the build state it holds beyond the base: the overlay
upper layer, or for reflink clones the files rewritten since the fork (shared
extents cannot be seen through stat), or the whole copy. STATUS is
running while any process started through ferry run, ferry shell or the
cargo shim for the agent is alive (JSON busy, with the pids Ferry can
attribute in running), incomplete
when spawning did not finish (remove it with ferry rm), worktree missing,
or idle.
JSON: { "bases": [...], "agents": [...] }. A base has id, ready,
current, meta, agents and disk_bytes; an agent has the spawn agent
fields plus base_is_current, worktree_exists, private_bytes and
busy and running (PIDs).
ferry rm
Usage: ferry rm [OPTIONS] <NAME>
Remove an agent’s worktree and build state.
Refuses if the worktree has uncommitted changes or processes are still running in the agent’s sandbox. The branch is kept if it has new commits.
| Option | Meaning |
|---|---|
--force | Discard uncommitted changes and ignore running processes |
Untracked files count as uncommitted changes, except the Cargo.lock that
spawn copied from the base when the repository does not track one (as long as
it is unchanged). Processes started for the agent are detected on every
backend. The branch is deleted only if it still points at the
commit the agent started from; otherwise Ferry keeps it and says how many
commits it carries.
JSON: { "name", "worktree", "branch", "branch_deleted", "branch_commits", "rescued_ref" },
where rescued_ref names the ref that keeps commits a --force removal
found on no branch (otherwise null).
ferry gc
Usage: ferry gc [OPTIONS]
Delete bases that no agent uses and that are not current.
| Option | Meaning |
|---|---|
--dry-run | Show what would be deleted |
First removes abandoned agents created by the Claude Code hook: clean
worktree, no new commits, nothing running, and not git worktree locked by
Claude Code. Agents created with ferry spawn are never removed. Then deletes
every base that is neither current nor used by an agent, including incomplete
ones left by interrupted builds, and skips bases that are being built.
The human-readable output lists reaped agents and removed and skipped bases.
JSON: { "removed": [...], "busy": [...], "kept": [...], "reaped_agents": [...], "warnings": [...], "freed_bytes": <number>, "dry_run": <bool> }.
ferry doctor
Usage: ferry doctor [OPTIONS]
Check platform capabilities and print fixes for what is missing.
| Option | Meaning |
|---|---|
--apparmor-profile | Print an AppArmor profile that allows this ferry binary to use user namespaces (Ubuntu 24.04+) |
Prints one line per check (✓ ok, · information, ! warning with a fix:
line, ✗ failure), then the plan: which fork backend agents will use and
whether their first edit can reuse incremental state. See
Platform support for every check.
--apparmor-profile prints the profile for the binary’s resolved path, and
warns on stderr if that path, or a directory above it, is writable by a
non-root user; it refuses paths with characters AppArmor reads as patterns.
With --json it prints { "binary", "profile", "warning" }. See
AppArmor on Ubuntu.
JSON: { "checks": [{ "name", "status", "detail", "fix" }], "capabilities": {...}, "plan": {...} }.
capabilities has os, kernel, store_fs, reflink, userns, overlay,
sysctls and probe_label (the namespace probe’s AppArmor label, such as
ferry (enforce), or null); plan has backend, backend_reason, isolation,
freshness and incremental_reuse.
ferry shim
Usage: ferry shim [OPTIONS]
Print the directory holding Ferry’s cargo shim.
Put it first on PATH (export PATH="$(ferry shim):$PATH"): every cargo call
made inside an agent worktree then runs in that agent’s sandbox with its own
target dir, and cargo elsewhere runs unchanged.
The directory is <store parent>/bin; ferry shim creates it and points its
cargo link at the running ferry binary. Run it inside a repository.
JSON: { "shim_dir": <path> }.
ferry hook
Usage: ferry hook [OPTIONS] <COMMAND>
Hook handlers for agent tools (reads the tool’s JSON on stdin).
| Command | Meaning |
|---|---|
claude-worktree-create | Claude Code WorktreeCreate: spawn an agent, print its worktree path |
claude-worktree-remove | Claude Code WorktreeRemove: remove the agent unless it has uncommitted work |
claude-session-start | Claude Code SessionStart: put the cargo shim on PATH via $CLAUDE_ENV_FILE |
See Claude Code for what each handler reads and does.
ferry setup apparmor
Usage: ferry setup apparmor [OPTIONS]
Let ferry create user namespaces on Ubuntu 24.04+: install it to a root-owned
path and load its AppArmor profile, with one sudo call.
| Option | Meaning |
|---|---|
-y, --yes | Run without asking; the plan is still printed |
--dry-run | Print the plan and exit without changing anything (not with --yes) |
--uninstall | Unload and remove Ferry’s profile instead |
--bin-dir <DIR> | Where to install ferry when its current path cannot be profiled (default /usr/local/bin) |
It first checks whether anything is needed. It does nothing, and exits 0, on
macOS, when the running ferry can already create user namespaces, when the
kernel has no kernel.apparmor_restrict_unprivileged_userns sysctl, and when
that sysctl is 0. It refuses, and exits 1, when run as root, inside a Ferry
agent or a container, without the AppArmor 4.0 userspace, or when a file in
/etc/apparmor.d that Ferry did not write already defines a profile named
ferry or unpriv_ferry. It never reads the current repository or its
ferry.toml.
Otherwise it prints, on stderr: why the setup is needed (the sysctl, the
probe’s failure and its AppArmor label), each step, what the profile allows,
the full profile, the root script and the exact sudo command. The steps,
each listed only when needed:
- Install the running binary as
<bin-dir>/ferry, owned by root with mode 0755, unless it already sits at a root-owned path a profile can name or<bin-dir>/ferryis already a root-owned copy of it. A copy your user owns, or a symlink, is replaced. Root copies the binary from/proc/<pid>/exe, so nothing your user can write is copied. - Write
/etc/apparmor.d/<path with . for />(for/usr/local/bin/ferry,usr.local.bin.ferry), check that it parses, and remove older Ferry profile files such as/etc/apparmor.d/ferry. - Load it with
apparmor_parser -r; AppArmor loads it again at every boot.
Then, without sudo, it checks that the profiled binary can create a user
namespace, points the cargo shim at it, and warns if the first ferry on
PATH is another copy.
| Situation | What happens |
|---|---|
stdin is a terminal, no --yes | asks Run it? [y/N]; only y or yes runs it |
--yes | runs without asking; with stdin not a terminal, sudo gets -n, so a needed password fails at once |
stdin is not a terminal, no --yes | refuses: refusing to run sudo without confirmation: stdin is not a terminal |
--json without --yes | refuses, like stdin not being a terminal |
--dry-run | prints the plan and exits 0 |
Running the upgraded copy is the upgrade path: after cargo install
upgrades ~/.cargo/bin/ferry, run ~/.cargo/bin/ferry setup apparmor. It
installs the new binary and leaves the profile alone. A bare ferry setup apparmor with /usr/local/bin first on PATH runs the old installed copy.
That copy is set up already, so it reports already-done and says when
~/.cargo/bin/ferry is a different build. With
--uninstall it unloads and removes every Ferry profile file (one sudo call,
same confirmation rules); it leaves the installed binary and any
/etc/apparmor.d/local/ferry in place and says so.
JSON: { "action": "install" | "uninstall", "outcome", "reason", "binary", "profile_path", "steps": [...], "command": [...], "verified": { "userns", "overlay", "label" } | null, "warnings": [...] }. outcome is one of
not-applicable, not-needed, already-done, refused, planned (a dry
run), done and failed; command is the full sudo argv. Exit status is 1
for refused and failed. See
AppArmor on Ubuntu.
Ferry under other names
The ferry binary also runs as:
cargo, through the shim link in<store parent>/bin. It resolves the agent from the working directory and runs the real cargo (the nextcargoonPATHthat is not Ferry) in that agent’s sandbox, or unchanged outside agent worktrees. It exits with status 127 if no real cargo is found.ferry-rustc, aRUSTC_WORKSPACE_WRAPPERused only by the experimental remap-paths mode.
Platform support
Ferry gives every agent a warm, private target dir everywhere it runs. What varies is whether an agent’s first edit can reuse rustc’s incremental state, which needs the Linux namespace sandbox (or the experimental remap-paths mode).
- Crate-level reuse: crates whose sources match the base are not rebuilt,
the workspace’s included. An edit rebuilds the edited crate and its
dependents from scratch. Without a canonical path, the few crates that
captured the base’s absolute paths (a test baking in
env!("CARGO_MANIFEST_DIR"), users of a proc macro reading it) also rebuild once after spawn. - Incremental reuse: on top of that, rustc recompiles only the parts of an edited crate that changed. In the prototype on ripgrep (2 vCPU) the first edit after spawning took 1.75 s with incremental reuse and 5.3 s with crate-level reuse only.
Agents in the namespace sandbox get both crate-level and incremental reuse.
After an agent’s first build, its own incremental state exists and later edits are incremental either way.
Summary
| Platform | Fork backends | Sandbox | First edit after spawn |
|---|---|---|---|
| Linux with unprivileged user namespaces (Debian, Fedora, Arch, NixOS, RHEL without STIG hardening, …) | reflink, overlay, copy | namespace sandbox with canonical paths | incremental reuse |
| Ubuntu 24.04 and later, with Ferry’s AppArmor profile installed | reflink, overlay, copy | namespace sandbox | incremental reuse |
| Ubuntu 24.04 and later, default policy | reflink, copy | none | crate-level reuse |
| Docker, default seccomp profile | reflink, copy | none | crate-level reuse |
| macOS (APFS) | reflink (clonefile), copy | none | crate-level reuse; experimental remap-paths mode |
| Windows | not supported |
Ferry does not build on Windows: compiling it there stops with a
compile_error! saying so.
Linux
The full feature set: agents build in a private user and mount namespace, at canonical paths. Forks are copy-on-write clones where the store’s filesystem supports them; elsewhere the overlay backend forks in constant time. This needs:
- unprivileged user namespaces, which are on by default on Debian 12 and 13,
Fedora, RHEL 9 and 10, Arch and NixOS, but not on Ubuntu 24.04 and later
(AppArmor) or in Docker’s default seccomp profile, and are disabled by the
DISA STIG for RHEL (
user.max_user_namespaces = 0); - for the overlay backend, Linux 5.11 or newer and a store filesystem that supports user extended attributes (ext4, btrfs, XFS; tmpfs from 6.6).
When namespaces are unavailable, sandbox = "auto" falls back to building at
host paths, and ferry base warns about it. See
Troubleshooting for every case.
Ubuntu 24.04 and later
Ubuntu restricts unprivileged user namespaces with AppArmor
(kernel.apparmor_restrict_unprivileged_userns = 1). Creating one succeeds,
but the process gets no capabilities inside it, so mapping its uid and
mounting fail. Ferry detects this and either:
- uses the namespace sandbox once
ferry setup apparmorhas installed ferry to a root-owned path with its AppArmor profile (one confirmedsudocall; CI runs Ferry’s sandbox tests on Ubuntu 24.04 this way), or - falls back to crate-level reuse.
Ubuntu on WSL2 runs Microsoft’s kernel, which has no such sysctl, and Linux
Mint 22 ships the restriction switched off; ferry setup apparmor reads the
sysctl and says when nothing is needed.
See AppArmor on Ubuntu. Ubuntu 23.10 ships the restriction switched off.
Containers
Docker’s default seccomp profile allows unshare and mount only with
CAP_SYS_ADMIN, and on AppArmor hosts Docker’s default AppArmor profile
denies mounts, so Ferry falls back to crate-level reuse inside Docker.
Rootless Podman’s default profiles allow them. On Ubuntu 24.04 and later
hosts, the host’s restriction applies to container processes too, and only
the host can lift it. If you run Ferry in a container with namespaces
allowed, put FERRY_HOME on a volume, because overlayfs cannot use the
container’s overlay root filesystem as its upper layer. See
Containers for the seccomp profile and a
devcontainer example. These container facts come from the profiles’ sources
and standalone probes; Ferry itself has not been tested in containers.
Filesystems
Where the store lives decides which fork backends work and what they cost.
| Filesystem | reflink backend | overlay upper layer | auto picks | Notes |
|---|---|---|---|---|
| btrfs | yes | yes | reflink | overlay copy-up is a reflink |
| XFS | yes (reflink=1, the mkfs default since xfsprogs 5.1) | expected to work, not tested | reflink | overlay copy-up is a reflink |
| OpenZFS | 2.2.0, and 2.3.0 or newer; off by default in 2.2.2 to 2.2.x | not tested | reflink where block cloning is on | |
| ext4 | no | yes | overlay | overlay copy-up copies the whole file |
| tmpfs | no | Linux 6.6 or newer | overlay (6.6+) | |
| NFS, CIFS, FUSE | not recommended | not recommended | ferry doctor warns: slow builds, unreliable mtimes |
Without the namespace sandbox auto never picks overlay: it uses reflink
where clones work and copy elsewhere.
ferry doctor probes reflinks and overlayfs on the store’s actual filesystem
rather than trusting this table.
macOS
Agents fork the base’s target dir with one clonefile(2) call on APFS (the
reflink backend) and build at their own paths. macOS has no per-process
mount namespaces, so there is no canonical path: crates are reused except the
few that captured the base’s absolute paths, which rebuild once after spawn,
and the first edit of a crate loses its incremental state.
The experimental remap-paths mode is designed for this case. On Linux it reuses 12 of 13 library codegen units after a fork to a new path, against 0 of 13 without it. That it works the same on macOS is predicted from rustc’s source but has not been verified on a Mac.
Bases and forks live in ~/Library/Caches/ferry; Ferry excludes base target
dirs and agent build state from Time Machine backups. Agent worktrees live in
~/Library/Application Support/ferry/worktrees, because macOS may purge
caches.
What ferry doctor reports
ferry doctor runs these checks and ends with the plan for new agents.
| Check | Status | What it means |
|---|---|---|
git | ok or fail | git --version; fails if git is missing |
repository | ok or fail | the main checkout Ferry will use; fails outside a git repository |
config | ok or info | the ferry.toml that was loaded, or that defaults are in use |
toolchain | ok or fail | the rustc and cargo the repository selects |
clippy | ok or info | whether cargo clippy works, when the toolchain works and the warm commands include the generated clippy command; bases skip it when it does not |
freshness | ok | checksum freshness (content hashes) or mtime stamping of files proven identical to the base |
store | ok or warn | the store path, its filesystem and free space; warns on NFS, CIFS and FUSE, and on overlayfs (a container’s root filesystem) |
reflink | ok or info | whether copy-on-write clones work on the store’s filesystem |
namespaces | ok or warn (Linux) | whether an unprivileged user and mount namespace with a bind mount works; the fix names the cause it found (AppArmor, with ferry setup apparmor as the fix or the path of a different ferry binary a Ferry profile covers; a sysctl; or a container) |
overlayfs | ok or warn (Linux) | whether overlayfs mounts inside such a namespace, with upper and work dirs in the store |
sandbox | info (macOS) | that macOS has no mount namespaces |
remap-paths | info | when bases would remap paths (remap-paths on, no namespace sandbox, rustc 1.98 or newer): cargo clippy runs are not remapped |
The plan line reads like:
→ agents fork with reflink (btrfs supports copy-on-write clones) and build in a namespace sandbox at canonical paths (incremental reuse on the first edit)
on ext4:
→ agents fork with overlay (overlayfs works in the namespace sandbox on ext4; no copy-on-write clones on ext4 (FICLONE: Operation not supported (os error 95))) and build in a namespace sandbox at canonical paths (incremental reuse on the first edit)
or, without namespaces:
→ agents fork with reflink (btrfs supports copy-on-write clones) and build at their own paths (only crates that captured the base's paths rebuild; no incremental reuse on the first edit)
ferry --json doctor adds the raw probe results and the sysctls that explain
namespace failures: user.max_user_namespaces,
kernel.unprivileged_userns_clone,
kernel.apparmor_restrict_unprivileged_userns,
kernel.apparmor_restrict_unprivileged_unconfined and
kernel.yama.ptrace_scope, where they exist, and probe_label, the namespace
probe’s AppArmor label (ferry (enforce) when Ferry’s profile attached).
Benchmarks
Ferry’s benchmark harness is in bench/ in the repository. Results are
published here as they are measured. This page never shows estimated or
extrapolated numbers: every figure comes from a recorded run.
What is measured
| Metric | What it is |
|---|---|
| Spawn time | from ferry spawn (or the equivalent setup) until the agent can build |
| First edit after spawn | a fixed source edit in a new agent, then a build; wall time and crates compiled |
| Steady-state edit | the next edit in the same agent, once it has its own incremental state |
| Disk per agent | bytes the agent holds beyond the base after the edits (ferry list DISK) |
| N concurrent agents | N agents making different edits and building at the same time: wall time, and whether each artifact contains only its own change |
Every result records the hardware (CPU model and count, memory), the OS and kernel, the filesystem of the store, the Rust toolchain, the workload and its commit, and the versions of the other tools compared. Numbers from one machine are not projected onto another, and results from small workloads are not scaled up to large ones.
Results
ripgrep, Ferry at d7fb5cb
| Workload | ripgrep at 6892bc2, debug cargo build: 44 units (34 crates; each build script counts as its own unit) |
| Edit | appends a documented pub fn to crates/ignore/src/lib.rs, which forces ignore and rg to rebuild; every run uses a new function |
| CPU | AMD Ryzen 5 3600X (6 cores, 12 logical CPUs, governor powersave) |
| RAM | 31 GiB |
| OS, kernel | NixOS 26.11, Linux 6.18.55 |
| Filesystem | btrfs (zstd:1 compression), which holds the store and the work dirs |
| Toolchain | rustc 1.99.0, cargo 1.99.0 |
| Other tools | kache 1.2.2, hyperfine 1.21.0 |
| Runs | 3 per step; 2 for steps that are, or prepare with, a cold build |
| Date (UTC) | 2026-10-24, 20:53 to 21:01 |
| Load average | 1.21 before, 4.86 after (1 min), on 12 logical CPUs: a desktop in use, not an idle machine |
First edit in a new agent (spawn, edit, build). Every artifact check passed: the edited crate’s rlib held exactly that agent’s own edit.
| Scenario | Median | Min | Max | Units compiled |
|---|---|---|---|---|
| Cold worktree, empty target | 7.063 s | 7.019 s | 7.107 s | 44 |
Copied warm target (cp -a), fresh checkout | 3.675 s | 3.598 s | 3.687 s | 10 |
| kache 1.2.2, warm local store | 2.863 s | 2.836 s | 2.876 s | 10, of which 8 restored from its store |
| Ferry, sandbox off | 3.204 s | 3.107 s | 3.480 s | 2 |
| Ferry, sandbox off, remap-paths | 2.000 s | 1.975 s | 2.033 s | 2 |
| Ferry, namespace sandbox, overlay backend | 1.252 s | 1.251 s | 1.274 s | 2 |
| Ferry, namespace sandbox, reflink backend | 1.130 s | 1.126 s | 1.137 s | 2 |
Other steps from the same run (medians):
| Step | Result |
|---|---|
ferry spawn | 0.103 s (overlay), 0.156 s (reflink), 0.169 s (sandbox off) |
| First build after spawn, no edit | 0.06–0.08 s, 0 units compiled, in every Ferry scenario |
| 3 agents with different edits building at once (wall) | 1.409 s (reflink), 1.572 s (overlay), 2.753 s (remap-paths), 4.130 s (sandbox off); 6.663 s for 3 worktrees sharing one CARGO_TARGET_DIR |
| Second, different edit in the same agent | 0.878 s (reflink), 1.066 s (overlay), 0.890 s (remap-paths), 0.940 s (sandbox off), 0.912 s (kache) |
ferry base after a commit, seeded from the previous base | 1.338 s, 2 units |
ferry base --cold after the same commit | 7.414 s, 44 units |
What these numbers show:
- The gain is in a new agent’s first edits. Later edits take about the same time with kache and in every Ferry scenario.
- The overlay backend is slower than reflink on later edits (1.07 s
against 0.88 s). Every timed build here is its own
ferry run, and an overlay build waits for its sandbox’s teardown, about 0.2 s per build on ripgrep (see How it works). That matches the gap. - Sandbox off loses the incremental state. The agent reuses every
unchanged crate, but
ignoreandrgrecompile from scratch. kache compiles the same two (the table’s 10 counts the 8 it restores from its store, which Cargo reports as compiled), and is faster here. Remap-paths keeps most of the library’s incremental state at the agent’s own path; on Linux thergbinary, built with debug info, still recompiles from scratch. - These are one workspace’s results. This page does not scale them to larger workspaces; those are measured below. macOS results are not published yet.
All steps, the disk usage, and the raw run times are in
bench/results/ripgrep/
(results.md, results.json, env.json, and hyperfine’s exports).
Larger workspaces, Ferry 0.1.2
Measured on the same machine as ripgrep (AMD Ryzen 5 3600X, 6 cores and 12
threads, 31 GiB RAM, btrfs, NixOS 26.11, Linux 6.18) on 2026-10-27, with the
release binary of Ferry 0.1.2 (5be5ee1). The commits in each env.json
differ from it only in docs and benchmark files. Each value is a median: 3
runs per step for rust-analyzer (2 for cold builds), 2 runs (1 for cold
builds) for the others. The machine was a desktop in use; each env.json
records the load. Crates compiled are in brackets.
Every scenario runs the same command. First build is an agent’s first
build with no edit; first edit is a new agent’s first build after one edit;
next edit is a later edit in the same agent. Ferry’s rows use the namespace
sandbox. Reflink is what auto picks on this btrfs; overlay is what it picks
on ext4.
rust-analyzer
0c4d746, 281 build units cold, edit crates/ide-db/src/lib.rs (rebuilds 9
units), cargo build, rustc 1.99.0.
| Scenario | First build | First edit | Next edit | Spawn |
|---|---|---|---|---|
| Cold worktree | 91.9 s (281) | 74.0 s (281) | – | – |
| Copied warm target | 61.6 s (51) | 67.0 s (51) | – | – |
| kache 1.2.2 | 2.90 s (58) | 26.0 s (58) | 7.40 s (9) | – |
| Ferry, sandbox off | 27.3 s (12) | 29.1 s (12) | 7.44 s (9) | 1.38 s |
| Ferry, remap-paths | 9.27 s (12) | 9.58 s (12) | 7.73 s (9) | 1.10 s |
| Ferry, overlay | 0.24 s (0) | 7.38 s (9) | 7.29 s (9) | 0.45 s |
| Ferry, reflink | 0.21 s (0) | 7.69 s (9) | 6.59 s (9) | 1.28 s |
Refreshing the base after a commit: 9.06 s seeded from the previous base, 92.5 s cold.
uv
f8bec5e, 624 build units cold, edit crates/uv-resolver/src/lib.rs,
cargo build, rustc 1.99.0.
| Scenario | First build | First edit | Next edit | Spawn |
|---|---|---|---|---|
| Cold worktree | 80.0 s (624) | 79.5 s (624) | – | – |
| Copied warm target | 36.8 s (98) | 45.0 s (98) | – | – |
| kache 1.2.2 | 34.4 s (113) | 40.6 s (113) | 7.09 s (15) | – |
| Ferry, reflink | 0.46 s (0) | 9.00 s (15) | 6.97 s (15) | 1.25 s |
Refreshing the base after a commit: 10.0 s seeded from the previous base, 79.9 s cold.
ruff
8aeb8e2, 505 build units cold, edit crates/ruff_linter/src/lib.rs,
cargo build, rustc 1.99.0. A Ferry agent’s first build recompiles 6 units
here, where the other workspaces recompile none; the cause is not
investigated yet.
| Scenario | First build | First edit | Next edit | Spawn |
|---|---|---|---|---|
| Cold worktree | 131.6 s (505) | 142.1 s (505) | – | – |
| Copied warm target | 89.7 s (68) | 106.0 s (68) | – | – |
| kache 1.2.2 | 24.6 s (70) | 40.3 s (70) | 31.0 s (11) | – |
| Ferry, reflink | 3.24 s (6) | 9.97 s (14) | 9.02 s (11) | 2.21 s |
Refreshing the base after a commit: 13.5 s seeded from the previous base, 162.9 s cold.
polars
d0a98d9, edit crates/polars-lazy/src/lib.rs, cargo build -p polars-sql
(212 build units cold), on the nightly polars pins (rustc 1.100.0-nightly
2026-08-31), so Ferry uses checksum freshness.
| Scenario | First build | First edit | Next edit | Spawn |
|---|---|---|---|---|
| Cold worktree | 71.2 s (212) | 71.0 s (212) | – | – |
| Copied warm target | 56.6 s (34) | 54.8 s (34) | – | – |
| kache 1.2.2 | 2.32 s (212) | 9.63 s (212) | 1.62 s (2) | – |
| Ferry, reflink | 0.26 s (0) | 2.00 s (2) | 1.62 s (2) | 0.80 s |
Refreshing the base after a commit: 2.89 s seeded from the previous base, 70.7 s cold.
Raw results, including min and max of every step and disk usage:
bench/results/.
CI runs (macOS)
macOS has no namespace sandbox, so its numbers come from a reduced benchmark
that the Bench workflow runs on a GitHub-hosted macOS runner when started
by hand (see bench/README.md, “CI runs”). These runners are shared virtual
machines. Their results are published in their own table, never next to the
results above, and each table states the runner label, image version, run
link and date; the Ferry commit, rustc, Xcode and linker versions; the runs
per step with the minimum and maximum of each step; and the load before and
after. Every such table carries this note, with the hardware taken from that
run’s env.json (logical_cpus, cpu_model and ram_bytes), because it
depends on the runner label:
Measured on a shared GitHub-hosted VM (logical CPUs vCPU CPU model, RAM GiB). Not comparable with the local results above or with other CI runs; ratios within one run are indicative, absolute seconds are not.
Claims about how much of rustc’s incremental state survives on macOS come from the test suite’s codegen-unit counts, which do not depend on timing noise, and not from these times.
No CI results have been published yet.
Prototype results
Before Ferry was written, a bash prototype tested the approach. These are its
measured results: prototype (bash) on ripgrep, 2 vCPU. Setup: ripgrep at
commit 6892bc2 (39 crates, debug cargo build), 2 vCPU, 7 GB RAM, ext4,
rustc 1.97.0. The edit adds a documented function to crates/ignore, which
forces ignore and ripgrep to rebuild. It is a small workspace on a small
machine, so the ratios and crate counts say more than the absolute times.
| Scenario (new agent, then one edit) | Time | Crates compiled |
|---|---|---|
| Cold worktree, fresh target | 17.1 s | 34 |
| Copied warm target, fresh checkout mtimes | 7.2 s | 9 (spurious) |
| Copied warm target, stamped mtimes, different path | 5.3 s | 2 |
| kache 1.2.2 (warm cache, new worktree) | 5.5 s | 2 |
| Prototype fork (stamped mtimes, canonical path) | 1.75 s | 2 |
The prototype also measured:
- spawning with no edit: 0.10 s and 0 crates for the fork, 0.45 s for kache (which restores 34 crates);
- from the second edit on: about 1.1 s for the fork and 1.0 s for kache, which converge once the agent has its own incremental state;
- 3 agents making 3 different edits and building concurrently: 2.7 s wall time, each artifact containing only its own change, the base untouched.
Path remapping without namespaces
Experimental. remap-paths recovers most of rustc’s incremental reuse
for agents that build without the namespace sandbox: on macOS, and on Linux
where user namespaces are unavailable. It is off by default.
The problem it solves
Without the namespace sandbox, each agent builds at its own path. rustc’s incremental cache is tied to absolute paths in two ways:
- the working directory is a tracked option: if it differs, rustc discards the whole cache;
- the absolute
--out-dirthat Cargo passes is hashed into every codegen unit that has debug info, so even a loaded cache is invalid.
So the first edit of a crate in a new agent recompiles it from scratch. Cargo itself is fine with the move (it hashes workspace paths relatively); Ferry still rebuilds the crates that captured the base’s absolute paths once after spawn, because Cargo never checks those again (see How it works).
What it does
With remap-paths, the base and every agent build through ferry-rustc, a
RUSTC_WORKSPACE_WRAPPER that is a link to the ferry binary in
<store parent>/bin. Cargo uses a workspace wrapper only for workspace
members. For each member compile that uses incremental compilation, the
wrapper:
- adds
-Zremap-cwd-prefix=., so the absolute working directory never enters rustc’s tracked options; - sets
RUSTC_BOOTSTRAP=1in that rustc process only, because-Zflags are otherwise nightly-only; - rewrites absolute paths under the worktree and the target dir in rustc’s
arguments (
--out-dir,-C incremental=,-L,--extern, …) into relative ones.--remap-path-prefixrules are left alone.
Builds reach the target dir through a .ferry-target link inside the source
root, so the relative path from sources to target is the same in the base and
every agent. Ferry hides the link from git status through the repository’s
.git/info/exclude. Every other rustc call, including Cargo’s version probes,
passes through unchanged.
Enabling it
[sandbox]
remap-paths = true
or FERRY_REMAP_PATHS=1, then build a new base with ferry base and spawn
agents from it. Notes:
- It needs rustc 1.98 or newer (the release with
rust-lang/rust#157348; before
it,
-Zremap-cwd-prefixstill stored the absolute working directory). With an older toolchain,ferry basewarnsremap-paths disabled: ...and builds without it. - It only applies to bases built without the namespace sandbox. On Linux with
working user namespaces, the sandbox is used and this setting has no effect;
set
sandbox = "off"as well to try remapping there. - Whether a base uses remapping is part of its identity, and its agents always
match it: a mismatch in the wrapper or in
RUSTC_BOOTSTRAPmakes rustc discard the cache.
Measured results
Ferry’s test tests/remap.rs builds a workspace with a library and a binary,
copies the source tree and the target dir to new paths, deletes the
originals, edits both crates and rebuilds. It counts codegen units whose
object files rustc reused. On Linux with rustc 1.99.0:
| Crate | With ferry-rustc | Without |
|---|---|---|
| library | 12 of 13 codegen units reused | 0 of 13 |
| binary, default debug info | 0 of 19 | 0 of 19 |
binary, debug = 0 | 17 of 19 | 0 of 19 |
Binaries with debug info get nothing on Linux because every codegen unit of a binary embeds a gdb scripts section, and computing it reads the absolute paths of all its dependencies. Apple targets do not emit that section, so on macOS binaries are predicted to be reused like libraries. That prediction comes from rustc’s source and an experiment with custom Linux target specs; it has not been verified on a real Mac.
A second test runs the same edit through Ferry itself: ferry base and
ferry spawn with the sandbox off, the edit, then the agent’s first build.
On Linux with rustc 1.99.0 it reuses the same 12 of 13 library units (0 of 13
without remapping). On macOS both tests also assert the binary prediction
above, and a further test checks that debug info and backtraces survive
remapping. They run on every macOS CI leg whose rustc can remap (1.98 or
newer), including the required test (macos-15, stable) check, so a wrong
prediction fails that check before a merge, not only the remap-paths leg
on macOS 26 that runs the whole suite in remap mode. Their results will
replace the prediction here once CI has run them.
Limitations
RUSTC_BOOTSTRAP=1is set for workspace-member compiles. It lets those compiles use unstable flags on a stable toolchain, which some policies forbid, and code that detects nightly through it may behave differently.- Stable wrapper path. Cargo hashes the workspace wrapper’s path into member artifact names. Builds with and without the wrapper do not share artifacts, and moving the store parent rebuilds every workspace member.
- Clippy.
cargo clippysetsRUSTC_WORKSPACE_WRAPPERtoclippy-driver, replacingferry-rustc, in the base and in agents alike. Clippy runs are not remapped: workspace members re-lint after the move without incremental reuse. Dependencies are unaffected, since both wrappers apply only to workspace members.ferry doctornotes this. profile.trim-pathsadds--remap-path-prefixrules with absolute paths, which rustc tracks. Ferry does not rewrite them, so trim-paths and remap-paths together are expected to lose the incremental reuse.- Not tested:
cargo docand rustdoc,--timings, build scripts that parse rustc’s arguments, proc macros that read paths, coverage tools. - Relative paths in output. rustc’s artifact notifications carry the
rewritten relative paths.
cargo build,cargo testandcargo runwork with them. - Debuggers. Debug info records
.as the compilation directory, so rungdborlldbfrom the worktree root, or map the path in the debugger. - Paths captured by code such as
env!("CARGO_MANIFEST_DIR")keep the real absolute path. As in the normal no-namespace mode, every local crate rebuilds once after spawn; with remapping those rebuilds reuse the base’s incremental state. .ferry-target. Ferry refuses to build if the source root contains a real file or directory with that name.- The approach relies on rustc hashing every path-dependent input into its incremental state, the same assumption remapped CI caches make.
Architecture
This chapter is the contract the code is written against. If code and this document disagree, one of them has a bug.
The problem in one paragraph
Every parallel agent needs its own git worktree. A fresh worktree builds from
cold (minutes on a real workspace). Pointing every worktree at one shared
CARGO_TARGET_DIR serializes builds on Cargo’s lock and, worse, produces
silently wrong builds: Cargo’s freshness check compares source mtimes against
its fingerprint files, so agent B’s older edit looks “fresh” next to agent A’s
newer artifact and B ships A’s code with zero crates rebuilt. Ferry forks one
immutable warm build per agent instead, and makes the fork look fresh to Cargo
only where that is provably true.
Invariants
Each one is enforced in code (module named) and covered by tests.
- Never stamp a file whose content differs from the base (
freshness). A tracked file’s mtime is set to the fixed stamp time only if it is a regular file at merge stage 0, its mode and blob id equal the base commit’s tree entry, its working-tree bytes equal the base checkout’s bytes, its metadata did not change between verification and stamping, and Cargo cannot reach it through a symlink that resolves differently in the agent and the base (both ends count: Cargo records a rustc input read through a link by its canonical path when that lies inside the input’s package or the target dir, but follows a build script’srerun-if-changedpath through it). Checksum freshness compares content instead, so the units that read through such a link (one with an end in the checkout) also get their dep-info invalidated (see Freshness modes). A directory is stamped only if its entry names equal the base checkout’s directory, because Cargo’srerun-if-changed=<dir>takes the recursive max mtime of the directory, its subdirectories and its files, and a deletion is only visible as a directory mtime change.StampPlancan only be built by the verifying function. The one other stamping function,stamp_base_checkout, stamps a fresh, unseeded base checkout, which is the reference itself; onlybase::preparecalls it. Every file and directory that is not stamped gets an mtime no older than the base (freshen_skipped), so a hook that restores commit times or a clock that went backwards cannot make it look fresh. Stamping happens once, at spawn; a live agent tree is never re-stamped. - Bases are immutable once built (
base). A base is keyed by commit plus a fingerprint of toolchain, warm commands and modes.base.jsonis written last, atomically, and doubles as the “ready” marker; nothing writes to a ready base. Refreshing builds a new base;gcremoves a base only when it is not current and no agent references it. A ready base’s target holds only units its own warm build compiled or found fresh (prune), so every unit an agent or a later refresh reuses was checked against the content stamping compares. There are three ways to get there: a cold build; a refresh seeded from a base with the same unit inputs (every manifest, lockfile, Cargo config and toolchain file of the commit, the Cargo configs above the store, Cargo’s flag variables,CARGO_INCREMENTAL, whetherCIorTF_BUILDis set, and a version of this list, so that no base recorded under an older list matches), which selects the same units; and a refresh seeded across unit inputs from a base whose warm build was observed, after which every unit the build neither wrote nor reported is deleted. When that observation is incomplete, the target is emptied and built again from scratch rather than published. - Each agent owns its build state (
fork,sandbox,agent). Forks are reflink clones (new inodes), overlayfs upper layers (base is a read-only lower), or full copies. There is no hardlink backend: Cargo rewrites fingerprint files withFile::create(truncating the existing inode), so a hardlinked fork would let an agent’s rebuild modify the base. Cargo always getsCARGO_TARGET_DIRandCARGO_BUILD_BUILD_DIRpointing at the fork, so a user’sbuild.build-dir(whose{workspace-path-hash}is identical for every agent under the canonical mount) cannot redirect intermediates into a shared directory. Each Cargo workspace gets its own target dir: Cargo hashes path packages by their path relative to the workspace root, so two workspaces sharing one would overwrite each other’s units (see the shim). - The agent is derived from the working directory, never from inherited
environment (
agent,shim). Agent tools copy their session environment into sub-agents that work in other worktrees; an inheritedCARGO_TARGET_DIRor agent name would point a sub-agent at someone else’s build state. Ferry never exportsCARGO_TARGET_DIRto anything but the cargo process it starts. - Refuse unsafe states with a clear error (everywhere): no base, base
checkout or target missing (stamping cannot be verified), agent not fully
created, overlay agent without a namespace, overlay upper dir still mounted
by an unjoinable namespace, store paths overlayfs cannot express, an agent
worktree inside the store or overlapping another agent’s, and on
rma dirty worktree, commits no ref reaches, running processes or an agent nested inside.
Module layout
src/
main.rs multicall: `cargo` (shim), `ferry-rustc` (remap), or `ferry` (cli)
lib.rs module list; compile_error! on non-unix
cli/ clap definitions, command handlers, human/JSON output (anyhow)
config.rs ferry.toml schema, platform overrides, FERRY_* env, Settings
context.rs Repo + Settings + Store for one invocation
git.rs shells out to git (worktrees, ls-tree, ls-files, status)
store.rs on-disk layout, ids, repo key
lock.rs flock(2) wrapper
fsutil.rs atomic JSON writes, disk usage, forced removal
toolchain.rs rustc/cargo version detection per worktree
freshness.rs stamping plan + application, freshness mode; relocation.rs:
what a fork built at other paths must rebuild
base.rs build / list / gc of immutable bases
agent.rs spawn / rm / list / env, session and cargo exec, reaping
shim.rs the cargo shim (PATH entry → per-call agent resolution)
hook.rs agent-tool hooks (Claude Code WorktreeCreate/Remove, SessionStart)
remap.rs experimental path-independent incremental builds without namespaces
fork/ Backend trait + selection; reflink (Linux FICLONE), clonefile
(macOS), overlay (Linux), copy; shared tree walker
sandbox/ Plan, enter(), users(); linux.rs: user+mount namespaces,
canonical bind mounts, overlay mount, join-or-create
platform/ Capabilities probes (fs type, reflink, userns, overlay, sysctls)
doctor.rs checks + fixes + "what spawn would do"; the AppArmor profile text
setup.rs `setup apparmor`: facts, a pure decision, the one sudo script
Git is driven through the git binary rather than libgit2 or gix: worktree
creation must honour the user’s hooks, filters, LFS, core.autocrlf and sparse
checkout exactly as git does, because any divergence changes file contents
relative to what the base was built from.
On-disk layout
<store parent>/ $FERRY_HOME, [store] dir, or the platform cache dir
bin/cargo -> /path/to/ferry the shim
bin/ferry-rustc -> ... experimental remap wrapper
<repo-name>-<hash>/ one store per repository (hash of git common dir)
store.json lock current
watch.lock held by `ferry base --watch`
mnt/src mnt/target canonical mountpoints (empty on the host)
bases/<commit>-<fp>/ immutable once base.json exists
base.json src/ target/
relocation.json built without the sandbox: what forks remove from target/
warm-units/ the shim's records of warm cargo calls, while building
bases/<commit>-<fp>.lock
agents/<name>/
agent.json lock
busy.lock held shared by every process run for the agent
target/ reflink / copy backends
ferry-ws/<path>/ target dirs of workspaces nested at <path>
upper/ work/ overlay backend
mount.lock ns.json overlay: in-use lock and live namespace identity
<worktree parent>/ FERRY_WORKTREES, [agent] worktree-dir, $FERRY_HOME/worktrees,
<repo-name>-<hash>/<name>/ or the platform data dir; --path overrides per agent
Build state lives in the cache dir; agent worktrees, which hold uncommitted
work, live in the data dir (~/.local/share/ferry, ~/Library/Application Support/ferry) because macOS may purge caches and users clear them casually.
All build state Ferry writes is under one directory, so an agent tool’s sandbox
needs one writable-root entry for it plus the worktree (its working directory).
Lifecycle
ferry base [--rev R]
- Resolve
R, detect the toolchain, resolve freshness mode and isolation (probe user namespaces in a child process whensandbox = auto). - Compute the base id; take
bases/<id>.lock. Ifbase.jsonexists, make it current and stop. Otherwise discard any partial directory. git worktree add --detachthe checkout andgit worktree lockit. Unless--coldis given, seed from the newest ready base with the same fingerprint at another commit that has the same unit inputs or (with[base] prune, the default) was observed: clone its target (reflink when possible), relocate the clone without the sandbox (see below), stamp only files proven identical to its checkout, and invalidate the units that read through a symlink retargeted within, into or out of the checkout, exactly as for an agent, so the warm build only rebuilds what changed. Otherwise stamp all tracked files and directories (the base is the reference, nothing to compare). A seed with other unit inputs loses its nested workspaces’ target dirs (ferry-ws/), which are not observed.- Create
warm-units/and take a write threshold: the mtime of a file created there that is strictly newer than one created just before, so every file written later is at least as new and every cloned file older. - Enter the sandbox (irreversible for this process; host paths stay
reachable), run each warm command with
sh -c, stdout redirected to stderr, the shim first onPATHso each workspace gets the target dir it gets in agents, andCARGO_BUILD_WARNINGS=warn. A failing command discards the base, except a generated clippy command, which only warns and is recorded inincomplete. Clippy commands are left out (before the base id is computed) whencargo clippy --versionfails. The shim records each build-type cargo call (build,check,test,run,clippy,doc, …) in a file of its own underwarm-units/: it adds--message-format=json-render-diagnosticsunless the call asks for JSON itself, records everycompiler-artifactandbuild-script-executedmessage with the inode, length and mtime of each named file as it arrives, and passes all other output through (and the messages too, when the call asked for them). A call asking forhumanorshortmessages, or a subcommand it does not know (nextest, aliases,install), is recorded as unobservable. - Work out which units the build used. Every profile dir (a directory with
a
.cargo-lock) is scanned in both build-dir layouts; a unit is used if Cargo wrote its fingerprint after the threshold, or if a message names it: by a path inside its own directory or carrying its hash, or, for files uplifted without a hash (binaries, examples, root proc macros and dylibs), as the only unit of that package with the fingerprint file Cargo names after the target, else the one whose output has the recorded inode (Linux hard links) or length and mtime (macOS copies). The observation is incomplete if a call was unobservable or did not finish, a message matches no unit or several, a rebuilt unit’s fingerprint predates the threshold, a written unit was reported by no message (a cargo call that bypassed the shim) although every call succeeded, a directory in the target cannot be read, or a profile dir has an unknown layout. After a call that failed (a clippy command under--keep-going, say), a written unit no message reported is one that failed to build (Cargo writes its fingerprint dir, with an empty hash, and reports nothing), and it counts as unused. - With other unit inputs than the seed: if the observation is complete,
delete every unused unit (fingerprint, build dir, hash-named outputs, and
hash-named outputs without a fingerprint), leaving uplifted files,
doc/andincremental/; otherwise warn, empty the target, and run the warm commands again into it without a seed. Recordobservedandpruned. - Record files the build wrote into the source tree (build scripts writing
outside
OUT_DIR) as warnings; without the sandbox, compute the relocation plan and writerelocation.json(see below); writebase.json; setcurrentbefore releasing the base lock.
ferry base --watch [--rev R] [--interval S] [--gc]
A foreground poller. It refuses an R that names no commit, takes
<store>/watch.lock (a second watcher is refused, naming the holder), then
every S seconds (default 2) resolves R with one git rev-parse,
skipping polls while the watcher’s worktree is in a rebase, merge,
cherry-pick, revert or bisect. A commit seen on two polls
in a row that is neither the current base’s commit at startup nor the last
one built or failed is built by a child ferry --json base --rev <commit>
(building enters the sandbox for the rest of the process, and the child
re-reads ferry.toml); its stderr is the watcher’s. A refresh is never
interrupted when R moves: the newest commit is built after it. A failed
build leaves the current base alone and is not retried until R moves.
--gc runs gc in the watcher after each new base.
ferry spawn NAME [--from REF] [--path DIR] [--branch B]
- Under the store lock: check name, branch and path are free; choose the base
(current if it is at
REF, else any ready base atREF, else current); decide isolation; writeagent.jsonwithstate: creating; take the agent lock. git worktree add -b <prefix><name>, copy the base’sCargo.lockif the repository does not track one, plan stamping against the base (invariant 1), stamp, raise the mtimes of what was not stamped,git update-index --refresh.- Select the backend, fork the base target, and when the agent runs without
the namespace sandbox, apply the base’s relocation plan (see below). If a
tracked symlink was retargeted within, into or out of the checkout,
invalidate the units that read through it (see
Freshness modes). Write
state: ready. A failure rolls back only what this spawn created: the worktree if it added it, the branch if it still points at the starting commit, and the agent dir.
ferry run [NAME] -- CMD and the shim
ferry run (and ferry shell) enter the agent’s sandbox and exec CMD with
the working directory left at its host path, PATH prefixed with the shim
directory, and FERRY_AGENT=<name> set for scripts. Host paths keep working
inside the sandbox; agent tools that key state on their cwd (transcripts,
memory) keep one entry per agent.
The shim (bin/cargo -> ferry) is where builds are redirected. On every call
it resolves the agent from the working directory’s git worktree: its root
must be exactly an agent’s worktree (a host path, or mnt/src whose mounted
inode is one), so a separate worktree nested inside an agent’s is not that
agent. For an agent it enters (or confirms it is already in) that agent’s
sandbox, moves to the canonical path, and finds the workspace the call
selects (cargo locate-project --workspace, honouring --manifest-path;
skipped for a call at the worktree root without one):
| Workspace | Target dir |
|---|---|
| rooted at the worktree root | the fork |
rooted at <path> inside the worktree | <fork>/ferry-ws/<path> |
outside the worktree (--manifest-path into another agent’s) | its own, as plain cargo; inherited target variables removed |
It sets CARGO_TARGET_DIR, CARGO_BUILD_BUILD_DIR and the freshness env
accordingly and execs the real cargo (the next cargo on PATH that is not
ferry). Outside agent worktrees it execs the real cargo unchanged, except
inside a base’s warm commands, where FERRY_WARM_SRC and FERRY_WARM_TARGET
apply the same table to the base checkout, FERRY_WARM_CARGO holds the
base toolchain’s cargo -V, and FERRY_WARM_UNITS names the directory the
shim records the call in (see ferry base, step 5).
With CI or TF_BUILD set, Cargo 1.99 and later turn incremental
compilation off after merging profiles, so ahead of [profile.*] incremental = true, and the switch enters every unit’s hash: an agent with
CI would rebuild every workspace member of a base warmed without it. So
wherever the shim sets a target dir, it also sets CARGO_INCREMENTAL=1 when
either variable is set and the base’s toolchain is Cargo 1.99 or later,
unless the call has one of the settings Cargo ranks above the switch:
CARGO_INCREMENTAL, CARGO_BUILD_INCREMENTAL, or build.incremental in a
--config value or in a .cargo/config{,.toml} in the working directory’s
ancestors or Cargo home (includes are not followed). CI and TF_BUILD
are passed on unchanged. Processes cargo starts inherit
CARGO_INCREMENTAL=1, so a cargo call they make on a workspace outside the
checkout builds incrementally too. Release and bench units still depend on
CI: by default they are incremental with it and not without. So whether
CI or TF_BUILD is set is a unit input (invariant 2). The costs are
listed under
Cargo configuration pitfalls.
ferry rm NAME [--force]
Refuses if processes still run for the agent (they hold busy.lock), if
another agent’s worktree is nested inside its worktree (even with
--force), or, without --force, if the worktree has uncommitted changes or
commits that no branch, tag or remote reaches. With --force such commits
are kept under refs/ferry/orphaned/<name>-<time>. Removes the worktree and
fork; deletes the branch only if it still points at the agent’s starting
commit and is not checked out elsewhere, otherwise keeps it and reports how
many commits it carries.
ferry gc [--dry-run]
Reaps abandoned hook-created agents (clean, no new commits, no commits only
their worktree reaches, no ignored files beyond build output, idle, not
git worktree locked by their tool), then deletes bases that are neither
current nor referenced, skipping any base whose lock is held. It never runs
git worktree prune, which would also drop the user’s own worktrees whose
directories are temporarily missing; Ferry unregisters only the worktrees it
removes.
Fork backends
#![allow(unused)]
fn main() {
pub trait Backend {
fn fork(&self, paths: &ForkPaths) -> Result<ForkStats, ForkError>;
fn target_spec(&self, paths: &ForkPaths) -> TargetSpec;
fn private_bytes(&self, paths: &ForkPaths) -> Result<u64, ForkError>;
}
}
| Backend | Platforms | Fork cost | Notes |
|---|---|---|---|
reflink | Linux (btrfs, XFS, OpenZFS 2.2+), macOS (APFS) | per-file FICLONE (Linux), one clonefile(2) of the tree (macOS) | target is a real dir; mtimes preserved |
overlay | Linux 5.11+, namespace sandbox required | O(1) | copy-up copies a file on its first write; on btrfs/XFS the copy-up is itself a reflink. The mount exists only inside the sandbox |
copy | everywhere | full copy | fallback |
auto picks reflink when the store filesystem supports copy-on-write clones,
else overlay when the agent runs in a namespace sandbox and overlayfs works
there, else copy. So auto picks overlay only on filesystems without clones
(ext4, tmpfs). Reflink comes first because the overlay costs every build
outside Cargo: the mount protocol below on entry, and on the per-call path
(shim, hooks) a syncfs of the whole upper filesystem when the last process
using the mount exits. An agent keeps the backend recorded at spawn. An
explicit backend that is unavailable is an error, never a silent fallback.
Sandbox (Linux)
sandbox::enter(plan) runs in the calling process, which must be
single-threaded (unshare(CLONE_NEWUSER) fails with EINVAL otherwise):
unshare(CLONE_NEWUSER | CLONE_NEWNS), writesetgroups=deny, map the caller’s uid and gid to themselves (not root: some agent tools refuse to run as root, and the util-linuxmountCLI misbehaves under that mapping, so Ferry callsmount(2)directly).- Make
/recursively private. - Bind-mount the worktree onto
mnt/src; bind-mount the target dir, or mount overlayfs (lowerdir= base target,upperdir,workdir, alwaysuserxattr), ontomnt/target.
Two overlay mounts must never use the same upper dir at once: the kernel only
warns, and the two views then diverge. Every process using an agent’s overlay
therefore holds a shared flock on agents/<name>/mount.lock through an
inherited file descriptor. Under the agent lock, enter tries an exclusive
lock on it. A free lock is not proof that nothing uses the overlay: a process
in a nested namespace (unshare, bwrap) may have closed its descriptor and
still have the overlay mounted. So before mounting a new one, enter also
scans every readable /proc/<pid>/mountinfo for an overlay on the agent’s
upper or work dir (one read per mount namespace and root, about 15 ms for 480
processes). Only when both checks come up empty does it mount and record the
namespace identity (and the upper and work dirs) in ns.json; otherwise it
setnses into a live process of the recorded namespace. An exiting session
holds the lock while the kernel unmounts its overlay, so enter re-checks
with backoff for up to 10 s before refusing, naming the processes that still
have the overlay mounted or hold the lock. ferry list, rm and gc count
those processes as users too.
Bind-mount-only sandboxes (reflink, copy) are safe to create per call.
Without namespaces (macOS; Linux with sandbox = off or where user
namespaces are blocked) builds run at host paths, and the first edit pays for
rustc’s incremental state being keyed on absolute paths. Cargo’s own unit
hashes survive the move (it hashes path packages relative to the workspace
root), but a unit can capture the base’s absolute paths where Cargo never
looks again: a build script writing its manifest dir into generated code or
a rustc-env value, a proc macro reading it from the environment (the
rust-embed pattern), a test that bakes in env!("CARGO_BIN_EXE_app"). Such
a unit would stay “fresh” while pointing at the base. So the fork gets a
relocation plan (freshness::plan_relocation): it keeps every unit unless
something recorded in the target shows an old path, and removes the rest.
| Rule | Reads | Flags a unit when |
|---|---|---|
| R1 | its dep-* and fingerprint JSON | they name any old path (a workspace file outside the package on Cargo <= 1.100, an absolute rerun-if-changed, the base checkout’s git dir) |
| R2 | rustc’s dep-info | an env! value or an absolute input names a checkout path (a vendored crate, or a path package outside the workspace root, rebuilds anyway, because Cargo hashes its absolute source path) |
| R3 | env! values in rustc’s dep-info | one names a target path that is not exempt (CARGO_BIN_EXE_*, CARGO_TARGET_TMPDIR, a profile dir) |
| R4 | a build script’s directives | they name an old path other than its own OUT_DIR spelled as in root-output, the string Cargo rewrites when it replays them; or a link argument (an rpath) names a dead OUT_DIR, which the linker already baked into the package’s binaries |
| R5 | every file and symlink in an OUT_DIR | one names an old path that is not exempt, or cannot be read |
| R6 | generated files outside OUT_DIRs that a compile included | one names an old path that is not exempt |
| R7 | proc-macro libraries, local macros’ sources | it can invoke a proc macro that may read a path from the environment |
Old paths are found with one needle per root, the longest suffix of the store
path and of the base checkout’s git dir made only of bytes that no escaping
rewrites ([A-Za-z0-9._/-]), so every spelling (Debug, JSON, Make, percent
encoded) contains it; what follows the needle tells a checkout from a target
dir, and <base src>/.ferry-target (remap mode) counts as the target.
Whether a target path may stay depends on whether that target is live. A
base built without the namespace is immutable and gc keeps it while an agent
references it, so a unit may keep a path into the OUT_DIR of a build-script
run it depends on: the base’s bytes there equal the fork’s until that run
reruns, and then the unit rebuilds in the same build. Every other target path
is dead: a namespace base’s mnt/target (an empty mountpoint on the host),
and the target of a seed, which gc may collect. A dead path is exempt only in
a script’s own directives other than link arguments, where Cargo rewrites the
exact root-output string, and where a registry crate includes from its own
OUT_DIR at compile time (serde’s include!(concat!(env!("OUT_DIR"), ...))).
A removed compile unit loses its fingerprint dir, so its dependents go stale
in the same build. Every build-script run that depends on a removed unit (a
build-dependency, a links upstream) is removed whole, run dir and OUT_DIR
included, so it reruns from an empty OUT_DIR instead of over the base’s
state, and Cargo rebuilds what depends on it in the same build; removing only
a run’s fingerprint would leave its dependents one build behind. Nested
workspaces’ target dirs (ferry-ws/) are planned like the rest and kept.
What cannot be read or parsed is removed, so a new Cargo fingerprint format
costs a cold build, never a wrong one; a store path without a plain suffix of
8 bytes empties the fork.
ferry base computes the plan of a base built without the namespace before
writing base.json and stores it as relocation.json, so a spawn only
deletes what it lists (it recomputes the plan after a Ferry upgrade). Files
are scanned a megabyte at a time, so a library of gigabytes in an OUT_DIR
costs time, not memory. For a namespace base, spawn computes the plan on the
base’s target, with every target path dead. A seeded refresh without the
namespace applies a dead plan to its clone of the seed’s target before the
warm build, so the new base never depends on the seed’s.
One rule is a heuristic. R7 assumes that a registry proc macro reading a path
from the environment names the variable literally (CARGO_MANIFEST_DIR,
OUT_DIR, PWD and the other path variables), which holds for every
path-embedding crate surveyed (rust-embed, rstest, manifest-dir-macros).
Crates on a small allowlist whose hits a human checked (document-features
0.2.12, xflags-macros 0.3.2) are exempt, by exact version. A local proc
macro also counts when its sources use the working directory, a subprocess or
dynamic variable names. What the plan cannot see:
- paths stored encoded: compressed, base64, hashed, UTF-16, or JSON with
\/; - registry proc macros that derive a path from the working directory, a
subprocess (
cargo metadata,git rev-parse) or a variable name built at run time, without naming a listed variable; - behaviour that depends on location without recording a path: existence
checks, or reads through
../..out of the worktree; - paths that a program derives at run time from an exempt live
OUT_DIRwith.., such asPath::new(env!("OUT_DIR")).join("../../../helper"), which leads to the base’s binary instead of the agent’s; - path variables outside the list read by proc macros, such as
[env]entries withrelative = true; - a registry crate that includes from a dead
OUT_DIRand also reads it at run time; - writes outside
OUT_DIRthat no compile includes (target/cxxbridgelinks,$HOME,/tmp); the source tree is reported asdirtied; - relative paths between the source and target trees, and absolute paths outside the store and git dir that an agent would compute differently;
- debuginfo, panic locations, cached diagnostics and doc output (cosmetic);
- run-time writes through a live
OUT_DIRpath, which modify the base; - literals that a proc macro built with optimizations only compares against.
Staleness that the namespace sandbox has too (a git hash baked in without
rerun-if-changed, a macro reading a file Cargo does not track) is not
relocation’s to fix.
The experimental remap mode ([sandbox] remap-paths, rustc 1.98+) recovers
incremental reuse without namespaces: builds reach the target dir through a
.ferry-target link inside the source root, and the ferry-rustc
RUSTC_WORKSPACE_WRAPPER adds -Zremap-cwd-prefix=. (with RUSTC_BOOTSTRAP=1
for workspace-member compiles only, and on non-nightly toolchains
-Zallow-features= so that #![feature] stays an error) and rewrites
absolute worktree and target paths in rustc’s arguments to relative ones.
Freshness modes
| Mode | When | What Cargo compares |
|---|---|---|
mtime | default on stable | source mtime vs fingerprint (strictly newer = dirty) |
checksum | auto on nightly toolchains, or explicit | rustc dep-info inputs by size + blake3; build-script inputs, outputs-vs-deps and rustdoc stay mtime-based |
checksum needs CARGO_UNSTABLE_CHECKSUM_FRESHNESS=true and (nightly
2026-09-02 and later) CARGO_BUILD_FINGERPRINT=content; Ferry sets both for the
base and every agent. Switching mode rebuilds every unit, so the mode is a
property of the base. Stamping runs in both modes because build scripts stay
mtime-based.
Leaving a file unstamped cannot show checksum freshness a symlink that leads
elsewhere than in the base. When the canonical path of a rustc input read
through a link lies inside the input’s package or the target dir, Cargo
records the input by that path (otherwise by the path through the link) and
compares the content of the base’s file there, which the agent has too. So
when a tracked symlink leads to another place in the checkout than in the
base, or into or out of it, spawn (and a seeded refresh, against its seed)
also empties the Cargo dep-info file (dep-* in the fingerprint dir, written
to an overlay fork’s upper dir) of these units (freshness::link_rebuilds):
- each compile unit whose rustc dep-info names an input in the checkout that
resolves differently in the two checkouts, unless it resolves into the
store in both (remap mode’s
.ferry-targetlink, below which Cargo tracks paths relative to the target dir); - each compile unit without readable rustc dep-info, except rustdoc’s, which compares mtimes.
Build-script runs compare mtimes too, so leaving their inputs unstamped is enough. Cargo cannot parse an emptied file, so it rebuilds the unit and its dependents in the same build.
Concurrency
Locks are flock(2) on files in the store. Exclusive locks are never held
while an agent’s build runs; the base lock is held for its own warm build:
| Lock | Held by | Duration |
|---|---|---|
<store>/lock | spawn (registry step), rm, gc, setting current | milliseconds to seconds |
bases/<id>.lock | ferry base for that id (exclusive), a refresh seeded from it (shared); gc and spawn only try-lock | the base build |
<store>/watch.lock | ferry base --watch (try-locked) | the life of the watcher |
agents/<name>/lock | spawn (whole), rm, sandbox entry for runs | spawn; ~1 ms per run |
agents/<name>/busy.lock | shared by every ferry run, shell and shim call through an inherited fd; rm, list and gc try it exclusively | the life of the process and its children |
agents/<name>/mount.lock | shared by every process using an overlay mount, as above | the life of the mount’s users |
Lock order is store, then agent, then base. ferry base also takes the
store lock while holding a base lock (to set current before gc can see the
new base unreferenced); that is safe because gc only ever try-locks bases.
Errors and output
Library code returns typed errors (thiserror); the CLI wraps them with
anyhow and prints error:, caused by: and a hint: from
Error::hint(). Data goes to stdout, progress to stderr, colour only on a
terminal. Every read command and every mutating command accepts --json,
including doctor --apparmor-profile.