Benchmarks

Ferry’s benchmark harness is in bench/ in the repository. Results are published here as they are measured. This page never shows estimated or extrapolated numbers: every figure comes from a recorded run.

What is measured

Every result records the hardware (CPU model and count, memory), the OS and kernel, the filesystem of the store, the Rust toolchain, the workload and its commit, and the versions of the other tools compared. Numbers from one machine are not projected onto another, and results from small workloads are not scaled up to large ones.

Results

ripgrep, Ferry at d7fb5cb

First edit in a new agent (spawn, edit, build). Every artifact check passed: the edited crate’s rlib held exactly that agent’s own edit.

Other steps from the same run (medians):

What these numbers show:

• The gain is in a new agent’s first edits. Later edits take about the same time with kache and in every Ferry scenario.
• The overlay backend is slower than reflink on later edits (1.07 s against 0.88 s). Every timed build here is its own ferry run, and an overlay build waits for its sandbox’s teardown, about 0.2 s per build on ripgrep (see How it works). That matches the gap.
• Sandbox off loses the incremental state. The agent reuses every unchanged crate, but ignore and rg recompile from scratch. kache compiles the same two (the table’s 10 counts the 8 it restores from its store, which Cargo reports as compiled), and is faster here. Remap-paths keeps most of the library’s incremental state at the agent’s own path; on Linux the rg binary, built with debug info, still recompiles from scratch.
• These are one workspace’s results. This page does not scale them to larger workspaces; those are measured below. macOS results are not published yet.

All steps, the disk usage, and the raw run times are in bench/results/ripgrep/ (results.md, results.json, env.json, and hyperfine’s exports).

Larger workspaces, Ferry 0.1.2

Measured on the same machine as ripgrep (AMD Ryzen 5 3600X, 6 cores and 12 threads, 31 GiB RAM, btrfs, NixOS 26.11, Linux 6.18) on 2026-10-27, with the release binary of Ferry 0.1.2 (5be5ee1). The commits in each env.json differ from it only in docs and benchmark files. Each value is a median: 3 runs per step for rust-analyzer (2 for cold builds), 2 runs (1 for cold builds) for the others. The machine was a desktop in use; each env.json records the load. Crates compiled are in brackets.

Every scenario runs the same command. First build is an agent’s first build with no edit; first edit is a new agent’s first build after one edit; next edit is a later edit in the same agent. Ferry’s rows use the namespace sandbox. Reflink is what auto picks on this btrfs; overlay is what it picks on ext4.

rust-analyzer

0c4d746, 281 build units cold, edit crates/ide-db/src/lib.rs (rebuilds 9 units), cargo build, rustc 1.99.0.

Refreshing the base after a commit: 9.06 s seeded from the previous base, 92.5 s cold.

uv

f8bec5e, 624 build units cold, edit crates/uv-resolver/src/lib.rs, cargo build, rustc 1.99.0.

Refreshing the base after a commit: 10.0 s seeded from the previous base, 79.9 s cold.

ruff

8aeb8e2, 505 build units cold, edit crates/ruff_linter/src/lib.rs, cargo build, rustc 1.99.0. A Ferry agent’s first build recompiles 6 units here, where the other workspaces recompile none; the cause is not investigated yet.

Refreshing the base after a commit: 13.5 s seeded from the previous base, 162.9 s cold.

polars

d0a98d9, edit crates/polars-lazy/src/lib.rs, cargo build -p polars-sql (212 build units cold), on the nightly polars pins (rustc 1.100.0-nightly 2026-08-31), so Ferry uses checksum freshness.

Refreshing the base after a commit: 2.89 s seeded from the previous base, 70.7 s cold.

Raw results, including min and max of every step and disk usage: bench/results/.

CI runs (macOS)

macOS has no namespace sandbox, so its numbers come from a reduced benchmark that the Bench workflow runs on a GitHub-hosted macOS runner when started by hand (see bench/README.md, “CI runs”). These runners are shared virtual machines. Their results are published in their own table, never next to the results above, and each table states the runner label, image version, run link and date; the Ferry commit, rustc, Xcode and linker versions; the runs per step with the minimum and maximum of each step; and the load before and after. Every such table carries this note, with the hardware taken from that run’s env.json (logical_cpus, cpu_model and ram_bytes), because it depends on the runner label:

Measured on a shared GitHub-hosted VM (logical CPUs vCPU CPU model, RAM GiB). Not comparable with the local results above or with other CI runs; ratios within one run are indicative, absolute seconds are not.

Claims about how much of rustc’s incremental state survives on macOS come from the test suite’s codegen-unit counts, which do not depend on timing noise, and not from these times.

No CI results have been published yet.

Prototype results

Before Ferry was written, a bash prototype tested the approach. These are its measured results: prototype (bash) on ripgrep, 2 vCPU. Setup: ripgrep at commit 6892bc2 (39 crates, debug cargo build), 2 vCPU, 7 GB RAM, ext4, rustc 1.97.0. The edit adds a documented function to crates/ignore, which forces ignore and ripgrep to rebuild. It is a small workspace on a small machine, so the ratios and crate counts say more than the absolute times.

The prototype also measured:

• spawning with no edit: 0.10 s and 0 crates for the fork, 0.45 s for kache (which restores 34 crates);
• from the second edit on: about 1.1 s for the fork and 1.0 s for kache, which converge once the agent has its own incremental state;
• 3 agents making 3 different edits and building concurrently: 2.7 s wall time, each artifact containing only its own change, the base untouched.