Other benchmarks · measured on the reference machine
Everything else
an agent waits on.
A coding agent spends time on more than tokens. It also waits for searches, builds and tests to run. This page holds benchmarks of those tools, measured on the DGX Spark that serves the LLM rankings. Each entry links to its raw numbers and a script you can rerun.
Ranking coding agents instead? LLM setups — harness, model, thinking →
Code search · 2026-10-12 · DGX Spark
grep vs ripgrep vs ugrep.
Agents search code all the time, and harnesses use different tools for it. In Claude Code 2.1.296 on this machine, the shell's grep is a function that runs the ugrep built into the Claude Code binary, not GNU grep. ripgrep is the de facto fast code-search tool; VS Code's search runs on it. This benchmark measures all of them on the corpus ripgrep's own benchmarks use: the Linux kernel source.
The standard Unix grep from the Ubuntu package. Recursive search with -r runs on one thread, and case-insensitive search is slow under a UTF-8 locale.
Andrew Gallant's line-oriented search tool, built on Rust's regex engine. It searches directories in parallel and skips gitignored, hidden and binary files by default.
Robert van Engelen's grep-compatible search tool. 7.8.5 was built from the upstream tag. 7.8.4 is the copy built into Claude Code, which the shell's grep runs.
Results · median seconds · lower is faster
ripgrep leads. ugrep is close behind.
Across the whole 1.8 GB kernel tree, ripgrep is 11–20× faster than GNU grep and ugrep is 8–16× faster. In a single 1.4 GB file, ripgrep is 4.8–7.0× faster than GNU grep and ugrep is 3.1–6.4× faster. Every tool returned the same number of matches in every row. Green marks the fastest time in each row.
| Search | GNU grep | ripgrep | rg -j1 | ugrep | ugrep -J1 | ugrep · Claude Code | Matches |
|---|---|---|---|---|---|---|---|
| tree · literal PM_RESUME | 1.188 | 0.097 | 0.636 | 0.132 | 0.612 | 0.129 | 38 |
| tree · case-insensitive -i | 2.082 | 0.103 | 0.705 | 0.134 | 0.628 | 0.131 | 697 |
| tree · word regex -w [A-Z]+_SUSPEND | 1.153 | 0.096 | 0.631 | 0.132 | 0.600 | 0.129 | 577 |
| tree · 4-way alternation | 1.820 | 0.103 | 0.724 | 0.141 | 0.734 | 0.137 | 181 |
| tree · no match | 1.007 | 0.094 | 0.621 | 0.131 | 0.587 | 0.128 | 0 |
| file · literal PM_RESUME | 0.861 | 0.164 | — | 0.236 | — | 0.231 | 29 |
| file · case-insensitive -i | 1.694 | 0.241 | — | 0.263 | — | 0.246 | 672 |
| file · regex \w+_init\( | 0.887 | 0.183 | — | 0.250 | — | 0.256 | 87,033 |
| file · no match | 0.681 | 0.119 | — | 0.217 | — | 0.217 | 0 |
| startup · tiny file | 0.002 | 0.003 | — | 0.003 | — | 0.004 | 0 |
tree is the Linux kernel at d6d3455d8: 96,080 files, 1.8 GB. file is every *.c and *.h joined into one file: 1.45 GB, 37.7 M lines. -j1 / -J1 limit a tool to one thread. A single file is searched on one thread anyway, so those cells are empty.
GNU grep searches the tree on one thread. Limited to one thread, ripgrep is only 1.6–3.0× faster than grep. Most of its 11–20× lead comes from searching on all 20 threads.
On one thread, ugrep matches or slightly beats ripgrep (0.59–0.73 s vs 0.62–0.72 s). On all cores, ripgrep's parallel directory walk scales better, which gives it a lead of about 1.35×.
The ugrep built into Claude Code runs within noise of the standalone build, with startup of 4 ms vs 3 ms. Typing grep there gives near-ripgrep speed, not GNU grep speed.
Method & references
How it was measured.
Each command ran once as a warmup, then 7 timed runs. The table shows the median. The page cache was warm, and the corpus fits in RAM. Output went to wc -l, not /dev/null, because GNU grep stops at the first match when writing to /dev/null. grep and ugrep ran with -rnI --exclude-dir=.git to match ripgrep's defaults, so all three searched the same files. In ripgrep, -E means --encoding, so regex cases pass the pattern to ripgrep without it. Machine: NVIDIA DGX Spark, Cortex-X925 + A725 with 20 threads, Ubuntu 24.04 aarch64, LANG=en_US.utf8. These are single-machine numbers, so treat gaps under about 5% as noise.
Source, docs and the benchsuite this method is based on. The tool was introduced in “ripgrep is faster than {grep, ag, git grep, ucg, pt, sift}”.
Source and the release built for this benchmark, v7.8.5. The project publishes its own ugrep-benchmarks.
results.tsv holds every median and match count, run.py reruns the benchmark, and the README lists exact versions and the kernel commit. GNU grep: gnu.org/software/grep.