concept

Harness Engineering — Coding Agent Operations Cheatsheet

Practical guide to harness engineering: design the context, tools, feedback loops, evaluations, and operating controls that make coding agents reliable.

2026-09-25 (3 weeks ago)
5 steps
~100 min total

harness-engineering
agents
evaluations
context
feedback-loops
operations

Harness Engineering — Coding Agent Operations Cheatsheet

One-line: Harness engineering is the practice of surrounding AI coding agents with the context, tools, constraints, evaluations, and feedback loops they need to deliver reliable software work.

Installation

Harness engineering is a discipline, not a single tool, so there is no standalone installer. Start by choosing a coding agent and documenting the harness around it.

# Example repository bootstrap for a harnessed agent workflow
mkdir -p .agent-harness/{context,evals,runbooks,logs}
touch .agent-harness/context/project.md
touch .agent-harness/evals/checklist.md
touch .agent-harness/runbooks/issue-loop.md

First-run checklist:

Pick one agent surface: Claude Code, Codex, OpenCode, Pi, or another approved tool.
Write the project’s durable context and forbidden actions.
Define the checks that prove work is done.
Capture handoff notes after every agent session.

Step-by-Step Setup & Optimization

Step 1 — Define the work envelope

Goal: Written answers for what the agent may know, do, store, and be judged byTime: ~15 minLevel: intermediate

A harness starts by specifying what the agent is allowed to know and do.

Surface	Harness question
Context	Which docs, files, issues, and decisions load by default?
Tools	Which tools can read, write, call APIs, or run shell commands?
State	Where are plans, logs, checkpoints, and handoffs stored?
Feedback	Which tests, reviews, evals, and metrics judge progress?

Each row of the table has a concrete answer — context, tools, state, and feedback are all specified.

Step 2 — Convert intent into operating instructions

Goal: A concise runbook that turns goals into repeatable loopsTime: ~20 minLevel: intermediate

Write a concise runbook that turns vague goals into repeatable loops.

# Issue loop
1. Restate acceptance criteria.
2. Inspect only relevant files.
3. Propose a plan with risks.
4. Implement one slice.
5. Run the narrowest check, then the full required check.
6. Write handoff notes and open a PR.

Your runbook names the check to run after each slice and how a task is handed off.

Step 3 — Build feedback loops before autonomy

Goal: Fast checks in place before the agent runs unsupervisedTime: ~30 minLevel: intermediate

An agent without feedback drifts. Add fast checks first:

Static validation for formatting, schema, or lint rules.
Unit tests or golden fixtures for expected behavior.
Reviewer prompts focused on acceptance criteria and regressions.
Handoff notes that make the next session cheaper.

A failing check actually blocks the agent — run one and confirm it catches a regression.

Step 4 — Add memory and artifacts deliberately

Goal: Durable artifacts carrying the facts that must survive sessionsTime: ~20 minLevel: intermediate

Use durable artifacts for facts that should survive sessions:

Artifact	Contents
`AGENTS.md` or `CLAUDE.md`	Project conventions and safety rules.
Decision records	Why a design or tradeoff was chosen.
Eval fixtures	Inputs and expected outputs for recurring tasks.
Handoff notes	What changed, what failed, and what remains.

Each artifact type in the table exists and is being updated — rules, decisions, fixtures, handoffs.

Step 5 — Measure the harness, not just the model

Goal: Metrics that tell you the harness — not the model — is the bottleneckTime: ~15 minLevel: advanced

Track metrics that the team can improve without changing models:

Time from issue pick to reviewed PR.
Percentage of tasks passing checks on the first run.
Rework caused by missing context.
Commands blocked by policy.
Follow-up issues created after merge.

You can state your current first-run pass rate and rework rate from the metrics above.

Best Practices

Do

✅ Treat prompts, config, tests, and handoffs as production assets.
✅ Keep the agent’s default context small and high signal.
✅ Use checklists and evals before increasing autonomy.
✅ Separate planner, implementer, reviewer, and release responsibilities when tasks grow.
✅ Record failures as harness improvements, not just model mistakes.
✅ Make rollback and checkpointing part of the loop.

Don’t

❌ Assume a stronger model fixes unclear tools or missing tests.
❌ Hide critical constraints in one-off chat messages.
❌ Give broad filesystem, network, or deployment access without policy gates.
❌ Optimize for impressive demos instead of repeatable outcomes.
❌ Let long-running agents continue without resumable state and handoff artifacts.

Quick Command Reference

Command or artifact	Use
`.agent-harness/context/project.md`	Durable project facts and constraints.
`.agent-harness/evals/checklist.md`	Human and script checks for done-ness.
`.agent-harness/runbooks/issue-loop.md`	Repeatable workflow for issue resolution.
`git diff --check`	Cheap whitespace and patch sanity check.
`pnpm test`	Example project-level feedback loop.
`gh issue view <n> --json ...`	Structured issue input without parsing prose output.

Expected Outcomes

After setup, you should have:

A documented harness that defines context, tools, state, and feedback.
A repeatable issue loop for planning, implementation, review, and handoff.
Fast checks that catch common agent mistakes before human review.
Durable artifacts that survive context-window resets and agent restarts.
Metrics that guide harness improvements over time.

Reference

Sources & deeper reading