Skip to main content

Introduction to Harness Engineering: An AI Agent's Reliability Is Decided by the Design Around It

· 27 min read
rasshii
Software Engineer

"I upgraded the model to the latest version, but it still repeats the same mistakes." "No matter how I tune the prompt, I hit a wall somewhere." When you try to put an AI agent to work, you always run into this wall. In many cases this is not a problem with the model itself; it stems from the design quality of the environment surrounding the model (= the harness). In this post I organize the idea of harness engineering — which was rapidly systematized starting in February 2026 — together with configuration examples from my own ~/.claude/ directory.

First, a quick note on the term's origin. "Harness" is an English word meaning "equipment / horse tack / (as a verb) to control and put a force to use." In the AI context, Mitchell Hashimoto (co-founder of HashiCorp) introduced it in February 2026 to mean "the full set of environment that supports the model." The equation at its core is very simple.

Agent = Model + Harness

Starting from this equation, let's look at what a harness is, how to design it, and how to grow it incrementally.

How to read this post

This post mixes official / industry-wide concepts with my own operational examples. The latter are enclosed in :::info My setup ::: blocks and clearly marked, so read along while distinguishing "does this apply directly to your Claude Code too?" from "is this rasshii's own way of running things?" All code templates are also placed inside the my-setup blocks.

What is harness engineering​

Harness engineering refers to the practice of designing the entire environment in which an AI agent (a coding agent like Claude Code / Codex / Cursor) operates. Mitchell Hashimoto proposed the concept on his personal blog in February 2026, and a few days later OpenAI's Ryan Lopopolo followed up with a field report. Lopopolo's team reported building a codebase of roughly one million lines in five months with 0 lines written by hand / 1,500 automated PRs, which is what drew the industry's attention. After that, several companies and authors — Anthropic, Augment Code, Birgitta Böckeler (published on Martin Fowler's site), SIG, and others — systematized it with their own vocabularies, and as of May 2026 it is becoming established as an industry-wide term.

Harness engineering is a concept that is hierarchically distinguished from the earlier "prompt engineering" and "context engineering."

LayerQuestionScope
Prompt EngineeringWhat do you askA single interaction
Context EngineeringWhat do you showWithin one context window
Harness EngineeringHow do you design the whole environmentAcross multiple sessions / environment design / phase gates

Whereas prompt engineering "polishes a single instruction," harness engineering targets the entire environment the agent will operate in from now on. The claim is that even with the same model, an agent's reliability changes dramatically depending on the design quality of the harness.

The core equation: Agent = Model + Harness​

Let me write the equation again.

Agent = Model + Harness

Here, Model is the LLM itself — Claude Opus / Sonnet / Haiku, GPT, Gemini, and so on. Harness refers to everything else. Specifically, it includes things like:

  • The system prompt
  • Always-injected instruction documents (CLAUDE.md, in Claude Code terms)
  • Definitions of rules / skills / subagents
  • MCP (Model Context Protocol) servers / various tools / permissions
  • Hooks (shell integrations that fire at specific moments before and after tool execution, such as PreToolUse / PostToolUse / Stop; details below)
  • Memory (persistence across conversations)
  • Context management (strategies for compaction [compressing conversation history] and reset)
  • Verification loops (CI / lint / type-check / automated review)

In other words, the harness bundles together "the paths through which the agent interacts with the world" and "the mechanisms by which the agent disciplines itself." More than the accuracy of the model alone, whether these mesh together determines the reliability needed to withstand real work.

The dichotomy framework: Guides x Sensors​

The most practical frame for organizing harness design is the Guides + Sensors dichotomy by Birgitta Böckeler (Thoughtworks Distinguished Engineer, published on Martin Fowler's site).

  • Guides (feedforward): Steering "before" the action. Setting direction before a failure happens.
  • Sensors (feedback): Self-correction "after" the action. Detecting and fixing mistakes.

Each of these is further cross-classified as Computational or Inferential. Computational is deterministic — verification that mechanically returns the same result every time (lint / type-check / tests, etc.). Inferential is LLM-based — probabilistic verification that depends on the LLM's reasoning (review agents / proposal skills, etc.). This gives four quadrants.

The important implication of this frame is that the harness will not function with only one side.

  • Feedback only (Sensors only): The agent can only fix mistakes after repeating them many times.
  • Feedforward only (Guides only): Even if you write rules, the LLM violates them probabilistically, and you have no way to verify the effect.
  • Computational only: Mechanical lint and types may pass, but they cannot detect design soundness or spec conformance.
  • Inferential only: LLM review is probabilistic, and it can even miss clear violations like // eslint-disable.

A good harness aims for a state where you have done something in all four quadrants.

CLAUDE.md / MCP / memory: the entrance to the harness​

Before diving into the fine-grained directories under ~/.claude/, let's cover the three elements that serve as the entrance to the harness provided by Claude Code out of the box. These are officially provided by Anthropic, and they work the same way for anyone's Claude Code.

  • CLAUDE.md: A Markdown instruction document placed at the project root or directly under ~/.claude/. It is always injected into the context at startup. Because you can auto-load other files with the @path/to/file syntax, it serves as a hub.
  • MCP (Model Context Protocol): A means of accessing external tools / data sources. It is like a plugin for Claude Code. Examples: context7 (latest library docs), aws-knowledge (official AWS docs), terraform, sequential-thinking (structured reasoning), serena (LSP-based code search), and so on.
  • auto-memory: Information persisted across conversations under ~/.claude/projects/<project>/memory/ (<project> is derived from the git repository). Build commands, debugging insights, architecture notes, coding-convention preferences, workflow habits, and the like accumulate here.

These three are the "outer entrance" to the harness. CLAUDE.md tells the agent "things I want you to follow this time too" every time, MCP provides "a means to access the latest primary sources," and memory plays the role of "reminding it of what it learned up to last time."

My setup

In the Quality & Accuracy Policy section of my CLAUDE.md, I explicitly write "MCP first choice -> WebSearch fallback." This is because relying on WebSearch alone tends to pull in stale information, so I assign MCP by purpose: library specs to context7, AWS to aws-knowledge, Terraform to the terraform MCP, and complex judgment to sequential-thinking. Furthermore, I define triggers in CLAUDE.md so that when certain keywords (e.g., "ultrathink," "architecture design") appear in my message, sequential-thinking starts automatically.

The structure under ~/.claude/ (the overall map)​

From here on, I introduce my own ~/.claude/ directory as a concrete example of a directory structure. The table below contrasts Claude Code's built-in features with my own custom usage in two columns.

DirectoryClaude Code built-in featureMy usage
rules/Yes, part of the Memory system (auto-load target)Split into 29 files, scoped per-path via the paths: frontmatter
skills/Yes, the Skill system (triggered by utterances)23 of them, with MCP startup enforced in Step 0
agents/Yes, the Subagent system (launched in isolated context)7 of them, intentionally separating the Generator (implementer) from the Evaluators (code-reviewer / architect / test-strategist)
hooks/Yes, the PreToolUse / PostToolUse / Stop hook system2 of them, for suppression detection / protected-file defense
teams/A meta orchestration pattern I defined myself, not an official featuremulti-perspective-review and the like, spawning multiple agents from the main agent
references/My own naming, not an official featureFor progressive disclosure, read on demand to save context
scripts/My own usage, not an official featureMonthly cron for consistency check / self-test
commands/Yes, the Custom slash command systemLightweight general-purpose commands (init / review / security-review, etc.)
output-styles/Yes, the Output style systemFor response tuning, minimal only

Directories marked "Yes" are storage locations for official Claude Code features; anyone who places them in the same spot gets the same behavior. The ones not marked official are operational conventions I carved out personally — the names could just as well be notes/ or meta/. You do not need to imitate these in your own Claude Code, but in the following chapters I introduce why I carved out such directories.

The overall layered structure goes from the entrance (CLAUDE.md / MCP / memory) to Inferential Guides (rules / skills), to Generator/Evaluator (agents). Beyond that come Computational Sensors (hooks), Orchestration (teams), and supporting pieces (references / scripts).

rules/: the core of Inferential Guides​

Claude Code has a mechanism called the Memory system, which loads ~/.claude/rules/*.md (and each repository's CLAUDE.md) into the context at startup. Two kinds are officially supported: rules that are always loaded, and path-scoped rules (loaded only when you read a file at a specific path) that have a paths: key in their frontmatter (the metadata section delimited by --- at the top of Markdown).

A rule is a "normative instruction aimed at the LLM," so in the dichotomy framework it is classified as an Inferential Guide. Unlike a compiler or linter, it cannot mechanically stop a violation, but it is the most basic means of conveying "this is how I want you to behave" to the agent.

My setup

I run my ~/.claude/rules/ split into 29 files. If you cram too much into one file, it pressures the context window (the amount of information the LLM can handle at once), and it puts everything in an "always read" state.

  • Always loaded: Things you want to reference in any task, such as coding conventions (code-style.md), the Git workflow (git-workflow.md), and harness concepts (harness-engineering.md)
  • Path-scoped: Things needed only when you enter a specific domain, such as Terraform conventions (terraform.md, paths: ["**/*.tf"]) and the Ratchet workflow (ratchet-workflow.md, only when reading related paths)

The structure of each rule is roughly the following Markdown.

---
name: code-style
description: Code readability / naming conventions / comments / secret management
---

# Code Style

- Priority: readability > maintainability > performance > brevity
- Variable / function / class names: English / comments: Japanese (explaining the "why")
- Documentation format: JSDoc
- Credentials via environment variables only, never hardcoded

## Example (Good)

```ts
const MAX_RETRY_COUNT = 3;
function groupUsersByRole(users: User[]): Map<Role, User[]> { /* ... */ }
const apiKey = process.env.MY_SERVICE_API_KEY;
```

## Example (Bad)

```ts
function f(u: any[]) { /* ... */ }                // naming violation
const apiKey = "hardcoded-value-here";            // hardcoding is forbidden
```
General TypeScript code-quality topics (how to write the code itself, not the harness version of code-style.md above) are organized on this site's Docs as Clean Code (TypeScript). For naming, function design, SOLID, and so on, see there.

On the other hand, conventions you only want to reference for specific extensions or paths become a path-scoped rule simply by adding paths: (a minimatch glob) to the frontmatter. For Terraform conventions, it looks like this:

---
name: terraform
description: Terraform coding conventions (provider versions / resource naming / secret management)
paths:
  - "**/*.tf"
  - "**/*.tfvars"
---

# Terraform

- Pin the provider version in the `~> X.Y` form
- Resource naming: snake_case of `<service>_<purpose>`
- Secrets go through AWS Secrets Manager / SSM Parameter Store (hardcoding is forbidden)
- Make module inputs/outputs explicit, and prefer official Verified modules where possible

## Example (Good)

```hcl
provider "aws" {
  region  = "ap-northeast-1"
  version = "~> 5.0"
}

resource "aws_s3_bucket" "app_logs" {
  bucket = "${var.project_name}-app-logs"
}
```

This rule is added to the context only when you read a .tf / .tfvars file. On days when you only touch the frontend, it is not loaded and the context window stays free, so splitting by domain pays off.

In either form, I always attach Good / Bad examples. The LLM learns a "pattern" more reliably from concrete examples than from explanatory text alone.

:::

skills/: utterance-driven skills​

Skills are a mechanism officially provided by Claude Code. You describe a workflow triggered by a specific utterance (for example, "review this," "make a proposal," "debug this") in ~/.claude/skills/<name>/SKILL.md. You declare the utterance trigger / available tools / effort and so on in the SKILL.md frontmatter, and describe the execution steps in the body.

In the dichotomy framework, things that inspect and fix after output — like /review or /debug — function as Inferential Sensors, while things that steer in advance — like /propose — function as Inferential Guides.

My setup

I run 23 skills, the main ones being /propose / /review / /debug / /learn / /explain / /compare / /tec-doc / /grill-me / /next / /checkpoint. As common operating points:

  • Enforce MCP startup in Step 0: I write in SKILL.md to always load sequential-thinking for complex judgment, context7 for library specs, and aws-knowledge for AWS
  • Iteration protocol: /review converges Critical / Major to 0 in at most 3 iterations, /propose does adversarial self-critique in at most 2 iterations, /debug does Investigate -> Fix -> Verify in at most 3 iterations：I set the iteration count per skill

A SKILL.md template example:

---
name: review
description: Review code changes from 7 perspectives (correctness / readability / performance / security / maintainability / conventions / regression), weighted by severity
when_to_use: Triggered by "review," "review," "take a look," "check"
allowed-tools: Read, Grep, Glob, Bash, WebSearch
effort: max
---

## Step 0: Start MCP

- sequential-thinking (enforced for complex judgment)
- context7 (when verifying framework usage)

## Step 1: Grasp the scope of changes

Get the target files with git status / git diff.

## Step 2-3: 7-perspective review -> severity weighting

Flag issues at four levels: Critical / Major / Minor / Nit.

## Exit condition

Finish when Critical / Major reach 0, or upon reaching the maximum of 3 iterations.

agents/: Subagents (Generator-Evaluator separation)​

Claude Code has an official feature called Subagents, which lets you launch agents with independent roles in an isolated context. You define them in ~/.claude/agents/<name>.md and call them from the main agent via the Agent tool (the main agent's tool for launching a subagent).

Here one of the important principles of harness engineering appears: Generator-Evaluator separation. It is known, as shared knowledge across Anthropic and the industry, that "when the agent that implemented something evaluates it itself, a self-evaluation bias kicks in and quality gets overestimated." The point is to make the implementer (Generator) and the evaluator (Evaluator) different agents.

My setup

My ~/.claude/agents/ defines 7 subagents.

  • Generator side: implementer (TDD [test-driven development] + type safety), refactorer (behavior-preserving refactoring), debugger (autonomous bug resolution)
  • Evaluator side: code-reviewer (7-perspective review, read-only), architect (design review + adversarial verification), test-strategist (test strategy), infra-reviewer (TF / IaC review)

For the Evaluator side, I set disallowedTools: Write, Edit to pin them to read-only roles without edit permission. This structurally prevents the accident of "a reviewer rewriting the code on its own."

A subagent definition template:

---
name: code-reviewer
description: A code reviewer for pair programming. Provides review and guidance after code changes
tools: Read, Grep, Glob, Bash, WebSearch
disallowedTools: Write, Edit
---

## Step 0: Start MCP (required)

- sequential-thinking (enforced for verifying multiple hypotheses)
- context7 (when verifying framework usage)

## Review perspectives

Correctness / readability / performance / security / maintainability / conventions / regression

## Output format

- Weighted by severity: Critical / Major / Minor / Nit
- Pinpoint locations by file name + line number
- Present a fix (as a concrete diff or instruction)

hooks/: Computational Sensors​

Claude Code's Hook system is an official feature that lets you run shell commands on specific events (PreToolUse: before tool execution, PostToolUse: after tool execution, Stop: at response end, UserPromptSubmit: when user input is received, and so on). You register them in the hooks field of ~/.claude/settings.json.

In the dichotomy framework, a hook is a typical Computational layer: a deterministic mechanism whose inputs and outputs follow the same shell script every time. It can be used for both Guide purposes (blocking before execution) and Sensor purposes (detecting after execution).

My setup

I run 2 hooks.

  • check-suppression.sh (PostToolUse): Detects the new addition of lint-suppression comments such as // eslint-disable / @ts-ignore / # type: ignore / # noqa. On detection it returns feedback to Claude with exit 2 and forces a root-cause fix (use PreToolUse if you want to block the edit itself preemptively)
  • check-protected-files.sh (PreToolUse): Preemptively blocks writes to .env / private keys / credential files

A minimal implementation of check-suppression.sh:

#!/bin/bash
# ~/.claude/hooks/check-suppression.sh
# PostToolUse hook: block new additions of lint suppression comments

set -eu

# Emergency override (can be skipped via an environment variable)
if [ "${CLAUDE_HOOK_SKIP_SUPPRESSION:-0}" = "1" ]; then
  exit 0
fi

# Extract the newly added content from the hook input (JSON)
INPUT=$(cat)
NEW_CONTENT=$(echo "$INPUT" | jq -r '.tool_input.new_string // .tool_input.content // empty')

# Detect suppression patterns
if echo "$NEW_CONTENT" | grep -qE '(eslint-disable|@ts-ignore|# type: ignore|# noqa)'; then
  echo "ERROR: Suppression comment is not allowed. Fix the root cause instead." >&2
  exit 2   # exit 2 in PostToolUse is returned to Claude as feedback
fi

exit 0

A registration example in settings.json (the matcher field lets you narrow the target tool names by regex):

{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "Edit|Write",
        "hooks": [
          { "type": "command", "command": "~/.claude/hooks/check-suppression.sh" }
        ]
      }
    ]
  }
}

The point is the premise that "just writing 'no any' in rules means the LLM will violate it probabilistically." The more important the norm, the more you stop it mechanically with a deterministic hook — that is one of the core philosophies of harness engineering.

Supporting directories (teams / references / scripts)​

From here are directories that are not Claude Code built-in features, carved out as my own operational conventions. You do not need to imitate them directly, but I will share why you might want such boxes.

teams/: Multi-agent Orchestration​

There are cases where you want multiple subagents to work in coordination. For example, "I want to review a change of 10+ files in parallel from 3 perspectives: code-reviewer (quality) + architect (design) + test-strategist (test)."

My setup

In ~/.claude/teams/multi-perspective-review/team.md, I describe in Markdown the steps for the main agent to spawn 3 subagents in parallel and integrate the results. The operation is that when an utterance contains "multiple perspectives," "comprehensive review," or "3 perspectives," the main agent reads team.md and runs it.

This is not an official Claude Code feature; it is a meta orchestration (coordinating multiple agents) pattern layered on top of the subagent feature.

references/: detailed protocols (progressive disclosure)​

If you write detailed steps / templates / application examples into rules/, you pressure the context every time they are auto-loaded. So I write only the normative rules (a summary) in rules and place the detailed protocols elsewhere, reading them only when needed — separating them this way (this is called progressive disclosure).

My setup

I keep 25 files of detailed protocols in ~/.claude/references/. The rules side references them as "Detailed protocol: ~/.claude/references/xxx.md (read when needed)." This saves the context window while letting me access the details when I want to dig in.

This is also not an official feature, and the naming is arbitrary. It could be docs/ or manuals/.

scripts/: measurement and verification​

A place for shell scripts that mechanically check the integrity of the harness itself.

My setup

In ~/.claude/scripts/ I keep an internal consistency check (check-internal-consistency.sh) and a self-test (self-test.sh), run via a monthly cron (a scheduled job). They are Computational Sensors that detect drift in the harness itself, such as "are the cross-links inside rules broken?" and "do the skills' utterance triggers collide?"

The Phase 0-6 staged adoption path​

So far I have introduced individual directories, but the next issue is in what order to adopt them. If you try to write "29 rules / 23 skills / 7 agents / 2 hooks" all at once, the context window overflows, the norms become hollow, violations are mass-produced, and the standards themselves collapse.

The Phase numbers are an organizing axis I assigned myself

The Phase 0-6 staging below is a label I attached for convenience to organize my own harness; it is not an official or industry-standard division. However, the individual concepts handled in each Phase (Computational Guards / Architecture Fitness / Generator-Evaluator separation / Spec-First / Ratchet / Metrics) are not my originals. They come from official / industry sources such as Hashimoto / Anthropic / Neal Ford / Birgitta Böckeler.

PhaseNameMain deliverablesOrigin of the concept
0Concept organizationDocumenting the harness norm ruleHashimoto / Lopopolo / Böckeler
1Computational GuardsHooks (suppression detection / protected files) + pre-commitAnthropic / industry-wide
2Architecture FitnessVerifying structural constraints with dependency-cruiser / Conftest / tflintNeal Ford (Building Evolutionary Architectures)
3Generator-Evaluator separationSubagents (implementer / code-reviewer / architect ...)Anthropic
4Spec-First PipelineCONTEXT.md (terms) / docs/adr/ (decisions) / docs/specs/ (specs)Hashimoto
5Ratchet WorkflowA procedure for promoting failures into permanent fixesHashimoto's principle + my own operational organization
6Metrics & Health CheckA monthly aggregation script for 4 Quality metricsHashimoto / Lopopolo

The flow of evolution drawn as a diagram looks like this.

The aim of each Phase is as follows.

  • Phase 0: First, document what a harness is. Write a normative document like rules/harness-engineering.md and align the vocabulary for what you want to enforce in later Phases
  • Phase 1: Of the norms written as rules (Inferential Guides), stop the mechanizable ones deterministically with hooks / lint / type-check. The first line of defense complementing the LLM's probabilistic compliance
  • Phase 2: Detect relationships between files (circular dependencies / layer violations / cross-feature dependencies [unnecessary dependencies between features]) in CI, not just single files. Concretely, use tools like dependency-cruiser (JS/TS dependency-direction verification) / Conftest (OPA Rego-based policy verification) / tflint (Terraform linter). The source is Neal Ford's Architecture Fitness Functions
  • Phase 3: Split the implementer and evaluator into separate subagents. Intentionally separate the Generator (implementer) and Evaluators (code-reviewer / architect / test-strategist) to eliminate self-evaluation bias
  • Phase 4: Make term definitions (CONTEXT.md) / important design decisions (ADR) / feature specs (docs/specs) required inputs to the harness. Do not let it implement without reading the spec
  • Phase 5: Persist failure patterns into rules / hooks / fitness functions to make recurrence structurally impossible (based on Hashimoto's principle; details below). I use learnings/ as an intermediate staging area and promote only the failures that recur
  • Phase 6: Measure the harness's own reliability with 4 Quality metrics (below) and run a health check via a monthly cron

The point is to "not start from Phase 6." Even Phase 0 and Phase 1 alone produce results in most projects.

Ratchet Workflow: turning failures into permanent fixes​

The idea of the Ratchet that appeared in Phase 5 is, even within harness engineering, a core principle at the "you can go home having learned just this" level, so I treat it as an independent section. Its foundation is the principle Hashimoto laid out on his blog, which, summarized faithfully to the original, is as follows.

When you notice that an agent made a mistake, each time, engineer a solution so that the same mistake can never happen again.

I call this principle the Ratchet and have shaped it into operation as: "when the same failure is observed twice, always add a permanent fix to one of rules / hooks / fitness function / learnings, making recurrence structurally impossible." The name "ratchet," the "two-times" threshold, and the four-way classification of fix destinations are all my own operational organization, not in Hashimoto's article itself.

A ratchet is a tool term for "a gear that only turns one way." It is a metaphor that overlays onto the harness's learning loop the property of "once it advances, it does not go back." "I'll be more careful" or "I'll watch out next time" will always repeat the same failure, so the point is to build it into a mechanism that does not occur even without being careful.

My setup

With the /learn skill, I append insights gained during a conversation to ~/.claude/learnings/<category>/<topic>.md on the spot, and at the end of a session I extract them in bulk with the /record-learnings skill. Furthermore, when the same failure is observed multiple times, I promote it from learnings into a rule or hook. For example, if learnings like "tends to miss type violations" pile up, I add an item to rules/typescript-strict.md or update the TypeScript config to be stricter.

The 8 normative rules​

I summarize the main norms worth following in the practice of harness engineering into 8. The sources mix official / industry-wide ones with my own operating policy.

  1. Anything not in context doesn't exist (Hashimoto / Lopopolo): Put information you want to convey to the agent in repository-local, versioned artifacts. Slack, the spoken word, and what is in someone's head are invisible to the agent
  2. Do not trust probabilistic compliance (industry-wide): Even if you write "no any" in a rule, the LLM violates it probabilistically. Enforce important norms deterministically with computational sensors (lint / hook / CI)
  3. Separate Generator and Evaluator (Anthropic): Do not let the agent that implemented something evaluate itself. Review with a separate subagent
  4. Do not allow inline-disable suppression (industry-wide): Suppressing a sensor with // eslint-disable-next-line / @ts-ignore is the biggest anti-pattern that hollows out the harness. Address it with a fix
  5. Ratchet principle (my operational organization based on Hashimoto's principle): When you make the same failure twice, add a permanent fix. Make recurrence structurally impossible
  6. Do not make Volume metrics your main indicator (Hashimoto / Lopopolo): "Lines the AI wrote" and "number of accepted proposals" do not reflect harness reliability. Use the Quality metrics below
  7. Context Resets > Compaction (Anthropic / Hashimoto): Do not keep a long task alive with compaction; fully reset with a structured hand-off and wake a fresh agent
  8. Detect spec drift with sensors (Hashimoto): Divergence between CONTEXT.md / ADR / docs/specs and the implementation cannot be caught by downstream tests. An Evaluator agent judges spec conformance

Anti-patterns​

Let me also list the failure patterns that often occur when building a harness.

  • One-shot completion: Trying to finish a complex task in one shot. Not inserting hand-offs (handing over to another agent / the next session) or iterations along the way
  • Rules files alone: Writing only rules without adding computational sensors. With Inferential Guides alone, violations occur probabilistically
  • Over-constraining: Constraints so strict they impede even legitimate refactoring. "Forbid everything" and no one follows it anymore
  • Declaring completion too early: Saying "done" without verification. No check of tests / types / lint
  • Leaving partial implementations: Handing off mid-state. The next session cannot pick it up
  • Self-evaluation only: Letting the Generator agent self-evaluate. Not going through a separate agent like code-reviewer / architect

The 4 Quality metrics​

Finally, let me organize the indicators that measure the harness's own quality. The claim of Hashimoto / Lopopolo is that you should make Quality metrics — directly tied to reliability — your main indicators, rather than Volume metrics like "lines of code" or "acceptance rate."

MetricContentWhy it matters
Task Resolution RateThe accuracy verified by automated testsThe actual pass rate of what the agent "self-declared complete"
Code Churn RateThe proportion of code discarded within 2 weeksThe amount rewritten in the short term. An indicator of whether the harness guided a stable design
Verification TaxTime spent auditing AI-generated codeHuman review load. The lower this is, the more trustworthy the harness
Defect Escape RateThe defect rate that reaches productionThe final user impact. The overall score of the whole harness

The contrast with Volume metrics is as follows.

  • Volume metrics measure "usage" (e.g., lines the AI wrote, number of accepted proposals): how much the agent worked
  • Quality metrics measure "reliability": how useful the agent was

Unless you evaluate the harness on the Quality side, "wrote a lot but rewrote all of it two weeks later" ends up scoring high.

The first step (practical guidance)​

If, having read this far, you feel "doing it all at once is impossible," that is correct. Harness engineering is something you grow incrementally. Let me list 3 minimal steps you can start today.

  1. Write one CLAUDE.md file: Put "things I want followed in this project" in Markdown, 5-10 lines, at the project root or in ~/.claude/CLAUDE.md. Just putting into words what you usually convey verbally："no any types," "comments in Japanese," "write tests before implementing" — is enough
  2. Record failures in learnings: When the same mistake appears twice, persist it in CLAUDE.md or a separate rule file (Ratchet Workflow). Lean toward "a mechanism so it does not occur next time" rather than "I'll be careful"
  3. Make the important ones deterministic with Computational Sensors: Type safety (TypeScript strict) and lint (ESLint) can reject violations regardless of the LLM's intent. What works best are pre-commit hooks (checks that run before a Git commit) and CI

"Do not write 29 rules / 23 skills all at once" is the biggest way to avoid the pitfall. Starting from one file and adding more as the need arises produces a stronger harness in the long run.

Term proposal and core concepts​

Anthropic official (Long-Running Agents)​

The dichotomy framework​

Architecture Fitness Functions​

Claude Code official docs​

Tool official pages​

Etymology​

Wrap-up​

Finally, let me confirm the core equation once more.

Agent = Model + Harness

If accuracy plateaus even after upgrading the model to the latest version, suspect the harness. Design the harness along two axes, Guides (feedforward) and Sensors (feedback), each cross-classified by Computational / Inferential. Do not put everything in at once; start from Phase 0 (concept organization) and Phase 1 (Computational Guards). Do not be careful about failures — make them structurally impossible with a Ratchet.

If, having finished reading this post, you feel "in my own project too, let me first write one CLAUDE.md file," then I think this article has done its job.

Feel free to share your thoughts and feedback on X

Tweet your thoughts on 𝕏