Harness Engineering: Matching Agents to the Work | Interloom

Introduction

Most agent architecture debates start in the wrong place.

They ask whether tool-calling agents are better than coding agents, or whether a CLI is a better interface for AI than structured tool calls.

That framing is too narrow.

In production, the useful question is not which interface is more advanced. The useful question is:

Which harness gives the model the right amount of freedom, structure, and accountability for the work at hand?

This is what I mean by Harness Engineering.

An agent harness is the execution environment around the model: the tools it can access, the runtime it operates in, the files and state it can inspect, the permissions it must respect, the way work is observed, and the way results are validated.

Different work needs different harnesses.

The mistake is trying to force all agent work through one interface.

We kept seeing the same pattern: the more exploratory the task, the worse the fit between the problem and the tool catalog. Tool catalogs scale poorly when the set of possible user problems keeps expanding. At the same time, pure sandbox freedom is the wrong fit for high-volume, permission-sensitive operations.

This article is not a step-by-step implementation guide and not original research. It is a production design principle: match the agent harness to the shape of work.

Two work streams: Field Agent and Assistant

The distinction became clearer once we separated two work streams that often get mixed together.

Work stream Best description Best harness
Field Agent A role-specific agent that performs known actions inside a business process. Predefined tools with tight permissions.
Assistant A more exploratory agent that helps a user investigate, compose, debug, or create a workflow. Sandboxed CLI, files, logs, and runtime state.

A Field Agent is closer to an operational worker. It may classify support tickets, apply an approved procedure, draft a reply, update a CRM field, request missing documents, or create follow-up tasks. The work is bounded. The allowed actions should be visible. The outcome should be easy to audit.

An Assistant is closer to an exploratory operator. It may inspect files, compare data sources, discover which workflow commands exist, write a small script, recover from an error, and then decide the next step. The path is not known upfront.

This distinction matters because the two work streams optimize for different things.

Dimension Field Agent Assistant
Primary goal Reliable execution Exploration and composition
Interface Tool calls CLI, files, shell, logs
Permission model Narrow and role-specific Broader but sandboxed
Best for Large-scale repeatable workflows Ambiguous long-running tasks
Failure mode Wrong tool call or bad input Unbounded exploration
Validation Structured result checks Evidence, logs, intermediate artifacts

The Field Agent should be constrained and predictable enough to run at scale. The Assistant should be flexible enough to figure out what needs to happen.

The two short demos below show that difference in practice.

The first demo shows a bounded tool harness. A Field Agent named ClaimAnalysisAgent has one narrow goal: extract problems from a claim, then append a new problem to a note only if that problem is not already present. The agent has a small tool surface: append_to_note, read_file, create_note, use_skill, and view_task. It then processes 100 cases and extracts recurring problems from them. This is a good fit for a Field Agent because the work is repetitive, permissioned, and constrained to a small set of known actions.

The second demo shows the sandbox harness on an analytics task. The Assistant inspects 100 synthetic IT department cases, builds an analytics summary by status, priority, department, location, and recurring issue theme, then identifies the biggest bottlenecks and produces a short operations report.

Why tool catalogs stop scaling

For a while, the default way to build an AI agent was simple: define a set of tools, describe them with schemas, and let the model decide which one to call.

That is still a good design for bounded operations.

If the user wants to create a task, assign an owner, apply a tag, send a draft reply, or trigger a known workflow, a tool is a clean contract:

The trouble starts when the work is not one action but a chain of decisions.

# Tool-based field agent
create_task("Task 1")
create_note("Task 1 note")
create_task("Task 2")
create_note("Task 2 note")

# Assistant in a sandbox
for i in range(1, 51):
    create_task(f"Task {i}")
    create_note(f"Task {i} note")

Take a simple example: a user asks an agent to create 50 tasks and 50 matching notes from a generated pattern.

Repetition alone does not make a task exploratory. A Field Agent can still be the right harness if each action needs permission checks, auditability, or human review. But when the user wants to generate or transform many related operations from a pattern, a sandbox gives the model a cheaper way to express that pattern as code instead of repeated tool calls.

That difference gets more important when the task is less repetitive and more ambiguous.

Consider a finance operations request like this:

Compare this month’s unpaid invoice export with payment-provider data, check whether support already promised an extension, and prepare a list of accounts that need follow-up.

There usually is no single perfect tool for that. The agent may need to inspect CSV files, query different systems, compare mismatched identifiers, derive intermediate results, and adjust its approach once it sees the real data.

You can try to model all of that as tools, but the system gets noisy quickly:

There is also a context cost.

Tool calls are often attached directly to the agent definition. As the tool catalog grows, the model has to carry more schemas, more descriptions, and more possible actions in context. That increases cost, slows down responses, and can reduce quality because the model spends more attention choosing from the menu than reasoning about the problem.

Dynamic tool selection can help, but it introduces a different tradeoff: every invocation now needs a routing or retrieval step, and prompt-cache reuse becomes harder because the effective tool surface changes between calls.

This is the core scaling problem:

A large tool catalog is a good interface for software, but a poor working memory for an agent solving an open-ended problem.

The tool harness fits bounded work

The tool harness is not obsolete. It is the right harness when the work is bounded, repeatable, and permission-sensitive.

That is why Field Agents still matter.

A support Field Agent might be allowed to:

A sales operations Field Agent might be allowed to:

These agents should not have general problem-solving freedom. They should have a carefully designed action surface.

That matters for governance. Tools are not only capabilities. They are responsibility boundaries.

In a production system, these should not be the same permission:

A workflow architect may be allowed to define the template. A frontline operator may only be allowed to apply it.

The sandbox harness fits exploratory work

The main change with sandboxed Assistants is not that the model becomes smarter. It is that we stop handing the model a long tool menu and give it a workspace.

Inside a sandbox, the Assistant can inspect files, run commands within a scoped environment, call a CLI, write a short script, recover from errors, and build intermediate structure around the task.

That makes a big difference for work where the path is not known upfront.

Three capabilities matter most:

  1. Explore before committing. The Assistant does not have to pretend the plan is obvious from the first token. It can look around, gather evidence, and decide what to do next.
  2. Compose steps naturally. Many real requests are not one business action. They are ten small actions with judgment in between. A sandboxed Assistant can transform data, branch based on results, retry with a different approach, and keep going.
  3. Recover better. When a rigid tool call fails, the interaction often stops at the error boundary. A sandboxed Assistant can inspect the failure, fix the input, add a missing transformation, or choose another route.

This is why the sandbox harness is a better fit for tasks that sound like:

Those are not just actions. They are investigations.

Example: turning IT cases into an analytics workflow

Imagine a user asks:

Analyze these 100 IT department cases. Build an analytics summary by status, priority, department, location, and recurring issue theme. Then identify the biggest bottlenecks and create a short operations report.

This is not a single business action. It is a small analytical workflow.

The Assistant has to discover where cases live, inspect the available fields, decide how to classify themes, write analysis code, run it, check the output, and turn the result into an artifact a manager can read.

$ npx -y @interloom/cli@latest --help
$ npx -y @interloom/cli@latest cases --help
$ npx -y @interloom/cli@latest cases list --help
$ npx -y @interloom/cli@latest cases list -o json > /private/tmp/interloom-case-list.json

The important part is not the CLI syntax. The important part is the harness.

The Assistant can discover the available interface, notice that list output is incomplete, fetch full records, write code, run analysis, track progress, and produce durable artifacts.

The sandbox harness still needs boundaries. A workspace is not a license for unlimited action. Production sandboxes need command allowlists, network controls, scoped credentials, approval gates for side effects, artifact retention, and logs that make the Assistant’s path reviewable. Otherwise the same flexibility that makes the sandbox useful for exploration can become difficult to audit or reproduce.

From exploration back to bounded capabilities

The product boundary became clearer over time.

Once sandboxed Assistants solved more difficult tasks, the valuable output was often not only the final answer. It was the working path the Assistant discovered.

If managers keep asking for the same IT inbox report, the sandbox run should not remain a one-off. The analysis script can become a governed workflow with typed inputs, fixed grouping logic, permission checks, test cases, and a reusable report artifact.

Once the path is known, we can capture it, parameterize it, and turn it into something reusable.

This reusable unit may become:

One platform, multiple harnesses

Supporting multiple harnesses cleanly requires more than adding a sandbox next to a tool system. The harnesses need one shared layer underneath them.

The key architectural decision is simple: UI, APIs, predefined tools, CLI commands, and external integrations should all reach the same business logic.

Conclusion

Harness Engineering is about matching the agent’s runtime to the work.

Tool-based Field Agents remain the right answer for bounded, governed, repeatable operations. They are easier to debug, easier to trust, and easier to align with business roles.

Sandboxed Assistants are the better answer for ambiguous work. They can inspect, compose, recover, and discover the path to an outcome when the user cannot specify every step in advance.

The most useful systems need both.