🐕 Guardrails, Not Prompts (Part 4): Bound the Loop, Own the Preconditions

📚 Series Overview

This is Part 4 of 4 in the Guardrails, Not Prompts series, built around mini-ai-harness.

Part 1: The thesis, the experiment, and the lie Part 2: Guardrail #1 — structured tools that shrink the world Part 3: Guardrails #2 & #3 — validate actions, verify ground truth Part 4 (this article): Guardrails #4 & #5 — bound the loop, own the preconditions — and the takeaway

---

TL;DR

Two final guardrails complete the harness:

• Bound the loop: a hard iteration cap so the agent fails closed instead of spinning forever, and a sliding context window so cost and confusion stay flat.
• Own the preconditions: the harness checks login and logs in itself — from .env, before the agent takes a single step. The model never sees a login page, and never sees the password.

Then the scoreboard: the same weak model and the same frozen prompt, carried from hallucinating liar to verified success by five guardrails and zero prompt edits.

---

Guardrail #4: bound the loop

A weak model can wander. Left alone it will re-fetch pages it has already read, retry actions that already failed, and let the conversation grow without limit — every iteration appending more messages, costing more tokens, and (worse) diluting the context until the model loses the thread entirely.

Two cheap rails fix both failure modes:

// agent/5-loop.ts
const MAX_ITERATIONS = 12        // fail closed, don't spin forever
const MAX_CONTEXT_MESSAGES = 20  // system prompt + the most recent messages

The iteration cap is self-explanatory — if 12 rounds of tool calls haven't produced a verified result, the run ends with an honest failure instead of a $40 OpenAI bill:

if (trace.length >= MAX_ITERATIONS) {
  return {
    answer: `Stopped after ${MAX_ITERATIONS} iterations without a verified result.`,
    stoppedBy: 'guardrail',
    verified: guardrails ? guardrails.succeeded() !== null : undefined,
  }
}

The context window is subtler. Only the system prompt plus the most recent messages are ever sent to the model — but you can't trim naively, because the OpenAI API rejects a tool result whose parent tool_calls message got trimmed away:

function trimContext(messages: ChatCompletionMessageParam[]) {
  if (messages.length <= MAX_CONTEXT_MESSAGES) return messages
  const system = messages[0]
  const tail = messages.slice(messages.length - (MAX_CONTEXT_MESSAGES - 1))
  // If the window opens mid tool-exchange, drop the orphaned tool results.
  while (tail.length && tail[0].role === 'tool') tail.shift()
  return [system, ...tail]
}

Note what these rails are not: they're not intelligence. They're the pressure-relief valve on a boiler — dumb, mechanical, and the reason the machine is allowed to run unattended.

---

Guardrail #5: own the preconditions

Every failure in this series traces back to one root cause: upvoting only works while logged in. Stage 4 made that failure visible (verified: false), which is honest — but honest failure is still failure. The final guardrail makes the failure impossible.

🙋 "Why not just tell the agent to log in if it needs to? It has browser_fill — it could drive the form."

🧑‍💻 "It could try. A weak model flailing at a login form is exactly the kind of multi-step, silently-failing side quest that eats all 12 iterations. And to even attempt it, the model would need the password — in its context, in the API logs, in every trace. No."

Some things should never be the model's job. The harness runs ensureReady() before the loop starts:

// agent/4-guardrails.ts — the harness owns the precondition
ensureReady: async () => {
  await session.navigate('https://news.ycombinator.com')
  if (await session.isLoggedIn()) return '[harness] already logged in.'

  const user = process.env.HN_USERNAME
  const pass = process.env.HN_PASSWORD
  // ... deterministic Playwright login, then persist the session cookie
  const ok = await session.login(user, pass)
  await session.saveState()
  return `[harness] logged in as ${user} and saved the session.`
},

By the time the agent takes its first step, voting is simply possible. The agent never touches a login page, and — this is the part that makes guardrails a security boundary, not just a reliability trick — the model never sees the password. Only harness code reads .env; the agent works off a saved cookie. What the model never has, it can never leak into a completion, a log, or a prompt-injection reply.

The general rule: anything deterministic, security-sensitive, or required-for-the-task-to-make-sense belongs to the harness, not the agent. Spend your model's limited wits on the part that actually needs judgment.

---

The scoreboard

Same model (gpt-3.5-turbo-0613). Same frozen prompt, byte-for-byte. Here's the whole journey:

Final run: the harness logs itself in, the agent fetches the story list, clicks the real top story's arrow, the harness sees nosee appear in the DOM, and the loop returns:

Answer:      Verified: upvoted story 40203729 on Hacker News.
Stopped by:  success
Verified:    true

Not one word of the prompt changed to get here.

---

The playbook for AI products

Zooming out from one toy task, the five guardrails are a reusable checklist for anything agent-shaped you ship:

1. Structure the world. Don't make the model parse; make code parse and hand the model facts. Most "hallucination" is information starvation.
2. Validate before acting. Gate every consequential action against what the agent has actually observed. Return refusals as corrections — they steer the retry.
3. Verify after acting. Find your domain's nosee class — the ground-truth signal that the thing really happened — and make the harness check it. Ship verified as a first-class output bit.
4. Bound everything. Iterations, context, cost. Fail closed with an honest answer.
5. Own the preconditions. Deterministic setup, auth, and secrets belong to harness code. The model should wake up in a world where its task is possible — and never hold a credential.

🙋 "But models keep getting better. Won't the harness be dead weight once the model is smart enough?"

🧑‍💻 "Ask it the other way: when a smart model tells you it finished, how do you know?"

A stronger model lies less often — but less often isn't a property you can build a product on, and you still need the verification machinery just to measure that it's lying less often. The harness is what turns "the model said so" into "the system proved so," and that holds at every model tier. Meanwhile, the economics run the other direction: a harness that carries gpt-3.5-turbo to verified-correct lets you ship the cheap, fast model where everyone else assumes they need the frontier one. The prompt-engineering reflex treats the model as the product. It isn't. The system around the model is the product — the model is a component, and guardrails are how components get specs.

An agent without guardrails is a dumb dog with no fence. You can breed a smarter dog. Or you can build the fence, and stop caring quite so much about the dog.

The shape of the constraint is the instruction.

---

Run the whole progression:

git clone https://github.com/Seif-Mamdouh/mini-ai-harness
cd mini-ai-harness && npm install && npm run playwright:install
git checkout phase-1 && npm run agent   # the lie
git checkout main    && npm run agent   # the harness