Moiz Ali
Tetovo Available for projects
RUN #003Building since 2026Live on Railway · 329 tests

Cold outreach agent

Give it a company website and a person's role. It researches the company and drafts a short email, and code rejects any claim it cannot source.

Client
Own product · B2B sales teams
Role
Design · build · evaluate
Timeline
2 weeks
Stack
Claude · Python · FastAPI · Postgres · Redis
Status
Building

01 — The system

6 nodes · 5 edges

02 — Results

Measured over 10 real leads
10/10drafts passed the claim verifier
$0.14 maxcost per researched lead ($0.08 at the low end)
34–78sfrom lead to draft

03 — The problem

Sales reps know a researched email beats a template, but researching by hand is slow, so most teams send templates. Asking a model to "personalise it" is fast, but it invents facts. One made-up funding round in a cold email costs the sender their credibility. I wanted the speed of a model with the honesty of a person who actually read the page.

04 — Decisions & tradeoffs

  • Code makes the final call. The pipeline is a plain Python state machine. A fact is kept only if its URL was fetched in this run and its quote appears in the page text. The verifier maps every claim in the email to a fact, and code decides pass or fail.
  • One autonomous agent, with no dangerous tools. Only the research step loops on its own. It has three tools, and none of them can send, write or delete. Fetched pages are wrapped as untrusted data.
  • A hand-written agent loop on the Anthropic SDK, with no framework, so cost, retries and turn limits stay visible.
  • Cheap checks first. A poor-fit lead is rejected for about $0.0013, before any research is paid for.
  • A person approves every email. An opt-out in a reply is caught by plain code, even if the model is down. An email is claimed in one conditional UPDATE before sending, so it goes out at most once.

05 — What broke

  • 240 tests passed, and the API could not start. The tests injected their own dependencies and never ran real startup. Now a smoke test boots the real server, worker, Postgres and Redis.
  • The LLM judge said 92% faithful; reading the facts myself said about 75%. Changing only the judge moved the same 24 facts from 0.92 to 0.67. I rebuilt it so the model names a quote for each detail, and code derives the verdict.
  • A rate limit let 5 of 6 requests through. "Check, then create" is two steps. A Postgres advisory lock makes it exactly one of six.

06 — What's next

One real email through the provider, measuring the reply classifier, a review screen, and a fit signal such as "do they already use a competitor?". The sample is still small. How real sales people rate the emails, and reply rates, are not measured yet.

Code on GitHub

Next run · RUN #002Concord