How I build
with AI.
Claude Code writes the code. Everything around the code is mine: the problem, the decisions, the checks, and the call on whether it works.
The short answer
If AI writes the code, what do I do? Five things AI does not do for me:
- Decide what is worth building, and what "done" means.
- Choose the model, service and data rules for each job.
- Demand the test or measurement that proves a change works.
- Notice when something is wrong even though nothing reports an error.
- Run it in production and answer for it.
Ten steps, and who does each one.
This is the loop behind every project on this site. The AI does a lot of the typing. It does not own a single step on its own.
Find the problem
A real pain I or a business has, stated in one sentence.
MeDefine the outcome
What a user can do afterwards, and how I will know it works.
MeResearch the options
Existing tools, APIs and models, with their limits and costs.
Me + AIWrite the requirements
Rules, edge cases, data access, what must never happen.
MeChoose the approach
Stack, hosting, which model does which job, what gets tested.
Me decides · AI proposesDirect the build
Claude Code writes the code, migrations and tests in small, reviewable pull requests.
AI implementsReview and test
Tests, type checks, a real production build, and looking at the actual screen.
Me + AIInvestigate failures
Read the logs and data, form a cause, then prove it with a measurement.
Me + AIRefine
Fix the cause, not the symptom, and add a test so it cannot come back quietly.
Me decides · AI implementsDeploy and operate
Ship, check the live result, monitor, and keep it running.
Me
One feature, start to finish.
PublishBench, July to August 2026. A routine model upgrade turned into a month of measured fixes. Every step below is in the git history. The quotes are from the actual commit messages.
- 29 Jul · Requirement
Move to the newer model
The goal was simple: move every AI generator to Claude Sonnet 5 and stop paying twice for the same prompt.
9ea1b4c · "Move the default model to Sonnet 5, and stop paying twice for the same prompt" - 29 Jul · Failure
Every generator went offline
The new model rejects two settings the old one silently ignored. For about four and a half hours, nothing generated. The fix switched every generator to a forced tool call, so the model must answer in a fixed shape.
f058f10 · "Fix every Anthropic generator, which Sonnet 5 had taken offline" - 31 Jul · Investigation
A quieter failure underneath
Replies now arrived, but often in the wrong shape: a list wrapped inside a string. The content was right; the encoding was wrong. Instead of guessing, I asked for a probe that measures how often it happens.
7876f3a · "Add a probe for the rate at which the model malforms a tool argument" - 31 Jul · Evaluation
Test the obvious fixes, and reject them
Four prompt changes looked plausible. Each was measured and each scored 0 out of 10. The one change that worked was a single sentence in the schema itself.
2ee1a9b · "Tell the model in the schema that an array must arrive as an array" - 11–14 Aug · Hardening
Make the remaining failures harmless
The repair step moved into its own tested module. A malformed reply now retries with its data intact, and a reply cut off at a token limit is no longer counted as a model failure.
e86af13 · #124 · #139 "A cut-off reply is not a model that failed" - 23 Aug · Final fix
Rescue on the first attempt
One prompt was still failing every time. The belief that rewording the query fixed it turned out to be luck: 15 of 15 runs were malformed, including the reworded control. A tolerant decoder now recovers the broken shapes.
42c06bd · #178 "Rescue the pseudo-XML tool-argument spill that burned contentBrief jobs"
The measurement that decided it
Clean replies out of 10, before and after the one-sentence schema change. Source: commit 2ee1a9b.
| Prompt | Before | After |
|---|---|---|
| chapters | 0 | 10 |
| summary | 3 | 9 |
| replication brief | 3 | 9 |
| thumbnail critique | 9 | 10 |
| titles | 9 | 9 |
| channel name | 5 | 6 |
Across the six main prompts: about 52% malformed before, 10% after. The control mattered: naming the property alone scored 1 of 15; adding "must be a JSON array, not a string containing one" scored 15 of 15.
From the commit message
# what was tried, and rejected Each of the plausible prompt-side explanations was tested and changed nothing at all — 0/10 clean every time. # what worked What fixes it is one sentence in the schema, and the control says which part does the work. # what it does not fix It does not fix everything ... so the repair layer stays.
- Measure the failure rate before changing anything.
- Reject fixes that do not move the number, even plausible ones.
- Stay on the new model rather than switch providers, because switching changes the product's writing voice and its cost model.
- Repair broken replies for free instead of paying for retries.
- Distrust a fix seen at 15 samples until it holds at 25.
- Wrote the probe, replay and repair code, and their tests.
- Ran each experiment and reported the counts.
- Diagnosed the reply shapes it could and could not decode.
- Wrote the commit messages that record what was tried.
Problems nothing reported.
The most valuable part of the job is noticing what is wrong when every log says fine. Four real examples:
Scheduled jobs that never ran
Property Station: the host registered none of three scheduled jobs, with no error anywhere. Found because the newest notification was a week old. Now one schedule, a run log, and a test that fails the build if it happens again.
Every signup refused
Thumbnail Bench: ads brought clicks but zero signups. Analytics showed no one ever reached the signup step, which traced to a security setting configured without its pair.
A reviewer that never says no
PublishBench: a cheaper grading model looked fine on average, but it never rejected a single article, including one with a known flaw. The grader stayed on the stronger model.
Audio at the wrong speed
Infinity Dictate: an independent multi-agent code review found that imported audio files were fed to the transcriber at the wrong sample rate. Fixed with 45 other findings from the same review.
How I decide something works.
Rules I keep because breaking them cost me something.
- Run the real build. Type checks and tests passing is not the same as the deploy's own build passing.
- Look at the screen. Reading the page code is not verifying a user interface. I take a screenshot.
- A green log line is not the outcome. Open the thing that was supposed to be produced.
- A prompt change is a hypothesis. Measure it over several runs before calling it a fix.
- Check the platform that serves users. Green CI does not mean the live deploy succeeded.
How the AI work is organised.
Claude Code
- Custom skills
- 20+ reusable instruction sets for recurring jobs, from case-study builds to video pipelines.
- Subagents
- Parallel agents for research and review, each given its own files so they cannot overwrite each other.
- MCP servers
- Connections to email, browsers, hosting and data tools, with read-only limits where data is sensitive.
Safety rails
- Pull requests + CI
- Small changes, each with tests, type checks and a build before merge.
- Read-only by default
- Live business systems such as the real-estate MLS are only ever read, never changed.
- Spending limits
- Every paid AI call is costed, and products have switches to stop paid calls fast.
What I do not claim
- Hand-written code
- I do not write code by hand, and I do not pretend to. AI coding agents write it.
- How I judge it instead
- By evidence: tests, builds, measurements, independent reviews and the live result.
Let's talk about
what you need built.
Open to AI roles: full-time, contract or consulting. I build AI products, internal tools and automations with AI coding agents.