TIM SCHROEDER
Work / PublishBench
AI SaaS · Creator workspace

PublishBench

A research-to-publish workspace for YouTube creators, with an unattended AI content engine and an AI reviewer that is allowed to say no.

Live productJul–Oct 2026
PublishBench homepage showing its workspace for researching, scripting and publishing YouTube videosReal screenshot · public homepage

The problem

Making one YouTube video means research, a script, a thumbnail, titles, a description and chapters, usually spread across five or six tools that do not talk to each other.

Who it's for

YouTube creators and small channel teams who publish every week.

Where AI is used

Claude writes scripts and publishing packages, a stricter Claude model grades content and can reject it, and OpenAI and Kie models make images. Plain statistics, not AI, find outlier videos.

Built with

  • Next.js 15
  • Postgres + Drizzle
  • Railway
  • Claude
  • OpenAI
  • Stripe
  • Cloudflare R2
01 · The problem

Every step of a video lives in a different tool.

A creator researches in one app, writes in another, designs the thumbnail in a third and pastes titles and chapters into YouTube by hand. Nothing carries over, so each step starts from zero.

I wanted one workspace where the research becomes the project, the project becomes the script, and the script becomes the thumbnail and the publishing package.

  • Research results do not flow into the script or the title.
  • Titles, description, chapters and tags are written separately and often disagree with each other.
  • Finding which videos are actually outperforming a channel means doing math by hand.
  • AI writing tools will happily invent experience and statistics, which a creator then publishes under their own name.

Why custom software

No single tool chained these steps, and none had a quality gate I trusted. Building it let me control what the AI may claim, what each action costs, and what happens when a model misbehaves.

02 · The solution

One workspace, from idea to publishing package.

Each tool hands its output to the next. Behind the product, a separate content engine writes, reviews and revises articles on a schedule, and a person approves each one with a single click.

AI

Bench Optimize

One pass over a script, transcript or video produces title options, a description, chapters, tags and a checklist, all written against the same source so they agree.

Channel watchlist and outliers

Each upload is compared with the median of its own channel's recent uploads. Videos at 2× or more are flagged. This is statistics, not AI, and the page says so.

AI

Script writer

Scripts autosave with full version history, so an AI rewrite never destroys the creator's last good draft.

AI

Six-stage content engine

Discover, brief, write, illustrate, grade and revise run unattended. The only human step is reading the result and clicking publish.

AI

An AI reviewer that can say no

Seven adversarial readings score each article. A failing score on a fatal lens, such as internal consistency, vetoes the article no matter how good the rest is.

Credits tied to real cost

Every AI action is metered in credits that were re-weighted so each one costs about the same in model spend, which keeps the worst case predictable.

03 · My role

What I did. What the AI did.

Claude Code wrote the code. I decided what to build, how it should behave, which model does which job, and what counts as done.

Tim · human judgement

I owned

  • Product concept and scope: research → project → script → thumbnail → publishing package.
  • Chose a model for each job, then measured the choice (for example, Opus over Sonnet for the reviewer).
  • Set the rules the AI must not break: no invented first-hand experience, no sentence rewrites by automatic repair.
  • Pricing decisions after a cost audit: smaller allowances at the same price, rather than higher prices.
  • Launch and operation on Railway: environment setup, live checks, and reading production failures.
  • Decided when a fix was proven, and asked for the measurement that would prove it.
Claude Code · AI implementation

AI did

  • Wrote the application, database migrations and the worker jobs.
  • Wrote the test suite (176 test files, about 1,900 test cases).
  • Built probe and replay scripts that measure how often a model misbehaves.
  • Proposed diagnoses and drafted fixes, then ran the tests and the build.
  • Wrote the commit messages and pull requests that document each change.

92% of non-merge commits carry a Claude co-author line. The decisions in those commits, and the call on whether each one worked, were mine.

04 · How I built it with AI

From idea to production.

Three months from first commit to a six-stage engine that runs on its own. Dates and references come from the git history.

  1. 25 Jul 2026 · Concept

    First commit: a research, scripting and optimisation workspace

    The first version set the shape: research, projects, scripts and a publishing package in one app.

    commit 05e28b9
  2. 27 Jul 2026 · Requirements

    Cost audit before launch

    An audit showed credit costs varied more than 30× between actions and the free plan promised a refill that did not exist. I chose to right-size the allowances and fix the copy.

    migrations 0005–0007
  3. 29 Jul 2026 · Implementation

    Model upgrade, and the outage it caused

    Moving to Sonnet 5 took every generator offline for about 4.5 hours because the new model rejects settings the old one ignored. The fix moved every generator to forced tool calls.

    commits 9ea1b4c, f058f10
  4. 31 Jul 2026 · Testing

    Measure the model before trusting it

    A probe script measured how often the model sent a broken tool argument. A one-line schema hint took it from 52% to 10%.

    commits 7876f3a, 2ee1a9b
  5. 13 Aug 2026 · Architecture

    The content engine, gate first

    The evidence store, the editorial gate and the brief stage were built before the writer, so the gate could be calibrated on articles I already trusted.

    PRs #112, #114, #116
  6. 14 Aug 2026 · Iteration

    Six stages, unattended

    Discover, brief, write, illustrate, grade and revise chained into one scheduled job. One human click remains: publish.

    PRs #127–#134
  7. 23 Aug 2026 · Maintenance

    Rescue replies instead of retrying them

    A tolerant decoder now recovers malformed model replies on the first attempt, so a failed reply costs nothing extra.

    PR #178
05 · Technologies

What it runs on, and why.

Every item here is in the project's code or configuration. I chose and connected them; AI coding agents wrote the code that uses them.

AI models

Claude Sonnet 5
Default writer for scripts, packages and articles.
Claude Opus 5
The editorial gate. Measured against Sonnet and kept because Sonnet never vetoed.
Claude Haiku 4.5
Small, cheap tasks.
OpenAI gpt-image-2, embeddings, Whisper
Thumbnails and illustrations, duplicate detection, and transcription.
Kie.ai (nano-banana)
Additional image generation.

Application

Next.js 15
The web app and its API routes.
Postgres + Drizzle
All product data. 69 SQL migrations.
Custom auth
Hashed passwords, session rows and CSRF protection, with Cloudflare Turnstile on signup.
Worker service
A separate Railway service that runs the 13 scheduled jobs and the content pipeline.

Data and services

YouTube Data API
Public channel and video data, read-only.
Supadata
Video transcripts.
Exa
Web research for the content engine.
Stripe
Plans and credit packs.
Cloudflare R2
Stored images and files.

Quality and operations

Vitest
176 test files, about 1,900 test cases.
GitHub Actions
Lint, type check, tests and build on every change.
Health checks + email alerts
An admin health page whose checks also send alerts.
Umami
Privacy-friendly analytics.
06 · System architecture

How the pieces fit.

A web service handles creators, a separate worker runs scheduled jobs and the content engine, and both share one Postgres database and the same AI providers.

PublishBench architectureA web service handles creators, a separate worker runs scheduled jobs and the content engine, and both share one Postgres database and the same AI providers.requestswrite · graderead · writequeue · jobspipelineimagesrefreshwebhooksfile refsCreator workspacebrowserWeb serviceNext.js 15 · RailwayClaudeSonnet writes · Opus gradesOpenAI + Kieimages · embeddingsStripeplans · credit packsPostgresDrizzle · 69 migrationsWorker service13 jobs · content engineYouTube + Supadatapublic data · transcriptsCloudflare R2images · files
InterfaceApp / serverAI serviceData / storageExternal serviceScheduled job
Technical detail for engineers
  • Every model call is a forced tool call validated against a zod schema. A repair layer decodes common malformed shapes before any retry.
  • The web and worker are separate Railway containers, so the worker clears the web cache through a signed internal endpoint.
  • The content engine is switched by an environment variable (off, discover, draft, publish), not a feature flag, because the flag system fails open.
  • First-hand claims can only come from an evidence row whose author is taken from the signed-in session. The request has no field to set it.
  • Every AI step records its cost, which is what the credit prices were set from.
07 · Challenges

Problems worth telling you about.

Real incidents from the project's history, each with how it was found, fixed and checked.

CHALLENGE 01

A model upgrade broke the AI in two different ways.

Moving to Sonnet 5 first took every generator offline. After that fix, the model often sent tool arguments in the wrong shape, such as a list wrapped inside a string. On one prompt, every reply was malformed.

VERIFIEDThe schema hint cut malformed replies from 52% to 10%. On 25 captured bad replies, the decoder now rescues 20 on the first attempt (was 8). Live, 9 of 10 runs of a known-bad query succeeded.
Why it was hard, what I decided, how AI helped
Why it was hard
The content in the broken replies was correct; only the encoding was wrong. Some retries happened to succeed, which made a reworded query look like a fix when it was luck.
My approach
I asked for a probe that measures the failure rate instead of guessing, and for every idea to be tested at a sample size large enough to trust.
How AI helped
Claude Code wrote the probe and replay scripts, tested the candidate fixes, and wrote the tolerant decoder.
The decision
Stay on Sonnet 5 and make the bad shapes harmless, rather than switch models and change the product's writing voice.
Evidence
commits 7876f3a, 2ee1a9b · PR #178
CHALLENGE 02

Choosing the reviewer model by measurement, not price.

The editorial gate was the largest cost in the engine. A cheaper model would cut that cost by roughly ten times, but only if it still caught bad articles.

VERIFIEDSonnet vetoed 0 of 8 articles and rated the known-flawed article among its best. Opus vetoed it with six quoted findings.
Why it was hard, what I decided, how AI helped
Why it was hard
Both models produce plausible scores. The difference only shows on an article you already know is flawed.
My approach
I had both models grade the same 8 articles on all 7 lenses, including one article with a known contradiction.
How AI helped
Claude Code ran the comparison and tabulated the 56 scores per model.
The decision
Keep Opus for grading. Save money instead by fixing replies that were cut off at a token limit, which had nearly doubled the attempts.
Evidence
measured 14 Aug 2026 · PR #139
CHALLENGE 03

Making fabricated experience impossible to publish.

An AI writer will invent first-hand stories and expert claims. In a creator's name, that is a trust problem, not a style problem.

VERIFIEDThe only route that creates a first-hand record takes the author from the session; the request schema has no field for it. With an empty store, a first-hand claim cannot pass.
Why it was hard, what I decided, how AI helped
Why it was hard
A prompt that says do not invent things is a request, not a guarantee. It fails quietly.
My approach
I asked for a structural rule: a first-hand claim must point to an evidence record that a signed-in person created.
How AI helped
Claude Code built the evidence store, the claim audit and the single admin screen that can create a first-hand record.
The decision
Treat fabrication as a hard failure. Automatic repair may fix links and formatting, but it may never rewrite a sentence.
Evidence
PRs #112, #134
08 · Results

What is true today.

PublishBench is live and running in production. It has no paying customers yet, so these results are about what was built and proven, not revenue.

562
commits in 3 months
Source: git log, 2 Nov 2026
198
merged pull requests
Source: GitHub, 2 Nov 2026
~1,900
automated test cases in 176 files
Source: test files in repo
13
scheduled jobs on a separate worker
Source: src/server/jobs/schedule.ts
6
AI stages running unattended
Source: PRs #127–#134
20 / 25
malformed AI replies rescued on first try (was 8)
Source: replay test, PR #178
  • Live at publishbench.com, with separate web and worker services on Railway.
  • The content engine discovers topics, briefs, writes, illustrates, grades and revises with no one watching. A person approves each article with one click.
  • Every AI action is metered and costed, so pricing is based on measured spend.
  • Tests, type checks and a production build run on every change.
09 · Lessons

What this project taught me.

  1. A prompt is not an enforcement mechanism. If a rule has one right answer, a code check enforces it. If it needs judgement, the work fails and a person looks.
  2. Measure before claiming a fix. One change looked like a cure at 15 samples and disappeared at 25.
  3. A cheaper model is only cheaper if it still says no. Test the grader on a case where you already know the answer.
  4. Check the live result, not the log line. My first live captcha test passed only because the setting had not deployed yet; a forged token exposed it.
10 · Demo

A guided tour of the workspace.

Four real screens, in the order a creator meets them. Each step says where AI is involved and where it is not.

Guided tour · real screenshots, not a live app
  1. PublishBench public homepage
    STEP 1 OF 4

    The promise

    PublishBench covers the whole video, not one step of it: research, project, script, thumbnail and publishing package.

  2. Watchlist upload feed with two videos flagged as outliers
    STEP 2 OF 4

    Find what is working

    The watchlist compares every upload with its own channel's median and flags 2× outliers. A creator can save one as an idea or start a project from it.

    WHERE AI IS INVOLVEDNone here. This is deliberate: outlier detection is plain statistics, so it is cheap, explainable and never invents a number.
  3. Bench Optimize form with source, video, format and language selectors
    STEP 3 OF 4

    Build the publishing package

    Pick a script, transcript or video. One pass returns titles, a description, chapters, tags, a pinned comment and a checklist.

    WHERE AI IS INVOLVEDClaude writes the whole package from one source in a single forced tool call, so every part agrees. A repair layer fixes malformed replies before any retry.
  4. PublishBench pricing page with Starter, Creator and Studio plans
    STEP 4 OF 4

    Pricing set from measured cost

    Every tool is on every plan; plans differ by volume. Credit prices come from the measured AI cost of each action.

    WHERE AI IS INVOLVEDThe AI cost of every step is recorded, which is what made the credit prices possible to set.

Let's talk about
what you need built.

Open to AI roles: full-time, contract or consulting. I build AI products, internal tools and automations with AI coding agents.