TIM SCHROEDER
Work / AI Craftbook
AI Implementation Platform

AI Craftbook

A tested, sourced guide that turns "what do you want AI to do?" into a recommended setup of skills, MCP servers, automations and models.

Live siteOct 20 – Nov 1, 2026
AI Craftbook homepage with the headline 'What do you want AI to do?', a plain-language task search box and a featured AI playbook for real estate lead follow-upReal screenshot · public homepage

The problem

People know what they want AI to do but not which skill, MCP server, automation or model to use. Most directories list tools with a badge and a link, and no evidence that they work.

Who it's for

Builders using AI coding agents, business owners who want a working automation, and people learning how agents work.

Where AI is used

The public site never calls a model; its search and Finder are rules-based. AI does the work behind the pages: Claude Code built the site, agents draft and review pages from sources, and skills are tested in isolated runs scored by a blind Claude grader.

Built with

  • Next.js 16
  • Supabase
  • Vercel
  • Claude Code
  • Anthropic Batch API
  • Vercel Sandbox
  • Resend
  • GitHub Actions
01 · The problem

Plenty of AI tools, little proof

Agent skills and MCP servers appeared faster than anyone could check them. A builder who wants a working setup has to guess which tool fits, how to install it, and whether it is safe.

AI Craftbook answers one question: what do you want AI to do, and what is the best way to make it happen? Every page is sourced and dated. Every page links to the other pieces you need to finish the job.

  • Directories show a title, a one-liner and a "verified" badge with no evidence behind it.
  • Third-party skills and MCP servers can carry hidden instructions aimed at the agent that loads them.
  • Pages go stale fast as tools, prices and models change week to week.
  • Nobody shows whether a skill does better than the same agent with no skill at all.

Why custom software

No existing directory stored evidence per record, linked tasks to tools, or blocked unreviewed third-party text. That needed its own database, publishing gate and test lab.

02 · The solution

One database, two front doors

Visitors start from a task or from a resource type. Both read the same Supabase database, and every record carries its source, its check dates and its verification level.

Plain-language task search

Describe a job in plain words and get an AI playbook. It compares the ways to do it, recommends one, and links the guide, automation, skills, MCP servers and model it uses. The matching is rules-based and never calls a model.

AI

Skill pages backed by tests

Skills are run in isolated tests with and without the skill. A blind Claude grader scores the answers, and each page shows a verdict, its limits and what was not tested.

AI

MCP server pages with security reviews

Each MCP server page carries a static security review and a verdict. For key servers, the lab ran the server in a sandbox and checked which permissions it really respects.

A publishing gate

A page is published only when it meets the written standard. Third-party skill text is shown only after a named reviewer approves it. The database enforces both rules, not just the admin screen.

AI

Ten open-source skills

Original agent skills, released under MIT on GitHub. The release script builds GitHub and the site from the same source, so both always show the same version.

Weekly freshness monitor

A scheduled job checks the upstream sources of hosted skills and listed servers, and emails an alert when one changes. A security change withdraws the affected file at once.

03 · My role

What I did. What the AI did.

Tim set the product, the evidence rules and the approval gates. Claude Code wrote the code and ran the research, tests and reviews he asked for.

Tim · human judgement

I owned

  • Defined the concept and audience: one question, "what do you want AI to do?", with task and resource-type entry points.
  • Picked the design direction from working prototypes. Asked for a real search, a stronger visual identity and the rename from "Build sheets" to "AI Playbooks"; later chose the full "Annotated Index" redesign.
  • Set the publish rule for original skills: they ship only if they beat the plain agent in a blind evaluation.
  • Set the test budgets: $200 a month for skill testing on API billing only, and $10 a month for the MCP lab.
  • Reviewed and approved third-party skill text himself before any of it went public.
  • Held back the Tavily MCP page because its keyless tool output offered agents payments, sign-ups and bonus credits.
  • Decided when to launch on the live domain and allow search indexing.
Claude Code · AI implementation

AI did

  • Wrote all application code, the 10 database migrations and the test suites.
  • Researched and drafted resource pages from public sources, then had a second agent review each batch.
  • Built and ran the evaluation harnesses: headless Claude Code runs, the Vercel Sandbox lab and the blind batch grader.
  • Ran the readiness passes: a signed-in admin test, live page checks at three widths and Lighthouse runs.
  • Diagnosed production errors from Vercel logs and drafted the fixes as pull requests.

224 of the 226 non-merge commits carry Claude's co-author line; Tim's work shows up in the decisions log, the approvals and the go/no-go calls.

04 · How I built it with AI

From idea to production.

The whole build ran in under two weeks. Each stage ended with a written report and a decision from Tim before the next one started.

  1. Oct 20, 2026 · Concept

    Research, structure and four prototypes

    Competitor research, information architecture and four working HTML design concepts, plus a first Next.js and Supabase site with an admin area.

    commit 6a8ae6c · docs/00–08
  2. Oct 21, 2026 · Harden

    Pre-launch readiness pass

    A signed-in admin test found and fixed a data leak. The pass also tightened the database gate and added a weekly monitor with email alerts.

    commit 481e418 · migration 0006 · docs/16
  3. Oct 21, 2026 · Launch

    Live on aicraftbook.com

    Moved to the live domain with indexing allowed. The same day, an independent evaluation of two original skills ran 54 scored, blind-graded runs.

    memory/log.md 2026-10-21 · docs/17
  4. Oct 22, 2026 · Open source

    Public skills repo and release flow

    Ten original skills published on GitHub under MIT. CI validates and install-tests each release.

    aicraftbook-skills commit af0c2be · PR #11
  5. Oct 23, 2026 · Evidence

    Skill and MCP evidence labs

    Skills and MCP servers were tested in Vercel Sandbox microVMs, with blind grading and an independent review for each page.

    PR #18 · PR #20 · docs/30
  6. Oct 26 – Nov 1, 2026 · Scale

    MCP library in reviewed batches

    Batches of 51, 55 and 57 new server pages, each checked by a second agent. The library grew to 220 MCP pages.

    PR #54 · PR #64 · PR #101
  7. Oct 28 – 31, 2026 · Operate

    Redesign and a production fix

    Shipped the search-first redesign. Then fixed database timeouts that caused 5xx errors on busy pages.

    PR #90 · PR #98 · PR #99
05 · Technologies

What it runs on, and why.

Every item here is in the project's code or configuration. I chose and connected them; AI coding agents wrote the code that uses them.

AI models

Claude Code
Wrote the code. Also the agent under test in every skill and MCP evaluation.
Claude Sonnet 5.5
The model inside the sandbox test runs and the blind batch grader for the evidence labs.
Claude Opus
The blind grader in the independent evaluation of the first original skills.

App

Next.js 16
App Router site, admin area and API routes, with a request guard in src/proxy.ts.
Tailwind v4 + zod
Styling and input validation.

Data

Supabase
Postgres content database with row-level security and gate triggers, Auth for editors, and a private storage bucket for security evidence.

Testing

Vercel Sandbox
Isolated microVMs for running untrusted skills and MCP servers, with network denied by default.
Anthropic Message Batches API
Runs the blind rubric grader at half price.
Vitest + Playwright
Unit tests and browser tests at desktop, tablet and phone widths.
gitleaks
Secret scan of the full git history on every CI run.

Operations

Vercel
Hosting, deploys from main, and the weekly monitor cron.
GitHub Actions
CI, a weekly monitor that opens a pull request, and daily IndexNow pings.
Resend
Newsletter with double opt-in, alert emails and inbound support mail.
Umami
Page and event analytics.
06 · System architecture

How the pieces fit.

A Next.js site on Vercel reads one Supabase database. Content reaches it only through a gated seed, and an offline lab tests the skills and servers the pages describe.

AI Craftbook architectureA Next.js site on Vercel reads one Supabase database. Content reaches it only through a gated seed, and an offline lab tests the skills and servers the pages describe.requestsevery /admin requestis_editor checkreadsconfirm emailsgated contentanswers, names removedevidence recordswithdraw / flagalertsPublic siteNext.js 16 on VercelEditor adminSupabase AuthAPI routessearch, newsletter, downloadsRequest guardsrc/proxy.tsSupabase PostgresRLS + gate triggersSeed gatescanner + gitleaksWeekly monitorVercel Cron + GitHub ActionResendopt-in, alertsEvidence labVercel Sandbox + Claude CodeBlind graderAnthropic Batch API
InterfaceApp / serverAI serviceData / storageExternal serviceScheduled job
Technical detail for engineers
  • Content lives in the repo and reaches Postgres through a seed step. The seed removes withheld text and refuses records that fail the written page standard.
  • Ten migrations define about 21 tables. Row-level security plus triggers stop editors from writing gate or security fields, and stop withdrawn text from being put back.
  • /admin is gated in src/proxy.ts before rendering: anonymous visitors are redirected to sign-in, and signed-in non-editors get a 403 with no page data.
  • The security scanner is a set of static pattern rules plus gitleaks. It reads bytes only and never runs what it scans. A clean scan means no known pattern matched, not that a file is safe.
  • Lab runs keep keys out of the test machine: the sandbox firewall injects the model key, and only the Anthropic API is reachable during skill runs.
  • Spend caps are enforced in the database (migration 0008): skill testing is refused once monthly spend nears the cap.
07 · Challenges

Problems worth telling you about.

Real incidents from the project's history, each with how it was found, fixed and checked.

CHALLENGE 01

Admin data reached non-editors

In a signed-in test, an account with no editor rights opened /admin. The screen said "Not an editor", but the page source still carried the admin board data.

VERIFIEDRe-tested every admin page as a signed-in non-editor: all answer 403 with no data. An access-check script re-runs this after any auth or RLS change.
Why it was hard, what I decided, how AI helped
Why it was hard
The page looked correct. The check lived in a shared layout, and the Next.js App Router can still stream a page's data to the browser while the layout shows a refusal.
My approach
Launch waited on a written readiness pass that ended in a go/no-go verdict for Tim. A signed-in admin test with two disposable accounts was part of it.
How AI helped
Claude Code ran the browser test, read the page source, found the leak, and moved the editor check into the request guard.
The decision
Check editor status at the request, before any rendering, and answer 403. Never rely on a layout check alone.
Evidence
commit 481e418 · docs/16 section 2 · web/src/proxy.ts
CHALLENGE 02

Database timeouts under load

Vercel flagged 5xx errors on /models/compare rising from 1.65% to 13.48%. The logs showed Supabase statement timeouts on several busy pages.

VERIFIEDMeasured reads after the change: models 2.4 MB in 0.5 s, MCP servers 12 MB in 0.9 s. The Vercel deploy for PR #98 failed; the deploy for PR #99 succeeded. The 5xx rate after the fix is not recorded.
Why it was hard, what I decided, how AI helped
Why it was hard
The first fix shared one in-flight read and served a cached copy on failure, but the next production build failed with the same timeout. The real cause was hidden: each read pulled the whole table, about 38 MB, and the Next.js data cache refuses items over 2 MB, so every new instance read the full table again.
My approach
Tim merged the two fixes as separate pull requests, and the second one only after the first one's production deploy failed.
How AI helped
Claude Code read the Vercel logs, wrote the first fix, found the 2 MB cache limit in the Vercel log, and changed list and get reads to fetch one content type at a time.
The decision
Read only the content type a page needs, and keep the stale-copy fallback so a slow read serves older data instead of an error.
Evidence
PR #98 · PR #99 · Vercel deploy statuses on both merge commits
CHALLENGE 03

Sharing third-party skills safely

The site shows and hosts skill text written by other people. A skill is a set of instructions an agent will follow, so a hidden instruction in one could steer a reader's agent.

VERIFIEDIn the admin test, a direct API attempt to approve was refused, and putting withdrawn text back through the edit form was refused by the database.
Why it was hard, what I decided, how AI helped
Why it was hard
Pattern rules miss novel injection phrasing and malicious logic that looks normal. A clean scan cannot prove a file is safe.
My approach
Tim set the rule: third-party text ships only after a named reviewer approves it, the submitter cannot approve, and any file change voids the approval. He reviewed and approved the first five himself.
How AI helped
Claude Code built the scanner, the approval flow and the database trigger that enforces it, and removes withheld text at seed time.
The decision
Tim later moved approvals from a terminal command to an admin page. He accepted that a stolen admin session could approve, and kept the trigger, hash and self-approval checks.
Evidence
docs/16 sections 1–2 · migration 0007 · memory/decisions.md 2026-10-22
08 · Results

What is true today.

A live, indexed site with a test lab behind it, built and run in under two weeks.

338
commits, Oct 20 – Nov 1 2026
Source: git log, 2 Nov 2026
91
merged pull requests
Source: gh pr list --state merged, 2 Nov 2026
54
blind-graded skill evaluation runs
Source: docs/17-independent-skill-evaluation.md
220
MCP server pages
Source: memory/log.md 2026-11-01 · PR #101
39
test files (32 unit, 7 browser)
Source: web/tests folder count, 2 Nov 2026
10
open-source original skills
Source: github.com/aicraftbook/aicraftbook-skills README
  • Live on aicraftbook.com since 21 Oct 2026, with search indexing allowed.
  • MCP Mechanic passed every file check in 15 of 15 runs and leaked no secrets, against 12 of 15 and 3 leaks for the plain agent.
  • Signed-in non-editors get a 403 with no data on every admin page.
  • Every pull request runs a full-history secret scan, the seed gate, skill lint, lint, unit tests and a production build.
09 · Lessons

What this project taught me.

  1. A screen that says "no" is not access control. In the App Router, gate the request, then check the page source, not just what the page shows.
  2. Always test against no skill. The plain model found all 17 planted flaws without Blueprint Probe, and in the first skill batch only one skill clearly beat the plain agent.
  3. A clean scan is not proof of safety. Pattern rules catch known tricks; a named person still approves anything written by someone else.
  4. Read the platform's limits before patching symptoms. The first timeout fix was reasonable, but the real cause was a 38 MB read meeting a 2 MB cache cap.
10 · Demo

See the evidence pages

Three real screenshots: the task-first homepage, an MCP server verdict built from a sandbox test, and a skill page that compares the skill with no skill.

Guided tour · real screenshots, not a live app
  1. AI Craftbook homepage with a plain-language task search box and example tasks
    STEP 1 OF 3

    Start from the job

    Describe the task in plain words, or pick an example. The matching is rules-based and only recommends published, reviewed records.

  2. GitHub MCP Server page with a 'Use with conditions' verdict card listing best use, biggest limitation and security findings
    STEP 2 OF 3

    A verdict from a real test

    The GitHub MCP Server page, 23 Oct 2026. The verdict comes from a test with two tokens on a throwaway repository. It names the biggest limitation and the security risk, not just the features.

    WHERE AI IS INVOLVEDClaude Code ran the test in a Vercel Sandbox microVM; Tim created and stored the test tokens.
  3. Teach skill page with an AI Craftbook verdict card comparing the skill with Claude Code's default behaviour
    STEP 3 OF 3

    Skill versus no skill

    The Teach skill page from a local test build, 22 Oct 2026. Its verdict compares the skill with the same agent and no skill, and says plainly what the test could not show.

    WHERE AI IS INVOLVEDAnswers were scored by a blind grader that never saw which condition produced them.

Let's talk about
what you need built.

Open to AI roles: full-time, contract or consulting. I build AI products, internal tools and automations with AI coding agents.