In January, I asked Claude to stop being so predictable. I would eventually come to learn that there is a right way to do this and a wrong way to do this. Here's what the wrong way looks like:
When you think of an answer, ask yourself: "Is this one of the top 5 most famous examples in this category?" If yes, give yourself only a 10% chance of using it. Mentally roll a d10, and only use it on a 1.
Yes, this was actually in the prompt. Would you believe me if I told you it sort of made the questions better? Not really, though.
But I had to find a way to stop it from writing questions only about Back to the Future. Eighties movies are close to my heart, and I wanted to broaden the horizons of my wide-eyed little trivia engine to see all the fruits the decade had to offer.
Six months, 1,021 commits, and a small ocean of tea later, trivai is a live multiplayer trivia game with a 188,000-record retrieval pipeline behind it, fueling a question generation engine that never stops surprising me as I continue to improve it. Its creation story is one of trial and error: systems that stood for weeks or months before being torn down and rebuilt, each teardown leaving behind a lesson that still shapes the game today. The liberating part was learning to recognize when it's time to stop patching and rethink the approach.
It all started with a little repo called ai-trivia, which I worked on for four days before coming up with the "trivai" name and realizing the limitations of my initial approach.
The architecture could not have been simpler. You type in six categories. One API call dreams up the whole board from nothing but the category names. A fuzzy string match decides whether your answer was close enough. No database, no accounts, no server state at all. And you know what? It worked. The first commit landed on January 10, 2026, and by the end of that weekend I had a playable Jeopardy-style game.
It had a soundtrack, too: intense Who Wants to Be a Millionaire strings that made you sweat over questions you definitely knew, and classic cartoon womp-womp horns when you got one wrong. It was deeply corny and that's the way I liked it.
The questions even seemed surprisingly good. For about an hour. Then I started noticing something.

An LLM with no grounding will reach for the most famous answer to everything, every time. Ask for 70s movies, you get Taxi Driver. Ask for 60s musicals, you get The Sound of Music. Animals? Lions, tigers, and bears. Oh my! The model isn't wrong, exactly. It's just relentlessly, boringly right, and good trivia lives in the second tier of fame, on the answers you know but wouldn't have thought of first.
By day three, the prompt resembled a Rube Goldberg probability machine. I had spent most of a day writing it, and surely one more instruction would do the trick.
When you think of an answer, ask yourself: "Is this one of the top 5 most famous examples in this category?" If yes, give yourself only a 10% chance of using it - mentally roll a d10, and only use it on a 1. Otherwise, dig deeper and find something from the second or third tier of popularity - still recognizable, but not the obvious choice. For example: If generating 70s movies and you think "Taxi Driver" - that's top 5 famous, so 90% of the time pick something else like Network, Dog Day Afternoon, or Chinatown instead. Same for Sound of Music (60s), Lion King (90s), Back to the Future (80s), etc.
The imaginary dice roll you saw up top was just the start. Answers were required to begin with randomly chosen letters. Random numbers got pasted in, on the theory that seeing something different might shake loose something different.
STARTING LETTER CONSTRAINT: For entertainment categories (music, movies, TV,
books, etc.) AND nature/animal categories, you must follow these rules:
- At least 2 of the 5 answers must start with a letter from SET A: ...
This forces variety - if your first instinct doesn't match either set, find
alternatives that do.
Random seed for this generation: ${Math.floor(Math.random() * 10000)}Near the bottom, under a capitalized heading for emphasis, sat a rule spelling out that Breaking Bad is a TV show and not a movie.
MEDIUM MATTERS: Movies and TV shows are DIFFERENT categories. If the category says "movies" or "films", only use theatrical films - never TV shows, miniseries, or streaming series. If it says "TV" or "television", only use TV shows - never movies. Breaking Bad is a TV show, not a movie. The Godfather is a movie, not a TV show.
Each hack seemed to help a little, but it was nowhere close to what I was looking for. Eventually, it clicked that I had the wrong approach entirely. I was asking the model to add a level of detail that it did not have access to. If I wanted questions to be specific, personal, and interesting, I was going to have to provide the information to make them that way.
The challenge became clear. What I was really building was an "interesting information" retrieval system. That meant a database, embeddings, retrieval. A real system. So on January 14, I started a new repo.
What is the best way to start over? One of the first commits in the new repo contains the answer: "delete everything." It was as cathartic as it sounds. After a day of setup, I started working on vector search. Retrieval would be the core of the new app, not some optimization I bolted on later.
The core idea is something I started calling seeds: little structured records of facts, stored in Postgres with pgvector embeddings. A movie and its description. An animal and its weird habits. A historical event and its dates.
Now when you type a category, the pipeline embeds your text, goes hunting through the seed tables for the closest matches, grabs a few dozen candidates, and randomly samples about ten per source. Those get handed to the model with a much humbler request: here are some facts, write six questions about them.
"give yourself only a 10% chance" "mentally roll a d10" "must start with a letter from SET A" "Random seed for this generation: 4821" "Breaking Bad is a TV show, not a movie."
Reference Facts (use these as source material for questions):
1. {seed.name}: {seed.description}
Quotes: 1. "text" - speaker, source (year) [context]
Stats: 1. statistic (= value unit) - source [context]
Pairs: 1. Work A (1982) & Work B (1988) - shared director: person [blurb]
All the tricks were removed. We didn't need them anymore. If the sample happens to surface Dog Day Afternoon instead of Taxi Driver, then the question is about Dog Day Afternoon. There are still questions about the hits, but now they appear at an appropriate ratio. The randomness moved out of the model's imagination and into the database query, where it belonged.
This endeavor would grow into the biggest engineering effort of the whole project. The "interesting information" half of the system turned out to be almost the whole job. Anyone can call a model and ask for trivia questions. Not everyone has a warehouse of interesting facts ready to hand it.
Instead of requesting variety, the prompt now encodes editorial judgment about what makes a question good. As much as things had improved, though, the early-day quirks still peeked through. One of the funnier rules was born from a real failure in a real game, and the prompt still preserves the crime scene:
CRITICAL - THE ANSWER MUST NOT APPEAR IN THE QUESTION: not verbatim, not inside
a title, name, or quote the question mentions, and not as an obvious variant or
substring. The most common failure is naming the answer entity while asking for
its name. Example of a BROKEN question: "The 1988 anime film 'Akira' centers on
a secret government experiment - what single word is the name of that
experiment, also the film's title?" - the answer ("Akira") is printed right
there in the question.On several occasions I saw questions where the answer was embedded in the question itself. Then when I introduced a rule to try to prevent it, I had to add a counter-rule so the AI wouldn't be coy where it was completely unnecessary ("in the 1984 film where Soviet troops invade a Colorado town..." instead of just saying Red Dawn). There's now a whole family of rules dedicated to catching answer leakage.
There isn't really a limit when it comes to the quality of question generation. I had already spent a lot of time on it, and it occurred to me that I had to start focusing on other aspects of the project if I wanted to get something released. Multiplayer was the feature that intimidated me the most, as I had never built anything like it before. It was the beginning of February when I decided to get started.
In 48 hours, I had a quick and dirty version. I took the most direct path I could see. The game ran in the active player's browser, and their client broadcast whatever happened to everyone else over Pusher.
The limits of that design are structural. Clients that broadcast state are clients that can lie about state. Pusher client events silently fail if a dashboard toggle isn't set. There's no server-side record of what happened, so every desync becomes an archaeology dig. That's the right trade for a proof of concept, and the wrong one for a product — which made the next rewrite a question of when, not whether.
Then the commits stop for ten weeks. Work got busy, other projects took priority, and trivai went on the shelf. It also wasn't a technical wall I'd hit but a design one: the game worked, yet it was missing some key mechanic I couldn't name. I'd tried ideas on and taken them back off — wagering came and went — and rather than force it, I let it sit.
The version that went quiet was already fun to play, and that mattered more than I realized at the time. When I came back, I wasn't dragging myself back to an obligation. I was coming back to a game I missed.
The comeback commit, dated May 8: "add stealing." Apparently three months of not thinking about the problem produced an answer that three weeks of staring at it hadn't: the missing ingredient was offense.
Adding the steal mechanic is what finally broke my patience with the client-driven architecture. Steals are adversarial by design: players race to answer after someone misses, wrong guesses cost points, and everything hangs on a timing window. Every piece of that needs a referee, and "whichever browser broadcasts first" is not a referee.
It was the January lesson all over again, one layer down: stop patching, rebuild the layer. So in late May I wrote up a plan for a clean-cutover, server-authoritative rewrite, and over one very tea-fueled stretch, with Claude Code carrying a lot of the keystrokes, it landed in seven phases.
The heart of it is a pure reducer. The entire game (turns, answering, evaluation, steals, reveals) is one state machine: reduce(state, action, context) goes in, new state comes out. No network calls inside, no database, no randomness. Even the clock gets passed in.
export type EnginePhase = | "setup" // MP pre-start: questions are generated but the host hasn't dispatched `start-game` | "idle" // active player chooses a tile | "answering" // a tile is open; awaiting the active player's answer | "evaluating" // open-text answer submitted; awaiting the AI verdict | "steal_window" // MP: active player was wrong; window open for buzz-in | "steal_answering" // MP: a stealer buzzed; awaiting their answer | "steal_evaluating" // MP: stealer's answer submitted; awaiting the AI verdict | "steal_reveal" // MP: steal sequence done; awaiting the active player's Continue | "reveal"; // turn-ending direct answer resolved; awaiting the active player's Continue
And because it's pure, I can test every gnarly timing edge case on my laptop, instead of trying to reproduce it live with two phones and a prayer.
Around that reducer sits a transactional harness. Every gameplay action goes through one endpoint, which locks the game's database row, loads the state, asks the reducer whether the move is even legal, saves with a version guard, and only broadcasts after the commit sticks. If two actions hit the same game at once, the database sorts them out. Pusher got demoted from source of truth to messenger; if a broadcast gets lost, the state is already safe and clients simply refetch it.
And the clients? No longer trusted. The old relay endpoint got locked down to a short whitelist of harmless cosmetic events (typing indicators, mostly), and everything that carries actual game state now comes from the server. Clients went from being authors of the game to being renderers of it.
const ALLOWED_EVENTS = new Set<string>([ PUSHER_EVENTS.ANSWER_SUBMITTED, PUSHER_EVENTS.ANSWER_EVALUATED, PUSHER_EVENTS.PLAYER_LEFT, ]);
The trickiest part was AI answer evaluation. The model judges the open-text answers, and an API call can take a few seconds; you cannot hold a database lock while a language model sits there thinking. So evaluation happens in two phases: lock, note that we're evaluating, unlock. Ask the model with no lock held. Re-lock, make sure the game hasn't moved on without us, and either commit the verdict or toss it. The subtle bugs were all timing: at one point the steal window was expiring before anyone ever saw it, because the model thought too long.
Stealing had cracked the design wall, but it turned out to be only half the answer.
Even stealing had to earn its place first. It entered the game as a chargeable ability, a limited resource you spent like a reroll. It kept being the best part of every playtest, so it got promoted to a core rule: when someone misses a question, anyone can jump in and steal it, one attempt each, with risk as the only limiter. Steal right and the points are yours. Steal wrong and you pay the same amount out of your own score.
The other half of the answer arrived in June. Traps. On someone else's turn, you quietly arm a tile. The server tells no one but you; the secrecy is enforced by the engine itself, not just hidden in the UI. When an opponent opens that tile, it springs into a Wildcard with stacking multipliers.
The board got livelier around them too: bonus multipliers arranged in a ring pattern, hot streaks for stringing correct answers together, and a reshuffle mechanic where the 6×6 grid compacts down to 5×5, then 4×4 as questions run out, with the tile animation built by hand because the off-the-shelf animation library and my question modal could not agree on timing.
With traps in place, the game finally felt whole. What remained was the montage phase, the unglamorous work that sits between "it works" and "it's a product." Zod schemas now check every API input, every Pusher payload, every JSON column I read back out of my own database, and every AI response.
The test suite grew up too. Unit tests had covered the core logic since the early weeks, but the montage took things much further: around 574 tests across 42 files, coverage on 51 of 52 API routes, and Playwright runs that pilot two real browsers through an actual multiplayer game over actual Pusher. CI gates every merge, and it has saved me from myself more than once.
The biggest late addition was Trivia Studio. For all the work that went into the retrieval corpus, it can never cover the categories people most want to play with the people they know: the friend group's inside jokes, the family vacation, your hometown. Studio lets players build their own categories by hand and drop them onto the same board, right alongside the AI-generated ones. It shipped about a week before launch — tight, but the test suite and the server-authoritative engine were what made that call safe to make.

Studio also broke the economy, in a good way. Pricing had started as simple as it gets: one token per game — which quietly assumed every game costs the same to run. Studio made that false: a board of hand-made categories costs me almost nothing, while a board full of AI-generated ones costs real money. Rather than guess at a new number, I built the instrumentation first. Every generation call logs its token usage, and an admin dashboard aggregates cost per game, so I could measure real games and simulated ones against each candidate price. The data showed the flat rate was underpricing AI-heavy boards, so I refactored the economy to charge for what actually costs money: 4 tokens per game, plus 1 for each AI-generated category. That landed before launch, which is the whole reason I wanted the measurements — margins are something I can now watch on a chart instead of discover on a bill.
// v8: seed-fetch cold-cache mitigation - categories pre-warm Neon's // local file cache as they land on the board, and the ivfflat indexes // get REINDEXed to shed bloat. // // v9: vector fetch LIMITs cut (100/50 -> 40/30) - beyond reading fewer // pages, the smaller LIMIT keeps the planner on the ivfflat index for // mid-size category filters instead of a Bitmap Heap Scan that read // ~113MB per query (measured: GEOGRAPHY on prod). export const GENERATION_VERSION = "v9-lean-fetch";
The same instrumentation carried the last fight before launch: performance. Question generation was sometimes fast and sometimes brutally slow, and the culprit wasn't the AI calls but the vector search in front of them; the indexes had outgrown the database's cache, so a cold lookup could eat whole seconds before the AI was even invoked. The fixes were pure montage material: quietly pre-warm the cache while the player is still picking categories, rebuild the indexes after bulk updates, and fetch less so the query planner doesn't wander off the index. Every attempt got a version tag stamped onto its cost and latency rows, which turned "I think it's faster now" into an actual chart.
trivai went public on July 11, 2026, with live Stripe payments. Six months and 1,021 commits after a prompt asked Claude to roll imaginary dice.
Since then: GDPR data export and erasure, an in-app feedback system, and a whole new respect for the deploy button now that real people's money is on the line.
And the sound came back. The corny soundtrack didn't survive the rewrite, but in late July the game finally got a proper soundscape: 29 sound effects, all AI-generated from written prompts and tuned by ear against the game's real animations. It turns out describing a sound in words is its own strange adventure — the best prompt I wrote asked for a whoosh "like an owl landing" — but that story deserves a post of its own. The womp-womp horns didn't make the cut. Their spirit lives on.
Prompts are a prototype tool. Retrieval is an architecture. If you find yourself writing ever-sterner instructions to make a model act against its own instincts, that's the system telling you the knowledge belongs outside the model. No amount of rewording gets you there; a database does.
Shortcuts are loans, so know the interest rate going in. Letting the clients run the game got multiplayer working in 48 hours, and that version taught me what the game needed. But anything a client can broadcast, a client can fake, and there was no server-side record to debug against. I'd take that loan again to learn the shape of the thing — I'd just plan the rewrite into the schedule from the start.
The model is a trust boundary. Validate AI output like it's hostile user input, because structurally that's exactly what it is: text from outside your system that you're about to act on.
The dataset is the product. The engine had to be right, and getting it right took a full rewrite. But it's the 188,000 curated, embedded, cross-linked records that make the questions worth answering, and building that corpus took more sustained effort than any code in the repo.
Instrument before you optimize, and version your experiments. A one-line version constant made every generation experiment comparable against real production data.