AI builds the product right. Who checks it's the right product?
A playtester played one round of my game and changed it with one sentence of feedback. The code was fine. The process was rigorous. The plan itself was wrong.
Category:
DevelopingAuthor:
Steffen M. FureliPublished:
A while back, my colleague Claes Gill wrote that experience decides whether AI-generated code should exist. That got me thinking. What happens when the code is fine and the plan is not? Rebuilding my old game prototypes with AI taught me that a disciplined AI workflow is excellent at building the thing right. It does nothing to check that you are building the right thing, and the faster it builds, the more you stack on assumptions nobody has tested. Imagine pushing the pedal to the metal, but the driver is asleep at the wheel. You’re getting nowhere fast.
The game
The game in question is simple. You steer a character swinging a yo-yo around itself. The string cuts vegetables, but every hit makes the yo-yo bounce and lose speed and string length. You fight while trying to keep the yo-yo’s spin alive.
The spin rule was mine. I wrote it by hand in the original prototype, long before any AI touched the project. Between assignments, I picked the prototype up again for a simple test: could an AI agent move it to another engine? It could. So I kept going and let the agent build the game further, one well-defined task at a time, on top of my systems.
Building it right
AI does solid work on a clear, specific task and messy work on a vague wish. The serious answer to that is spec-driven development: a written specification becomes the source of truth the agent builds from. I work this way on larger projects, with documents meant to remove every ambiguity before an agent writes a line of code. On the yo-yo game there was no spec document. My own code was the source of truth. Either way, the game came together fast and did what I asked.
Notice the phrase “source of truth”, though. My beliefs and assumptions go into the spec, or in my case into my own code, and the agent treats them as true. When the agent follows the spec, it amplifies my assumptions, right or wrong. And it couldn’t have caught this one: it can’t experience the game, and language models tend to agree with the user anyway.
Software engineering has a classic distinction for this, usually credited to Barry Boehm. Verification asks whether we are building the product right. Validation asks whether we are building the right product. In my workflow, specs, tests, task breakdowns and code review all pointed at verification. AI made every one of them faster and better. None of them was checking validation.
One sentence
Then I handed the game to a playtester. Remember the rule: every vegetable you cut cost the yo-yo speed, so the harder you fought, the closer you came to losing. After one round he said: “What if hitting enemies increased the spin? It might feel better to earn something by playing aggressively than to be punished for it.”
That one sentence flipped the core loop. Hitting enemies went from a cost to a reward. The game stopped being about defending what you have and became about earning it. It was much more fun, and everything I’ve built since stands on the flipped loop.
The fix took the agent a couple of minutes. The systems were already there, so it mostly flipped numbers. Changing the game was trivial. Seeing that it needed changing was not.
Why I didn’t see it
I’d like to blame the AI, but the spin rule was mine from the start. My original prototype was a programming challenge: I tested whether the code ran well and the concept worked, not whether the game was fun. Agents and LLMs aren’t that good at fun. When the agent built the actual game, the rule came along untested. The person who designs a rule is often the last to feel what’s wrong with it. What the AI workflow changed was how much got built on top of the rule, how fast, and how sure I felt while it happened.
With an agent doing the building, my job became setting goals, handing over tasks and checking that each one was done as planned. Checking against the plan only tells you whether the plan was followed. The spin rule was the foundation every task stood on, so no task ever asked whether it was right. I need to have the taste, as the hyperlogical AI doesn’t have it any more than Spock.
The loss of judgment through automation isn’t a new problem. In 1983, the psychologist Lisanne Bainbridge described the ironies of automation in industrial control rooms. Automate the work, and the human is left with the hardest part: watching whether the system is doing the right thing, and stepping in when it isn’t. At the same time, automation removes the hands-on practice that builds the judgment for that. It is a paper about process plants, and it describes my workflow uncomfortably well. I’ve become a software factory manager.
Knowing better doesn’t protect you, either. Game designers tell a classic story about this exact mistake. In World of Warcraft’s beta, a rest system cut experience points for players who played too long. Players hated it, and Blizzard reworked it into the rested bonus, which rewards you for coming back. “Reward, don’t punish” is about as basic as game design gets. I’ve taught that story myself, and I still made the same mistake in my own game. Knowing the principle didn’t help when I was looking at the plan instead of playing the game.
What I do differently now
AI makes change cheap, but it doesn’t make the need for change visible. My bottleneck was a real person playing the game. So I’ve changed how I work.
- I mark the load-bearing assumptions. Every spec now names the few assumptions everything else depends on, and treats them as hypotheses, not requirements.
- I shake up the foundations on purpose. When a change costs two minutes, trying a different take on a core assumption and playing both is cheaper than defending it.
- I play it, and get it played. Reviewing the agent’s work means using it, not only reading the diff. The hours AI saves on building go to getting the build in front of someone who didn’t write the spec.
None of this is specific to games. Swap the yo-yo for a checkout flow, an onboarding sequence or a pricing rule. Every product has a spin rule: an assumption the rest is built on, which nobody on the team is in a position to question.
Why experience matters more, not less
Bainbridge’s final irony is the one I keep coming back to: the most reliable automated systems may need the greatest investment in the people operating them. It’s the best argument I know against the idea that AI makes experienced people less necessary. With an agent doing the building, the builder’s real work is deciding what is worth building.
Claes wrote that experience decides whether AI-generated code should exist. My story adds a less flattering point: it was a playtester, not my experience, that found the problem. What experience gives you is knowing that you won’t see it yourself, and building the outside view into the work before the agent has stacked a whole game on top of your mistake.
I’ll keep using AI, heavily. But the person who wrote the plan is the one least likely to question it. In my case, it took someone else one sentence.