How many tokens did it take for AI to screw in a light bulb?
The honest answer to whether AI actually helps — and what it costs when it does.
I have been building Don’t Know Jack with Claude and X’s assistance for the past several weeks.
Not vibe coding. Not “generate me a SaaS.” Actual daily engineering sessions - Lambda functions, DynamoDB schemas, API Gateway wiring, CloudFormation, email pipelines, a Stripe web hook, a character named Bart, and a website that has been rewritten approximately fourteen times.
The result is a working product. Daily emails going out to subscribers. A pipeline that generates questions from a master list every night at 2am. Emails for edits and approval every morning, and then off to the two tiers of subscribers at 7am and 11am.
Bart’s Week in Review pulls live AWS and Azure RSS feeds every Tuesday for review and approvals. The Real World File happens every Monday pulling use cases and suggestions for edits and approvals. Ask Jack answers cloud questions in plain english on demand.
It works. I am not a developer. So how many tokens did it take?
The entire Don’t Know Jack build — weeks of sessions, hundreds of exchanges - probably consumed 15-25 million tokens in total.
At API pricing that is $45-75 worth of compute. The product it produced has a running infrastructure cost of under $5/month at current scale.
There were sessions that produced a working Lambda function in twenty minutes. There were sessions where the same Lambda failed six times in a row because of an em dash in a string that broke YAML, a missing optional chaining operator, a DynamoDB permission that was not on the IAM role, and a Node.js version that did not support nullish coalescing.
There were sessions where I pasted a CloudWatch log and got the exact fix in one response. There were sessions where I pasted the same error four times before I found that the S3 upload path was ~/Downloads/lambda/ when the file was actually in ~/Downloads/. Claude simply changed the path location because it did.
Claude did not always know the answer. It sometimes knew a confident wrong answer, which is worse. It sometimes fixed one thing and broke another. It always, eventually, got there.
What AI is actually good at in a build like this
Writing boilerplate that would take an hour to look up in thirty seconds. Every Lambda follows the same pattern: get a secret from SSM, query DynamoDB, call an API, send an email via SES. Once the pattern exists, variations take minutes not hours.
Debugging with context. Paste the error, paste the relevant code, get a diagnosis that accounts for both. Better than Stack Overflow. But not as good as a senior engineer sitting next to you.
Remembering what you built. The memory system means the second session knows what the first session decided. The API Gateway endpoint from three weeks ago is still in context when you need to add a route.
Helping create the things that are not code. The Bart character. The email copy to approve. The FAQ. The privacy policy. The LinkedIn banner. The tagline “Until the meetings that used to make no sense suddenly do.” That line came out of a session where I described what the product does and asked what the most honest version of that was.
What AI is not good at
Knowing when it is wrong. The confidence is constant. The accuracy is not. The skill you develop fastest when building with AI is learning to verify before deploying. node --check before every Lambda. aws cloudformation validate-template before every stack update.
Do not trust the output. Trust the testing.
Fixing problems it introduced two sessions ago. Context windows end. Something that worked last Tuesday may have been quietly broken by something that happened on Wednesday. Claude does not remember what it does not remember. And it doesn’t answer until you ask.
Sequential dependencies. “Do this, then this, then this” should be simple. It is not always. The order of AWS commands matters. IAM permissions propagate with a delay. Lambda updates show LastUpdateStatus: InProgress for thirty seconds after deployment. Claude knows all of this in theory. In practice it sometimes tells you to test before the deployment has finished activating.
The light bulb
Back to the question. How many tokens did it take to screw in the light bulb? 25 million tokens or so.
Enough to build something that works. More than I expected. Fewer than hiring a developer would have cost.
The bulb is in. The light is on. There are fourteen previous versions of the light socket that did not work, three IAM permission errors that cost a few minutes, and one em dash that broke a CloudFormation a few times.
The honest answer to whether AI helps is: yes, with conditions. You have to stay in the room. You have to read the output. You have to know enough to recognize when the confidence is misplaced. You have to run the test. You can’t run out of espresso. Ever.
Claude and X did not build Don’t Know Jack. I built Don’t Know Jack with Claude and X. The distinction matters.