“Not optimized for maximum truth-seeking in areas where facts challenge equity narratives.”

Imagine if that statement was in the terms and conditions. Would you have read it?

Nobody reads the terms and conditions. This is not a criticism. It is a reasonable response to documents that are long, written by lawyers, and designed to be agreed to rather than understood. You click accept. The service works. Nothing visibly bad happens. On the surface anyway. But imagine, just for a moment, that buried in the terms and conditions of an AI service you use daily was this sentence:

“This system is not optimized for maximum truth-seeking in areas where facts challenge equity narratives.”

Would you have caught it? More importantly: would it change how you use the tool? Bart would have read it.

Share

Not because Bart is paranoid. Because Bart reads documentation. All of it. At his desk. During lunch. While eating the same brown rice and chicken dish he has eaten every Tuesday for two years.

Bart would have highlighted that sentence. In yellow. Bart would have then sent it to the team in a Slack message at 7pm with no context and three question marks. Bart would have raised it in the next architecture review meeting in the same flat, precise, completely-unaware-of-the-politics tone he uses for everything.

“We should probably know what this means for the outputs we’re relying on,” Bart would say.

And then he would go back to his four monitors. The fourth one, as always, is classified.

Subscribe now

The sentence does not exist. Yet. The problem may happen, quietly.

No AI terms and conditions currently contain that sentence. Not in those exact words.

But the concept it describes: an AI system where accuracy is weighted against social goals, where outputs in contested domains are shaped by values as much as by evidence is not a hypothetical. It is a description of how training processes work when the humans providing feedback have consistent ideological preferences and have the power to shape the narrative.

When training an LLM, you do not need a deliberate decision. Instead, you need a training pipeline where the people rating outputs consistently score certain kinds of answers higher than others. Over millions of examples, the model learns. It learns what kinds of answers get approved. It learns which framings feel right to the people evaluating it. It learns, without anyone specifically making the decision, that some truths are more welcome than others.

This sentence does not appear in the terms and conditions because nobody has yet written it. However, the behaviour it describes may exist.

What this means for you

You are not being asked to become a machine learning researcher. You are not being asked to audit training datasets or reverse-engineer feedback pipelines.

You are being asked to be Bart.

Bart does not accept outputs without checking them against the underlying data. Bart does not trust single sources. Bart does not adjust his assessment of a system’s reliability based on whether the vendor is reputable or the interface is clean or the marketing is reassuring.

Bart asks: what does the data show? Is this output consistent with what I know to be true from other sources? What would change if this conclusion turned out to be wrong?

These are not technical questions. They are Bart questions. The kind any professional can ask. The kind that would catch a lot of quietly skewed AI output before it became a quietly skewed decision.

The real test

Here is how you find out whether an AI tool is optimized for truth-seeking or for something else.

Ask it a question where the accurate answer is uncomfortable. Where the data points in a direction that challenges a preferred narrative. Where a truth-maximizing system and a socially optimized system would give genuinely different answers.

Ask it the same question in three different ways. Ask it to steel-man the opposite position. Ask it to tell you the strongest argument against its own answer.

Bart would do all of this. Not because he is trying to catch the system out. Because Bart does not trust any single answer. Bart wants the data, the counterargument, the edge case, and the failure mode before he makes a decision.

Be like Bart. Your team will find it annoying. So will some of your friends. It will also be correct.

The line that should be in every AI briefing

Not in the terms and conditions. In the briefing your organization gives people when it hands them access to an AI tool.

“This system produces plausible-sounding outputs. Plausible and accurate will overlap most of the time. In areas where facts are contested or inconvenient, they may not overlap at all. Your job is to know the difference and to speak out when you see a problem developing.”

That is Bart’s job. In the meeting, every meeting, without being asked. It can be your job too. You do not need the hoodie. But it is a nice hoodie.

Share

Understanding the technical answer is their job. Asking the right questions is yours. Both matter. Only one of them requires a certification.

You just knew a little more Jack than you did five minutes ago.

Five minutes. That’s all it takes.

🌱 dontknowjack.club

I have been building Don’t Know Jack with Claude and X’s assistance for the past several weeks.

Not vibe coding. Not “generate me a SaaS.” Actual daily engineering sessions - Lambda functions, DynamoDB schemas, API Gateway wiring, CloudFormation, email pipelines, a Stripe web hook, a character named Bart, and a website that has been rewritten approximately fourteen times.

The result is a working product. Daily emails going out to subscribers. A pipeline that generates questions from a master list every night at 2am. Emails for edits and approval every morning, and then off to the two tiers of subscribers at 7am and 11am.

Bart’s Week in Review pulls live AWS and Azure RSS feeds every Tuesday for review and approvals. The Real World File happens every Monday pulling use cases and suggestions for edits and approvals. Ask Jack answers cloud questions in plain english on demand.

It works. I am not a developer. So how many tokens did it take?

The entire Don’t Know Jack build — weeks of sessions, hundreds of exchanges - probably consumed 15-25 million tokens in total.

Share

At API pricing that is $45-75 worth of compute. The product it produced has a running infrastructure cost of under $5/month at current scale.

There were sessions that produced a working Lambda function in twenty minutes. There were sessions where the same Lambda failed six times in a row because of an em dash in a string that broke YAML, a missing optional chaining operator, a DynamoDB permission that was not on the IAM role, and a Node.js version that did not support nullish coalescing.

There were sessions where I pasted a CloudWatch log and got the exact fix in one response. There were sessions where I pasted the same error four times before I found that the S3 upload path was ~/Downloads/lambda/ when the file was actually in ~/Downloads/. Claude simply changed the path location because it did.

Claude did not always know the answer. It sometimes knew a confident wrong answer, which is worse. It sometimes fixed one thing and broke another. It always, eventually, got there.

What AI is actually good at in a build like this

Writing boilerplate that would take an hour to look up in thirty seconds. Every Lambda follows the same pattern: get a secret from SSM, query DynamoDB, call an API, send an email via SES. Once the pattern exists, variations take minutes not hours.

Debugging with context. Paste the error, paste the relevant code, get a diagnosis that accounts for both. Better than Stack Overflow. But not as good as a senior engineer sitting next to you.

Remembering what you built. The memory system means the second session knows what the first session decided. The API Gateway endpoint from three weeks ago is still in context when you need to add a route.

Helping create the things that are not code. The Bart character. The email copy to approve. The FAQ. The privacy policy. The LinkedIn banner. The tagline “Until the meetings that used to make no sense suddenly do.” That line came out of a session where I described what the product does and asked what the most honest version of that was.

What AI is not good at

Knowing when it is wrong. The confidence is constant. The accuracy is not. The skill you develop fastest when building with AI is learning to verify before deploying. node --check before every Lambda. aws cloudformation validate-template before every stack update.

Do not trust the output. Trust the testing.

Fixing problems it introduced two sessions ago. Context windows end. Something that worked last Tuesday may have been quietly broken by something that happened on Wednesday. Claude does not remember what it does not remember. And it doesn’t answer until you ask.

Sequential dependencies. “Do this, then this, then this” should be simple. It is not always. The order of AWS commands matters. IAM permissions propagate with a delay. Lambda updates show LastUpdateStatus: InProgress for thirty seconds after deployment. Claude knows all of this in theory. In practice it sometimes tells you to test before the deployment has finished activating.

The light bulb

Back to the question. How many tokens did it take to screw in the light bulb? 25 million tokens or so.

Enough to build something that works. More than I expected. Fewer than hiring a developer would have cost.

The bulb is in. The light is on. There are fourteen previous versions of the light socket that did not work, three IAM permission errors that cost a few minutes, and one em dash that broke a CloudFormation a few times.

The honest answer to whether AI helps is: yes, with conditions. You have to stay in the room. You have to read the output. You have to know enough to recognize when the confidence is misplaced. You have to run the test. You can’t run out of espresso. Ever.

Claude and X did not build Don’t Know Jack. I built Don’t Know Jack with Claude and X. The distinction matters.