Day 9/100: AI Is a Tool, Not an Authority

Day 9/100: AI Is a Tool, Not an Authority

Yesterday, Karan Balaji sat down with Ram K Pari, a friend since school who is now finishing a PhD in neural engineering, working on speech perception for elderly people. They did not plan a debate. They wanted to answer one question that keeps coming up in different forms: why does the same AI give a beginner and an expert completely different outcomes?

The conversation kept circling back to one answer. AI is a tool, not an authority. And the difference between a good outcome and a bad one is almost never the model. It is the person holding it, and whether they check its work.

AI remixes. It does not invent.

Ram still codes optimization by hand. That surprised Karan at first, until Ram explained why. When you are trying to build something genuinely new, not a remix of something that already exists, a large language model is the wrong instrument. It is trained on what has already been done. It recombines it, rephrases it, and finds clever ways around the edges of a problem. It does not have the intuition to push a field forward.

Ram gave the example of the Navier-Stokes equations. There is a well known 3Blue1Brown video on the subject, and the AI approaches that made headlines around it mostly found edge cases, weird ways of getting around the problem rather than solving the core of it. Ram's point was blunt: an approach like that reveals nothing the field can build on. If you are in a field that specific, you need hyper specific knowledge to move forward, and no prompt replaces it.

This is the part most of the public conversation misses. For the average person on an average day, remixing is enough. Most of what we do is not new. It is a website, a report, a piece of code that already exists somewhere in a different shape. AI is excellent at that. The mistake is assuming that because it remixes well, it also creates. It does not, at least not yet.

The axe and the chainsaw

Karan offered Ram an analogy during the call, and it stuck. Give an expert lumberjack an axe and a beginner a chainsaw, and ask them to cut down a tree. The expert wins. Not because the axe is better, but because he knows where to cut, at what angle, and how deep. The beginner with the chainsaw will cut a circle all the way around the trunk, and the tree will not move.

It is the same with Photoshop, which Karan has used as an analogy for years. Photoshop did not kill photographers. Photographers kept their jobs because they knew what a good photo looked like and how to direct the tool toward it. The tool made them faster. It did not make them unnecessary.

AI is the same story at a bigger scale. A designer who knows UX can direct a model to a one step workflow instead of five. A researcher who knows their domain can spot the moment a model starts confidently drifting. A beginner who cannot tell the difference gets an output that looks finished and is not.

The Dunning-Kruger trap

Ram manages masters and bachelors students, and this is where the conversation got serious. He watches students paste an entire problem into an AI tool, receive a solution, and walk away believing they understand it. They do not. They have an answer, not knowledge. The learning outcome, which was the whole point of the assignment, never happened.

A couple of model generations ago, Ram had a trick for this. He made assignments hard enough that the AI would fail to produce a perfect solution. Students had to debug why the AI went wrong, and that debugging was where the actual learning lived. He is not sure that trick still works with frontier models, but the principle behind it is more important than ever: if the tool does the thinking, the student does not.

This is the Dunning-Kruger effect in its purest form. The less you know, the more you think you know, because you cannot see the depth of what you are missing. AI makes this worse, because it hands you fluent, confident output. Fluency feels like understanding. It is not.

The one place a beginner might win

Ram did name one edge case where a beginner could theoretically outperform an expert, and it is worth naming honestly: medical diagnosis. A doctor is trained to work through likely causes first and rare ones last, in a specific order. An AI tool has no such order. It might surface a rare diagnosis for a strange set of symptoms that the standard sequence would reach last, or never.

But Ram put a large caveat on it, and Karan will repeat it louder: talk to your doctor first. Using AI for medical decisions without expert knowledge is a massive risk. The edge case is real but rare, and the statistical distribution exists for a reason. Most of the time, the common diagnosis is the right one, and the expert's priors are a feature, not a bug.

Design still needs a designer

Karan shared his own version of this from design engineering. Tell a model to improve a screen and it will do visible things: nicer fonts, a more modern look, a cleaner layout based on a design system. What it will not do on its own is real UX. It will not decide that five steps should become one step, or that a sidebar experience fits the workflow better than a multi page flow. You have to tell it that, precisely, and then you have to check whether it actually did it.

Karan has run this loop at the frontier model level. The first output is rarely directly usable by a human. You direct it, you review it, you redirect it, and only then does it become good. The work did not disappear. It changed shape, from producing to directing and verifying.

The PHP story: a million tokens, no patch

Ram's best example came from security. Recently a large PHP vulnerability made headlines, one that touched a huge share of the web. Before the patch dropped, people rushed to find the exploit with AI. Ram described users burning a million tokens in two days, and a wave of people on Twitter claiming they had found the exact exploit. They had not. Their models were exploiting older vulnerabilities, or finding tricks that only worked on their own local machines.

This is the validation problem in its most expensive form. You will always get an output. The model will tell you it did the task. The question is whether it actually followed your instructions, and whether what it produced is true. Those are two different questions, and only a human can answer them.

Ram put it well: AI is like a colleague who tells you the work is done. There is only one way to find out if it is. You check.

The bias that scares Karan most

Here is the pattern Ram and Karan both keep seeing, and it is the one that worries Karan most. When nine out of ten AI tasks go fine, people quietly stop validating the tenth. The tool feels reliable, so the habit of checking fades. Then one task lands in a critical environment, a system exposed to the internet, a piece of code that touches real users, and the skipped check is the difference between a bad day and losing the business.

That is not a hypothetical. Failing to validate AI generated code in production is a business risk, full stop. It can cost revenue, trust, and in the worst case the company itself.

Validation is also recursive, which is the part people forget. You can write validation tests with an AI agent too. But then you have to verify that the tests themselves do what you think they do. There is no point in the chain where checking becomes unnecessary. The chain just moves.

Working more, not less

One more observation from the call, because it cuts against the loudest narrative. Ram mentioned a study finding that people are working more because of AI, not less. Everyone Karan has talked to in industry says the same thing. Expectations rise to fill the new capacity. The tool lets you do more, so more is expected of you, with the same hours.

That reframes the jobs question. Bulk, middle tier coding work is genuinely at risk. But domain specific roles, the ones where the knowledge itself is the job, look safe for now. Not because AI cannot touch them, but because directing AI well requires the very expertise people feared would be replaced.

The rule to take into tomorrow

So here is the rule Karan is taking from this conversation, and the one he would hand to anyone starting out with AI:

Treat every AI output as a draft from a confident colleague. Verify it against what you actually asked for. Verify it against reality. And the more it matters, the more carefully you verify.

That habit is the difference between an expert and a beginner, and it will stay the difference no matter what the models are called next year. Ram joked that we will probably be calling it superintelligence by then. The name will change. The rule will not: the tool does the work, but you are still responsible for the outcome.