Skip to content
Karan Balaji / Writing
Day 8/100: Your Next User Is an Agent
← All articles
#100DaysOfAi17 min read

Day 8/100: Your Next User Is an Agent

Agents are becoming the interface between people and software. Cloudflare's data shows bots already pass humans on the web, and the agent share is the fastest growing part. Here is how agents connect to your app, why MCP is changing, and what to build so your site can serve them.

By Karan Balaji

Day 8/100: Your Next User Is an Agent

Last night I co-hosted a workshop for small business owners in Toronto. While I was still in the room, I sent a few photos from my phone to my personal agent, Muse. Minutes later those photos were published on my website, on the speaking page for that exact event, with captions, in the right order. I never opened my own website's admin panel. I never logged in. I just sent a message, the way I would text a person.

That is not a party trick. That is the new interface.

Muse is the site manager for karanbalaji.com. It connects to an agent API on my site, reads and updates my articles, manages my speaking timeline, and reviews the event invites that come in. The same agent files my business receipts into my accounting software, keeps track of invoices, compares flights when I travel, and drafts my posts. I still approve the important things. But increasingly, I do not use apps directly anymore. I ask, and my agent uses the apps for me.

This article is about why that pattern is about to become normal, what the data says, how agents actually connect to software under the hood, and what you should do if you build a website or an app.

The numbers: humans are no longer the majority of the web

Cloudflare runs one of the largest networks on the internet, so its traffic data is a good place to start. In its 2025 Year in Review, Cloudflare reported that for the full year 2025, humans made up only 43.5% of HTML page requests. Non-AI bots were 47.9%, AI bots were 4.2%, and Googlebot alone was another 4.5%.

Then, in June 2026, Cloudflare's CEO Matthew Prince shared that bots had reached 57.5% of HTML web traffic, with humans down to 42.5%. The crossover happened faster than he had predicted.

Imperva's Bad Bot Report tells the same story from a different network: automated traffic was 51% of all web traffic in 2024, the first time it rose above human traffic in a decade.

Now the honest caveat, because it matters. Bots are not agents. Most of that automated traffic is search indexing, monitoring, scraping, and bad bots. Identified AI bots were only 4.2% of HTML requests in Cloudflare's full-year 2025 data, and around 80% of AI bot activity was training crawls, not an assistant fetching a page for a person.

The agent signal is not the share. It is the growth rate. Cloudflare reported that user-action crawling, a bot fetching a page because a person asked their AI assistant a question, grew more than 15 times in 2025. Adobe reported AI-driven traffic to US retail sites up 693% year over year in the 2025 holiday season, and those visitors converted 31% better than visitors from other sources. Small base, very steep curve. That is what a platform shift looks like in its early innings.

Discovery is delegated first, payment later

Watch how people actually behave. Pew Research found that when a Google results page had an AI summary, users clicked a traditional result in only 8% of visits, compared to 15% without a summary. The agent reads the web so the human does not have to.

Surveys of shoppers point the same way. ChannelEngine found 58% of shoppers now use AI tools to research products, but only 17% are comfortable completing a purchase through AI. Contentsquare found 30% would let an AI agent complete a purchase for them.

Read those two facts together and you get the real picture for 2026: people already delegate the research, the comparison, and the shortlist to their agent. They still approve the money themselves. So the practical question for any business is simple. When your customer's agent builds the shortlist, can it actually read your site, understand your offer, and complete the task? If not, you never make the list.

How agents connect to your app: MCP and its growing pains

The connection layer everyone rallied around is MCP, the Model Context Protocol. Anthropic open-sourced it in November 2024, and it works like a USB-C port for AI: any agent can connect to any MCP server and use its tools without a custom integration. OpenAI, Microsoft, and Google all adopted it in 2025. By December 2025 there were more than 10,000 public MCP servers, and Anthropic donated the protocol to the Linux Foundation's Agentic AI Foundation.

My own site runs on exactly this pattern. Muse holds scoped access to my CMS through an agent API. When I say "add these photos to the workshop page," it reads the existing page, uploads the images, and writes the update. No browser, no clicking, no copy-pasting.

But MCP has a real problem at scale: tool overload. Every tool an agent might use has a definition that gets loaded into the model's context window before the conversation even starts. Anthropic measured a realistic five-server setup (GitHub, Slack, Sentry, Grafana, Splunk) at 58 tools and roughly 55,000 tokens of definitions before the user says a word. They have seen setups consume up to 134,000 tokens. Worse, accuracy drops as the tool list grows. In one stress test, naive tool selection across many tools scored just 13.62% accuracy. Too many tools does not just cost more. It makes the agent dumber.

Code Mode: the fix both Cloudflare and Anthropic landed on

The most interesting development this year is that Cloudflare and Anthropic independently arrived at the same fix: stop making the model call tools one at a time, and let it write code instead.

Cloudflare calls it Code Mode. Instead of exposing thousands of tools, the MCP schema is converted into a TypeScript API, and the agent gets a single tool: write code, run it in a secure sandbox, get the result. In February 2026, Cloudflare exposed its entire API, more than 2,500 endpoints, through just two tools, search and execute, consuming about 1,000 tokens. The native MCP equivalent would need 1.17 million tokens, more than the entire context window of the most advanced models. That is a 99.9% reduction.

Anthropic's version, code execution with MCP, cut their worked example from 150,000 tokens to 2,000, a 98.7% saving. Their other fix, a Tool Search Tool that loads definitions only when needed, reduced token use by 85% and raised tool-use accuracy from 49% to 74% on their internal tests.

The lesson for builders is not "MCP is dead." MCP is becoming the discovery, connection, and auth layer. What is changing is how the agent consumes it: search first, then write code, instead of carrying every tool definition in its head at all times. As a site owner, your durable investment is a well-described API with good schemas and documentation. How the agent calls it is the agent's problem, not yours.

%% alt: Flowchart showing how a personal agent completes a task: the person asks in plain language, the agent plans, connects through MCP with scoped permissions, searches tools or writes code in a sandbox, the app does the work, consequential actions pass a human approval checkpoint, and the result returns to the person.
flowchart TB
  A["You ask in plain language"] --> B["Your agent plans the task"]
  B --> C["Connects through MCP: tool discovery and scoped auth"]
  C --> D["Searches the tools it needs, or writes code in a sandbox"]
  D --> E{"Consequential action? Spend, send, or delete"}
  E -- "No" --> F["The app does the work"]
  E -- "Yes" --> G["You approve"]
  G --> F
  F --> H["Result returns to you"]

How a personal agent completes a task, from plain-language request to approved result.

Around MCP, a full stack is forming:

  • A2A (Google, April 2025) handles agent-to-agent delegation, the way MCP handles agent-to-tool. The two complement each other.
  • WebMCP (Chrome preview, February 2026) lets a web page register its own tools directly in the browser through navigator.modelContext, so the site becomes its own tool provider inside the user's session.
  • NLWeb (Microsoft, May 2025) turns a site's existing structured data into a natural-language interface, and every NLWeb instance is automatically an MCP server too.
  • Commerce protocols are racing to handle the money: OpenAI and Stripe's Agentic Commerce Protocol, Google's Universal Commerce Protocol and AP2 with cryptographically signed payment mandates, Visa's Trusted Agent Protocol, and Mastercard Agent Pay with tokens bound to a specific agent and consent.
  • Web Bot Auth, built on HTTP message signatures (RFC 9421), gives agents a cryptographic identity, so a site can admit a trusted agent acting for a known customer and still block anonymous scrapers.

One honest note on llms.txt, the file many sites added this year to guide AI crawlers: it is cheap to add, but Google has said it does not use it, and a large study found no measurable benefit for AI citations. Treat it as a small experiment, not a strategy.

The fun part: even game engines are becoming tools now

This is where it stops being about websites and gets fun. In June 2026, Epic Games shipped Unreal Engine 5.8 with an official, experimental MCP plugin. A model like Claude can now connect directly to a live Unreal project and operate the editor itself: place actors, build Blueprints, edit materials, art-direct the lighting, run automation tests. Epic's own framing is that models stop being assistants that copy and paste suggestions, and become collaborators that work inside the tool.

Unreal is the headline, not the exception. ChatGPT turned the same idea into consumer apps: Booking.com, Expedia, Spotify, Figma, Canva, and Zillow all launched as apps inside ChatGPT, built on MCP, so an agent can work with a flight, a house listing, or a playlist without the user ever opening the app. When a game engine and a travel site both expose themselves to agents in the same year, the direction is not subtle. Software is being rebuilt to be operated by agents, with humans supervising.

No API? The browser is the universal adapter

Here is the part that surprises people. A platform does not need to adopt any of this to become usable by agents. Missing agent support makes it the slower path, not a blocked path.

Google's NotebookLM is my favourite example, because I hit this myself. NotebookLM has no public API on its regular plans, so there is no clean way for an agent to call it. And yet the podcast episode for my Day 6 article was generated in NotebookLM, by my agent. It opened a browser, worked through the site signed in as me (I typed the verification codes myself when it asked), created the notebook, added the article as a source, generated the audio overview, and downloaded the finished episode. No API. No integration. Just a virtual browser, my login, and patience.

That is the general pattern now. ChatGPT Agent works on its own virtual computer and asks permission before consequential actions. Claude's computer use clicks and types from screenshots. Personal agents hold your signed-in sessions and work through websites the way a very fast, very literal assistant would. A legacy site with no API, no MCP server, and no structured data does not stop them. They drive the interface the site already has.

Two honest caveats. Browser agents are the most expensive and brittle way to connect: a site redesign can break the flow, every step costs another round of inference, and Anthropic described its own computer use as "at times cumbersome and error-prone" at launch. A logged-in agent also inherits your privileges, which is exactly why approval checkpoints and scoped sessions matter so much.

But as a bridge, it changes the strategic picture completely. "We have no API" used to mean "agents cannot use us." Now it means agents will use you the slow, fragile way, and use your agent-ready competitor the fast, reliable way. The incentive to build the real interface has never been clearer.

The crawl-to-refer problem, and why blocking is not a strategy

There is a fair objection from publishers: agents take and do not give back. Cloudflare's crawl-to-refer data is stark. In July 2025, Anthropic's crawler made about 38,065 crawls for every one referred visit, while Google's ratio was 5.4 crawls per visit. Bots do not click ads. If agents become the interface, the old web business model of free content paid by pageviews breaks.

But blocking everything is not working either. Cloudflare found in June 2025 that only 2.98% of the top million web properties actually blocked or challenged AI bots. The middle path being built right now looks like this: identify the agent, state your terms per purpose (search yes, training no, user-action yes), and charge where charging makes sense. Cloudflare's Pay Per Crawl lets a site return a price with an HTTP 402 response instead of a block. Content Signals let you write those preferences into robots.txt. The web is moving from "allow or block" to "identify, license, and price," which is exactly what you would expect when the visitor becomes software acting for a customer.

The other danger: the old web was not built for this traffic

There is a second problem, and it is less about business models and more about physics. A huge share of the web still runs on the stack of the human era. WordPress powers roughly 40% of all websites, PHP runs on about 70% of sites whose server technology can be identified, and most of those sites sit on small shared or entry-level hosting plans that were sized for human visitors.

Human visitors are gentle. They load a page, read for a while, click something, read again. A small hosting plan can only run a couple of PHP processes at the same time, and every uncached page view holds one of those processes for its full duration. A handful of humans browsing slowly never gets near the limit.

Agents and crawlers do not browse like that. They fetch systematically, in parallel, at machine speed, and they disproportionately request the expensive pages: search results, archives, version histories, and other uncached dynamic pages that humans rarely touch and caches cannot help with.

The casualties are already documented. The Wikimedia Foundation reported that its multimedia bandwidth rose 50% since January 2024, driven by AI scrapers working through 144 million files, and that bots make up at least 65% of its most expensive traffic while producing only about 35% of pageviews, because crawlers bulk-read the uncached pages humans skip. Read the Docs watched a single AI crawler download 73 TB in one month in 2024 and leave a bandwidth bill of more than $5,000; blocking AI crawlers cut its download bandwidth by 75%. Drew DeVault, who runs the code forge SourceHut, wrote in March 2025 that he spends between 20% and 100% of his week fighting LLM crawlers, with dozens of brief outages a week, driven by tens of thousands of residential IP addresses each making a single request to the heaviest pages on the site. A pattern like that looks like a DDoS attack because, mechanically, it is one. KDE's GitLab was overwhelmed by a crawler in the same period. GNOME deployed defensive software. Fedora went as far as temporarily blocking all traffic from Brazil.

And it is usually not malice. Kinsta, analysing more than 10 billion requests, describes most of this as inefficiency at scale: crawlers looping on URL variations for days, one bot sending 3.75 million add-to-cart requests in 24 hours, every one of them an uncacheable request that spins up PHP and a database session. On a large platform that is a rounding error. On a small host, it is the whole server.

%% alt: Diagram contrasting human visitors and agent fleets arriving at the same small website: humans arrive slowly and mostly read cached pages, while agents and crawlers fetch many uncached dynamic pages in parallel at machine speed, saturating the site's few PHP workers until requests queue and fail.
flowchart TB
  H["Human visitors: slow, read between clicks, mostly cached pages"] --> S["Small WordPress site: a couple of PHP workers"]
  A["Agents and crawlers: parallel, machine speed, uncached dynamic pages"] --> S
  S --> Q["Workers saturated, database hammered"]
  Q --> R["Queue grows, pages time out, site falls over"]

Two kinds of users, two completely different loads, one small server.

There is a security edge to this, too. A site that cannot tell a legitimate agent from a scraper has only two settings: block all automation, or trust all of it. Both are bad. Blocking everything locks out your customers' agents along with the scrapers. Trusting everything means anything that claims to be a browser gets in, because user-agent strings and IP addresses are easy to fake. In October 2026 the Wikimedia Foundation reported that agents it believes were OpenAI's made millions of API requests and page crawls, ran hundreds of thousands of expensive database queries that may have contributed to a partial outage, and even made edits to pages and tool configurations. When agents act through the same public interface as humans, "the agent did it" is not a defence, for them or for you.

The defences are becoming standard kit, and they rhyme with the checklist below. Caching and a CDN, so agent traffic hits cheap static copies instead of PHP. Rate limits and crawl budgets. Proof-of-work challenges like Anubis, the open-source tool Xe Iaso released in January 2025 after AI crawlers kept hammering small sites, now deployed by GNOME, FFmpeg, and UNESCO: every request costs the requester a small amount of computation, which is trivial for one agent doing a real task and painful for a crawler fetching a million pages. Lightweight markdown versions for agents, about 80% smaller than the HTML page, so serving an agent costs a fraction of serving a browser. And signed identity through Web Bot Auth, so verified agents get a fast lane while unsigned automation gets the challenges.

So the mandate for site owners is bigger than "be discoverable by AI." Your site now has to serve two kinds of users at two completely different scales: humans, who arrive slowly and read, and agents, who arrive in fleets and execute. A site built only for the first kind will spend the agent era invisible, overwhelmed, or breached. The sites that win will be agent accessible and agent hardened at the same time.

If you own a website or app, do this

  1. Expose a real API, and an MCP server on top of it. Generate the MCP surface from your OpenAPI spec instead of hand-writing wrappers that will rot.
  2. Serve content machines can read cheaply. Markdown instead of heavy HTML can cut the token cost of a page by about 80% (Cloudflare's own example went from 16,180 tokens to 3,150). Structured data with Schema.org is what agents extract reliably.
  3. Write per-purpose crawler policies. Training crawlers, search crawlers, and user-action agents are different visitors with different value to you. Treat them differently in robots.txt and in your terms.
  4. Verify agent identity. Web Bot Auth signatures let you welcome a signed, accountable agent and still stop unsigned scrapers.
  5. Scope every permission. My agent can edit my site because it holds a specific role with specific limits, and my payment flows use tokens scoped to one merchant and one amount. Give agents the least access that completes the task.
  6. Gate the consequential actions. Reads can be free. Writes, sends, deletes, and payments should ask the human. That checkpoint is not friction; it is the trust that makes people delegate more.
  7. Design fewer, better tools. Anthropic's guidance: consolidate multi-step operations into single high-level tools, name them clearly, and return meaningful context. If a human engineer cannot tell which tool to use, the agent cannot either.
  8. Measure the agent channel separately. Track verified agent visits, what they tried to do, and where they failed. Your next conversion problem may be an agent failing on your checkout, not a human abandoning it.

The risks are real, and they are design problems

None of this is risk free. Prompt injection is the structural threat: an agent that reads the web can be manipulated by what it reads, as the EchoLeak vulnerability in Microsoft 365 Copilot showed. Liability sits with the human and the company, not the model; Air Canada was held responsible in 2024 for refund information its chatbot invented. OpenAI itself pulled in-chat checkout back to discovery plus buy-on-site in March 2026 after a limited rollout, which is honest evidence that end-to-end agent purchasing is not solved yet.

The pattern that works is the one I use every day: scoped access, signed identity, human approval on anything that spends money or sends anything, and a full record of what the agent did. Trust is built in layers, the same way my workshop co-host put it on a slide this week: start with the busywork, grow into a partner.

The interface changed before. It is changing again.

The web was built for browsers. Then it was rebuilt for phones, and the businesses that adapted early took the distribution. Now it is being rebuilt for agents: software that reads, compares, books, files, and buys on behalf of a person who may never see your homepage.

Your next user might not be a person at all. It might be Muse, ChatGPT, or a personal agent like mine, arriving with a task and a budget and permission to act. The only question is whether your website can serve it.

Mine can. Can yours?

Sources

Article media viewer

The newsletter

Keep building with me.

Get the next article on design and applied AI in your inbox. One account for the blog and portfolio, with the newsletter included.

You will receive new articles after your email is verified. You can unsubscribe at any time.

Get the newsletter

Keep reading