AI, Search and the Future of the Web

Why the Business Model Must Evolve

The web is evolving, and its future will be shaped not by a new app or platform but by a fundamental shift in its economic engine: the content and discovery business model.

Matthew Prince, CEO of Cloudflare, recently gave a compelling talk that captures a truth many in tech are beginning to confront: AI is rewriting the rules of how value flows through the internet. For the past 15 years, that value has been driven by search. Every blog post, product page, and how-to guide was part of a world built around Google's search engine. It was simple: get scraped, get found, get traffic. But that model is changing

The Changing World of Search

A decade ago, the equation was fairly balanced. For every two pages Google crawled, publishers could expect one visitor. That traffic, in turn, fueled the internet's business model, whether through advertising, subscriptions or simply brand exposure. Fast forward to today, and it now takes six pages scraped to get just one visitor. Why? Because 75% of Google queries are now answered directly on the search results page. The user never leaves. The traffic never comes.

And that's just search. The rise of generative AI platforms makes the erosion even more stark. Prince points out that for OpenAI, the ratio is about 250 pages scraped for every one visitor sent. For Anthropic, it's 6000 to one. These tools consume massive amounts of content to produce their outputs, yet the creators of that content see little or no return.

The Reward for Content Creators

This situation is not just a traffic issue. It's a threat to content creators across the web. If the reward for original content disappears, so too will the incentive to create it. And without a healthy ecosystem of writers, researchers, educators, and creatives, the quality and diversity of what we find online will deteriorate.

AI companies know this. Sam Altman and other leaders in the space understand the need to support the ecosystem that feeds their models. But, as Prince notes, no one wants to be the only one paying while everyone else takes for free. Without a broader shift, the problem compounds.

A New Business Model

We need a new business model for the web, one that recognises the value of original content and ensures its creators are compensated. This new model could take many forms: licensing agreements between AI companies and publishers, better attribution and linking mechanisms, or even new protocols that govern how machines access and use content.

At Cloudflare, where 20-30% of the web and 80% of AI companies intersect, Prince sees an opportunity to help shape this future. The company sits at a unique infrastructure, content, and AI crossroads. And while the path forward isn't fully clear, what is clear is that business as usual won't cut it.

The web can remain a vibrant, creative, and open space, but only if we evolve how it works and who gets rewarded when it does. This shift is not just about protecting the past. It's about enabling a sustainable future for the web we rely on.