A high-stakes war has just broken out over the future of the internet. In one corner is Cloudflare, a giant of web infrastructure that acts as a gatekeeper for a huge portion of online traffic. In the other is Perplexity, a darling of the AI world, a search engine threatening to upend Google’s dominance.

The accusation is explosive: Cloudflare claims Perplexity is a bad actor, a rogue bot that ignores the internet’s oldest rules to secretly scrape data from websites that have explicitly told it to stay away. Perplexity’s response is just as fiery: it says Cloudflare is either dangerously incompetent or engaged in a publicity stunt, fundamentally misunderstanding how modern AI works.

The feud is the first major battle in a conflict that will define the next era of the web: Who gets to access online information, and who gets to decide the rules?

The Accusation: A Rogue Bot in Disguise

For decades, the internet has operated on a “gentleman’s agreement” called the robots.txt file. It’s a simple text file that website owners use to post a digital “Do Not Enter” sign for automated web crawlers or “bots.” Well-behaved bots, like Google’s, respect this sign.

In a scathing blog post, Cloudflare alleges that Perplexity is ignoring it. The company claims that when its declared bot, “PerplexityBot,” is blocked, the AI search engine switches to stealth mode, using generic browser identities and rotating IP addresses to continue crawling and gathering data in disguise.

Cloudflare says it tested this by creating brand-new, private websites with strict “no bots allowed” rules. Despite this, they found that “Perplexity was still providing detailed information regarding the exact content hosted on each of these restricted domains.” Based on this “stealth crawling behavior,” Cloudflare announced it has now de-listed Perplexity as a verified bot and is actively blocking its undeclared crawlers.

The Rebuttal: “You Don’t Understand How AI Works”

Perplexity’s response was swift, accusing Cloudflare of getting “almost everything wrong about how modern AI assistants actually work.” The company argues that it is not a traditional “bot” and that Cloudflare is misapplying old rules to new technology.

The core of their argument is the difference between a bot and a user agent. A traditional bot, like Google’s, systematically crawls billions of pages to build a massive index for later use. A user agent, Perplexity claims, acts on behalf of a real person in real-time. When you ask Perplexity a question, its AI agent fetches the necessary information from the web at that moment to answer you. It’s not stockpiling data; it’s acting as your personal research assistant.

“This is fundamentally different from traditional web crawling in which crawlers systematically visit millions of pages to build massive databases, whether anyone asked for that specific information or not,” Perplexity wrote in a detailed response. “When companies like Cloudflare mischaracterize user-driven AI assistants as malicious bots, they’re arguing that any automated tool serving users should be suspect—a position that would criminalize email clients and web browsers.”

Then came the bombshell counter-accusation. Perplexity claims Cloudflare “fundamentally misattributed 3-6M daily requests” from a third-party cloud browser service to Perplexity, calling it a “basic traffic analysis failure that’s particularly embarrassing for a company whose core business is understanding and categorizing web traffic.” Perplexity suggests this is either a “clever publicity moment” or a sign that Cloudflare is “dangerously misinformed on the basics of AI.”

Users on social media were divided. “Perplexity is just using a proxy to fetch something that’s already on the public web, to answer a user’s question. Framing it as some kind of attack is absurd. The public web should be public,” defended tech founder Andrej Radonjic. Another user was more critical: “Perplexity, pretending to be a search engine, pretending to be AI, yet neither.”

perplexity is just using a proxy to fetch something that’s already on the public web, to answer a user’s question.

framing it as some kind of attack is absurd. the public web should be public.

— Andrej (@0xdrej) August 4, 2025

Who Owns the Open Web?

This public feud lays bare the central tension of the AI era. AI startups like Perplexity need access to the vast ocean of data on the open web to function and compete with giants like Google and OpenAI. Without it, they can’t provide real-time, accurate answers. But website owners are growing increasingly wary of having their content scraped without consent or compensation to train and power these new AI models.

Cloudflare, by choosing to block Perplexity’s undeclared crawlers, has effectively appointed itself as the AI data police, making decisions about what constitutes “legitimate” web traffic. Perplexity warns this could lead to a “two-tiered internet” where access depends not on a user’s needs, but on whether their chosen AI tool has been “blessed by infrastructure controllers.”

The rules of the internet are being rewritten in real-time. The old gentleman’s agreement is breaking down, and the battle between the gatekeepers and the innovators has just begun. The outcome will determine not just the future of AI, but the future of the open web itself.