Cloudflare, Inc. Just Changes How AI Crawlers Scrape the Internet-At-Large; Permission-Based Approach Makes Way for A New Business Model

Published: 2025-07-01 13:52:08 ET NET

Published on 07/01/2025 at 13:52

Cloudflare, Inc. announced it is now the first Internet infrastructure provider to block AI crawlers accessing content without permission or compensation, by default. Starting July 1, 2025, website owners can choose if they want AI crawlers to access their content, and decide how AI companies can use it. For decades, the Internet has operated on a simple exchange: search engines index content and direct users back to original websites, generating traffic and ad revenue for websites of all sizes.

This cycle rewards creators that produce quality content with money and a following, while helping users discover new and relevant information. That model is now broken. AI crawlers collect content like text, articles, and images to generate fans, without sending visitors to the original source - depriving content creators of revenue, and the satisfaction of knowing someone is viewing their content.

If the incentive to create original, quality content appears, society ends up losing, and the future of the Internet is at risk.