Google’s dominance over the web’s traffic model is changing. For nearly three decades, Google’s search engine facilitated a business deal between content creators and publishers where creators could get paid for generating traffic through ads or subscriptions. However, with the rise of AI, this relationship is shifting.
The number of searches on Google is declining, and users are increasingly turning to AI systems like ChatGPT to answer questions. Google has also changed its interface, adding an answer box that provides answers without leaving the site, making it harder for content creators to get traffic.
According to Cloudflare, a recent AI tool called OpenAI makes it 750 times more difficult to get traffic compared to the old Google system, while another tool, Anthropic, makes it 30,000 times more difficult. This is because users are increasingly consuming derivatives rather than originals.
The problem lies in how the web rewards content creators. The old search-driven web model no longer applies, and content creators see almost no value from AI crawlers that strip-mine their content without paying them.
To address this issue, Cloudflare has launched “Content Independence Day” on July 1, blocking AI crawlers unless they pay creators for their content. This is a step towards creating a new marketplace where content creators and AI companies can come together to value content based on how much it furthers knowledge rather than just traffic.
By changing the way we score and value content, Cloudflare aims to create a better web with high-value content creation. The company believes that by doing so, they can help AI engines get better faster and potentially facilitate a new golden age of content creation.
Source: https://blog.cloudflare.com/content-independence-day-no-ai-crawl-without-compensation