Cloudflare is making it easy for website owners to block AI bots and scrapers from using their content with a new set of tools that provide control over how website content is used.
According to Cloudflare, AI models are typically trained on content across the web, often without the creator even knowing about it or being compensated for their creations.
The new offering, AI Audit, is now available for Cloudflare customers. It enables website owners to block AI bots in a single click and access analytics into why, when, and how often AI models are accessing their website.
Website owners will also be able to distinguish between bots that credit the source of the data and bots that give no attribution.
In addition, Cloudflare provides sample terms of use that website owners can add to their websites to even further protect their rights, the company explained.
AI Audit also provides analytics on the metrics that are often used in negotiations with AI companies to pay for using content in training, such as rate of crawling.
The company is also working on adding price setting and transaction flow capabilities to make it easier for model providers to find content to scan and content creators to be paid for that content and the value it provides. Interested site owners can join a waitlist to participate in this beta.
“AI will dramatically change content online, and we must all decide together what its future will look like,” said Matthew Prince, co-founder and CEO of Cloudflare. “Content creators and website owners of all sizes deserve to own and have control over their content. If they don’t, the quality of online information will deteriorate or be locked exclusively behind paywalls. With Cloudflare’s scale and global infrastructure, we believe we can provide the tools and set the standards to give websites, publishers, and content creators control and fair compensation for their contribution to the Internet, while still enabling AI model providers to innovate.”
