Cloudflare Introduces AI Audit, Helps Create More Interesting Content
Cloudflare()
Cloudflare, Inc, a leading cloud connectivity company, today announced the launch of AI Audit, a suite of tools that can help websites of any size analyze and control how their content is used by artificial intelligence (AI) models.
For the first time, website builders and content creators will be able to quickly and easily understand how AI model providers are using their content, and then control whether and how the models can access it. Additionally, Cloudflare is developing a new feature where content creators can reliably set fair prices for their content that AI companies use for model training and retrieval augmented generation (RAG).
Website owners, whether for-profit companies, media and news publications, or small personal sites, may be surprised to learn that AI bots of all kinds can scan their content thousands of times each day without the content creator knowing or being compensated, for whichcan cause massive amounts of value destruction for businesses large and small.
Even when website owners are aware of how AI bots are consuming their content, they don't have a more sophisticated way to determine which scans to allow and a simple way to take action. For society to continue to benefit from the richness and diversity of content on the Internet, content creators need tools to take back control.
“AI will drastically change online content, and we must all decide together what the future looks like,” said Matthew Prince, co-founder and CEO of Cloudflare.
"Content creators and website owners of all sizes deserve to own and control their content. Otherwise, the quality of online information will suffer or be locked exclusively behind paywalls," he said.
"With Cloudflare's global scale and infrastructure, we are confident in our ability to provide the tools and set standards for websites, publishers and content creators to provide fair control and compensation for their contributions to the Internet, while still enablingprovider of AI models for innovation," he said.
With AI Audit, Cloudflare aims to provide information to content creators and take back control so that transparent exchanges can exist between websites that want more control over their content, and AI model providers that need data sourcesnew, so that everyone benefits.
With this announcement, Cloudflare aims to help any website:
● Control AI bots automatically and for free:
AI is a rapidly evolving field, and many website owners take time to understand and analyze how AI bots impact their traffic or business. Many small sites don't have the skills or bandwidth to block AI bots manually. The ability to block all AI bots in one click puts content creators back in control.
● Leverage analytics to see how AI bots access their content:
Every site using Cloudflare now has access to analytics that can be used to understand why, when, and how often the AI model accesses their website. Website owners can now differentiate between bots – for example, text-generating bots that still list the data source they use when creating a response, versus bots that retrieve data without specifying or specifying the data source.
● Better protect their rights when negotiating with model providers:
A growing number of sites sign direct agreements with model providers to license training and content shoots in exchange for payment. Cloudflare's AI Audit tab will provide advanced analytics that can be used to understand metrics commonly used in these negotiations, such as the crawl rate of specific sections or entire pages. Cloudflare will also model terms of use that each content creator can add to their site so they can legally protect their rights.
● Setting fair prices in terms of content scanning rights and smooth transactions (under development):
Many site owners, whether future mega-corporations or high-quality individual blogs, don't have the resources, context, or expertise to negotiate the one-time deals that large publishers sign with AI model providers, and AI model providers don't have the bandwidthto do this with every site that wants to negotiate with them. In the future, even the largest content creators will benefit from Cloudflare's seamless pricing and transaction flow, making it easy for model providers to find new content to scan that they might not otherwise have access to, and making it easier for content providers totake control and get paid for the value they create.