Cloudflare Web Scanner is a Cloudflare crawler that checks domain liveness, content type, robots.txt directives, and other web properties to map the shape of the web. It identifies as CloudflareWebScanner/1.0. Cloudflare publishes Web Bot Auth signatures for verification; no scanner-specific exclusive IP ranges or reverse-DNS validation domains are published, so the user-agent or generic Cloudflare-network source alone is not sufficient to authenticate it.

At a glance

Operator: Cloudflare
Type: Search Engine
RSL category: search

How Centinel checks it

User agent: The request calls itself this crawler. Anyone can send the same string.

CloudflareWebScanner publishes nothing Centinel can check a source against, so a match reports the name and leaves the source unconfirmed. The match tokens, verification domains, and address feeds are not published here.

Allowing or blocking it

The crawler object in the /validate response sets access_allowed to true only for a verified source that your tenant allowlists. A policy rule can allow or block this crawler by its category, Search Engine.

Back to the crawler catalog