Centinel AnalyticaArquivo Web Crawler
Arquivo-web-crawler is the Portuguese web archive's bot that systematically crawls and preserves Portuguese websites for historical research, creating a comprehensive digital heritage of Portugal's web presence.
At a glance
- Operator: is the Portuguese web archive
- Type: Archiver
How Centinel checks it
- User agent: The request calls itself this crawler. Anyone can send the same string.
- IP ranges: The operator publishes the addresses it crawls from, and Centinel checks the client address against that list.
A request that passes one of these checks is Arquivo Web Crawler itself. One that only matches the user agent is reported as unverified. The match tokens, verification domains, and address feeds are not published here.
Allowing or blocking it
The crawler object in the /validate response sets access_allowed to true only for a verified source that your tenant allowlists. A policy rule can allow or block this crawler by its category, Archiver.