domains-distillery is a research crawler operated by Validus AI, the threat intelligence lab of Exaforce, Inc. It requests only the root page of a domain in order to classify what the site is, which the operator compares to the defensive scanning done by VirusTotal, urlscan.io and Google Safe Browsing, and it states the crawler does not attempt to log in or circumvent CAPTCHAs. The page at crawler.validus.ai documents the user agent, honors a robots.txt rule for the token domains-distillery, offers an email opt out, and names two static egress hosts, scan-egress-1.validus.ai and scan-egress-2.validus.ai; every source address seen in our own traffic forward-confirms into that zone.

At a glance

Operator: Exaforce, Inc.
Type: Security

How Centinel checks it

User agent: The request calls itself this crawler. Anyone can send the same string.
Reverse DNS: The client address resolves to a hostname the operator controls, and that hostname resolves back to the same address.

A request that passes one of these checks is domains-distillery itself. One that only matches the user agent is reported as unverified. The match tokens, verification domains, and address feeds are not published here.

Allowing or blocking it

The crawler object in the /validate response sets access_allowed to true only for a verified source that your tenant allowlists. A policy rule can allow or block this crawler by its category, Security.

Back to the crawler catalog