Centinel AnalyticaCrawlers
Crawler object schema returned by the /validate endpoint.
Overview
When /validate detects a known crawler and crawler metadata is enabled for your tenant, the response includes a crawler object. It identifies the crawler and reports whether the request meets source-IP verification and tenant allowlist requirements.
Crawler object
| Field | Type | Required | Description |
|---|---|---|---|
id | string | yes | Crawler identifier (UUID). |
name | string | yes | Human-readable name (e.g., Googlebot). |
access_allowed | boolean | yes | true only when the crawler source IP is verified and the crawler is on your allowlist. |
category | string | no | The catalog crawler type, copied unchanged. This is not the policy crawler.category value. |
rsl_category | string | no | The catalog RSL category, copied unchanged. It is independent of category and policy matching does not use it. |
When the response contains a crawler object, id, name, and access_allowed are present.
Either optional field can be absent when the crawler catalog has no value. access_allowed
identifies the crawler's standing; enforce the /validate result from the top-level decision
field.
Category metadata
The API does not publish or enforce a fixed mapping between category and rsl_category. A
crawler can return category: "LLM" and rsl_category: "ai-all". Both fields use open string
values, copied from the catalog without normalization.
The policy matcher field crawler.category is different from the API response field. Policy
normalizes the catalog crawler type into a closed set of values. It does not read
rsl_category, and it does not compare the API category value unchanged. See
Policy rules for the allowed policy values and operators.
When a catalog type has no policy normalization, a crawler.category not_in rule can match the
request. Do not assume that an unknown API category is other.