Centinel AnalyticaCrawlers
Crawler object schema returned by the /validate endpoint.
Overview
When /validate detects a known crawler and crawler metadata is enabled for your tenant, the response includes a crawler object. It identifies the crawler and reports whether the request meets source-IP verification and tenant allowlist requirements.
Crawler object
| Field | Type | Required | Description |
|---|---|---|---|
id | string | yes | Crawler identifier (UUID). |
name | string | yes | Human-readable name (e.g., Googlebot). |
access_allowed | boolean | yes | true only when the crawler source IP is verified and the crawler is on your allowlist. |
category | string | no | The catalog crawler type, copied unchanged. This is not the policy crawler.category value. |
rsl_category | string | no | The catalog RSL category, copied unchanged. It is independent of category and policy matching does not use it. |
When the response contains a crawler object, id, name, and access_allowed are present.
category and rsl_category are optional metadata. Either field can be absent when the crawler
catalog has no value. Do not derive one field from the other.
access_allowed is true only when the identified crawler source IP is verified and the crawler
is allowlisted for your tenant. Use the top-level decision field to enforce the /validate
result.
Category metadata
category and rsl_category are independent catalog values. The API does not publish or enforce
a fixed mapping between them. For example, a crawler can return category: "LLM" and
rsl_category: "ai-all".
Both fields use open string values. The API copies the catalog values without normalization.
The policy matcher field crawler.category is different from the API response field. Policy
normalizes the catalog crawler type into a closed set of values. It does not read
rsl_category, and it does not compare the API category value unchanged. See
Policy rules for the allowed policy values and operators.
When a catalog type has no policy normalization, a crawler.category not_in rule can match the
request. Do not assume that an unknown API category is other.