Fundus is an open-source Python news scraper developed by the FlairNLP group at Humboldt-Universität zu Berlin and presented at ACL 2024. It crawls news publishers with handcrafted per-site parsers to extract articles for NLP research and training-corpus construction. The library is MIT-licensed, so the user agent is shared by anyone who installs it — including the original FlairNLP researchers, academic collaborators, and individual users.

At a glance

Operator: FlairNLP
Type: Research
RSL category: ai-train

How Centinel checks it

User agent: The request calls itself this crawler. Anyone can send the same string.

Fundus publishes nothing Centinel can check a source against, so a match reports the name and leaves the source unconfirmed. The match tokens, verification domains, and address feeds are not published here.

Allowing or blocking it

The crawler object in the /validate response sets access_allowed to true only for a verified source that your tenant allowlists. A policy rule can allow or block this crawler by its category, Research.

Back to the crawler catalog