Venvo Scraper Bot

Venvo Scraper Bot discovers and indexes publicly accessible web pages to build the Venvo search index and provide search results. Venvo independently operates the crawler. Crawled content is not used to train or fine-tune AI models.

Scheduled by Venvo to discover pages and refresh its search index. The crawler accesses public pages for search indexing. It does not bypass logins or paywalls, complete transactions, or evade a website owner’s access restrictions.

Our crawling policy requires the following protections on every target website:

• Respect robots.txt. Apply rules for VenvoScraperBot and applicable wildcard rules before crawling, including after a redirect.
• Keep requests paced. Wait at least 5 seconds between requests to a host (at most 0.2 requests per second across crawler workers). Honor a longer Crawl-delay.
• Back off on errors. Honor Retry-After and reduce traffic on HTTP 429 or 503. Stop when access is denied; do not bypass challenges.
• Respect indexing preferences. Apply noindex, nofollow, nosnippet, and X-Robots-Tag directives when pages can be fetched. Do not follow links from pages marked nofollow.
• Honor owner requests. Blocklist requests are actioned within 24 hours. Request crawl exclusion or removal from the search index at bots@venvo.co.

To disallow this crawler, add the following group to your website’s /robots.txt. The stable robots token has no version number.

User-agent: VenvoScraperBot
Disallow: /

To allow crawling at a slower pace, use:

User-agent: VenvoScraperBot
Allow: /
Crawl-delay: 10

To exclude a page from search results while allowing it to be fetched, serve a noindex robots meta tag or X-Robots-Tag: noindex response header. If crawling is disallowed, the crawler cannot read a page’s indexing directives; email us to request removal of an already indexed URL.

User-Agent names can be imitated. Use a valid signature to authenticate a request; allowlisting the User-Agent alone is not proof of identity.

Venvo’s verification method is Web Bot Auth: Ed25519 HTTP Message Signatures, with public keys published at the standard key-directory URL.

Signature-Agent

"https://scraperbot.venvo.co"

Public key directory

https://scraperbot.venvo.co/.well-known/http-message-signatures-directory

Algorithm

Ed25519

Signed crawler requests include Signature-Agent, Signature-Input, and Signature. A verifier checks the signature, key, destination authority, and validity period. A valid signature identifies the operator; site access rules still apply.

Publishing these keys does not itself confer Cloudflare Verified Bot status. Cloudflare approval is a separate review. See Cloudflare’s Web Bot Auth documentation.

For crawler questions, abuse reports, site opt-outs, index removal, or security concerns, use our single contact address:

bots@venvo.co

Include your domain, affected URLs, and relevant request timestamps or logs. For opt-outs, tell us whether you want crawling stopped, existing search results removed, or both. Please omit passwords and other secrets.

Operator: Venvo · Blocklist requests are actioned within 24 hours.