Agent directory / CCBot

CCBot

AI · TRAINING CRAWL AI SURFACE

Observed live across sites Botfall meters — measured, not copied from docs

Last seen2 days ago

First seen59 days ago

Fetches · 24h0

Fetches · 30d299

Sites seen on3

Identity verified no published ranges

Networks observed AMAZON-AES - Amazon.com, Inc. · EUGameHost-LTD EUGAMEHOST LTD · GOOGLE-CLOUD-PLATFORM - Google LLC · HOSTODO - Hostodo · INFRALY-LLC - Infraly, LLC · NextGenWebs-NL NextGenWebs, S.L.

Operator	Common Crawl
Purpose	AI · training crawl
Identifies as	User-agent substring: `ccbot`

What the operator publishes

Their documentation, not our measurements — re-checked automatically every day.

robots.txt stance	Operator documents that it honors robots.txt
Operator documentation	https://commoncrawl.org/ccbot link checked 2026-09-30

Provenance

What establishes this record, and when it was last checked — no speculative entries.

Observed in live storefront telemetry (classifier lineage)checked 2026-07-30

Want these numbers for your own site — which agents read it, what they took, and what it cost? Get started free →