Google-Extended

AI · TRAINING CRAWL AI SURFACE

Not yet observed on any site Botfall meters. This record exists because the operator documents it — when it shows up in real traffic, its vitals appear here.

Operator	Google
Purpose	AI · training crawl
Identifies as	Nothing — this entity never appears in logs. It exists only as a robots.txt token; no user-agent carries it.
robots.txt token	`Google-Extended`
Notes	robots.txt token controlling Gemini/AI training use of Googlebot-fetched content. No UA of its own — a top source of merchant confusion.

Provenance

What establishes this record, and when it was last checked — no speculative entries.

Google crawler docs (blind-spot verification)checked 2026-08-01

Want these numbers for your own site — which agents read it, what they took, and what it cost? Get started free →