Training
ClaudeBot
Training crawler
ClaudeBot is the main Anthropic crawler associated with model training and broad public web access decisions.
Anthropic crawlers
Does Claude read your website?
Claude visibility depends on more than one token. Anthropic uses separate agents for training, live user fetches, and search-backed access, plus older tokens that still appear in logs.
Anthropic
The Claude agents to separate
ClaudeBot, Claude-User, and Claude-SearchBot describe different access patterns. Separating them helps you see whether Claude can train from, fetch, or search your public pages.
Live fetches
Claude-User
Live Claude fetch
Claude-User appears when Claude retrieves a page for a concrete user action. It is the live-read signal website owners usually want to prove.
AI search
Claude-SearchBot
Search-backed Claude access
Claude-SearchBot is tied to search and retrieval flows that can support current answers.
Legacy Anthropic tokens
Older or compatibility tokens such as anthropic-ai and Claude-Web can still matter in log analysis. KI-Console keeps them in the training group so old evidence is not lost.
anthropic-ai
Claude-Web
robots.txt guidance
Name the current Claude agents explicitly and include the legacy tokens if your policy is to allow Anthropic access. Keep private paths blocked as usual.
Claude starter rules
User-agent: ClaudeBot
Allow: /
User-agent: Claude-User
Allow: /
User-agent: Claude-SearchBot
Allow: /
User-agent: anthropic-ai
Allow: /
User-agent: Claude-Web
Allow: /
Sitemap: https://www.example.com/sitemap.xmlHow KI-Console proves visits
KI-Console detects ClaudeBot, Claude-User, Claude-SearchBot, anthropic-ai, and Claude-Web in logs and tracker reports, then groups them so live proof and training traffic stay separate.
What Claude access does not promise
Allowing Claude crawlers improves technical access but does not guarantee that Claude will crawl, cite, recommend, or rank the website.
Straight answers
What skeptical website owners usually ask first
KI-Console does not sell magic visibility. It makes your website more readable for AI systems and shows evidence where it can.
Which Claude token proves a live read?
Claude-User is the clearest live-read token because it is tied to a concrete user action. ClaudeBot is a different training signal.
Should I still care about anthropic-ai and Claude-Web?
Yes for historical logs and compatibility. They may appear less often than current tokens, but keeping them mapped avoids missing older Anthropic evidence.
Does KI-Console show Claude traffic separately?
Yes. KI-Console recognises the Claude and legacy Anthropic tokens and separates live, training, and search traffic in the crawler proof workflow.
Check your own website
Find out which AI signals are working right now
Run the free scan first. With an account, you can verify the domain, keep history, generate files, and document crawler visits over time.