Short answer
Claude-SearchBot is Anthropic's crawler for search quality and Claude search results.
AI crawler guide
Claude-SearchBot: set robots.txt correctly
Claude-SearchBot is Anthropic's crawler for search quality and Claude search results.
Crawler facts
Facts about Claude-SearchBot
User-agent token
Claude-SearchBot
Operator
Anthropic
KI-Console group
Live fetches
Usually appears when a user-triggered AI fetch or citation path reaches the page.
robots.txt
Yes, documented
Documentation
Open source documentationWhat is Claude-SearchBot?
Claude-SearchBot crawls online content to make Claude search answers more relevant and accurate. It is not the ClaudeBot training crawler.
Purpose and SEO meaning
The KI-Console group is live fetch. Bot-specifically, this is about search indexing and answer quality for Claude search.
Visibility and privacy note
Blocking Claude-SearchBot can reduce how systems from Anthropic see, index, train on, or retrieve your public content. Allowing it improves technical access, but it never guarantees crawling, ranking, citation, or a positive AI answer.
Anthropic documents Claude-SearchBot as controllable through robots.txt. Disallowing it may reduce visibility in Claude search results.
robots.txt for Claude-SearchBot
Use the exact token Claude-SearchBot in your robots.txt. Keep private paths blocked separately if only part of the site should be available.
Allow :bot
User-agent: Claude-SearchBot
Allow: /Block :bot
User-agent: Claude-SearchBot
Disallow: /Straight answers
What skeptical website owners usually ask first
KI-Console does not sell magic visibility. It makes your website more readable for AI systems and shows evidence where it can.
What is Claude-SearchBot?
Claude-SearchBot is a crawler or robots.txt control token operated by Anthropic. The facts above show its purpose, robots.txt behavior, and exact allow or block rules.
How do I block Claude-SearchBot in robots.txt?
Use User-agent: Claude-SearchBot and Disallow: /. robots.txt is a directive for compliant crawlers, not a technical access barrier.
Does blocking Claude-SearchBot reduce AI visibility?
It can reduce visibility in systems from Anthropic if those systems can no longer crawl, index, train on, or retrieve your content. The effect depends on the bot type and product.
Check your own website
Find out which AI signals are working right now
Run the free scan first. With an account, you can verify the domain, keep history, generate files, and document crawler visits over time.