Claude
-
Robots.txt Examples: Copy-Ready Rules for AI Crawlers
Copy-paste robots.txt examples for allowing AI search, blocking AI training, blocking all AI crawlers, disallowing folders and more, with what each rule does.
-
AI Crawler List 2026: Every AI User Agent (Updated Monthly)
A living list of AI crawlers and user agents from OpenAI, Anthropic, Google, Perplexity, Meta, Apple, Amazon and more, with what each does and how to control it.
-
ClaudeBot: What It Crawls and How to Allow or Block It
ClaudeBot is Anthropic’s training crawler. Learn how it differs from Claude-SearchBot and Claude-User, its user agent, and robots.txt rules to allow or block it.
-
Best AI Visibility Tools in 2026 (Free and Paid, Compared)
Compare the best AI visibility tools in 2026, free and paid: Profound, Peec AI, Otterly, Semrush, Ahrefs Brand Radar and more, with pricing, engines tracked and who each one fits.