Skip to main content

Discover answers and insights from the Anthropic team

Search results for:extension
Does Anthropic crawl data from the web, and how can site owners block the crawler?
To limit crawling activity, we support the non-standard Crawl-delay extension to robots.txt
Can I use Claude in different languages?
Claude also has extensive knowledge of common programming languages