Home › Knowledge base › AI crawlers in detail

Amazonbot

Amazonbot collects content for Amazon's assistant services, including Alexa answers and the shopping assistant Rufus.

What the data is used for

Amazon uses the crawled content for question-and-answer features and product context. For shops and product pages, visibility here can be relevant; for pure content sites, less so.

Managing access

"User-agent: Amazonbot" with "Allow: /" or "Disallow: /". Amazonbot obeys robots.txt and identifies itself with a verifiable user agent and IP ranges.

Rate-limit instead of block

If you want to allow Amazonbot in principle but reduce the load, you can throttle the frequency via "Crawl-delay" or server configuration.

Checking

citeglass shows the applicable robots.txt rule for Amazonbot and tests whether the server delivers content to it.

Check your own site

citeglass shows in about 30 seconds where your website is unreadable for AI systems — free and without login.

Read on

• Applebot-Extended
• Controlling AI crawlers in robots.txt
• Block all AI crawlers

General information, not legal advice. GEO is a young field — recommendations may change.