Home › Knowledge base › AI crawlers in detail

PerplexityBot and Perplexity-User

Perplexity shows several sources with title and domain for every answer. Without access for its bots, you don't appear there.

PerplexityBot and Perplexity-User

PerplexityBot builds the search index. Perplexity-User fetches a page live when a user's question requires it. Both need read access so your domain can appear as a source.

Allowing them

One block each for "User-agent: PerplexityBot" and "User-agent: Perplexity-User" with "Allow: /". The content must also be in the HTML without JavaScript — Perplexity renders only to a limited extent.

The debate about compliance

Perplexity has been criticised for bypassing robots.txt rules. A robots.txt block is still the right first step; if you want to hard-stop access, add a server or WAF rule on the user agent.

Testing visibility

citeglass checks whether PerplexityBot and Perplexity-User are allowed to load your page per robots.txt and whether your server actually delivers content to them.

Check your own site

citeglass shows in about 30 seconds where your website is unreadable for AI systems — free and without login.

Read on

• ClaudeBot and the Anthropic crawlers
• Visibility in Perplexity
• Controlling AI crawlers in robots.txt

General information, not legal advice. GEO is a young field — recommendations may change.