Checked 9/4/2026 · https://consentglass.com/
The technical prerequisites for AI discoverability are met.
AI crawler matrix
For each bot: what your robots.txt allows — and what your server actually returns to the bot's user agent. Rows highlighted in red: robots allows it, but the server blocks it (usually a WAF or bot-protection rule).
| Bot | Operator | robots.txt | Server response |
|---|---|---|---|
| GPTBot | OpenAI | allowed | 200 + content |
| OAI-SearchBot | OpenAI | allowed | 200 + content |
| ChatGPT-User | OpenAI | allowed | 200 + content |
| ClaudeBot | Anthropic | allowed | 200 + content |
| Claude-SearchBot | Anthropic | allowed (via User-agent: *) | 200 + content |
| Claude-User | Anthropic | allowed (via User-agent: *) | 200 + content |
| PerplexityBot | Perplexity | allowed | 200 + content |
| Perplexity-User | Perplexity | allowed | 200 + content |
| Google-Extended | Google (Gemini-Training) | allowed | — |
| Googlebot | Google (KI-Übersichten) | allowed (via User-agent: *) | 200 + content |
| CCBot | Common Crawl (Trainingsdaten vieler LLMs) | allowed | 200 + content |
| Bytespider | ByteDance (Doubao) | allowed | 200 + content |
| Amazonbot | Amazon (Alexa/Rufus) | allowed | 200 + content |
| Applebot-Extended | Apple (Intelligence-Training) | allowed | — |
| Meta-ExternalAgent | Meta (Llama/Meta AI) | allowed | 200 + content |
How is the score calculated?
| No author or authorship signal | -4 |
| No visible or marked-up date | -3 |
| No FAQ or HowTo markup | -2 |
| Result | 91 / 100 |
Structure & machine understanding
No FAQ or HowTo markup
Question-and-answer and step-by-step content is harder for AI systems to recognise as directly citable answers without FAQPage or HowTo markup.
This is what you should do: Where the page answers real questions or gives instructions, mark it up as FAQPage or HowTo (no markup without visibly matching content).
Trust and entity signals (E-E-A-T)
No author or authorship signal
No author is recognisable (neither meta tag, rel=author nor JSON-LD Person). For "experience" and "expertise" in the E-E-A-T sense, named, traceable authorship matters.
This is what you should do: For editorial content, name an author and link an author page; also mark them up as JSON-LD "Person".
No visible or marked-up date
No publication or modification date is recognisable (neither <time>, article:published_time nor datePublished in the JSON-LD). For many questions, AI systems prefer current sources.
This is what you should do: For content with a time reference, show a date visibly and mark up datePublished / dateModified in the JSON-LD.
OK (10)
- llms.txt present and plausibly structured.
- The main content is in the HTML without JavaScript.
- Structured data (JSON-LD) is present. — Organization
- The organisation is marked up as JSON-LD.
- Exactly one H1 heading.
- A canonical URL is set.
- The sitemap is reachable.
- The page title has a sensible length.
- Fast server response time.
- No noindex — the page may be indexed.
Next steps
- No author or authorship signal. For editorial content, name an author and link an author page; also mark them up as JSON-LD "Person".
- No visible or marked-up date. For content with a time reference, show a date visibly and mark up datePublished / dateModified in the JSON-LD.
- No FAQ or HowTo markup. Where the page answers real questions or gives instructions, mark it up as FAQPage or HowTo (no markup without visibly matching content).
Weekly re-scan with an email alert on every change. In preparation — add your address.
Method & limits
Checked 9/4/2026. citeglass fetches consentglass.com and its related files (robots.txt, llms.txt, sitemap.xml) over HTTP — once as a normal browser, once per AI crawler user agent. No JavaScript is executed. Time to first byte: 195 ms.
What this is not: No rank or citation tracking, no statement about whether a model actually names you, and no check of content loaded via JavaScript. A snapshot from the perspective of one server IP.