Checked 9/23/2026 · https://rehau-termopane.ro/
There are 1 critical blockers for AI discoverability — fix these first.
AI crawler matrix
For each bot: what your robots.txt allows — and what your server actually returns to the bot's user agent. Rows highlighted in red: robots allows it, but the server blocks it (usually a WAF or bot-protection rule).
| Bot | Operator | robots.txt | Server response |
|---|---|---|---|
| GPTBot | OpenAI | not named | blocked (403 / WAF) |
| OAI-SearchBot | OpenAI | not named | 200 + content |
| ChatGPT-User | OpenAI | not named | 200 + content |
| ClaudeBot | Anthropic | not named | 200 + content |
| Claude-SearchBot | Anthropic | not named | error / timeout |
| Claude-User | Anthropic | not named | 200 + content |
| PerplexityBot | Perplexity | not named | error / timeout |
| Perplexity-User | Perplexity | not named | error / timeout |
| Google-Extended | Google (Gemini-Training) | not named | — |
| Googlebot | Google (KI-Übersichten) | not named | 200 + content |
| CCBot | Common Crawl (Trainingsdaten vieler LLMs) | not named | 200 + content |
| Bytespider | ByteDance (Doubao) | not named | 200 + content |
| Amazonbot | Amazon (Alexa/Rufus) | not named | 200 + content |
| Applebot-Extended | Apple (Intelligence-Training) | not named | — |
| Meta-ExternalAgent | Meta (Llama/Meta AI) | not named | blocked (403 / WAF) |
How is the score calculated?
| Server blocks bots although robots.txt allows them | -12 |
| No llms.txt | -5 |
| No sitemap.xml found | -4 |
| Heading levels are skipped | -3 |
| No sameAs links | -3 |
| No robots.txt found | -2 |
| No FAQ or HowTo markup | -2 |
| Result | 69 / 100 |
Access for AI crawlers
Server blocks bots although robots.txt allows them
For 2 bot(s) from OpenAI, Meta (Llama/Meta AI), robots.txt says "allowed", but the server responds with 403 or a bot-protection page. Usually a WAF, firewall or CDN rule that locks the crawler out past robots.txt.
This is what you should do: Add the user agents and/or IP ranges of the desired AI crawlers to an allow list in your WAF / Cloudflare / firewall. Verify them via reverse DNS lookup, not just the user-agent string.
No robots.txt found
No valid file was reachable at /robots.txt. Crawlers then assume "everything allowed" — it works, but you have no control and no sitemap signal.
This is what you should do: Create a robots.txt in the root directory, at least with a sitemap reference and your desired allow/disallow rules.
User-agent: * Allow: / Sitemap: https://your-domain.com/sitemap.xml
Structure & machine understanding
No llms.txt
There is no file at /llms.txt. llms.txt is a short Markdown overview of your key content that some AI systems use for orientation.
This is what you should do: Create an /llms.txt: an H1 with the name, a short paragraph about the offering, and a link list to the central pages.
# Your company > One-sentence description. ## Key pages - [Product](https://your-domain.com/product): … - [Pricing](https://your-domain.com/pricing): … - [About](https://your-domain.com/about): …
Heading levels are skipped
The heading sequence skips at least one level (e.g. H2 straight to H4). This makes it harder to reconstruct the content structure automatically.
This is what you should do: Use the heading levels without gaps and in order (H1 → H2 → H3 …).
No FAQ or HowTo markup
Question-and-answer and step-by-step content is harder for AI systems to recognise as directly citable answers without FAQPage or HowTo markup.
This is what you should do: Where the page answers real questions or gives instructions, mark it up as FAQPage or HowTo (no markup without visibly matching content).
Trust and entity signals (E-E-A-T)
No sameAs links
The JSON-LD is missing "sameAs". It lets AI systems clearly map your brand or person to known entities (Wikidata, LinkedIn, industry directories).
This is what you should do: Add a "sameAs" in Organization or Person with the URLs of your official profiles and directory entries.
Technical foundation
No sitemap.xml found
No valid XML sitemap was reachable at /sitemap.xml and robots.txt names none. A sitemap helps crawlers find all relevant URLs.
This is what you should do: Generate a sitemap.xml with all indexable URLs and reference it in robots.txt.
OK (8)
- The main content is in the HTML without JavaScript.
- Structured data (JSON-LD) is present. — HomeAndConstructionBusiness, Organization, WebSite, ImageObject, WebPage, Person, Article, VideoObject
- The organisation is marked up as JSON-LD.
- Exactly one H1 heading.
- A canonical URL is set.
- The page title has a sensible length.
- An author / authorship signal is present.
- Fast server response time.
Next steps
- Server blocks bots although robots.txt allows them. Add the user agents and/or IP ranges of the desired AI crawlers to an allow list in your WAF / Cloudflare / firewall. Verify them via reverse DNS lookup, not just the user-agent string.
- No llms.txt. Create an /llms.txt: an H1 with the name, a short paragraph about the offering, and a link list to the central pages.
- No sitemap.xml found. Generate a sitemap.xml with all indexable URLs and reference it in robots.txt.
Weekly re-scan with an email alert on every change. In preparation — add your address.
Method & limits
Checked 9/23/2026. citeglass fetches rehau-termopane.ro and its related files (robots.txt, llms.txt, sitemap.xml) over HTTP — once as a normal browser, once per AI crawler user agent. No JavaScript is executed. Time to first byte: 774 ms.
What this is not: No rank or citation tracking, no statement about whether a model actually names you, and no check of content loaded via JavaScript. A snapshot from the perspective of one server IP.