User-agent: * Allow: / # Block things you REALLY don't want indexed Disallow: /cart Disallow: /checkout Disallow: /account Disallow: /wishlist Disallow: /chat/ Disallow: /ajax/ Disallow: /s/ Disallow: /search.php # ── AI assistants and answer engines ────────────────────────────── # Explicitly welcome. These already crawl heavily (ClaudeBot, Amazonbot # and meta-externalagent together account for ~111K hits/48h) and the # catalog is exactly the kind of price/stock/spec data they answer with. # Named here so the policy is deliberate rather than inherited from the # wildcard, and so a future tightening of "User-agent: *" cannot cut # them off by accident. Same Disallow set as everyone else. User-agent: GPTBot User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: ClaudeBot User-agent: Claude-User User-agent: Claude-SearchBot User-agent: PerplexityBot User-agent: Perplexity-User User-agent: Google-Extended User-agent: Applebot User-agent: Applebot-Extended User-agent: Amazonbot User-agent: meta-externalagent User-agent: DuckAssistBot User-agent: cohere-ai User-agent: MistralAI-User Allow: / Disallow: /cart Disallow: /checkout Disallow: /account Disallow: /wishlist Disallow: /chat/ Disallow: /ajax/ Disallow: /s/ Disallow: /search.php # Plain-language summary of the business and key entry points for LLMs: # https://www.aztekcomputers.com/llms.txt Sitemap: https://www.aztekcomputers.com/sitemap/sitemap-index.xml