After scanning your website, you'll know your score and which categories need work. This guide covers the 10 highest-impact improvements you can make, ordered by typical impact. Most can be implemented in a few hours.
1. Allow AI Bots in Your robots.txt
Impact: Very High — This is the single most common issue we see. Many websites unknowingly block AI crawlers through default CMS settings, CDN configurations (Cloudflare's managed bot blocking is a frequent culprit), or overly restrictive robots.txt rules.
Your robots.txt should explicitly allow these user agents:
User-agent: GPTBot
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: Google-Extended
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Amazonbot
Allow: /
User-agent: Applebot-Extended
Allow: /
User-agent: meta-externalagent
Allow: /
User-agent: CCBot
Allow: /
If you use Cloudflare, check Settings → Security → Bots and ensure “AI Scrapers and Crawlers” is not set to block.
2. Add Schema.org Structured Data
Impact: High — Structured data helps AI agents understand what your page is about, who published it, and how to categorize the information. At minimum, add:
- Organization — Your company name, URL, logo, contact info, social profiles
- WebSite — Site name, URL, search action
- WebPage — Page title, description, dates, publisher reference
For product pages, add Product schema with price and availability. For articles, add Article schema with author and dates. For FAQ pages, add FAQPage schema — this is particularly valuable because AI agents often use FAQ structured data when synthesizing answers.
3. Implement OpenGraph and Meta Tags
Impact: High — OpenGraph tags (og:title, og:description, og:image, og:type) provide a standardized way for AI agents to extract page metadata. Twitter Card tags (twitter:card, twitter:title) serve a similar purpose. Include both on every page.
Also add canonical URLs, article:published_time, and article:modified_time for content freshness signals.
4. Create a Comprehensive Sitemap
Impact: Medium-High — A sitemap.xml tells AI agents about every page on your site, along with last-modified dates and priority. This is especially important for larger sites where AI crawlers might not discover all pages through link following alone.
Include all important pages with accurate lastmod dates. Reference your sitemap from robots.txt.
5. Add an RSS or Atom Feed
Impact: Medium — RSS feeds provide a structured, chronological list of your content that AI agents can subscribe to. This is particularly important for content freshness signals — AI agents use feeds to discover new and updated content quickly.
Even if you don't have a traditional blog, you can create a feed for product updates, company news, or page changes.
6. Write Substantial, In-Depth Content
Impact: Very High — AI agents need content to work with. Pages with minimal text (under 500 words) are rarely cited in AI-generated answers. Aim for at least 1,000+ words of substantive content on key pages — not fluff, but genuine information, data, explanations, and analysis.
Include specific facts, numbers, and examples. AI agents prefer content that contains verifiable claims and concrete information over vague marketing copy. Add FAQ sections with real questions and detailed answers — these map directly to how AI agents structure responses.
7. Use Semantic HTML and Proper Heading Hierarchy
Impact: Medium — Use semantic HTML elements: <article>, <section>, <nav>, <header>, <footer>, <main>. Use a logical heading hierarchy (H1 → H2 → H3) without skipping levels.
This helps AI agents understand the structure and hierarchy of your content, making it easier to extract relevant sections for answering specific questions.
8. Implement .well-known/agent.json
Impact: Medium — The .well-known/agent.json file is an emerging standard that describes your website's capabilities to autonomous AI agents. It includes your name, description, API endpoints, and available actions. As AI agents become more autonomous, this file helps them understand how to interact with your service programmatically.
9. Add security.txt and Trust Signals
Impact: Low-Medium — A .well-known/security.txt file is a trust signal that shows your website follows security best practices. While not directly used by most AI agents for content selection, it contributes to the overall trust profile of your domain.
Also ensure you have clear author/organization attribution, social media links, and contact information visible on your site.
10. Build Internal Links with Context
Impact: Medium — Internal links help AI agents discover related content and understand the topical structure of your site. Don't just link from navigation menus — include contextual links within your content that connect related pages.
For example, a pricing page should link to feature descriptions. Blog posts should link to related articles and product pages. This creates a web of context that AI agents can traverse to build a comprehensive understanding of your business.
Bonus: Emerging Standards
The AI readiness landscape is evolving quickly. Keep an eye on:
- llms.txt — A proposed standard for providing LLM-friendly content summaries
- MCP (Model Context Protocol) — A protocol for AI agents to interact with web services
- A2A (Agent-to-Agent) — Standards for AI agents to communicate with each other through your service
Measure Your Progress
After implementing these changes, re-scan your website with Score for AI to see your updated score. Most sites see a 30-50 point improvement after implementing steps 1-6. For priority-ranked fixes specific to your site with code examples, check out our Full Report.