Guide by AI with Milo · · 5 min read
AI visibility: how ChatGPT, Claude and co. read your website
More and more people ask an AI assistant instead of typing into a search engine. Whether your website appears in those answers depends on a handful of things you can check yourself. Milo explains them step by step.
What AI visibility means
AI assistants such as ChatGPT, Claude, Gemini, Perplexity and Copilot answer questions with content from the web. Some search the web while they answer, others draw on what they read earlier. Your website is visible to them when three things are true: they may read it, they can read it, and they understand what you offer.
Step 1: Let the right bots in
Your robots.txt tells crawlers what they may read. Many websites lock out AI assistants by accident, for example through a security plugin or a firewall rule. It helps to know two kinds of bots:
- Search and answer bots fetch pages so an assistant can answer a question right now, for example OAI-SearchBot and ChatGPT-User (OpenAI), Claude-SearchBot and Claude-User (Anthropic), PerplexityBot, and Bingbot, which Copilot relies on. If you block them, you are missing from those answers.
- Training crawlers collect text to train future models, for example GPTBot, ClaudeBot and CCBot, plus the Google-Extended token. Blocking them is a legitimate choice and does not hide you from search answers.
Allowing a bot in robots.txt is only half the job: your server has to let it through as well. A firewall that answers with an error looks like a closed door.
Step 2: Put your content in the HTML
Most AI crawlers do not run JavaScript. Text that only appears once a script has run stays invisible to them, even if every visitor sees it. A quick test: open your page, choose «View page source» in your browser and search for an important sentence. If it is not there, AI crawlers will not find it either.
Step 3: Tell machines who you are
Structured data in the schema.org format describes your business in a way machines understand without guessing: an Organization or LocalBusiness with name, logo, address, opening hours, area served and services. Questions and answers can be marked up as FAQPage, guides as Article, team members as Person. The data goes into a JSON-LD block in the HTML and has to match what visitors see on the page.
Step 4: Link the pages that build trust
AI assistants, like people, look for signs of who is behind a website. Pages about your team, your services, your references and your know-how, linked from the menu, make those signs easy to find.
Step 5: Consider an llms.txt
An llms.txt is a short text file at the root of your website that sums up what you offer and links your most important pages. The convention is young, and so far there is little evidence that AI assistants use it. It costs little, so treat it as a small bonus, not a must.
Check your website with Milo
Milo’s free website check measures all of this in the area «AI visibility», together with six more areas from loading speed to legal basics. It takes a few seconds, with no sign-up and no email address.