What AI search engines actually read on your site

A marketing site is built for a human with a browser. An AI search engine arrives with none of that patience: it makes a request, takes what comes back, and moves on. That gap is where AI visibility is won or lost.

The raw HTML, not the rendered page

Most crawlers do not execute JavaScript. They read what your server returns. If your copy is injected after hydration, the crawler sees an empty shell. You can check this yourself, and it takes one command. See our methodology for the full checklist.

curl -sL https://yoursite.com | grep -i "<h1"

# Nothing back? Then your headline is not in the model's input either.

Know your crawlers

Blocking GPTBot to stay out of training data is reasonable. Blocking OAI-SearchBot at the same time, which many sites do by accident, removes you from ChatGPT results entirely. Two decisions, two lines.

If the extractable answer is buried, a competitor who stated it plainly gets cited instead.

Where to start

1. Confirm reachability. It is binary and it invalidates everything else.
2. Fix extractability on your ten highest-intent pages.
3. Then measure per engine, so you can tell whether it worked.

Inline code looks like robots.txt and struck text looks like keyword density.