Your Page Is an Empty Shell to Every AI Crawler
AI crawlers fetch the initial HTML and never run your JavaScript. If your content is rendered client-side, they see nothing to cite.
Google can index your site. That does not mean AI can read it.
Googlebot renders JavaScript. GPTBot, ClaudeBot and PerplexityBot fetch the initial HTML and stop there. If your text arrives only after useEffect runs, they get a <div id="root"></div> and nothing else.
Why the two differ
Rendering JavaScript is expensive: it needs a browser engine per page. Search engines pay that cost because they index the whole web and can afford a render queue. AI crawlers fetch at retrieval time — the moment a user asks a question — so they take what comes back in the first response and move on.
The gap is invisible from inside a browser. Your page looks fine to you, and fine to Google Search Console. The only place it shows up is the raw HTML.
Check it yourself
Fetch your page the way a crawler does — no JavaScript, just the response:
curl -sL https://your-site.com/your-page | grep -c "<p>"
A count near zero on a page you know has ten paragraphs is the whole diagnosis. Two more things worth looking at in that output:
- Whether the body text appears at all (search for a distinctive sentence from your page)
- What sits inside the mount point —
<div id="app">,<div id="root">,<div id="__next">
If the mount point is empty and the text is missing, you have the problem.
Fixing it
The fix is to send content in the first response, not to add a fallback to it. In rough order of how much work each takes:
- Static generation. Content is rendered at build time. Best case, and it works for most marketing pages and blogs.
- Server rendering. Rendered per request on the server. Necessary when the content depends on the request, and still just as readable to a crawler.
- Prerendering. A service renders your SPA into HTML for crawlers. A stopgap when the app cannot be restructured, but it is a second system that can drift.
A <noscript> block is not a fix. It helps a little, but the text a crawler needs should be in the page, not in a warning aimed at browsers that already have JavaScript off.
You can see what your own page looks like to a crawler with the no-JS view tool — it shows how much body text survives and whether the mount point is empty.
If you want the whole picture rather than one check, run a full audit: it covers all 32 rules, including the rendering ones, and returns a prioritised fix list.