Growth Notes

Rendering and SEO

Tags: web-dev concept

Rendering and SEO

Date: 2026-08-16

Google renders JavaScript, eventually, on a queue. Almost nothing else does — AI crawlers read raw HTML and stop. So content that only exists after a script runs is content some crawlers will never see at all.

What it is

Rendering and SEO is the question of whether a crawler sees your content, given how your page is built. It follows directly from your rendering strategy and it’s the SEO consequence of that decision.

Two-wave indexing

Google processes JavaScript pages in two passes:

WAVE 1   raw HTML crawled and indexed         immediate
           ↓
         page queued for rendering
           ↓
WAVE 2   headless Chromium executes the JS    hours to weeks later
         index updated with rendered content

The rendering service runs an evergreen headless Chromium, so modern JavaScript is supported. The problem isn’t capability, it’s the queue. Wave 2 is resource-intensive, so it’s scheduled by site importance and crawl budget — and for lower-authority sites, some JavaScript-rendered content is never reliably indexed at all.

[CHECK: two-wave behaviour and rendering delays are Google’s documented model but the timings are anecdotal — verify against Google Search Central before quoting durations.]

The thing that changed

AI crawlers do not render JavaScript. GPTBot, ClaudeBot, PerplexityBot, Bytespider and Meta-ExternalAgent read the HTML they’re served and stop there.

That makes client-side rendering worse than it was two years ago, and for a new reason: it’s no longer a delay, it’s an absence. Content behind JavaScript simply doesn’t exist to a growing share of crawlers. What that costs commercially — and why you can’t see the loss — is Answer Engines.

There’s also a distinction worth knowing when you configure robots.txt: training crawlers and search crawlers are separate bots. GPTBot (training) is distinct from OAI-SearchBot (search); ClaudeBot from Claude-SearchBot. You can decline training while remaining visible in AI search, which is usually what a retailer wants.

[CHECK: crawler names and their training/search split change — verify against each vendor’s published bot documentation before writing rules.]

On llms.txt: as of early 2026 no major AI vendor had committed to reading it, and traffic to the file was negligible. It’s a proposal, not a standard. Don’t spend time on it yet.

What must be in the raw HTML

The test: view source — not DevTools, which shows the rendered DOM — and check for:

  • The main body content
  • <title> and meta description
  • Canonical tag — Canonicalisation
  • Internal links as real <a href> elements
  • Structured data — Structured Data
  • hreflang, robots meta, pagination signals

Links are the one people miss. A crawler discovers pages by following href attributes in HTML. A “link” that’s a <div> with a click handler, or an <a> with no href populated until JavaScript runs, is not a discoverable route — it’s a dead end, and it silently orphans whole sections of a catalogue.

<a href="/collections/wool">Wool</a>          ← discoverable
<div onclick="router.push('/collections/wool')">Wool</div>   ← invisible
<a href="#" data-to="/collections/wool">Wool</a>             ← invisible

Fixing it

  1. Server-render or statically generate anything indexable. This is the whole answer and everything else is mitigation
  2. Real anchors with real hrefs for every route you want crawled, even in a single-page app
  3. Meta tags in the initial response, not injected client-side
  4. Dynamic rendering — serving pre-rendered HTML to bots — as a last resort. It’s a maintenance burden, it risks cloaking accusations if the content differs, and it’s now widely discouraged
  5. Check what’s actually seen: Search Console’s URL Inspection shows Google’s rendered HTML; curl shows what a non-rendering crawler gets
curl -s https://example.com/product/wool-socks | grep -c "product-description"

If that returns zero, no AI crawler will ever see your product description.

Where it interacts

  • Faceted Navigation and Crawl Budget — client-rendered facets are invisible, which is sometimes accidentally the right outcome
  • Lazy Loading — content deferred until scroll may never render for a crawler, which doesn’t scroll. Use loading="lazy" on images, not JavaScript gating on text
  • Core Web Vitals — a separate, weaker ranking input. Indexability is binary; page experience is a tiebreaker
  • Site Migrations and SEO — a replatform that moves from server-rendered to client-rendered is the classic cause of a traffic collapse nobody predicted

Graph View

Backlinks