JavaScript SEO for Google and AI Crawlers: Rendering Issues and Fixes

JavaScript is not inherently bad for SEO. Google can render modern JavaScript applications, and websites built with React, Vue, Angular, Next.js, Nuxt, and similar frameworks can rank perfectly well.

The problem starts when important content exists only after client-side JavaScript runs.

A human visitor using a modern browser may see a complete article, product page, pricing table, or application. A crawler requesting the same URL may initially receive little more than navigation, script tags, placeholders, and an empty application container.

Googlebot can usually render that JavaScript later. Many AI crawlers, however, currently do not execute client-side JavaScript in the same way.

That difference makes JavaScript SEO increasingly important for websites that want visibility across both traditional search engines and AI-powered discovery systems.

This guide explains what crawlers actually receive, how Google and AI crawlers behave differently, which JavaScript rendering issues cause visibility problems, and how to diagnose and fix them without unnecessarily rebuilding an entire website.

Initial HTML and the Rendered DOM Are Not the Same Thing

When a browser requests a webpage, the server first returns an HTTP response containing HTML.

On a server-rendered page, much of the meaningful content may already be present in that response.

On a heavily client-rendered application, the initial HTML might instead contain something similar to:

<div id="app"></div>
<script src="/app.js"></script>

The browser then downloads JavaScript, calls APIs, updates application state, and constructs the visible page.

This creates several different representations of the same URL:

• Raw HTTP response: the HTML returned before browser-side JavaScript executes.
• Rendered DOM: the document after the browser has executed scripts and modified the page.
• Google-rendered DOM: the version Google sees after its Web Rendering Service executes the page’s JavaScript.
• Non-JavaScript crawler view: what a crawler may understand if it primarily processes the initial response.

The difference matters because crawler visibility is not necessarily binary.

The initial response may already contain:

• the title and meta description;
• canonical tags;
• structured data;
• framework-specific serialized data such as React Server Component payloads;
• navigation;
• internal links;

while the main article, product description, pricing information, or documentation content is still loaded later through JavaScript.

Structured data in the initial response can improve machine accessibility, but different search engines and AI systems may interpret or use that data differently.

For a quick technical baseline, CodeItBro’s LLM Content Visibility Scanner checks the first HTML response without executing page JavaScript and reports whether important copy, headings, metadata, structured data, and crawl signals are already exposed.

How Googlebot Handles JavaScript

Google Search processes JavaScript pages through three broad stages: crawling, rendering, and indexing.

Googlebot first retrieves the URL and processes the initial response. Pages that return a successful HTTP response can then enter Google’s rendering process, where the Web Rendering Service uses Chromium to execute JavaScript and inspect the resulting DOM.

Google’s JavaScript SEO documentation confirms that Google Search can run JavaScript.

Client-side rendering therefore does not automatically make a page unindexable.

The real risks are more specific:

• important JavaScript or API resources fail during rendering;
• content depends on cookies, local storage, or session state;
• runtime errors prevent critical components from loading;
• important links are available only through JavaScript event handlers;
• canonical or robots directives change incorrectly after page load;
• critical content is missing when Google’s renderer processes the page;
• a CDN, firewall, or authentication layer treats Googlebot differently from a normal browser.

If terms such as canonical URL, indexability, crawl budget, robots directives, soft 404, or structured data are unfamiliar, CodeItBro’s SEO Glossary provides concise explanations of common technical SEO concepts.

Google’s Rendering Queue Is Usually Faster Than the Old SEO Myth Suggests

Google’s documentation says pages can wait in a rendering queue because executing JavaScript requires additional resources.

That does not mean JavaScript pages normally wait several days before Google renders them.

In a large Vercel and MERJ analysis of more than 37,000 matched rendering observations, the median rendering delay was about 10 seconds.

The study found roughly:

• 10 seconds at the median;
• 26 seconds at the 75th percentile;
• around 3 hours at the 90th percentile;
• around 6 hours at the 95th percentile;
• around 18 hours at the 99th percentile.

Those numbers came from a specific dataset and should not be treated as guarantees for every website.

They do, however, show why the old assumption that Google normally needs days to render JavaScript is misleading.

For large websites, JavaScript can still affect crawl efficiency because rendering and fetching additional resources requires more work than processing straightforward HTML.

Googlebot Currently Fetches Only the First 2 MB of an HTML Resource

JavaScript execution is not the only technical limitation to consider.

In March 2026, Google clarified that Googlebot currently fetches approximately the first 2 MB of an individual non-PDF URL, including HTTP headers.

If the response exceeds that amount, Google processes the content it successfully fetched and ignores the bytes beyond the limit.

This can become relevant when pages contain unusually large amounts of:

• inline JavaScript;
• serialized application state;
• base64-encoded assets;
• inline CSS;
• navigation markup;
• embedded data.

If critical content or technical signals appear unusually late in a massive HTML response, Google may not receive them.

External scripts, images, stylesheets, and other resources have their own fetches, so keeping large assets outside the HTML document can also reduce unnecessary response size.

Many AI Crawlers Currently Do Not Execute Client-Side JavaScript

This is where JavaScript-heavy websites face a different challenge.

A Vercel and MERJ analysis of major AI crawlers found no evidence of JavaScript execution among several important crawlers in the dataset, including OpenAI’s GPTBot and OAI-SearchBot, Anthropic’s ClaudeBot, PerplexityBot, Meta’s external crawler, and ByteDance’s Bytespider.

Some of those crawlers still downloaded JavaScript files.

The study observed JavaScript requests in approximately:

• 11.5% of OpenAI crawler requests;
• 23.8% of Claude crawler requests.

Downloading a JavaScript file is not the same as executing it and using the resulting DOM.

That distinction is critical.

If an article, product description, documentation page, or pricing table appears only after client-side JavaScript runs, a crawler that primarily consumes the first response may receive much less useful information than a human visitor.

These findings represent observed behavior rather than permanent rules. Crawler implementations can change, so important assumptions should be periodically checked against provider documentation and your own server logs.

GPTBot and OAI-SearchBot Are Not the Same Thing

One common mistake in AI SEO discussions is treating every OpenAI crawler as a single “ChatGPT bot.”

OpenAI currently documents several different user agents with different purposes.

According to OpenAI’s publisher guidance, websites that want their pages discoverable through ChatGPT search should allow OAI-SearchBot.

Blocking GPTBot is therefore not the same as opting out of ChatGPT search visibility.

This distinction matters when writing robots.txt rules because an organization may want different policies for search discovery and model-training access.

Google AI Overviews and AI Mode Use Google Search Infrastructure

Google’s AI search experiences should also be distinguished from independent AI crawlers.

Google says AI Overviews and AI Mode use the existing Google Search infrastructure.

Pages generally need to be crawlable, indexed, and eligible to appear in Google Search before they can appear as supporting links in these experiences.

That means the same technical SEO fundamentals that make a page accessible to Google Search also matter for Google’s AI-powered search features.

There is not a separate “Gemini SEO crawler” that website owners need to optimize for in Google Search.

Google does provide a robots.txt product token called Google-Extended, but it is not a separate crawler user agent. It controls certain uses of Google-crawled content for Gemini training and grounding rather than deciding normal Google Search inclusion.

Common JavaScript SEO Problems That Affect Crawlers

A JavaScript website does not have to display a completely blank page to have a rendering problem.

Partial failures are far more common.

1. Main Content Is Missing From the Initial HTML

This is the most obvious issue for crawlers that do not execute JavaScript.

If the raw response contains only an application shell while the primary article, product description, category introduction, or documentation arrives later through an API call, some crawlers may never receive that content.

2. Canonical, Robots, or Metadata Signals Depend on JavaScript

Google can process JavaScript-generated metadata, but important SEO directives are generally safer when they are correct in the initial response.

This is particularly important for canonical URLs and robots directives.

A dangerous pattern is returning:

<meta name="robots" content="noindex">

in the initial HTML and then expecting JavaScript to change it to index.

Google may process the original noindex directive and skip rendering the page entirely, meaning the JavaScript change may never help.

Similarly, avoid declaring one canonical URL in the initial response and then changing it to a conflicting URL with JavaScript.

3. Important Links Are Not Real HTML Links

Search engines discover pages most reliably through standard anchor elements.

Critical navigation should generally look like:

<a href="/products/example">...</a>

rather than relying only on JavaScript click handlers without a crawlable destination.

4. Hash-Based Routing Represents Indexable Pages

Another SPA problem appears when important pages are represented only through URL fragments such as:

https://example.com/#/products/example

The fragment portion of a URL is primarily browser-side state and is not an ideal way to represent distinct indexable documents.

For public pages that should be crawled and indexed, use normal URLs such as:

https://example.com/products/example

and use the browser History API or framework routing to preserve a smooth single-page-app experience.

5. Required Scripts or APIs Are Blocked

A page can return HTTP 200 while still failing during rendering because a JavaScript file, API endpoint, stylesheet, or other dependency is unavailable.

Possible causes include:

• robots.txt rules;
• WAF policies;
• CDN bot protection;
• rate limiting;
• authentication;
• expired API credentials;
• CORS restrictions;
• temporary backend failures.

6. Content Depends on Cookies or Local Storage

Crawler requests may not have the same browsing history, local storage values, login state, or cookies as a human visitor.

If important public content depends on those states, crawlers may see a different page.

7. Runtime Errors Break Critical Components

A JavaScript error does not automatically erase an entire page.

However, an uncaught error in a critical rendering path can prevent individual components—or in some cases much of an application—from loading while the server still returns HTTP 200.

That makes error boundaries, monitoring, and graceful fallbacks important on JavaScript-heavy websites.

8. SPAs Return HTTP 200 for Pages That Do Not Exist

Some single-page applications return the same application shell for every route, including invalid URLs.

If the client displays a “not found” message but the server still returns HTTP 200, search engines may interpret the page as a soft 404.

Serve meaningful HTTP status codes where practical, particularly 404 for missing pages and 410 for intentionally removed resources.

9. JavaScript Redirects Replace Proper Server Redirects

Google can process JavaScript redirects after rendering, but server-side redirects are usually more robust for permanent URL changes.

Use HTTP 301 or 308 responses when a URL has permanently moved rather than depending solely on browser-side redirect logic.

10. WAF or CDN Rules Block Crawlers

A robots.txt file can explicitly allow a crawler while a firewall or bot-management service still returns a 403 response, CAPTCHA, JavaScript challenge, or rate-limit error.

That is why crawler accessibility needs to be checked at both the robots layer and the infrastructure layer.

If you’re diagnosing 403 responses, bot challenges, rate limits, or JavaScript-dependent pages, CodeItBro’s guide to common web scraping and crawler blocking issues explains how robots rules, anti-bot systems, authentication, rate limiting, and JavaScript rendering can affect automated requests.

How to Audit JavaScript SEO and AI Crawler Visibility

You do not need to guess whether a page depends too heavily on browser-side rendering.

The problem can be tested systematically.

Step 1: Inspect the Raw HTML Response

Start with what the server returns before JavaScript execution.

You can use View Source in your browser or fetch the page directly:

curl -L https://example.com/page

Search the response for:

• the H1;
• a distinctive sentence from the primary content;
• product or category descriptions;
• canonical URL;
• meta robots;
• structured data;
• important internal links.

For easier inspection, paste the response into CodeItBro’s Source Code Viewer to add syntax highlighting, line numbers, and search.

Step 2: Compare Raw HTML With the Rendered DOM

Open the page normally and inspect the Elements panel in browser DevTools.

If the rendered DOM contains substantial content that does not appear in View Source, that content depends on JavaScript execution.

This is not automatically a Google SEO problem, but it tells you that crawler rendering behavior matters for that page.

Step 3: Test Google’s Rendered Version

Use Google Search Console’s URL Inspection Tool or Google’s Rich Results Test.

Inspect:

• rendered HTML;
• JavaScript console errors;
• blocked resources;
• HTTP status;
• structured data;
• whether the primary page content appears after rendering.

This gives you a much better picture of Google’s view than looking only at the page in your own browser.

Step 4: Check the Initial Response for AI Crawler Visibility

Run the URL through CodeItBro’s LLM Content Visibility Scanner.

The scanner intentionally analyzes the initial HTML without running client-side JavaScript, making it useful for identifying pages where the first response exposes too little meaningful content to non-rendering crawler workflows.

Step 5: Test robots.txt for Individual Crawlers

Do not assume the same robots rule applies to every search or AI crawler.

CodeItBro’s Robots.txt Tester lets you test specific URLs using Googlebot or a custom user agent such as OAI-SearchBot, GPTBot, ClaudeBot, or PerplexityBot.

Remember that robots.txt and actual network access are different layers. A crawler can be allowed in robots.txt and still be blocked by a CDN, firewall, or bot-management product.

Step 6: Check Server Logs

Server logs show whether a crawler actually requested a URL and what your infrastructure returned.

Useful information includes:

• user agent;
• requested URL;
• HTTP status;
• response size;
• response time;
• request frequency;
• verified crawler IP information where applicable.

Human-focused analytics platforms often provide little insight into crawler activity, so logs remain one of the most useful sources for confirming real bot requests.

Step 7: Run a Broader Technical SEO Audit

Rendering is only one part of technical SEO.

If a website has indexing problems, also check redirects, canonicalization, internal links, metadata, robots directives, HTTP status codes, structured data, duplicate content, and performance.

CodeItBro’s comparison of SEO audit tools with JavaScript rendering support covers platforms that can crawl JavaScript-driven pages alongside more traditional technical SEO checks.

How to Fix JavaScript Rendering Problems

The right solution depends on which content is missing and how the application is built.

You do not necessarily need to rewrite the entire website.

Option 1: Static Generation

For articles, documentation, landing pages, category pages, and other content that does not need to be generated uniquely for every request, static generation is often one of the simplest solutions.

The build process generates complete HTML ahead of time, allowing users and crawlers to access the primary content immediately.

Option 2: Server-Side Rendering

Server-side rendering generates HTML on the server for the request or through a framework caching layer.

This works well when the page requires fresh server data but the main content should still appear in the initial response.

Frameworks such as Next.js, Nuxt, and Angular provide server-rendering options without giving up rich client-side functionality.

Option 3: Incremental or Cached Static Rendering

Many content-heavy websites do not need to choose between rebuilding everything statically and dynamically rendering every request.

Incremental generation and server-side caching can deliver pre-rendered HTML while refreshing pages periodically or when content changes.

Option 4: Hybrid Rendering

Often the most practical solution is to render SEO-critical information on the server and leave highly interactive features to the client.

The initial response might contain:

• the H1;
• primary body copy;
• product information;
• important internal links;
• metadata;
• canonical signals;
• structured data.

Filters, dashboards, editors, charts, personalization, and account tools can then hydrate or load after the browser starts.

Option 5: Managed Prerendering or Dynamic Rendering

Some teams cannot easily modify an existing client-rendered application.

A rendering service can run the application in a headless browser, capture the resulting HTML, and return a cached representation to compatible crawlers.

Commercial platforms offering JavaScript rendering for SEO are one option for teams that prefer to offload that infrastructure rather than operate their own rendering layer.

However, this architecture should be distinguished from normal server-side rendering.

SSR or static generation makes crawlable HTML part of the normal application architecture and generally serves the same HTML foundation to users and crawlers.

Dynamic rendering, by contrast, detects certain crawlers and serves them a separately rendered representation.

Google currently treats dynamic rendering as a workaround rather than the preferred long-term solution. Where practical, Google recommends server-side rendering, static rendering, or hydration instead.

If bot-specific rendering is used, the primary content should remain equivalent to what normal users receive. Rendering should solve a technical accessibility problem, not become a mechanism for showing search engines materially different content.

JavaScript SEO Checklist for Google and AI Crawlers

Final Thoughts

JavaScript itself is not the enemy of search visibility.

The real issue is whether important information remains accessible across crawlers with different rendering capabilities.

Googlebot can execute JavaScript and is capable of processing sophisticated web applications. Independent AI crawlers may behave differently, and current research shows that several major AI crawlers do not execute client-side JavaScript in the same way.

That makes the initial HTML response an increasingly useful technical baseline.

For critical public pages, expose enough information in the first response that a crawler can understand what the page is about even before browser hydration: the main topic, important copy, crawlable links, metadata, canonical signals, and relevant structured data.

Then use JavaScript for what it does best: interactivity and application behavior rather than making every crawler depend on successful browser execution just to discover the primary content.

When visibility problems appear, compare the raw response with the rendered DOM, inspect Google’s rendered version, test crawler-specific access rules, and verify real requests in server logs.

That process usually reveals whether the real problem is JavaScript rendering, indexing directives, crawler access, infrastructure, or something else entirely.