Skip to content

All articles

One thick royal-blue line lying level along the ground whose right end, without a break, lifts off in orange and rolls up into one flat spiral; beneath it the words RAW HTML and NOT RUN are stacked.

Do AI crawlers render JavaScript? What the tests since 2024 found, and what ranking pages send without it

· 11 min read

Do AI crawlers render JavaScript? In the tests we found, the crawlers and page fetchers behind ChatGPT, Claude and Perplexity read the raw HTML a server sends and, apart from one isolated case in one site's log, did not run the page's scripts, while Googlebot does run them. Two things qualify that answer. A September 2026 report from one site's logs says OpenAI's crawlers began executing pages' JavaScript, though OpenAI has not confirmed it. And the AI companies' own crawler pages say nothing about JavaScript either way; the one vendor statement we found covers one of Anthropic's developer tools, so every answer about the crawlers themselves, including this one, rests on outside tests.

The pages that rank for this question, mostly SEO and marketing blogs, repeat a December 2024 study. We add three things: the tests published since, what the vendors' documents do and do not say, and a count of our own, of what the pages that rank for questions about AI search actually send to a request that runs no JavaScript. Our interest: we sell a tool that watches which pages AI assistants quote, and the fetcher behind our count belongs to it.

What did the 2024 Vercel study measure?

Vercel and MERJ watched AI crawler requests on Vercel's own network and found none of the major AI crawlers running JavaScript. The study, published on 17 December 2024, drew on monitoring nextjs.org and the Vercel network. In one month on Vercel's network, GPTBot made 569 million requests and Anthropic's Claude crawler 370 million.

The finding names the bots. OpenAI's three (OAI-SearchBot, ChatGPT-User and GPTBot), ClaudeBot, Meta's crawler, ByteDance's and PerplexityBot all fetched pages without executing their scripts. OpenAI's and Anthropic's crawlers did download JavaScript files, 11.50% and 23.84% of their requests, but in the study's words "they don't execute them." Gemini was the exception, because it "leverages Googlebot's infrastructure, enabling full JavaScript rendering", and AppleBot renders through a browser-based crawler.

The most practical line in the study is about what still gets through: "content included in the initial HTML response, like JSON data or delayed React Server Components, may still be indexed." What a bot that runs no scripts can see is whatever the server put in the first response.

This is one company's network, the bots as they behaved in late 2024, and a record of what they requested. It tells you nothing about what any assistant later quoted.

Has anyone tested it since?

Two tests published in 2026 found the page fetchers behind ChatGPT, Claude and Perplexity reading only the HTML, and one site owner's logs suggest OpenAI may have changed since. These are the tests we found in a dated search on 19 October 2026, and each is small.

Three identical plates in a row with empty gaps between them, labelled FETCH, EXECUTE and USE IN ANSWER, the middle plate outlined in orange; above the row the words SEPARATE STEPS.
Fetching a script, executing it and using its output in an answer are drawn apart because the tests found them apart. A request for a script file shows it was fetched, not that it ran, and in Search Engine World's test a fetcher that ran the script still answered from the page's HTML. Rendering is the middle step.

Search Engine World's test, published on 9 June 2026, put a decoy number in a page's HTML and had a script replace it with the real one. Asked to report the number, ChatGPT, Claude, Gemini, Perplexity, Meta AI and Copilot all gave the decoy. DeepSeek, ERNIE, Qwen, Kimi and Mistral gave the number the script wrote. Grok's fetcher executed the script on one node, yet its answer still quoted the decoy, so executing a script and using what it wrote in an answer turned out to be separate steps. The article calls it "a single-pass, live-fetch test: one prompt per assistant". It tested the fetch an assistant makes when someone asks about a page, not background crawlers such as GPTBot.

Forge Creative, a New Zealand firm that sells a bot-logging plugin, published 30 days of one news site's log on its Forge AI Bot Log site on 24 September 2026. ChatGPT-User fetched no script files across 37,045 page requests on that site, and Claude-User none across 10,028, counted by bot name and not checked against the companies' published addresses. OAI-SearchBot fetched 10 script files across 1,738 page requests and once loaded an article the way a browser would; Googlebot and Applebot fetched scripts as well. A log shows that a script was fetched, not that the bot went on to execute it, and the article says so itself: "a log cannot tell reading from running."

On 30 September 2026 one developer reported that from 25 September the logs of the developer's own sites showed OAI-SearchBot and GPTBot sending Next.js prefetch requests, which only a page whose scripts have run produces. The developer checked them against OpenAI's published IP ranges and wrote: "I couldn't find any announcement from OpenAI". It is one site's logs, and OpenAI has not confirmed it.

The tests do not all agree. searchVIU's October 2025 page test, the one our schema markup page reads, saw Gemini return a price written by JavaScript, where Search Engine World's June test saw Gemini return the decoy. Links are a separate question: a 41-day Search Engine Land test, as reported by scout7, found that, apart from Googlebot at about 2 in 100, none of the crawlers it named discovered the pages on a 1,062-page site that were linked only through JavaScript.

What do the AI companies document?

OpenAI, Anthropic and Perplexity document what each of their bots is for and how to block it, and none of their crawler pages mentions JavaScript. OpenAI's page covers its search, training and live-fetch bots; Anthropic's covers the same three roles; Perplexity's covers its search bot and its live fetcher. Silence is not a no. For the crawlers, it means the answer comes from the tests above, not from the companies. The one vendor statement we found is about a developer tool instead: Anthropic's web fetch tool documentation says the tool, which developers call through the Claude API, "currently does not support websites dynamically rendered with JavaScript." It describes that tool, not how ClaudeBot or Claude's other fetchers treat a page's scripts.

Google is the one that documents rendering. Its JavaScript SEO basics describe crawling, rendering and indexing as separate phases, and say that once resources allow, "a headless Chromium renders the page and executes the JavaScript." The same page adds that server-side or pre-rendering is "still a great idea" because "not all bots can run JavaScript." Google-Extended is not a bot with behaviour of its own: the common crawlers list says it has no separate user agent and that its robots.txt token is used "in a control capacity."

Where do the ranking pages' figures come from?

Three of the five readable pages that rank for this question carry the 2024 study's figures, and none of the five reads a later test. Every one of the five answers no, or mostly no. Averi writes that the main AI bots "fetch raw HTML"; scout7 answers "Mostly not" and leads with the link test; aivisibility.systeme.io titles its page "JavaScript Is Killing Your AI Visibility"; and two titles put it as a warning, digitalposition's "Why Your Site Might Be Invisible to AI" and amicited's "My JavaScript-heavy site is invisible to AI crawlers".

Two details change as they travel between pages, and the primaries settle both. A figure circulates as "more than 500 million GPTBot fetches"; Vercel's own unit is 569 million requests across its own network in one month. And a line says Google-Extended "renders JavaScript"; Google's crawler list says it has no user agent of its own. None of the five counts what the pages that rank actually send.

What do pages that rank actually send without JavaScript?

Most of them send their text: of 217 organic search results for 26 of our own keywords, captured on 12 and 19 October 2026, 132 came back readable to a plain request with no JavaScript and 18 came back as empty shells. 16 of the 18 shells were forum threads or video pages. As our platform describes its fetcher on 19 October 2026, each page "is fetched once (requests with a desktop Chrome user agent, 25 s timeout)", and it does not run a page's JavaScript on the production server today.

The full split: 49 of the 217 answered with a 403, 26 of them on Reddit. The 18 shells were 8 OpenAI community forum threads, 7 YouTube video pages, 1 thread on another forum and 2 pages on small websites. 10 were Facebook or Instagram pages recorded as social posts, with nothing read, and 8 were not read for other reasons (HTTP errors, a fetch error, a PDF and one unknown). Of the 143 results on ordinary websites, outside the forum, video and social platforms, 115 returned their text to a plain request, 22 answered with a 403, 2 were JavaScript shells and 4 were not read for other reasons.

Our fetcher is not an AI crawler: it identifies itself as Chrome, so a 403 to it tells you nothing about GPTBot or any other AI crawler. The two labels, readable and empty shell, are our platform's own: a page is readable when the platform got a word count back for it and recorded no problem, and an empty shell when it recorded the page as "a JS shell with no text". We did not open the 18 shells by hand, so the count cannot tell a page that sent nothing readable from one whose words the platform's extractor missed; the 7 YouTube pages each gave 10 words and were still recorded as shells. The 26 keywords are our own candidate questions about AI search and SEO, so the results lean towards marketing blogs, Reddit and LinkedIn, and they are not a sample of the web. Each keyword was captured once, in the United States, in English, on desktop. Our platform records a reason for each unread page only from 12 October, so earlier captures are left out.

What we think it shows: in this sample, 16 of the 18 pages that came back empty sat on forum and video platforms that build the page on the client side, in the browser, and ordinary articles almost always shipped their content in the initial HTML. It does not show what any AI crawler saw, and it says nothing about what any assistant cited. The count is small and early, our research page does not carry it yet, and we keep capturing.

How do you check what your own pages send?

Compare the page source with what you see in the browser, then read your logs for the AI bots' names. If your site is a single-page React, Vue or Angular app, this is the part that matters. Four steps, none of which needs a paid tool.

Two identical pages side by side: on the right, under ELEMENT INSPECTOR, four blue lines and AFTER SCRIPTS RAN beneath; on the left, under VIEW SOURCE, the third line missing and marked by an empty orange outline, with NO SCRIPTS RUN beneath.
One page in two views. The inspector on the right shows every line because the page's scripts have already run; the raw source on the left is missing one, the line a script writes in, and that is the line a fetcher without scripts never receives. A line that is in the source too reaches a fetcher whether or not it runs scripts.

First, open the page source (view source, not the element inspector, which shows the rendered page after its scripts ran) and search it for a sentence, a price or a heading you care about, on every page that matters to you. You can get the same raw HTML by fetching the URL with curl from a terminal, or by turning JavaScript off in the browser's settings and reloading. If you can see the sentence there, a fetcher that runs no scripts can read it.

Second, if the text is missing from the source, a script writes that content in on the client side. An app built that way usually needs server-side rendering (SSR), static generation (SSG) or pre-rendering to put the content in the initial HTML, which frameworks such as Next.js and Nuxt can do for a React or Vue site. Google's documentation calls dynamic rendering "a workaround and not a long-term solution" and recommends server-side rendering, static rendering or hydration instead.

Third, check your navigation. A link that exists only after a script runs may never be followed, which is what the link test above measured.

Fourth, read your logs for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User and PerplexityBot: which pages they ask for, whether they ever request your JavaScript files (a bot has to fetch an external script before it can execute it), and whether any request carries a URL that only a rendered page would produce. Check them against each company's published IP addresses, because a user agent string can be faked.

What the check does not promise: moving text into the HTML makes it readable to a fetcher that runs no scripts, and that is all. SSR has no published test we found showing it raises citations. Serving a separate Markdown copy is a different question again, and the clients documented as asking for one are coding agents, in what that page read.

Whether a rebuild is worth it depends on what your pages send today and on what content you need read. The check tells you the first part; the second is yours to decide.

Sources

See where you stand today.