Skip to content

field note

Googlebot's 15MB limit is now 2MB: what actually changed

Crawling and robots.txt

Shahid AliSeptember 12, 2026all posts

Googlebot's 15MB limit is now 2MB: what actually changed

The Googlebot 15MB limit is one of those numbers the industry learned once, in June 2022, and never checked again. It is now wrong. Google’s Googlebot documentation says 2MB, and has done since early 2026, while the blog post that taught everyone 15MB is still live, still says 15MB, and still ranks.

Both numbers are real. They describe different things. Getting them the wrong way round is how you end up auditing the wrong limit on a page that is nowhere near either.

What is Googlebot’s file size limit?

From Google’s Googlebot documentation:

“When crawling for Google Search, Googlebot crawls the first 2MB of a supported file type, and the first 64MB of a PDF file.”

Two details on the same page matter as much as the number:

“Once the cutoff limit is reached, Googlebot stops the fetch and only sends the already downloaded part of the file for indexing consideration.”

“The file size limit is applied on the uncompressed data.”

So gzip does not help you. The cap is measured on the bytes after decompression, not on what crossed the wire. And Google’s March 2026 post Inside Googlebot adds one more:

“Googlebot currently fetches up to 2MB for any individual URL (excluding PDFs). This means it crawls only the first 2MB of a resource, including the HTTP header.”

Including the header. A page with a heavy set of response headers spends part of its 2MB before the HTML starts.

So where does 15MB come from?

It is still a real Google number. It is just not Googlebot’s. From the crawler overview:

“By default, Google’s crawlers and fetchers only crawl the first 15MB of a file, and any content beyond this limit is ignored.”

And immediately after: “For example, a Google crawler like Googlebot may have a smaller size limit (for example, 2MB).”

The Inside Googlebot post says the same thing from the other direction: “For any other crawler that doesn’t specify a limit, the default is 15MB regardless of content type.”

So 15MB is the house default across Google’s fleet of crawlers and fetchers. Googlebot, the one that matters for Search, opts down to 2MB. The 2022 post Googlebot and the 15 MB thing was accurate when it was written and carries a 16 March 2023 addendum saying “each individual subresource fetch (in particular CSS and JavaScript) is bound to the 15MB limit”. Read that today, next to the current Googlebot page, and it is simply the older figure. Google never went back and marked the post superseded, which is why half the SEO web is still repeating it.

The caps Google publishes, and the HTML sites actually ship Bars are on a log scale, because the smallest number here is 4,000 times smaller than the largest.

WHAT LIMIT

Googlebot, one URL 2MB

Googlebot, one PDF 64MB

Other Google crawlers, default 15MB

robots.txt 500 KiB

One sitemap file 50MB or 50,000 URLs

Median mobile home page HTML 22 KB

90th percentile HTML 59 KB

Limits from Google's Googlebot page, crawler overview, robots.txt specification and sitemap documentation. Page weights from the Web Almanac 2025 Page Weight chapter.

Does the limit include images, CSS and JavaScript?

No, and this is the part most articles get wrong. From the Googlebot page:

“Each resource referenced in the HTML such as CSS and JavaScript is fetched separately, and each fetch is bound by the same file size limit.”

So a page with 900 KB of images and 600 KB of JavaScript is not a 1.5MB page as far as the cap is concerned. The HTML is measured on its own. Your images do not push your HTML towards the cutoff.

The sting is in the second half of that sentence. Every subresource gets its own 2MB budget, which means a single JavaScript bundle over 2MB is itself cut off mid-file. Truncated HTML loses content at the bottom. A truncated script is worse, because it is cut at an arbitrary byte and will not parse, so it does not run at all. If your page depends on that bundle to render, the rendered page is empty rather than short.

How do I check my page size?

Measure the HTML response on its own, uncompressed, and ignore everything else on the page. In a terminal:

curl -sH 'Accept-Encoding: gzip' -A 'Googlebot' https://example.com/ | gunzip | wc -c

Browser devtools will mislead you here, because the network panel’s transferred column shows the compressed size. Google measures the uncompressed one.

I ran that across this site while writing. The homepage is 84,433 bytes, about 82 KB, which is 4 percent of the cap. The bulk URL inspection tool page is 87.9 KB. The single heaviest page I have is the reviews page, which renders 413 client reviews into one document, and it comes to 273 KB. That is the worst case on a site built to dump a lot of records into one page, and it is still only 13 percent of the limit.

For scale, the Web Almanac 2025 Page Weight chapter puts the median mobile home page at 22 KB of HTML and the 90th percentile at 59 KB. The whole median page, images and scripts and fonts included, is 2,362 KB. Almost nobody is near 2MB of HTML alone.

So when does 2MB actually bite?

Rarely, and almost always for the same handful of reasons:

  • Base64 images inlined into the HTML. This is the fastest route to the cap, because an inlined image stops being a separately fetched resource and becomes part of the document.
  • Programmatic pages that dump their whole data payload into the page. A <script type="application/json"> block holding the full API response, a Next.js __NEXT_DATA__ payload, an over-embedded WordPress REST response. Related: Google’s crawler does not parse JSON it finds in the page the way people assume it does.
  • Unpaginated everything. A ten thousand row table, a comment thread with two thousand replies, an archive page with no pagination.
  • Machine-generated listings at scale, which is one more reason programmatic SEO gets the reputation it has.

If none of those describe your site, the 2MB limit is not your problem, and you are better off spending the time on crawl budget, which is a real constraint on far more sites.

What does Search Console show when a page is truncated?

Nothing. There is no status for it, no warning, no error. The page is fetched, the first 2MB is sent for indexing, and Google indexes what it received. In the page indexing report the URL will most likely sit happily under Indexed.

That is the actual hazard. A truncated page does not look broken. It looks fine and quietly ranks for less than it should, because the bottom of the document, which is where the FAQ block and the structured data and the internal links often live, never reached the index. The only place you can see it is the URL Inspection tool’s rendered HTML, and you have to know to look.

What I do about it

In an audit I do not check every page for this. I check the page families where the failure mode is plausible: the biggest listing template, anything programmatic, and anything that inlines images. One curl per template, not per URL. If a template is under a couple of hundred KB of HTML, the whole family is fine and I move on.

The other thing I changed is what I tell people about JavaScript. The 2MB cap on each subresource is the more likely failure of the two, and it interacts badly with how crawlers handle rendered content. A fat bundle is a rendering risk before it is a size risk.

Sources

  • Googlebot, Google Search Central. The 2MB and 64MB figures, the uncompressed-data line, the separate-fetch rule for referenced resources, and the stop-and-send-what-you-have behaviour.
  • Inside Googlebot, Google Search Central blog, 31 March 2026. The 2MB per URL figure including the HTTP header, and the 15MB default for crawlers that do not specify one.
  • Overview of Google crawlers and fetchers, Google Search Central. The 15MB default and the note that Googlebot may use a smaller limit.
  • Googlebot and the 15 MB thing, Google Search Central blog, 28 June 2022, with its 16 March 2023 addendum on subresource fetches. Still live, still says 15MB.
  • robots.txt specification, Google Search Central. The 500 KiB robots.txt limit.
  • Build and submit a sitemap, Google Search Central. The 50MB and 50,000 URL limits.
  • Page Weight, Web Almanac 2025. Median mobile home page HTML of 22 KB, 90th percentile 59 KB, median total page 2,362 KB.
  • Page sizes for shahidali.co measured by me on 12 September 2026 with the curl command above.