Server error (5xx) is the Page indexing status that means exactly what it says and still gets misread constantly. Google asked for the page, and your server answered with a 500-level status code instead of content. Google’s help entry for the status is a single sentence, which is part of the problem: it tells you what happened, and nothing about what happens next.
What happens next is documented, in a different place, and it is worth knowing precisely, because 5xx is the one not-indexed status that changes how Google treats your whole site rather than just the failing URL.
What Google does when a page returns a 5xx
Google’s crawling infrastructure documentation lays out the behaviour, and three parts matter.
First, the crawl slows down. A 5xx response prompts Google’s crawlers to temporarily reduce crawling for the site, and the documentation is specific that the decrease is proportionate to the number of individual URLs returning a server error. One template page failing occasionally is noise. A whole section of the site failing is a signal, and Googlebot backs off accordingly.
Second, whatever the server sent back is ignored. If your 500 page renders a nice branded error message, Google does not read it, does not index it, and does not treat it as content. A 5xx means “there is nothing here to process”.
Third, and this is the line people miss: already indexed URLs are preserved in the index, but eventually dropped. The exact wording in the documentation is that the indexing pipeline “removes from the index URLs that persistently return a server error”. A one-day outage does not delete your rankings. A URL that has answered 500 for weeks quietly disappears, and it does so without any separate warning beyond this status in the report.
Recovery is the mirror image. Once the server starts answering with 2xx again, Google gradually increases the crawl rate back to normal. Gradually is Google’s word, and it matches what I see after incidents: the report clears in days, the crawl rate takes longer.
Which codes count, and the odd one out
The report groups every 500-level response under one status: 500, 502 and 503 are all handled the same way. There is also one code from the other family that behaves like them: 429 (too many requests) is the only 4xx that Google treats as a server error, because it means the server is overwhelmed rather than the content missing. If your rate limiter answers Googlebot with 429, you have opted into the whole sequence above.
The special case that outranks all of this is robots.txt. If the page failing with a 5xx is your robots.txt file, the consequences are bigger and stranger than for any ordinary URL, because Google cannot tell what it is allowed to crawl. I wrote up that failure mode separately in failed: robots.txt unreachable, and it is the first thing to rule out when 5xx errors and crawl drops arrive together.
Where the errors actually come from
Google’s own troubleshooting list for server connectivity errors names the usual suspects: an overloaded or misconfigured host, slow dynamic pages timing out, DNS problems, and one it singles out politely as “a misconfigured firewall or DoS protection system”.
That last one deserves to be first. In the site audits I do, when a working site shows Server error (5xx) on pages that load fine in a browser, the application is rarely the culprit. The security layer is. Protection systems are built to block unusually high levels of requests from one source, and a crawler is exactly that. The site works for you, works for your visitors, and fails for Googlebot, which is the one visitor the report measures.
The diagnosis order I use on client sites:
- Check whether the errors are real. Open the Crawl Stats report and look at host availability, then at the response-code breakdown. A spike of 5xx there, with a matching dip in pages crawled per day, is a real server-side event. I covered where that report hides and what its gaps mean in Search Console crawl stats missing data.
- Fetch the failing URLs as Googlebot. URL Inspection’s live test does this properly. If the live test passes today, the report is showing you history, not a current problem; the report’s data always trails the incident.
- Ask when, not just what. The example URLs in the report carry a last-crawled date. If every failure clusters in one two-day window, you are looking at one outage, not a chronic problem, and the right response is to confirm recovery and move on.
- If it is chronic, look at the WAF, the rate limiter and the host’s resource limits before touching the application. Verify Googlebot’s identity properly instead of allowlisting by user agent string, since the user agent is trivially fakeable.
When to worry
A handful of URLs in this status after a deploy or a hosting blip is the web working as designed; the index preserves your pages through short outages precisely so that a bad afternoon does not cost you rankings. The status deserves real attention in two situations: when the count grows week over week, which means the cause is still live, and when important pages sit in it for weeks, because “persistently failing” is when removal starts. If pages have already begun dropping, the recovery path and its timelines are the subject of pages dropped out of the Google index, and the fix is the same in either case: make the server answer 200, then let the crawl rate rebuild on its own.
Server error (5xx) sits in the same not-indexed family as the statuses I have covered before, and the contrast is useful. A 404 tells Google the content is gone, and Google believes it. A 5xx tells Google the site is struggling, and Google waits. That patience is the one thing this status gives you that the others do not: time to fix the server before the index notices.
Sources
- Google Search Console Help, Page indexing report, the Server error (5xx) status and the fixing server connectivity errors section, checked 1 September 2026
- Google crawling infrastructure documentation, how HTTP status codes affect Google’s crawlers, for the 5xx and 429 handling, the proportionate crawl slowdown, and the persistent-failure removal rule, checked 1 September 2026
- Google Search Console Help, Crawl Stats report, for host availability and response-code breakdowns
- Google Search Central, verifying Googlebot, for identifying real crawler traffic before allowlisting