Bulk URL inspection tools, and the quota nobody explains

Every tool in this category calls the same endpoint. Google’s URL Inspection API returns one answer per URL, and Screaming Frog, Sitebulb, a Chrome extension and my free checker all receive exactly the same payload for the same URL at the same moment. That payload is the indexed version, which is also why it settles arguments with the Page indexing report; I set out the three datasets involved in URL Inspection tool vs Page indexing report.

So the accuracy comparisons you find in this space are measuring nothing. What actually differs between these tools is how they handle quota, how much setup they demand before the first inspection, and what they charge. This guide compares them on those three things, because those are the only three that vary.

The short answer

The API gives you 2,000 inspections per property per day, free, and returns the reason a URL is not indexed rather than a yes or no. If you already own a Screaming Frog licence, use it, because attaching inspection data to crawl data in one table is genuinely the best version of this. If you do not, there is no reason to buy one for this job, and my free bulk index checker does the inspection half in the browser with no key to generate and nothing to install.

If your site is large enough that 2,000 a day is the binding constraint, you are holding the wrong instrument, and there is a better one further down.

The quota, properly explained

This is the part that determines what is possible, and most write-ups state the number without the detail that matters.

2,000 inspections per property per day. Roughly 600 per minute. Both are Google’s, both are free, and no tool can raise either. A vendor advertising a 1,000 per day cap is applying its own limit on top of Google’s, which is worth knowing before you pay for it.

The budget is per property, not per account. This is the useful part. If you verify example.com as a domain property, that is one budget of 2,000. If you also verify example.com/shop/ and example.com/blog/ as separate URL-prefix properties, each of those carries its own 2,000, and the URLs inside them can be inspected against either. A large site split sensibly into verified sections has a daily ceiling several times higher than the same site verified once.

I would not build a permanent workflow on that, because it is a quirk of how the budget is scoped rather than a documented feature. For a one-off audit of a large catalogue it is the difference between three days and three weeks.

The minute limit is the one you will actually hit. 600 a minute sounds generous until a tool fires requests in parallel with no pacing, at which point you get rate limited a few hundred URLs in and the run reports failures that look like index problems and are not. Any tool worth using paces itself and retries. When results come back inconsistent between two runs of the same list, throttling is the first thing to suspect.

The setup tax

The second real difference between these tools is how far you get before you have inspected anything.

Most desktop tools and extensions authenticate with a service account, which means: create a Google Cloud project, enable the Search Console API on it, create a service account, download its JSON key, then go into Search Console and add that service account’s email as a user on the property. Five steps, one of which involves handling a credential file you now have to keep somewhere safe.

There is nothing wrong with this. It is the right design for something running unattended on a schedule. It is a lot of ceremony for the far more common case of wanting to know the status of 300 URLs once.

The alternative is browser OAuth against the account you are already signed in as, which is two clicks and generates no key at all. That is the design I chose for my own tool, because the ceremony was the reason I kept doing this by hand for clients.

The tools

Screaming Frog SEO Spider. The best of these if you already own it. The URL Inspection API integration pulls index status into the crawl table, so you get coverage state sitting next to the title, the status code, the canonical and the internal link count for the same URL. That combination is what actually diagnoses a problem, because “not indexed” plus “zero internal links” is an answer, and “not indexed” alone is a fact. It costs £199 a year, the integration needs the licence, and it is a desktop application that inspects at the same 2,000 per property per day as everything else.

Sitebulb. Added URL Inspection API support in version 5.7 and hangs it off the existing Search Console integration, so it is a checkbox in the crawl configuration rather than a separate exercise. Desktop plans run roughly $18 to $42 a month. Its reporting is more opinionated than Screaming Frog’s, which people tend to either like a lot or not at all. Same API, same quota.

GSCTool. A Chrome extension covering Search Console and Bing Webmaster Tools, with bulk inspection among its features. Free tier, $29 a year, $59 lifetime, or a $5 day pass, which is a genuinely sensible option for a one-off audit. Note that its bulk inspection is documented at up to 1,000 URLs a day, which is half of what Google allows. If your list is 1,500 URLs, that limit is the tool’s, not Google’s.

My free bulk index checker. Paste the list, authorise against your own property, get per-URL status from the same API. No service account, no JSON key, no extension, nothing stored, no cost. It does one job and it is the job most people arrive here needing.

The Search Console MCP server. The same access from inside an AI assistant, installed on your machine as gsc-indexer-mcp. It needs no API key and no Google Cloud project, and it exposes index status, unindexed discovery with reason codes, sitemaps and search analytics as tools an assistant can call directly, which means you can ask a question in English instead of exporting a CSV and pivoting it. It is open source and pre-alpha, so run it alongside the console rather than in place of it.

The gotcha worth knowing before you read your results

The API answers about the canonical Google selected, which is not always the URL you submitted.

Inspect a URL that Google treats as a duplicate and the response describes the page it kept instead. The URL comes back with a coverage state referring to a different address, and if you are skimming a spreadsheet of 800 rows it is easy to read that as indexed when the thing that is indexed is a different page.

This is not a defect. It is the single most useful signal in the whole response, because “duplicate, Google chose a different canonical” tells you precisely what happened and most other methods cannot see it at all. It just needs reading carefully. A site: query will never tell you this, which is a large part of why inspection beats scraping on a property you own.

When 2,000 a day is the wrong tool

At some size, per-URL inspection stops being the right instrument, and knowing where that line sits saves weeks.

A 50,000-URL catalogue takes 25 days to inspect once at the daily cap. By the time you reach the end, the beginning has changed. You will have spent a month collecting a number you cannot act on, because the answer to “how many of my 50,000 pages are indexed” was never going to be actionable at that resolution.

Read coverage at the sitemap level instead. Split the catalogue into sitemaps that mean something, by category, by template, by margin, then compare submitted against indexed per sitemap. That tells you which template or which section is failing, which is the thing you can actually fix. My free sitemap index coverage checker does that comparison, and Search Console’s own Sitemaps report gives you a coarser version of the same view.

Then spend your 2,000 daily inspections where they earn their keep: a sample from each failing group to find out why, and the specific URLs you are actively working on. That 2,000 is a diagnostic budget and not a submission one, which is a distinction worth being clear on: see how many URLs you can actually request indexing per day.

What the answer does not include

Bulk inspection tells you the state of every URL and the reason for it. It does not tell you what to change, and this is where the exercise usually stops being useful.

A list showing 340 URLs at crawled, currently not indexed is a complete, accurate, correctly gathered answer that leaves you exactly where you started. Google fetched those pages, decided they were not worth storing, and will keep deciding that until something changes. Inspecting them again next week returns the same 340. Neither will requesting indexing move them, because submission was never the missing step.

The causes are findable and boring: thin content, near-duplicate pages competing with each other for the same intent, weak internal linking that leaves a page with nothing vouching for it, or a technical block nobody noticed. Usually two of them together.

Naming which one is the actual work. Run your list through the free checker first so you have the evidence in hand, then send it to me. The diagnosis is free and you get the cause in plain English rather than another tool recommended. Where there is real work to do, indexing work starts at $25 with a per-URL ledger showing what changed and when.