What is the fastest way to get staging out of Google?

Use the Removals tool in Search Console for the staging host, then fix the cause. The tool removes pages from Google search results within a day, but the removal lasts about 6 months and hides rather than deindexes. The permanent fix is blocking the staging host so no crawler can fetch it.

Should I block the staging crawlers with robots.txt first?

No. If you block crawling first, the crawler cannot see the noindex on the pages, so the pages stay in the index without a description. The correct order is: allow the crawl, serve noindex or remove the pages, wait for reprocessing, and only then block. Google's own documentation warns against using robots.txt to block a page.

Does the Removals tool deindex a page?

No. Google's documentation says requests made in the Removals tool last for about 6 months. The tool hides the page from search results for that window and requires you to verify the property in Search Console. After the window, the page can appear again unless the cause is fixed.

Why does authentication beat noindex and robots.txt for staging?

A page behind HTTP authentication or an IP allowlist cannot be fetched by anyone unauthenticated, including crawlers. A page that cannot be fetched cannot be indexed, and it does not depend on a directive being read correctly. Noindex and robots.txt both rely on the crawler fetching the page first.

How did staging get indexed in the first place?

Usually one of: no authentication on the staging host, a robots.txt that was correct in one environment but not the other, an environment variable wrong at build time, a staging sitemap listing staging URLs, or external links to staging from a ticket or a Slack unfurl. Any of those can point a crawler at staging URLs.

What should I check after staging is out of the index?

That production is unaffected, that no canonical on production points at staging and none on staging points at production, and that the staging host does not appear in the production sitemap. Those are the three ways the two environments leak into each other.

A staging site got indexed by Google. How to get it out, and how to stop it happening again.

Your staging host is in the Google results, possibly duplicating your production pages. Get it out in this order: allow the crawl, serve noindex or remove the pages, wait for reprocessing, and only then block. The order is the whole trick, and getting it backwards is what makes the pages stay. This post covers why it happened, the order that fixes it, what the Removals tool actually does, and the fix that makes it stop.

Why it happens

A staging site gets indexed when a crawler can reach it and the pages give no signal that they should not be indexed. The usual causes:

No authentication on the staging host, so any crawler can fetch it.
A robots.txt that was correct in one environment and not the other. A rule that allows everything on staging, or a rule that blocks the live site’s paths only, leaves staging open.
An environment variable wrong at build time, so the live noindex rule never made it into the staging build.
A sitemap on staging that lists the staging URLs, giving the crawler a map of the pages.
External links to staging, from a ticket, a chat, or a Slack unfurl. A single public link is enough for a crawler to follow.

The urgent part, in the right order

Do the steps in this order, and know why. If you block crawling first, the crawler can no longer see the noindex on the pages. The pages then stay in the index without a description, which is worse than the original problem. The same trap as the robots.txt versus noindex confusion, and it catches almost everyone in a panic.

Allow the crawl. The crawler must be able to fetch the page and read the noindex on it.
Serve noindex or remove the pages. Add noindex to the pages, or take them down, so the response carries the signal.
Wait for reprocessing. Google has to re-crawl the page and see the noindex. Use Search Console’s URL Inspection to request a re-crawl, and confirm the page is reported as not indexed.
Only then block. Once the pages are out, you can disallow the staging host in robots.txt, or better, block it at the network level.

Google’s own documentation warns against the wrong order. From “Remove a page hosted on your site from Google”: “Don’t use robots.txt as a way to block your page.”

What the Removals tool actually does

The Removals tool in Google Search Console removes a page from Google’s search results quickly, but it is a temporary hide, not a deindex. Google’s documentation says: “Requests made in the Removals tool last for about 6 months.” After that window, the page can appear again unless the underlying cause is fixed.

The tool requires that you own and verify the property in Search Console. If the staging host is not a verified property, you cannot use the tool on it. And because the removal hides rather than removes, the permanent fix is the same: block the staging host so it cannot be fetched, or put noindex on the pages and keep it there.

The permanent fix, ranked

HTTP authentication or an IP allowlist on staging, so no crawler can fetch it at all. This is the best fix because it does not depend on a directive being read correctly. A page that cannot be fetched by anyone unauthenticated cannot be indexed, period. This is the fix Google’s own docs recommend first, as password-protecting your page.

Noindex on the staging pages, kept there. This works as long as the crawler can fetch the page and the directive survives deploys. It is the same approach as above but relies on the directive staying in place.

The staging host blocked in robots.txt, after the index is clear. This stops new crawling once the pages are gone, but it cannot fix pages already indexed, which is why it comes after the other two.

What to check after

Production is unaffected. Confirm production pages still index and no production traffic fell while you were fixing staging.

No canonical on production points at staging, and none on staging points at production. A canonical from production to staging hands the production page’s signals to a host that should not have them, and a canonical from staging to production does the reverse.

The staging host does not appear in the production sitemap. If the sitemap generation picked up the staging host, the crawler sees it as a source of pages. Remove it and resubmit.

A staging site that cannot be fetched cannot be indexed. Put the gate on the host and the problem stops being a recurring search problem, and the fix is the kind of thing that is worth checking on the production host and staging hosts alike.

Where these facts come from

The Removals tool behaviour is quoted from Google’s own help page, which states that “a successful request lasts only about six months” and that “blocking a URL does not prevent Google from crawling your page, only from showing it in Search results”. That second sentence is the reason the tool is a stopgap and not the fix: see Removals and SafeSearch reports tool.