Noindex

noindex is an instruction you place in a page’s HTML head (<meta name="robots" content="noindex">) or in an HTTP header (X-Robots-Tag). It tells search engines: you may read this page, but do not store it or show it in results. It is not the same as blocking in robots.txt — that stops crawling, which is a different job and often the wrong one.

Use noindex for pages that must exist for users but should not compete in search: thank-you and confirmation pages, internal search results, thin filter permutations, staging content, paginated archives you do not want ranking. One critical detail: never noindex a page and block it in robots.txt at the same time — if crawlers cannot fetch the page they cannot see the directive, so it may stay indexed indefinitely. And remember a noindexed page stops passing value onward once dropped, so do not use it on pages carrying meaningful internal links or backlinks; canonicalise or redirect those instead.

The recurring senior failure is scale: a noindex left in a template after a migration, silently de-indexing an entire section. Make robots directives part of deploy QA, monitor indexed-page counts as a health metric, and diff staging against production before release. Strategically, noindex is one lever among four — noindex, canonical, redirect, delete — and choosing correctly depends on whether the page has users, has links, and has intent. Reaching for noindex when the honest answer is “this page should not exist” only hides the mess from Google while leaving it in your architecture.