noindex
Controlling What Search Engines Should Not Index
Noindex is a search engine directive that tells crawlers not to include a page or resource in search results.
It is one of the most important tools in technical SEO because not every accessible URL deserves to be indexed. Some pages need to exist for users, systems, campaigns, testing, filtering, or account flows, but they should not appear as search results.
noindex does not hide a page from the internet. It tells search engines that the page should not be stored and shown in their index.What Is noindex?
noindex is a directive used to prevent a page from appearing in search engine results.
It can be added as a meta robots tag in the HTML of a page or as an HTTP X-Robots-Tag header. Google supports both methods. The meta tag is usually used for HTML pages, while the HTTP header is useful for non-HTML files such as PDFs or other resources. Google also states that a page must be crawlable for the noindex directive to be seen; if the URL is blocked by robots.txt, Google may not be able to detect the noindex instruction.
A common meta robots implementation looks like this:
<meta name="robots" content="noindex">
This tells search engines that support the directive not to index the page.
What noindex Does
noindex controls indexation. It does not control access, security, crawling permission, or user visibility.
When a search engine crawls a page and sees a valid noindex directive, it should remove that page from the index or avoid adding it in the first place. The page may still be accessible to users, linked internally, visited directly, tracked in analytics, or used as part of a website flow.
For example, a thank-you page after a form submission may need to exist so analytics can record a conversion. But it usually should not rank in search results because users should only reach it after completing the form.
noindex vs robots.txt
noindex and robots.txt are often confused, but they solve different problems.
Control | Main Purpose | What It Does | What It Does Not Do |
|---|---|---|---|
| Index control | Tells search engines not to show a page in search results | Does not prevent crawling by itself |
| Crawl control | Tells crawlers which URLs or paths they should not crawl | Does not reliably remove URLs from the index |
Password protection | Access control | Prevents unauthorized users and crawlers from accessing content | Requires authentication setup |
Canonical tag | Duplication signal | Suggests the preferred version of similar content | Does not guarantee non-indexation |
This distinction matters. If a page is blocked in robots.txt, search engines may not be able to crawl the page and see the noindex tag. For noindex to work reliably, the crawler needs access to the page or resource where the directive is present.
When to Use noindex
noindex should be used when a page has a valid purpose on the site but should not appear as an organic search result.
Common examples include internal search result pages, filtered or faceted pages with weak search value, staging or temporary pages, thank-you pages, login-related pages, account pages, duplicate utility pages, campaign pages that are not meant for organic discovery, and thin system-generated pages.
The goal is not to hide poor work. The goal is to keep the search index focused on pages that are useful, intentional, and appropriate for search users.
When Not to Use noindex
noindex should not be used as a shortcut for every SEO problem.
Do not use noindex on important pages just because they are not ranking well. If a page is strategically important, improve the page instead. Strengthen the content, search intent alignment, internal links, structured data, page experience, and technical foundation.
Do not use noindex when you actually need to protect private information. Noindex is not security. A noindexed URL can still be visited directly if someone has the link.
Do not use noindex when a canonical tag is the better solution. If two pages are similar and one should consolidate signals into another, a canonical tag may be more appropriate. If a page should not appear in search results at all, noindex is the clearer instruction.
How to Implement noindex
The most common method is the meta robots tag:
<meta name="robots" content="noindex">
This should be placed in the <head> section of the HTML page.
For non-HTML resources or server-level control, noindex can also be sent through the HTTP response header:
X-Robots-Tag: noindex
This is useful when you need to prevent indexing of files such as PDFs or when the directive is easier to manage at the server or application layer. Google documents both the meta robots tag and the X-Robots-Tag header as valid ways to apply noindex.
You can also target Google specifically:
<meta name="googlebot" content="noindex">
However, in most normal SEO cases, using robots is cleaner because it applies broadly to search engines that support the directive.
noindex and nofollow
noindex and nofollow can be combined, but they should not be treated as the same thing.
<meta name="robots" content="noindex, nofollow">
noindex tells search engines not to index the page. nofollow tells crawlers not to follow the links on the page. Google’s documentation shows that noindex can be combined with other robots directives, including nofollow.
In many cases, noindex, follow or simply noindex is more useful because the page can still pass crawl paths through its links. For example, a filtered listing page may not need to be indexed, but its links may still help crawlers discover useful products, articles, or category pages.
Use nofollow only when there is a clear reason to discourage link following.
noindex and Canonical Tags
noindex and canonical tags should be used carefully together.
A canonical tag tells search engines which version of a similar page should be treated as the preferred URL. Noindex tells search engines not to index the page. If both are used on the same page, the signals can become less clean because one instruction says “do not index this page,” while the other says “treat another page as the preferred version.”
For most cases:
Situation | Better Choice |
|---|---|
Duplicate or near-duplicate page should consolidate into another URL | Canonical tag |
Page should not appear in search results at all | Noindex |
Page is private or sensitive | Authentication, not noindex |
Page should not be crawled | Robots.txt or access control, depending on the case |
The decision should be based on the page’s role, not just whether the URL feels “unimportant.”
How noindex Affects SEO
noindex can improve SEO hygiene when it is used correctly.
It helps keep low-value, duplicate, temporary, or utility pages out of search results. This can make the indexed version of a site cleaner and easier to understand. It also reduces the risk of search users landing on pages that were never designed as entry points.
However, noindex can damage SEO if it is applied to important pages by mistake. A misplaced noindex tag on a homepage, category page, article, service page, or product page can remove valuable content from search results.
That is why noindex should be treated as a technical control, not a casual content setting.
How to Check If a Page Has Noindex
A page can be checked in several ways.
You can inspect the page source and look for:
<meta name="robots" content="noindex">
You can also check HTTP headers for:
X-Robots-Tag: noindex
In Google Search Console, the URL Inspection tool can show whether Google detected a noindex directive. Google also recommends checking the Page Indexing report to monitor pages where Google extracted a noindex rule.
For technical audits, it is also common to crawl the site with SEO crawling tools and filter URLs by indexability status.
Best Practices for noindex
Noindex works best when it is managed intentionally, documented clearly, and checked before major site changes.
Use noindex Only for Clear Page Types
Noindex should be applied to page types with a clear reason for being excluded from search.
Examples include internal search results, thank-you pages, test pages, staging pages, account pages, or thin system-generated pages. Avoid deciding page by page without a consistent rule, because this increases the risk of mistakes.
Keep Important Pages Indexable
Any page that is meant to attract organic search traffic should remain indexable unless there is a strong technical or strategic reason to exclude it.
This includes core service pages, articles, category pages, product pages, location pages, and other search-focused content.
Do Not Block Noindexed Pages in robots.txt
If search engines cannot crawl the page, they may not see the noindex directive. For noindex to work properly, the crawler needs to access the page and read the instruction.
Audit noindex Before and After Launches
Website launches, migrations, redesigns, CMS changes, and staging-to-production moves are common moments where noindex problems happen.
Before launch, check that staging pages are noindexed or protected. After launch, check that production pages are indexable.
Document Why noindex Was Used
noindex decisions should not be invisible.
For important templates or page types, document why they are noindexed, where the directive is applied, and who owns the decision. This helps prevent future teams from removing or adding noindex without understanding the impact.
Conclusion
noindex is a precise SEO control for keeping specific pages or resources out of search results.
Used correctly, it helps maintain a cleaner search index and prevents utility, duplicate, temporary, or low-value pages from appearing where they do not belong. Used carelessly, it can remove important pages from organic visibility and create serious indexing problems.