Skip to content
DevMarDe
Crawl Audit check 40Crawl AuditTechnical SEO

Meta robots nofollow tag: Crawl Audit check 40

Crawl Audit check 40 tests for nofollow in the robots meta tag, how it differs from rel=nofollow on one link, and when either is the right choice.

Zain Kazmi2 min read
On this page
Check
40 of 200
Module
Technical SEO & Indexability
Group
Crawlability & Indexing
Impact
Medium

Short answer: a page-level nofollow, in the robots meta tag, tells search engines not to follow any link on that page. On a normal page it cuts off every internal link from being followed, which is rarely what you want. Check 40 warns when the audited page has one. To stop search engines following one specific link, use rel="nofollow" on that link instead.

What Crawl Audit checks

  • Warning: <meta name="robots"> contains nofollow.
  • Pass: it doesn't.

Like check 39, it reads the name="robots" meta tag only, not the X-Robots-Tag header.

  • On the page (<meta name="robots" content="nofollow">): Google's definition is "Do not follow the links on this page." All of them, including your menu.
  • On one link (<a href="..." rel="nofollow">): only that link. Google also has more specific values: "sponsored" for "links that are advertisements or paid placements", and "ugc" for "user-generated content (UGC) links, such as comments and forum posts". It suggests nofollow "when other values don't apply".

Google says internal links are how it finds pages ("Google primarily finds pages through links from other pages it already crawled"), so a page-level nofollow on a hub or menu page can leave other pages harder to find.

Where it comes from by accident

  • WordPress: "Discourage search engines from indexing this site" in Settings › Reading adds noindex,nofollow to every page. If check 40 warns together with check 39, this is the first thing to look at.
  • Copied templates from staging or login pages, where nofollow made sense.
  • Next.js: robots: { follow: false } in shared metadata.

When page-level nofollow is fine

Login, cart and account pages, and pages full of user-submitted links you can't vouch for. For everything else, leave links followable and mark individual paid or user links with the right rel value.

Sources

All 200 Crawl Audit checks

See how your site does on every check.

Crawl Audit runs this check and 199 others on your domain. The full report is $7.89, one time. The free check on this site covers 25.