Robot.txt plays an important role when Google and other search engine crawlers index the website or store. The Robots Meta tag applied in the website pages controlling the correct index and served in search results. No need to serve all pages of the website on the search engines. In most cases, the search engine crawlers don’t need to crawl through your entire website or store. Therefore, this needs to carefully select which pages to index, follow, noindex or nofollow.
Magento 2 NoIndex NoFollow Extension allows the store owner to prevent products from indexed and follow from Google and other search engine crawlers. Also, it allows set noindex nofollow on the category pages and on all products of that category pages to prevent from the crawlers. noindex nofollow extension applied in CMS pages of the website to ignore from the crawlers. Also, the store owner set noindex nofollow extension for any custom URLs.
NoIndex NoFollow for Magento 2 extension enables or disable web crawlers to crawling, indexing and follow links on the web page. This extension prevents your store content from any replicate content, also restricts your confidential pages from indexing.
- Set noindex nofollow extension for any custom URLs
- Control search engines to pass the link juice to other pages, which prevents the SEO score of the store
- Prevent products from indexed and follow from crawlers
- Prevents the store content from any replicate content
- Restrict your confidential pages from the indexing
- Product Page: http://demo.cynoinfotech.com/d4m2/joust-duffle-bag.html
- Category Page: http://demo.cynoinfotech.com/d4m2/men/tops-men/jackets-men.html
- CMS Page: http://demo.cynoinfotech.com/d4m2/customer-service
- Username : ninft_demouser
- Password : ninft_demouser123
Any combination of Index & Follow Tag
With noindex nofollow Magento 2 extension store owner can set any four Meta robot.txt combinations to controlling search engine crawlers:
- Index, Follow – Allows web crawlers to INDEX the page & continues to FOLLOW the links in the webpage.
- Noindex, Nofollow – Control search engines from INDEXING the webpage and FOLLOWING the links.
- Index, Nofollow– Allow web crawlers to INDEX the web page but NOT FOLLOW the links in the webpage.
- Noindex, Follow – Controls web crawlers from INDEXING the webpage but FOLLOW the links in the webpage.
Add NoIndex & NoFollowTag to Products
prevent products from indexed and follow from Google and other search engine crawlers
NoIndex & NoFollow Tag for Categories
manage your category pages by update robot tags to noindex/nofollow Or noindex/follow and prevent them from being crawled or followed
Add Noindex/Nofollow Tag to CMS pages
NoIndex & NoFollow Tag for Custom URL
Set noindex/nofollow tags for custom URLs like the checkout page, Cart Page, Order Success Page, Custom Extension URL, etc.