Google and other search engines scan of the Internet for new data. To find new or updated content, crawlers, and spiders crawl web pages.
Shopify store owners may create robots.txt to stop search engines from crawling your website content. This will decrease your website’s search engine optimization.
Google’s guide on robots.txt states that robots.txt tells search engines which pages or files they can request. We will later discuss how to keep Shopify pages from Google.
This article will help you find your Shopify robots.txt file, hide pages from search engines, and whether it can be edited.
Search Shopify Robots.txt. File and Submit Your Sitemap
Shopify robots.txt is located in your root directory for the primary URL of your Shopify store. This will look like the domain name slashbots.txt in this example.
examplestore.com/robots.txt
Submitting Your Sitemap
You may think that robots.txt has blocked a specific page, file, or content from your Shopify store. You can edit it to make it indexable by Google and other search engines.
You don’t have to change your robots.txt for Google to index your website content.
Google can receive your sitemap and let you know everything about your website, including files and updates. Sitemaps are automatically generated and are located in your Shopify store’s root directory. I.e., Katies-apperal.com/sitemap.xml.
Shopify sitemaps contain links to all products and pages. Search engines use this file to index your website to ensure that your pages appear in search results.
- Basic Shopify Plan: This plan only applies to your primary domain. It is searchable by search engines but cannot be used elsewhere.
- Shopify Plus Plan, Advanced Shopify Plan: Sitemap files created for all domains you use.
Search engines can take some time to crawl and index your pages Google cannot guarantee how long. You can manually request indexing or submit your sitemap to Google Search Console.
How to submit your Shopify sitemap via Search Console
Step 1. Go to your Search Console account. Click ≡ and select Add Property.
Step 2. From the Select Property Type box, select URL Prefix. Then, enter the Shopify domain you want to add as a property. (Include the https:// part.)
Step 3. Click to Continue. Select HTML Tag on the Verify Ownership window.
Step 4. Highlight the full HTML tag and copy it to your clipboard.
<meta name=”google-site-verification” content=”IV7BPLESttSpBdxSWN1s4zlr4HIcuHkGQYmE3wLG59w” />
Step 5. Go to Shopify Admin > Online Store > Themes.
Step 6. Find the theme you wish to edit and click Actions, then Edit Code.
Step 7. See the Layout section, find, and click theme.liquid.
Step 8. Paste the tag you copied on a blank line directly below the opening <head> tag.
Step 9. Save and return to Search Console to click Verify
If you receive an error message, wait 15 minutes. You can click Verify again.
Edit Shopify Robots.txt
Your question “Can I edit my Shopify Robots.txt?” answered no. Shopify handles robots.txt files for you automatically. They will do it right, so you can be sure.
Shopify blocks the visibility of your shopping cart page in search engines. This improves SEO as it makes it easier for customers to find your product pages in search results.
This is to make sure that your product pages rank higher in search engines than your checkout pages.
How do I hide Shopify’s pages from search engines?
Shopify cannot modify your robots.txt files, but you can hide certain pages from them. Customize the Section in your store’s design.
1. Visit your Shopify admin page to view the themes.
2 Choose the theme you want to edit Click Actions> Edit Code.
3. Click theme.liquid.
4. To exclude the search template, copy the code below.
When the template includes search, % and % endif
5. To exclude a page, copy the code below. Section:
% if handle contains ‘page-handle-you-want-to-exclude’ % % endif %
- Don’t forget to replace page-handle-you-want-to-exclude with the correct page handle.
6. You can save changes.
Source Shopify
What is a Robots.txt File?
A robots.txt helps you manage crawler traffic to the website and sometimes to keep a page from Google, depending on what type of page.
Web page. While robots.txt may be useful in managing web page traffic, it cannot hide web pages from Google search result results.
Media File: The robots.txt file is used to manage traffic and stop images, videos, and audio files from appearing in Google search results.
Resource file: Robots.txt may block resource files such as unimportant scripts, images, or style files. It can also help you manage the traffic to these files.
How can you keep Shopify pages from being removed by Google?
To protect your website, you can use noindex directives or to password-protect it if it is not accessible by Google.
Learn about the Limitations of Robots.txt
- A page with robots.txt that has been linked to other websites can still be indexed by Google.
- Different crawlers index syntax differently. Some people may not understand certain instructions.
- The crawler has the option to follow your robots.txt instructions or not all search engines support robots.txt directives
Note: You must test your Shopify robots.txt files in all cases. To do this, you can use Google Robots.txt testing.
