Streaming live at 10am (PST)

NOINDEX pages / page protection problem

I published a webflow site (empathconnection.com) and checked the “Exclude this page from site search results” for several pages. The “auto generate sitemap” was turned on. We launched the site, submitted the sitemap google and then ran a search “site:empathconnection.com” and saw that the EXCLUDED pages were showing up in the list.

I then tried the suggested “noindex” code on the specific pages, but the problem with that is, the pages are still listed in the sitemap and therefore anyone who knows how to crawl for sitemaps (specifically the thieves who steal material downloadable after purchase) can gain access to the pages.

I understand that all pages need to be in the sitemap so that when google crawls the pages, it will see the “noindex” command – or do they???

Can anyone tell me if the answer to this problem is:

leave the noindex command on the individual pages
submit a custom sitemap to google that excludes the noindex pages

Advice?

ADDITIONAL QUESTION: What will happen if I add the Google Analytics tracking code to the header of a page that is “noindex”? My reason for doing this is to see how many people are actually accessing those pages versus the number of people who I know paid to access those pages.

Exclude this page from site search results, only excludes the page from, wait for it, site results. So the website’s own search function. The noindex code will make it clear to google to not show those pages anymore.
If you’re so concerned about the content being hidden, why not just add a password? That’s a built-in webflow feature, that way the only people accessing the content are the ones who paid and got the password.

Thank you, Sarah, for your feedback. I misunderstood “site search results” as search engine results. My bad.

We are going to use a password for those pages where a customer purchased some downloadables, but I guess the logic of submitting a site map to Google that includes the URLS of noindex pages doesn’t seem logical to me. They could still easily be accessed by anyone searching on a sitemap.

As would a user sharing a page URL or potentially just visiting one, with tracking scripts. If a page is published on the web, it is accessible. As @sarahfrison said password protect or don’t publish (draft).

1 Like