Streaming live at 10am (PST)

Page Blocked / Crawling Issue


#1

Hi there!

We blocked a subdirectory using robots.txt (on purpose) and later we removed it but this page is still not being crawled. Could you please tell what could be the reason?

URL: https://www.userlane.com/solutions/jira-software-training
Robots.txt: https://www.userlane.com/robots.txt.
Sitemap: https://www.userlane.com/sitemap.xml

FACTS:

  1. I have created a customized sitemap and including the page
  2. Sub-directory “solutions” has been removed from the robots.txt.
  3. Page is not blocked by the meta-noindex tag

Any help will be highly appreciated!


#2

It takes some days for Google to scan and re-index pages.

1-google-console Fetch the pages:

2-And also under google-console re-submit your robots.txt (“test”)

https://janit.iki.fi/force-update-robots.txt-google/

Summary:

Steps 1 & 2 - should make the procces faster (instead of waiting for an automatic update).


Anyway this URL already index:
use site: docs
URL: https://www.userlane.com/solutions/jira-software-training