Streaming live at 10am (PST)

Is there any standard format of Robots.txt file for WordPress website?


#1

Hello Friends,

I am having a website in WordPress, there are certain pages of the website which contains wp-include and gives 404 error in Webmaster tools.

Do I need to block these page from Robots.txt? Is there any standard format of Robots.txt file for website which is built in WordPress. I am asking for standard format so that I can block those page from rendering 404 error, before I get any 404 error in the webmaster.

Hope, you guys are clear with my issue. Please help me with the Solution.


#2

Here is a great discussion: https://wordpress.org/ideas/topic/wordpress-needs-a-default-robotstxt-file-and-more


#3