Hello fellas, i’ve got an issue… well, actually i’ve got several issues that i can’t fix on my own.
To give a bit of context, i work for a company that already had a website running (https://www.kimmy.pt) but they wanted to update the website with a new look and so i chose webflow to make a new one (hosted with webflow, of course), but now that i wanted to speed up the google indexing thingy so that it updates with the new links etc, ive came across a few issues. Ill describe them below.
Problem numero uno: In the google search console thingy, when i try to add a new sitemap.xml to it, it gives me an “http general error” which i can’t fix. In my webflow project settings i have the “auto-generate sitemap” option on, so google console should accept the new sitemap.xml right?
Second problem: In the google’s inspection url tab, when i make it inspect “https://www.kimmy.pt”, google detects it and it tells me it’s on google, but the actual content is refering to the old website (old links etc), and when i request a new indexing it literally rejects it and tells me that there were problems detected with the url indexing. When i click to see the test results it tells me the url is not on google… So… I don’t get it, the previous url was on google but the new one isn’t? What gives here? Or maybe i’m missing something…?
Problemo number three: When i open up google’s robot.txt tester, it tells me robots.txt doesn’t even exist, and when i try to submit a one it tells me to refresh in a minute, which i do, but then the same error happens. However when i click the link “See live robots.txt” it gives me the code which i added in the webflow project settings:
User-agent: *
Allow: /
So, it detects the code but it doesn’t detect the actual robots.txt file which should be created automatically by webflow?
Ive googled about these issues a lot but i haven’t been able to fix any of them. Any help would be highly appreciated
I was under the impression that i could use either the prefix option or the “entire” domain option. But anyways ive tried to use the “entire” domain option but it wouldn’t validate so i contacted the domain dns administrator and he added the google verification ID line, but i’m guessing now it takes time to actually update. And yes i’ve also added the google ID to my project’s settings.
As for the prefix option, which is where the issues described in the main post occurred, it “works” fine i guess, it detects data from the previous website but it doesn’t index the new pages.
How long ago did the DNS Record get added for validation? If it was very recently it might just still be propagating, if you’re lucky! You could check this using something like https://www.whatsmydns.net/ to see if it’s fully propagated yet.
You mean the google ID line? It was added 2 hours ago. I apologize if i seem to be rushing this, i’m not, it’s just that i’m not exactly an expert in this sort of areas.
Yeah that could be it. Don’t apologise it could just take a few hours. I have the same problem when I do anything DNS-related - I get impatient and have to leave my computer for a few hours haha.
So i’ve tried, again, to validate the option which wasn’t working, the option with the “entire” domain thingy, and it has validated successfully. So i guess enough time has passed and the DNS has updated.
The problem with that is, because webflow does not have email hosting, we had to keep the previous hosting just for emails, and so my dns admin changed i guess the IP (?) value and not the CNAME value because, as he explained it to me, it would “deactivate” the emails thingy.
Anyway, to be sincere, I don’t think that adding that CNAME value would affect email (I’m pretty sure about that). Try to tell to domain admin that he need to ADD the cname value and not to modify the existing ones (except for the www value, but it is not related to email settings).
@Kimmy, just to back up what @patunz is saying, this is correct. For example I currently have 4 CNAME records assigned to one of my client’s domains, covering different things like domain email, the website, and a forum mailing service.
it could just take a few hours. I have the same problem when I do anything DNS-related - I get impatient and have to leave my computer for a few hours haha.
