Streaming live at 10am (PST)

Webflow and Web Scraping

Hello,

My client is in contact with Indeed in an attempt to set up web scraping for their website.

Basically the idea is that Indeed would pull info from the “careers” page of my client’s website to add job listings to indeed.com.

However, Indeed has told him that they are not able to access the page due to geoblock settings on the website.

Is web scraping possible with Webflow? and if it is, what do I need to do to get it working?

Any help would be much appreciated.

Thanks,

Reid

Hi Reid,

Is your clients website hosted on Webflow?

Omar

Yes, I wouldn’t be here otherwise lol.

I designed and built the site myself using Webflow.

I’m not aware of Webflow having geo restrictions as an offered feature, and if they did I’d imagine you would remember setting it up for your client. First thing I’d do is ask Indeed what country their scraping IPs are in. I’d assume they’re using U.S. based IPs to scrape, being a U.S. company, so them being geoblocked is odd to me. Webflow certainly wouldn’t be geoblocking the U.S.

Okay, I’ll have my client reach out regarding this.

Will let you know what they say.

Thanks!