There is definitely something strange going on, the site never even finishes “loading” in Chrome—I get a white screen with a loading icon until I manually cancel it, then the page appears, but it continues to download infinite JPG files:

After a bit of poking around in the code, it looks like a JS file is the culprit—which hangs the site during the initial loading (where I’m getting a white screen) and then causes the excessive JPG file requests:

I’m not too sure why Chrome specifically has an issue with that (Safari and Firefox on MacOS handle it just fine) but it was definitely that specific JS file from Typekit and not anything to do with the previous SVG usage.
That said, I’d still argue that complex SVGs—while smaller in file size—can cause performance issues compared to high quality PNG files (which I’d use over JPGs in this case). Obviously it’s not fixing the root issue, but hopefully you’ve got a better idea how you can troubleshoot it.