Streaming live at 10am (PST)

Cant add unicode characters past 12000


#1

I've got characters in smaller ranges to come through e.g.

ℹ⌂⌂⏭⏭⏮⏮␡║

e.g. from … to  i've tested.

However from 👤 onwards they are getting stripped out of the HTML when i publish and inspect the HTML.
1. Why is this?

What am i trying to achieve? I make my own icon fonts, and when inserting the unicode characters into the webflow preview I get to see and style them, and then in the published site I can see the final output as i've got some custom code embedding the fonts and replacing the glyphs. This all works just fine, but wondering why the restriction on the unicode range?


#2

Very strange, maybe its just an oversight? I'm curious to see your icon work.


#3

I mainly take icon sets i've bought and arrange them in a compact font of only the icons needed for the brand to be used cross platform e.g. website & mobile. Some mobile app examples here http://dribbble.com/aidangreen

But prefer to use the unicode that describes the icon e.g. http://www.amp-what.com/unicode/search/ so annoying the full range its not available.


#4

So you also build native apps for IOS and Android? Here is a reference I often use. http://unicode-table.com/en/ for slider arrows and such.


#5

You're right @aidan Webflow seems to be stripping the higher number character. I tested this using http://www.ftrain.com/unicode/

The character shows up in the designer but get stripped out on publish and export. We'll look into this and get back to you.

There aren't many characters after 12000 that would seem useful unless you are making a Chinese website. Which characters are you trying to use? I'd say most of the useful symbols are below 12000.


#6

Thanks for the reply Sergie, when looking at the unicodes of a number of font sets, I found that the https://symbolset.com/ guys chose some great unicodes that described the icons, so i've been choosing codes that best describe the symbol e.g.:

http://www.amp-what.com/unicode/search/bust
http://www.amp-what.com/unicode/search/credit%20card
http://www.amp-what.com/unicode/search/computer
http://www.amp-what.com/unicode/search/mobile
http://www.amp-what.com/unicode/search/camera
http://www.amp-what.com/unicode/search/video
http://www.amp-what.com/unicode/search/speaker

I know there are not default symbols showing up for all of these, its more of a semantics of the unicode and choosing the ones that best describe the icon designed.


#7

Totally @aidan. Thanks for the additional feedback. We're looking into why characters after 12000 are stripped. I'll rally back with answers.


#8

Hi @aidan - the reason why we're stripping out unicode characters after a certain number is because of the way some people copy and paste text into a Webflow page. For example, if you're using Microsoft Word and copy paste the text into your Webflow canvas and publish it, the copy/pasting will carry some gnarly characters across. Some browsers will have difficulty parsing that document due to those foreign unicode characters (from the browser's perspective).

The way we've dealt with it is by stripping out entire ranges of unicode characters. If you're curious, this is the range we replace:

unicodeString.replace(/[\u0000-\u0008\u000B-\u000C\u000E-\u001F\uD800-\uDFFF\uFFFE-\uFFFF]/g, '');

#9

Thanks for the explanation, Ah the loverly joys of Microsoft word.

In this case I was able to get around the issue by pasting the unicode characters into the custom code section as CSS and then using content attribute to get them back onto the page.

e.g.: this in the CSS
.cc-alert:before,.cc-alert.right:after{content:'⚠'}

and the html something like this

It makes for more legible code too anyway in the case of icons, so no love lost. You just don't get to see the icons in the preview which is ok.


#10

Bugger, the published site is different to the exported code.

I thought adding the characters to the custom code section was ok, as it was working just fine in the published site, however when exporting the code and about to put the site up. The exported code is messing with the characters.

This is from the published site:

.cc-lock:before,.cc-lock.right:after{content:'🔒'}

this is from the exported site

.cc-lock:before,.cc-lock.right:after{content:'🔒'}

Any chance we can get the export to match the published site?


#11

Hi @aidan - could you share your sitename so I can take a look?

edit: found out that it has to do with the TextEncoder API we use to ZIP up the code: https://developer.mozilla.org/en-US/docs/Web/API/TextEncoder It seems to handle UTF in a special way.

Can you describe how it's translating your UTF codes? I don't have these characters installed locally so I can't really see what characters you're trying to use.


#12

Hi guys,

Any additional developments on this topic? I am struggling with this issue as well because I use Entypo webfont. This font seems to use Unicode above the 12000 range as well.

Thanks
Martin


#13

I haven't re checked in a few weeks but i got around it by uploading my css that references the unicodes above 1200 to an external server and referencing them in the custom code section e.g.

Obviously not ideal, but a work around to get you out of trouble.


#14

Hey, there was a change made for this, can you please check out if you are still experiencing the issue? There should be no more char stripped over 12000 unicode range.


#15

Works fine with my fonts now. Thanks!


#16