ImageFont and non latin characters in a path to font #3145

Activity

1. Jenyay

   changed the title [-]ImageFont and non latin characters in a path to fone[/-] [+]ImageFont and non latin characters in a path to font[/+] on May 29, 2018
2. aclark4life

   added

   Fonts

   on Jun 30, 2018
3. radarhere

   added

   Windows

   on Sep 1, 2018
4. radarhere commented on Sep 1, 2018

   

   radarhere

   on Sep 1, 2018

   Member

   More actions

   Can't reproduce on MacOS either.
5. hugovk commented on Nov 28, 2018

   

   hugovk

   on Nov 28, 2018

   Member

   More actions

   Info from duplicate #3480:

   Reproducible with Chinese characters with:

   • OS: Windows 10 Pro 1803
   • Python: 3.6.5
   • Pillow: 5.3.0

   But not in CentOS.
6. hugovk commented on Nov 28, 2018

   

   hugovk

   on Nov 28, 2018

   Member

   More actions

   Looks like FreeType's FT_New_Face doesn't support Unicode paths on Windows.

   ---

   core.getfont is getfont in _imagingft.c (making it hard to debug on Windows).

   getfont can raise if error is set, and they're all return values from FT_ calls: FT_New_Face, FT_New_Memory_Face, FT_Set_Pixel_Sizes, etc.

   Pillow/src/_imagingft.c

   Lines 268 to 302 in 080bfd3

   	if (filename && font_bytes_size <= 0) {
   	self->font_bytes = NULL;
   	error = FT_New_Face(library, filename, index, &self->face);
   	} else {
   	/* need to have allocated storage for font_bytes for the life of the object.*/
   	/* Don't free this before FT_Done_Face */
   	self->font_bytes = PyMem_Malloc(font_bytes_size);
   	if (!self->font_bytes) {
   	error = 65; // Out of Memory in Freetype.
   	}
   	if (!error) {
   	memcpy(self->font_bytes, font_bytes, (size_t)font_bytes_size);
   	error = FT_New_Memory_Face(library, (FT_Byte*)self->font_bytes,
   	font_bytes_size, index, &self->face);
   	}
   	}
   	if (!error)
   	error = FT_Set_Pixel_Sizes(self->face, 0, size);
   	if (!error && encoding && strlen((char*) encoding) == 4) {
   	FT_Encoding encoding_tag = FT_MAKE_TAG(
   	encoding[0], encoding[1], encoding[2], encoding[3]
   	);
   	error = FT_Select_Charmap(self->face, encoding_tag);
   	}
   	if (filename)
   	PyMem_Free(filename);
   	if (error) {
   	if (self->font_bytes) {
   	PyMem_Free(self->font_bytes);
   	}
   	Py_DECREF(self);
   	return geterror(error);

   And "cannot open resource" is a FreeType error:

     FT_ERRORDEF_( Cannot_Open_Resource,                        0x01,
                   "cannot open resource" )

   https://github.com/aseprite/freetype2/blob/ae3afbc47184db5c7753318b36237af261c66b09/include/freetype/fterrdef.h#L61-L62

   FT_New_Face is most likely, as it passes in the filename.

   ---

   From https://stackoverflow.com/a/10075289/724176:

   Since freetype is meant to be portable, it only uses fopen, which can only accept const char* filenames. So, either load file to memory (or memory map it) and then use FT_New_Memory_Face to create font or convert wchar_t pszFileName into 8-bit encoding, potentially losing characters due to impossible conversion.

   On linux, you could attempt to use setlocale so fopen will accept UTF8 strings, convert wchar_t string to UTF8. However on windows it won't work. So either load file to memory, or convert pszFileName to 8-bit encoding, then pass it to FT_New_Face.

   Both of these are similar issues and contain suggestions to load the file and instead pass it to FT_New_Memory_Face

   • https://freetype.nongnu.narkive.com/XIf73fcO/ft-ft-new-face
   • Refused to open font files with non-ascii names - Cannot marshal: Encountered unmappable character. at SharpFont.FT.FT_New_Face(IntPtr library, String filepathname, Int32 face_index, IntPtr& aface) Robmaister/SharpFont#89

   ---

   And Pillow already does something like this. ImageFont.truetype's font parameter:

   font – A filename or file-like object containing a TrueType font. Under Windows, if the file is not found in this filename, the loader also looks in Windows fonts/ directory.

   If a path is passed in, it uses FT_New_Face. If bytes, it uses FT_New_Memory_Face.

   ---

   I've not tested this on Windows (and it still works on Mac), please try something like this:

   from PIL import ImageFont
   from io import BytesIO
   
   FONT_PATH = "方正黄草简体.ttf"
   
   def open_filename(filename):
       font = ImageFont.truetype(filename)
       return font
   
   def open_data(filename):
       with open(filename, "rb") as f:
           font_bytes = BytesIO(f.read())
       font = ImageFont.truetype(font_bytes)
       return font
   
   # print(open_filename(FONT_PATH))
   print(open_data(FONT_PATH))

   Does that help?

   One of the threads above suggests not to use FT_New_Memory_Face by default, as it loads the whole font into RAM rather than a single font, so we wouldn't want to do that in all cases.

   However, perhaps it could be done on Windows when the filename contains Unicode/non-ASCII.

   👍React with 👍1Reacted by jdhao
7. jdhao commented on Dec 4, 2018

   

   jdhao

   on Dec 4, 2018

   More actions

   @hugovk , it works on my Windows machine. Thanks~

   👍React with 👍1Reacted by Hugo van Kemenade
8. radarhere commented on Apr 10, 2019

   

   radarhere

   on Apr 10, 2019

   Member

   More actions

   I've created PR #3785 to resolve this.
9. hugovk

   closed this as completedin #3785on May 4, 2019