Bug 22983 - The Koha copy of NotoSans font (in the new OPAC) lacks glyphs for (some) Roman letters with diacritics
Summary: The Koha copy of NotoSans font (in the new OPAC) lacks glyphs for (some) Roma...
Status: NEW
Alias: None
Product: Koha
Classification: Unclassified
Component: OPAC (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal (vote)
Assignee: Owen Leonard
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-05-23 19:26 UTC by Janusz Kaczmarek
Modified: 2022-10-28 03:40 UTC (History)
2 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
NotoSans rendered incorrectly (missing) for Polish letters (146.96 KB, image/png)
2019-05-23 19:26 UTC, Janusz Kaczmarek
Details
Firefox 67.0, Windows 10 (44.46 KB, image/png)
2019-05-24 13:02 UTC, Owen Leonard
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Janusz Kaczmarek 2019-05-23 19:26:44 UTC
Created attachment 90054 [details]
NotoSans rendered incorrectly (missing) for Polish letters

The patches of Bug 20554 introduced a new pretty NotoSans font family as basic font for OPAC.  A decision was taken to serve the font files from Koha installation, not from Google font API.  Unfortunately, the copy of the font files in Koha lacks non-standard (probably those outside iso 8859-1 set?) accented Roman letters (among others, Polish).  

As a result the user is presented with the page which is--typographically--hard to accept (cf. attached screenshot -- Windows 10, Chrome 74.0.3729.169; BTW, on my Android 9 everything seems to be OK -- probably thanks to the local, system copy of the font?).

Possible solutions could be:
1) update the Koha copies of the font files with the up-to-date version
2) let css get the font from Google font API (href="https://fonts.googleapis.com/css?family=Noto+Sans&display=swap")

Can be tested--for instance--with Polish letters: 
ACELNOSZZ
ĄĆĘŁŃÓŚŻŹ
acelnoszz
ąćęłńóśżź
Comment 1 Owen Leonard 2019-05-24 13:02:02 UTC
Created attachment 90060 [details]
Firefox 67.0, Windows 10

Curiously, it seems to work correctly in Firefox. I can reproduce the problem in Chrome.
Comment 2 Janusz Kaczmarek 2019-05-24 18:34:58 UTC
Thank you, Owen, for checking this.  I can confirm: the rendering of the fonts is (almost) OK in FF 67.0 (almost = in my case the non-bold paragraphs show the problem mentioned). 

After having investigated more deeply: it seems that FF and Edge use some local copy of the font:  Removing the part "local("Noto Sans *")," from each @font-face definition (e.g. with the developer tool of FF) causes the problem occur in FF and in Edge for every variant (regular and bold) as it is in Chrome. 

I have also used the following html file to test the issue fully locally (and had the Koha version of the font locally downloaded):

<!DOCTYPE html>
<html lang="en">
  <head>
    <title>Zażółć gęślą jaźń</title>
    <meta charset="UTF-8" >
    <meta name="description" content="Zażółć gęślą jaźń" >
  </head>
  <body>
    <style>

@charset "UTF-8";
@font-face {
 font-family:NotoSans;
 font-style:normal;
 font-weight:400;
 src:local("Noto Sans Regular"),
 url(NotoSans-Regular-webfont.woff2) format("woff2"),
 url(NotoSans-Regular-webfont.woff) format("woff")
}
@font-face {
 font-family:NotoSans;
 font-style:bold;
 font-weight:700;
 src:local("Noto Sans Bold"),
 url(NotoSans-Bold-webfont.woff2) format("woff2"),
 url(NotoSans-Bold-webfont.woff) format("woff")
}

body {
 font-family:"NotoSans";
}
    </style>

    <h1>Zażółć gęślą jaźń</h1>
    <h2>Zażółć gęślą jaźń</h2>
    <h3>Zażółć gęślą jaźń</h3>
    <p>Zażółć gęślą jaźń</p>
    <p><b>Zażółć gęślą jaźń</b></p>

  </body>
</html>


BTW, after inserting the line:
@import url('https://fonts.googleapis.com/css?family=Noto+Sans&display=swap');
in <style> and changing the font-family for body in "Noto Sans" (with space) = using the Google API version, the rendering is OK (also with no 'local' part in declaration).
Comment 3 Owen Leonard 2019-05-24 18:51:42 UTC
(In reply to Janusz Kaczmarek from comment #2)

> = using the Google API version, the rendering is OK (also with no 'local'
> part in declaration).

My goal will be to figure out how to solve the problem without pulling the data directly from Google. Doing so opens us up to Google data-collection.
Comment 4 Janusz Kaczmarek 2019-05-24 19:15:04 UTC
(In reply to Owen Leonard from comment #3)

> My goal will be to figure out how to solve the problem without pulling the
> data directly from Google. Doing so opens us up to Google data-collection.

https://google-webfonts-helper.herokuapp.com/fonts/noto-sans?subsets=latin-ext,latin

?

But how other alphabets? Greek, cyrillic and others?  Loading everything would not be create performance issues?

One could also download and insert to Koha the Google versions and use their css (after modifying URLs) (to be consulted here: https://fonts.googleapis.com/css?family=Noto+Sans).  Fonts can be downloaded, but I succeeded only with woff2.
Comment 5 Janusz Kaczmarek 2019-05-24 19:18:29 UTC
Forgot to say: I have checked the fonts downloaded from google-webfonts-helper.herokuapp.com -- seems to be OK (tested for Latin extended).
Comment 6 Jonathan Druart 2019-06-08 20:09:17 UTC
Owen, are you working on this?
Comment 7 Owen Leonard 2019-06-10 12:00:36 UTC
(In reply to Jonathan Druart from comment #6)
> Owen, are you working on this?

I am not.
Comment 9 Janusz Kaczmarek 2019-06-10 15:09:14 UTC
(In reply to Jonathan Druart from comment #8)
> Should we follow:
> 
> https://google-webfonts-helper.herokuapp.com/fonts/noto-sans?subsets=latin,
> greek-ext,devanagari,cyrillic,latin-ext,vietnamese,greek,cyrillic-ext
> 
> Download and include them all?

Entire WOFF2 set (Regular, Italic, Bold, BoldItalic) = 546 KB
Entire WOFF  set = 714 KB

Now it is eight times less (71 KB, 89 KB, respectively).

If it is OK (the browser would keep it normally in cache, wouldn't it? -- if so, then it would be downloaded only once, in general) -- let's do it.
Comment 10 Jonathan Druart 2019-06-23 01:57:24 UTC
(In reply to Janusz Kaczmarek from comment #9)
> (In reply to Jonathan Druart from comment #8)
> > Should we follow:
> > 
> > https://google-webfonts-helper.herokuapp.com/fonts/noto-sans?subsets=latin,
> > greek-ext,devanagari,cyrillic,latin-ext,vietnamese,greek,cyrillic-ext
> > 
> > Download and include them all?
> 
> Entire WOFF2 set (Regular, Italic, Bold, BoldItalic) = 546 KB
> Entire WOFF  set = 714 KB
> 
> Now it is eight times less (71 KB, 89 KB, respectively).
> 
> If it is OK (the browser would keep it normally in cache, wouldn't it? -- if
> so, then it would be downloaded only once, in general) -- let's do it.

I have no idea how to proceed here. 1Mo to open a library's home page, for the first time, in some countries/cities it will not be acceptable (I would close the page thinking it crashed).
Isn't there a best practice to follow? I am pretty sure we are not the only ones facing this problem.
Comment 11 David Cook 2022-10-28 01:49:59 UTC
Is this still an issue? It looks like the font family CSS has moved on a bit over the past few years...

--

I haven't seen this issue personally, but I mostly work in English, French, Arabic, and Chinese. English and French are included in the Noto Sans font, while Arabic is in Noto Sans Arabic font and Chinese has a few Noto Sans Chinese fonts. 

I'm not a font expert by any means, but recently I created a custom NotoSansTC-Regular TFF file for label printing in Chinese and English. That's a very different use case but I learned a lot about fonts in the process.

Little tip for troubleshooting fonts in Chrome: 
- Inspect the element
- Go to "Computed"
- Scrolldown to the very bottom to see a "Rendered Fonts" section. 

I'm looking at an OPAC with English and Arabic right now, and it's telling me that it's rendered 11 glyphs using a local file for Segoe UI and 2 glyphs using a local file for Noto Sans. A bit of trial and error proved the 2 Noto Sans glyphs were the space character and a punctuation mark. 

When using the Google Fonts tester, I type in some Polish and I can see that it is using "Noto Sans - Network resource" for the 16 glyphs. 

--

Now back to Koha... if "fonts/NotoSans-Regular-webfont.woff2" is being used, that would mean that "Noto Sans Regular" isn't available locally. 

But if the glyph isn't available in "fonts/NotoSans-Regular-webfont.woff2", it should move on to "sans-serif" (when this issue was raised, I don't think the CSS was "NotoSans, sans-serif" so maybe that was contributing to the problem).

But even without "sans-serif", it should still resort to a default system font (like "Times New Roman" on English Windows systems).

--

So ultimately this one sounds like a misconfigured local system to me.
Comment 12 David Cook 2022-10-28 03:30:19 UTC
As an aside, I find this file from Google to be interesting:
https://fonts.googleapis.com/css2?family=Noto+Sans&display=swap

Latin, Greek, Cyrillic, Vietnamese, Devanagari are listed... yet Noto Sans describes itself as a font for Latin, Cyrillic, and Greek at https://fonts.google.com/noto/specimen/Noto+Sans.

That said, it does say that it's "also suitable as the complementary choice for other script-specific Noto Sans fonts". If you look at https://fonts.google.com/noto/fonts?noto.continent=Americas, you can see that it says Noto Sans supports 855 languages. 

There are a lot of languages that Noto Sans doesn't support though. Notable exceptions are Chinese, Japanese, Korean, and Arabic. They have their own fonts. 

Even Devanagari has its own font, so it's interesting to see it included in that top link...

I've played with TTF and OTF files, but I don't have experience with WOFF files. Check out the imports apparently needed to support the Noto Sans TC font: https://fonts.googleapis.com/css2?family=Noto+Sans+TC&display=swap

That's 101 different URLs. 

Perhaps it's for performance reasons. As you can see with "unicode-range", the browser will determine which source file to use depending on the Unicode code point of the character it is rendering. By splitting up the files, you'd be able to download faster since your browser can use something like 8 concurrent fetches and probably lazy-load the source file you need. 

--

Overall, I think it's best to rely on local system fonts... although I can see wanting to use web fonts to have a more consistent font display across browsers.
Comment 13 David Cook 2022-10-28 03:40:48 UTC
Oh interesting wrinkle I hadn't thought about... the content of https://fonts.googleapis.com/css2?family=Noto+Sans&display=swap is dependent on your user agent. It's WOFF2 with Chrome but it's a different format for other browsers.

But for Janusz's case... well I wonder where the original NotoSans-Regular-webfont.woff2 files came from. 

I just noticed the screenshots and actually... this would need some more analysis using that "Rendered Fonts" I mentioned.

So at this stage I'd say it's more a support issue than a bug report per se.