Behdad Esfahbod's daily notes on GNOME, Pango, Fedora, Persian Computing, Bob Dylan, and Dan Bern!

My Photo
Location: Toronto, Ontario, Canada

Ask Google.

Contact info
Hacker Emblem Become a Friend of GNOME I Power Blogger
follow me on Twitter
July 2003
August 2003
October 2003
November 2003
December 2003
March 2004
April 2004
May 2004
July 2004
August 2004
September 2004
November 2004
March 2005
April 2005
May 2005
June 2005
July 2005
August 2005
September 2005
October 2005
November 2005
December 2005
January 2006
February 2006
March 2006
April 2006
May 2006
June 2006
July 2006
August 2006
September 2006
October 2006
November 2006
December 2006
January 2007
February 2007
March 2007
April 2007
May 2007
June 2007
July 2007
August 2007
September 2007
October 2007
November 2007
December 2007
January 2008
February 2008
March 2008
April 2008
May 2008
June 2008
July 2008
August 2008
October 2008
November 2008
December 2008
January 2009
March 2009
April 2009
May 2009
June 2009
July 2009
August 2009
November 2009
December 2009
March 2010
April 2010
May 2010
June 2010
July 2010
October 2010
November 2010
April 2011
May 2011
August 2011
September 2011
October 2011
November 2011
November 2012
June 2013
January 2014
May 2015
McEs, A Hacker Life
Friday, May 01, 2015
 How to use custom application fonts with Pango

I am at the Libre Graphics Meeting in Toronto this week, which means that I got to talk to GIMP and Inkscape developers after many years, and was reminded that Pango still does not make it easy to use custom (aka. application) fonts, and it still does not allow turning OpenType features on or off.  So I decided to give Pango some love.

OpenType features is bug 738505.  Akira and Matthias wrote the initial patch, but there are certain complexities in handling the attributes that needs to be fixed before this can go in.  I'll see if I can get myself to do that tomorrow.

Custom fonts is a different issue.  And by custom fonts I mean when an application wants to use a font file that it ships, but is not installed in the system or user font directories.  Most of the times when people have this request, they also don't want any system fonts, ie, they only want their custom fonts.  A font viewer is a basic example of this that we never had a good solution for.

Webfonts, and other embedded fonts, are another use case.  Eg, a document might bundle fonts that it uses.  Many times, the document would want to refer to the font using font family name, so in that case you want the font to be added to the system fonts and go through the font matching process.

Back in 2006 when we considered pangocairo the only backend we care about, I proposed that we add pangocairo API to use a certain cairo_face_t.  The thinking was that by doing it in the pangocairo layer, we don't have to do it in individual platform backends (pangofc, pangowin32, pangoatsui / pangocoretext).  That, however, never happened, because I was too lazy to implement it, but also because it was actually a lot of tricky work, to make the generic pangofc layer understand this custom object...

I have since changed my mind on how this should be done.  Over the years different groups asked how they do this, I've had suggested different variations of the same solution: custom Fontconfig config; and Win32 / CoreText APIs on other platforms.  Ie, do it at font host layer, not Pango layer.  It's a legitimate approach, if not most convenient.

Here is one way to do it, using fontconfig API:

Before calling into Pango, do this:
This way Pango will only see your fonts.  If you want to add your fonts but also see the system fonts, you can skip the first step.  That will make your custom fonts visible to all Pango users within the process, including Gtk+ and its font selection dialog.  If, instead, you want to limit it to a particular document, we need something more involved; we talk about further down.

Recently I needed to do something similar in Noto's test suite, which is written in Python.  Since we currently don't have fontconfig Python bindings, I either would have had to be bothered to do a ctypes binding of the few functions I needed, or find an even simpler solution.  Which is:

Before calling into Pango, do this:
That's it.  Here's are the two steps for Noto: 1 and 2.

Now, this sounds easy, and works for very limited usecases.  But for it to be useful in apps like The GIMP or Inkscape, there are a few issues that need to be handled.  And I'm writing this post to raise enough interest from Akira, Matthias, Khaled, and others, to help fix these so we can have a great custom-font experience.  Writing a tutorial when these are all fixed would be great, but for now, this post is documentation enough!

First, the custom XML currently has to have a cachedir element or fontconfig warns.  That's annoying to say the least.  Filed here.  For now, you can use "<cachedir prefix="xdg">fontconfig</cachedir>" and that should work with recent-enough versions of fontconfig that understand the prefix="xdg".  Older version will ignore it and try to create a fontconfig directory under the current directory and use as cache, so beware of that.

Python / JS / etc bindings for fontconfig will be useful if we are pushing in this direction.  Filed here.

Another problem with the suggested setups above is that it works for non-GTK use-cases, like a game, test suite, etc.  But in most usecases, you definitely don't want your menus and other GUI elements to use the custom fonts.  In the case of a document editor or graphics editor, you might want to add custom fonts per document.  You can do that by using a separate PangoFcFontMap instance for each custom need.  I outlined that like this before:
  1. FcConfigCreate()
  2. FcConfigAppFontAddFile()
  3. pango_cairo_font_map_new_for_font_type()
  4. Use that PangoFontMap to create your context, etc.
  5. Every time you want to use Pango with that context, FcConfigSetCurrent the above config.  Then reset it back to whatever it was before.
This will allow you to use Pango with custom-only fonts in parts of the application.  If you also want the system fonts to be visible in this private font map, you can use the obscurely named FcInitLoadConfigAndFonts() instead of FcConfigCreate().

That's not awfully bad.  Except that the last step is very cumbersome and a recipe for disaster.  That step can be avoided if we have API to attach a custom FcConfig to a PangoFcFontMap.  Owen and I talked about it years ago and I filed a bug, but never happened... until yesterday.

I added pango_fc_font_map_set_config() yesterday, and just added pango_fc_font_map_config_changed().  So you can attach your private FcConfig to a private PangoFcFontMap, inform the font map every time you change the FcConfig (eg, add more fonts to it), and you should be able to use that font map to create layouts and use normally, without switching FcConfig's all the time.  However, remember: I wrote the code, but I didn't test it.  So a brave first user is wanted.

Then there's the set of issues with Fontconfig; the fact that to add font or configuration to an FcConfig, you need to have those in files, and can't add from memory blobs.  Fixing that should be fairly easy, and I filed those last year.  Would be great if we can fix them soon.  Here's one for font blobs, and another for configuration XML.

Another issue with using custom fonts is that every time you add a custom font, Fontconfig has to scan it.  Ie, you get no benefit from the Fontconfig font cache.  This can be really slow for huge fonts.  And became five times slower when the Adobe CFF rasterizer was integrated in FreeType.  I tried a few different approaches to speed it up.  Removing the (prematurely added) FC_HASH helped.  But there's more that can be done.  That bug is here.

Now, here's the one remaining issue that I don't have a full answer for: by sidestepping the default fontconfig configuration, you will miss some essential features.  Right now those are: synthetic italics, synthetic bold, and scaling of bitmap fonts.  The reason is that these are encoded in configuration files instead of in the library.  That sounds slightly over-engineered, and I like to improve it, but as of right now, that is the way it is.  You also will lose system-wide and user's configuration.

Sometimes that's not a problem.  For example, in the Noto test suite, we don't want any system or user configuration or any synthetic emboldening or italics, so the FcConfigCreate() approach suites those kinds of scenarios perfectly well.  Same about the preview part of a font-viewer, or other use cases where being system-independent is a goal.  But in other situations it might not be desirable.

I can, of course, suggest that anyone creating a custom FcConfig to add the essential configuration to it; Eg. bitmap scaling, synthetic bold and italic, essential aliases, user's configuration.  But that still leaves out generic aliases, system config, etc.  And I don't like that approach, because that spreads our default configuration all around Fontconfig clients and will create a mess that we would have to clean up every time we change how we do configuration.  It's a leaky abstraction.

Quite fortunately, there's an almost-perfect solution for that.  The obscure FcInitLoadConfig() function creates a new FcConfig and loads the default configuration files, but does not load any fonts referred to from those configuration files.  As such, you can go ahead and add your own fonts to it, but still benefit from the configuration.  Don't complain, if the configuration does undesirable font aliasing or other stuff.  I'm hugely relieved that this might do exactly what we need.  I have not tested this, so if you test, please kindly let me know how it works.  I cannot think of obvious undesirable behaviors from this approach.

So, to wrap up, assuming you would use Pango to be released soon, this is how you do custom fonts with PangoCairoFc:
  1. Create a custom FcConfig:
    • Use FcConfigCreate() if you don't want any system / user fonts or configuration whatsoever, neither you want any synthetic or other manipulations,
    • Use FcInitLoadConfig() if you want system / user configuration, but no system / user fonts,
    •  Use FcInitLoadConfigAndFonts() if you want system / user configuration and fonts visible in your private font map,
  2. Call FcConfigParseAndLoad() to add any custom configuration you want to add.  Such configuration can add custom font directories, or you can use next step,
  3. Call FcConfigAppFontAddFile() or FcConfigAppFontAddDir() to add custom fonts,
  4. Call pango_cairo_font_map_new_for_font_type(CAIRO_FONT_TYPE_FT), to create a private PangoCairoFcFontMap, which is a subclass of PangoFcFontMap, which is a subclass of PangoFontMap,
  5. Call pango_fc_font_map_set_config() to attach your custom FcConfig to your private PangoFcFontMap,
  6. Call pango_fc_font_map_config_changed() whenever you add new fonts to your custom FcConfig,
  7. Use that PangoFontMap to create your context, etc, and use normally.
You can optimize the logic to use the default font map if there are no custom fonts involved, and do the above otherwise.

This can trivially be adapted to the PangoFT2 backend by the way.

If your application has its own font dialog, then you can implement that using the private font map as well, and things should work.  But if you use the GTK+ font dialog, currently you can't attach your private font map to the font dialog instance.  I filed a request for that.

While there, let me address the issue of having or not having to restart applications when new fonts are installed.  As I covered in my State of Text Rendering in 2009 and presented at the Gran Canaria Desktop Summit, online font addition/removal should Just Work in GNOME.  I implemented that in 2008.  Indeed, it does in simple applications.  They are a bit slow (~4 seconds before new font shows up), so I filed a bug to shorten the delay.  However, a lot is not working:
As for pangocairo API to use a cairo_font_face_t, I still think that would be useful, but much less so than before.  Also, it won't be very useful until cairo adds API to create a cairo_font_face_t from a font file or memory blob.  Right now the only way to do that is to write code for the FreeType, Win32, and ATSUI / CoreText font backends of cairo separately.

Even if you try to do that, you will hit a very well-known problem with AddFontMemResourceEx(); which is: if you try to add a custom font with a family name that matches that of an existing font on the system, when you try to use the font, you might end up using the version installed on the system.  Ie. you can add fonts to the system, but you can't reliably address them.  The hack that Firefox, HarfBuzz, and others use to work around this is to modify the font data before calling AddFontMemResourceEx() to set a unique font family name on it, so they can be sure there will be no collision.  Here's the HarfBuzz code for that.

Anyway, I hope this write up helps developers, after confusing them, implement custom fonts in their applications, and motivate others to help me fix remaining issues and generate examples and better documentation.

While writing this, I ended up doing some bug triage and closing obsolete issues around Pango, Fontconfig, and cairo, which is always nice :D.

Anyway, this alone makes me really happy that I attended LGM.  The HarfBuzz Documentation Sprint was also very productive, but takes a couple more weeks to get to a stage that we can show off what we produced.

Labels: , , ,

Tuesday, January 14, 2014

GLyphy is an experiment in high-quality glyph rendering using OpenGL ES2 shaders.  It's a project of mine at Google that I never really publicly announced.

I worked on it mostly in 2011 and 2012.  The work was then stalled when I refocused on HarfBuzz, but also because there were no immediate target platform for it.  That has changed recently as mobile GPUs have caught up and can run GLyphy fairly well now.  So I'll be spending more time on it this year.

Last week at I presented GLyphy which was recorded.  The video is here.  And there's a mailing list for it.  The source for the slides are here.

Labels: ,

Sunday, January 05, 2014

In the Summer of 2003 I visited Ottawa for a Computer Science workshop.  Before getting there I also noticed that there's Ottawa Linux Symposium going on after my workshop, so I made it there, and stayed at Carlton's Leeds College.  It was my first Linux conference.  I knew no one.  I met Keith Packard for the first time.  The closing keynote was by Rusty Russel, and left me energized about the community.

Since then Free Software conferences have taken me to Montreal, Barcelona, Glasgow, Wolverhampton, Birmingham, Canary Islands, Austin, Boston, San Francisco, Istanbul, Wroclaw, Vienna, Berlin, and Amsterdam,  among others.  I got to become part of a fantastic community.

Last night I arrived in Perth, Australia, home to the last major conference I wanted to speak at.

Today, at registration, I saw Keith, so went to say hi.  He asked:

  - You here to talk about your crazy glyph rendering stuff?
  - Yes, have you seen it?
  - Oh yeah, it's insane!

Then I headed to the first-timers' session run by Rusty.

Walking last night at University of Western Australia's Trinity college campus made me realize how much I had missed the Free Software conference scene, and made me feel great being here.  That, and the southern-hemisphere night sky.


Sunday, June 23, 2013
 Iran Presidential Election 2013 Vote Count per County

Just finished downloading the 125 pages of vote counts from the Ministry of Internal Affairs portal, massaged them into usable data and put into a public spreadsheet. Will add extra sheets for analysis in the coming days. Feel free to clone, do your own analysis, and share back with me to integrate. May Mebane run with it...


Thursday, November 22, 2012
 High-DPI, Subpixel Text Positioning, Hinting

What happens when an unstoppable bullet hits an impenetrable wall?

A while back I wrote a document about the interactions of high-density displays and font rendering options.  I just went ahead and made it public, so, enjoy!

High-DPI, Subpixel Text Positioning, Hinting

Update (July 18, 2013): It was brought to my attention that the document had disappeared. I have no idea how that happened, but I've uploaded it again. It also means that the URL has changed. :(

Labels: , , ,

Wednesday, November 02, 2011
 "Behdad's Calendar" now available at

Since is blocked in Iran these days, I went ahead and made my Iranian calendar thingy available at  Let me know if it doesn't work.

Labels: , ,

Saturday, October 08, 2011
 In Montréal for “Boston” Summit

After a crazy Oktoberfest party in Kitchener last night, I woke up at 6:45 this morning, drove back to Toronto, took the ferry to the Island, took a Porter flight to Montréal, took the 747 downtown, took the 165 up to Queen Mary, walked up the hill to Polytechnique to arrive at the GNOME “Boston” Summit. Found Matthias, Owen, Ryan, and Andreas in the hallway, shook hands and received hugs, and felt right at home!

Inside, saw Colin and Marina among a few other familiar faces and many new ones. Marina explained that the reason she looks so sleepy is that she was blogging Ada Lovelace Day last night at 3am. Which of course reminded me that it was Finding Ada yesterday. So I thought I hereby list my own pick of women that I have had the pleasure to work with, and who, in my opinion, have made a lasting contribution to GNOME. Now I don't have to preach these awesome women to this crowd, so I'll just summarize my own experience with them in a two lines. In no particular order:

Marina Zhurakhinskaya has been critical to the Women Outreach Program success and happening in recent years, so for that alone she deserves a special mention. That's independent of she being part of the every-awesome GNOME Shell team. Plus, she's so nice and great to hangout with in person.

Stormy Peters wrote in her job application for the GNOME Executive Director as part of her responsibilities to be the "mom". And she delivered! It was a pleasure being on the board when she was in charge. Plus, she's so energetic she brightens everyone around her whereever she goes!

Karen Sandler is awesome in spite of being a lawyer! I have not had the opportunity to work with her in her new role, but at the Software Freedom Law Center, she was a great resource to the GNOME board, and much easier to get hold of than, well, other Free Software lawyers. Now, I did not actually know about her DJ hobby (check her website!) and wedding invitation until today. Waiting to run into her here to learn more :-D.

Rosanna Yuen is hard to find on Planet GNOME, and that's a shame. Many may not know her, she's sometimes better recognized as zana. Fortunately she's been making more regular appearance at GUADEC. Anyway, it's hard to imagine anything in the Board / Foundation level getting happened without her back-office work. She moves the money, she keeps the book, she knows what happened in the board five years ago! Plus, sometimes researches and books the venue for Boston Summit too.

At GUADEC this year, and at the Summit today I had the opportunity to meet a few young ladies rather new to the GNOME family: Pat, Kat, Meg, and Nohemi: you girls rock! I hope I blog about you for the years to come!

Went for lunch with Marina and Owen, had a great Thai chicken green curry, and talked food. I was thinking about a small project to hack on while at the summit and I thought I pickup rewrapping lines in vte / gnome-terminal upon width change. It's a well-defined well-contained problem, I have a design in mind, and one of the most common requests against vte. I passed my design past Owen, we agreed that it should work, and I hope that's what I'm going to hack on. Stay tuned!

I want to close by a picture of my favorite GNOME artist with the coolest hair style:

[Woah, long post! Been a while since I last did that...]

Labels: , ,

Wednesday, September 28, 2011
 Events of significance for Sep 26 and Sep 27

1982: I was born.

1998: Google was born.

2011: My personal website was filtered in Iran.

2011: Roozbeh joined Google.

Labels: , , ,

Thursday, August 11, 2011
 Can I has intel tablet?

So I missed the intel party it seems. I assume the tablets are GLES2-ready and have a fair pixel density. If that is the case, I can make good use of one for OpenGL-based text rendering I'm experimenting with. If there are leftovers at the summit, I'd happily take one!

kthxbye :)

Labels: , , , ,

Sunday, August 07, 2011
 Arrived at the Desktop Summit

Just got to the Desktop Summit in Berlin. It's lovely seeing everyone after two years.

I'm running Text Layout Summit August 9th to 12th, so come find me for some font and text chat!

Labels: , , ,