| [ Return to Bugs & Features | SVN ⇄ GIT ]
STR #1903
Application: | FLTK Library |
Status: | 1 - Closed w/Resolution |
Priority: | 1 - Request for Enhancement, e.g. asking for a feature |
Scope: | 3 - Applies to all machines and operating systems |
Subsystem: | Unicode support |
Summary: | Check for missing functionalities from original UTF8 patch |
Version: | 1.4-feature |
Created By: | matt |
Assigned To: | AlbrechtS |
Fix Version: | 1.3.0 |
Fix Commit: | unknown |
Update Notification: | |
Trouble Report Files:
|
#1 | timothy.lee 03:10 May 15, 2008 |
| fltk-1.3-utf8.patch 1.9M | |
|
#2 | timothy.lee 17:43 Feb 09, 2009 |
| fltk-1.3-u8wrapper.patch 28k | |
|
#3 | timothy.lee 00:36 Feb 10, 2009 |
| fltk-1.3-xft.patch 17k | |
|
#4 | michaelbaeuerle 05:27 May 27, 2014 |
| str_1903_r10174_xft_fontset_v1.patch 28k | |
|
#5 | michaelbaeuerle 05:27 May 27, 2014 |
| str_1903_r10174_xft_fontset_v1.png 15k | |
|
#6 | ianmacarthur 16:20 Jun 04, 2014 |
| alt-xft-glyph-subs-V1.zip 46k | |
|
#7 | ianmacarthur 14:21 Jun 07, 2014 |
| alt-xft-glyph-subs-V2.zip 46k | |
|
#8 | ianmacarthur 16:05 Jun 08, 2014 |
| alt-xft-glyph-subs-V3.zip 47k | |
|
#9 | ianmacarthur 15:02 Jun 09, 2014 |
| alt-xft-glyph-subs-V4.zip 48k | |
|
#10 | ianmacarthur 14:59 Jun 11, 2014 |
| alt-xft-glyph-subs-V5.zip 54k | |
|
#11 | ianmacarthur 16:56 Jun 17, 2014 |
| alt-xft-glyph-subs-V6.zip 55k | |
|
#12 | ianmacarthur 15:58 Jun 19, 2014 |
| alt-xft-glyph-subs-V7.zip 55k | |
|
#13 | michaelbaeuerle 06:12 Sep 10, 2014 |
| alt-xft-glyph-subs-V7_for_r10299.tar.bz2 45k | |
Trouble Report Comments:
|
#1 | matt 13:42 Mar 27, 2008 |
| Merge UTF-8 support into the FLTK 1.3 source tree | |
|
#2 | timothy.lee 03:13 May 15, 2008 |
| I've posted FLTK-UTF8 patch updated for 1.3 branch. Hopefully this will speed up development.
The patch compiles and runs cleanly on linux, mingw and MSVC. Don't have access to Mac. | |
|
#3 | ianmacarthur 00:57 May 16, 2008 |
| I haven't had a chance to look at the patch yet. Is it based on one of the existing utf8 patches? And if so, which one? | |
|
#4 | ianmacarthur 04:46 May 17, 2008 |
| OK - I started to look at this patch, but it's taking more time than I had allowed... And I've only had a chance to try it on my linux box so far, so...
Initial comments: Patch applied OK to current (at time of writing) SVN 1.3 tree, and code builds.
Observations: the header files Xutf8.h and fl_utf8.H - patch seems to have included all the text twice in each file. Odd. Still builds OK.
The XFT fontopen code has been extended to handle a list of fonts, this seems like a Good Thing. There are some uses of Xft*16 functions - these always make me uneasy - I think I'd rather go the Xft*UCS4 route, at least on *nix, for anything that can't be readily done directly as utf8. I worry that at we need to handle glyphs that can't be represented in a single 16-bit field, and going to a UCS4 type is more future proof.
Something in the patch breaks GL text rendering - see this in the cube and fractals demos, where the FPS text in the GL window is not displayed (1.1.9 and 1.3.x both seem fine.) This is a nuisance for me, as I have need of that facility in my code!
Time permitting, more comments to follow. Also (possibly!) OSX testing...
-- Ian | |
|
#5 | timothy.lee 19:27 May 18, 2008 |
| The patch was initially extracted from http://www.oksid.ch/xd640/fltk-utf8_1.1.6-1.tar.gz. Since then, I've been adapting for each of the FLTK releases (up to 1.1.9).
This patch is different from oksid's in the following ways:
1. The -utf8 suffix have been removed from the output libraries.
2. No extra classes like Fl_Shaped_Window, Fl_Table or Fl_Tree. Just the bare minimum to get UTF-8 content working.
3. UTF-8 Xft support under X11.
And yes, OpenGL's probably broken -- I don't use GL, so haven't been testing that. Sorry. | |
|
#6 | ianmacarthur 01:06 May 19, 2008 |
| Ah, OK... I guess we've been working in parallel a bit then...
I had done somework on the Oksi'D patch to bring it into 1.1.8 (see here) http://www.imm.uklinux.net/fltk/fltk118-utf8-2008-02-24.tar.bz2
This of course differs from both the Oksi'D patch, and from your patch! In particular I did a chunk of work on Xft, GL and OSX support. I ripped out a lot of the code Oksi'D had added to support IM's on older winXX variants (I don't think we are supporting win9x anymore...) Also, I hacked around a lot of the utf utility functions to align them more closely with the fltk2 variants.
Anyway, some of my stuff is in 1.1.8/1.1.9 and 1.3.x, but not the utf8 specific parts... Can you take a look at my stuff and see where it fits with yours, and I'll try and do the same from this end? Mind you, it's been a while since I worked on this - I can't remeber what I did!
Cheers, -- Ian | |
|
#7 | astrand 06:54 Sep 30, 2008 |
| fltk118-utf8-2008-02-24.tar.bz2 was easy to build and the utf8 test looks good. Is this version supposed to work on all major platforms (Solaris/Linux/Windows/OS X)? Any known stability problems? | |
|
#8 | ianmacarthur 07:07 Oct 02, 2008 |
| The fltk118-utf8-2008-02-24.tar.bz2 tarball is deprecated now in favour of the fltk-1.3 svn which now incorporates most of the functionality from that tarball. (Missing some of the functionality that is in the timothy.lee patch though, that should probably be considered.)
From my testing, fltk-1.3 seems OK on OSX, win32 and Linux with XFT, others have tested Linux without XFT and reported it OK.
I can't vouch for Solaris - probably OK, as the X implementation is good on linux. -- Ian | |
|
#9 | fabien 01:51 Jan 02, 2009 |
| renamed this rfe to more accurate/useful title (was: add UTF-8 support) | |
|
#10 | timothy.lee 18:11 Feb 09, 2009 |
| I finally had time to merge my UTF-8 changes into the latest SVN trunk (r6639).
The flkt-1.3-u8wrapper patch changes all filename-related system calls (fopen, access, getenv, chdir, system) to their equivalent UTF-8 wrapper function. | |
|
#11 | timothy.lee 00:38 Feb 10, 2009 |
| The flkt-1.3-xft patch allows a Unicode string to be drawn with a font set (poor man's Pango). | |
|
#12 | timothy.lee 17:27 Feb 22, 2009 |
| Dear Ian,
> Is there a list of what functionalities this particular patch adds? > > Also, the other patches you have posted - do they need to be applied in > a particular order? > > These patches seem to be mainly linux oriented - have you had a chance > to test on other platforms? Are there any known issues? How has the > testing gone, generally?
My patches are independent of each other. I have not had any issues using these patches together.
The Xft and iconv patches are geared towards Linux, and I do not have the opportunity to test on other platforms. However since conditional compilation is used (ie: USE_XFT and HAVE_LIBC_ICONV macros), the new code should have no effect on other platforms. It would be great if others could verify this assumption.
Below is a summary of the patches I've posted so far:
fltk-1.3-u8wrapper (under STR #1903): This patch enforces the use of UTF-8 wrapper functions to ease future porting. All filename-related system calls (fopen, access, getenv, chdir, system) are changed to their equivalent UTF-8 wrapper function.
fltk-1.3-xft (under STR #1903): This patch allows a Unicode string to be drawn with an Xft font set (poor man's Pango).
fltk-1.3-iconv.patch (under STR #2148): This patch makes use of libc's iconv() to convert between UTF-8 and other encodings, instead of placing the conversion table inside libfltk. External iconv() implementations are not considered for such use.
How do I proceed to get these patches applied to the trunk? | |
|
#13 | matt 12:56 Nov 14, 2010 |
| Not essential for the 1.3.0 release. Temporarily moved to 1.4 . | |
|
#14 | andreldm 17:40 May 26, 2014 |
| This patch would be much appreciated if merge, as in my case, a music player needs to display text in different languages at the same time. | |
|
#15 | michaelbaeuerle 05:34 May 27, 2014 |
| The Xft fontset V1 patch is the first attempt to get glyph substitution working again for 1.3 current. Many things may still be horribly broken, but the attached picture show that FLTK now can at least mix latin, cyrillic and japanese characters from different fonts in a single string.
I have copied the strings from Greg's demo program and added a test file test/fontset.cxx for demonstration. | |
|
#16 | ianmacarthur 16:19 Jun 04, 2014 |
| I've posted my first pass at my proposed alternative way of achieving glyph substitution "on the fly" with fontconfig/XFT on X11 systems.
The attached zip alt-xft-glyph-subs-V1.zip is not a patch per se, it is the full modified files:
src/Fl_Font.H src/fl_font_xft.cxx src/fl_set_font.cxx src/fl_utf.c FL/Enumerations.H FL/fl_utf8.h test/unittest_text.cxx test/rotated_text.cxx test/utf8.cxx
My intent is to lower the barrier for testing - anyone can drop these files onto a recent weekly snapshot and rebuild, and it should Just Work. Probably...
Measurement of text extents is still a bit borked, but glyphs substitution and text rotation and so forth all seem to work "well enough".
You will need to have the ABI guard in Enumerations.H set though.
I have tweaked 3 of the test programs to illustrate the usage:
- unittest is tweaked just to use a "compound" string (EN, Ru, Jp) using the UTF8 values from Greg's demo code. View the "rendering text" page of this test.
- rotated_text is modified to use the "compound" string, and to draw a bounding box around the rotated string.
- utf8 is modified to use the substitution for the "whole Unicode" page... This can be Quite Slow the first time it loads and searches all the faces... Note that the UTF8 test calls Fl::set_fonts(*) to load *all* the fonts on the system and the glyph substitution logic then searches all of them...
The other tests I modified only load the fltk built-in fonts and one other CJK face that this box has installed; this speeds things up a lot since my glyph subs logic only tests fonts the app has loaded, *not* all the fonts on the system (unless you use set_fonts() to load all the fonts of course!)
In testing, you may need to tweak the "extra" CJK font I use to match something that exists on your system.
I have other code that uses fontconfig to locate a system font that is a Good Fit for a given body of text, but that is not ready yet and is not in this post. Once it is cleaned up, the idea is that you can pass it a chunk of UTF8 text and it will suggest which font that you can load that *should* be able to render that text... | |
|
#17 | ianmacarthur 14:20 Jun 07, 2014 |
| Revised version of my "patch" posted as alt-xft-glyph-subs-V2.zip
This *does not* resolve the issue with text_extents, nor does it include my functions for using fontconfig to "search" for a suitable font before loading it. Both these features are in work, but I just have not had the time to actually get them ready...
This patch does:
- Address a bug in the glyph substitution such that, if the composite string mixed both TT and Type-1 fonts, the rotation went awry.
- As per Michael B's suggestion, I have added a musical G clef to exercise glyphs from above the BMP
- As per Michael's suggestion, I have added a few more font options to the load path (FreeSerif and Kochi Mincho) so the test should work on a wider range of hosts
- As per Michael's suggestion, the files in the zip now ought to be path as per the fltk folder layout, so the zip file can be unziped over a stock fltk tree
Feedback welcome. | |
|
#18 | ianmacarthur 16:05 Jun 08, 2014 |
| Another revised version of my "patch", alt-xft-glyph-subs-V3.zip.
This *still* does not fix the text extents issue, but adds a demonstration version of the scheme I'm proposing for finding a font to fit a block of text.
The function fl_xft_font_finder_for_UTF8_text(...) is added (somewhat arbitrarily) to fl_font_xft.cxx, and if passed a block of UTF8 encoded text, it will then *print to stdio* a list of suggested fonts, and how many of the needed glyphs that *does not* have.
This is *not* useful as a run time feature, it is just to let us explore what fontconfig can do for us.
Note, also, that I have not implemented yet a way to create a *set* of fonts that, taken together, will cover all the needed glyphs...
I think this is possible, but... well, not yet understood!
Feedback welcomed. Also, fixes! | |
|
#19 | ianmacarthur 15:01 Jun 09, 2014 |
| Now posting alt-xft-glyph-subs-V4.zip, which implements a function fl_xft_font_set_find() in fl_font_xft.cxx that, when passed a corpus of UTF8 encoded text, returns an array of (up to 5) char* names for fonts to load.
This mechanism is used in the modified rotated_text demo to load a set of fonts sufficient to execute the demo without me loading any fonts by name, so should work on any host that has any suitable fonts available.
(If there are no CJK fonts on your system though, this will still fail I imagine!)
This works, but the actual code is a mess! The name mechanism is especially clunky...
Anyway, as ever, feedback on whether this is useful, or the right direction to take, is welcomed. | |
|
#20 | ianmacarthur 14:58 Jun 11, 2014 |
| Revised version of the font picking function: I have added the function fl_xft_font_set_find_similar(...) that takes in a Fl_Font value referring to some already laoded font, and then tries to find a font with simialr metrics to use for the replacement glyphs.
rotated_text modified to exercise this "new, improved" font finding scheme.
Note that I have not actually modified fl_font_xft to take advanatge of this yet however, so there's not much difference on the display...
See: alt-xft-glyph-subs-V5.zip | |
|
#21 | ianmacarthur 17:00 Jun 17, 2014 |
| V6 version of my patch that actually tries to use a glyph with "similar" metrics when substituting... i.e. it will try to follow the bold / italic / serif / sans nature of the base font when picking a replacemnt glyph.
The rotated_text and unittest demos, and the utf8 demo, should now be using this scheme in the patch.
The utf8 demo is *slow* to start the first time, a lot of font picking is going on there. The other two seem to be "OK" since they only load a few extra faces... | |
|
#22 | ianmacarthur 16:01 Jun 19, 2014 |
| Test alt-xft-glyph-subs-V7.zip posted: when searching for a substitute face this checks to see of the base font is AA, and if it is it tries to force things so that only AA substitutes are considered.
This may address an issue that Michael has been seeing (though I confess I have failed to reproduce it) where the picker chooses a bitmapped font with extensive glyph coverage, rather than other better looking AA fonts with less extensive, but still adequate, coverage... | |
|
#23 | michaelbaeuerle 06:16 Sep 10, 2014 |
| I have attached an archive with Ians V7 patches modified to work with r10299.
I have used a bzip2 compressed tar file because my zip utility don't accept the symbolic link. The content are whole files to copy over the old ones as before. | |
|
#24 | michaelbaeuerle 08:37 Sep 11, 2014 |
| Some results with r10299 and the V7 patch: http://micha.freeshell.org/tmp/flnews-0.9gs2_glyphen.png using this font set: | | Chose font: 0 Baekmuk Batang, still missing 5 glyphs | Chose font: 1 FreeSerif, still missing 4 glyphs | Chose font: 2 TakaoPGothic, still missing 3 glyphs | Chose font: 3 Lohit Kannada, still missing 2 glyphs | Chose font: 4 Lohit Telugu, still missing 1 glyph | Chose font: 5 AR PL KaitiM Big5, still missing 0 glyphs
"Kannada" failed for some unknown reason, but the rest is not looking bad. The clipping on the end of some lines should be the wrong calculated width that we had already seen on the bounding box of the "rotated text" demo. | |
|
#25 | ianmacarthur 14:52 Sep 13, 2014 |
| Thanks Michael - glad to hear the V7 patch more or less works as expected.
Dunno about the Kannada issue though.
Regarding the bounding box issue, I more or less know how to fix it, I think, but there's a big restructuring of my code needed to make it work; and I just can't bring myself to face the change - there's a hill to climb...! | |
|
#27 | AlbrechtS 11:39 Jan 15, 2023 |
| Fixed in Git repository.
I assume this is all already done, maybe in 1.3.x or later. Looking at the old patches doesn't seem useful anymore, after more than 14 or at least 7 years (last modified: Sep 18, 2014).
Changed status to "4 - Pending".
Can we close this STR? It appears to be obsolete.
I'll close this if the original authors agree or in one week, whichever comes first. | |
|
#28 | matt 06:31 Jan 16, 2023 |
| The current 1.4 implementation has been working well for the last 14 years. SO we can assume that this is done. Most (all?) patches no longer apply anyway. | |
[ Return to Bugs & Features ]
|
| |