STR #2961

GitHub FLTK Project   FLTK News RSS Feed  
  FLTK Apps      FLTK Library      Forums      Links     Login 
 Home  |  Articles & FAQs  |  Bugs & Features  |  Documentation  |  Download  |  Screenshots  ]
 

Return to Bugs & Features | Roadmap 1.3 | Post Text | Post File | Prev | Next ]

STR #2961

Application:FLTK Library
Status:5 - New
Priority:3 - Moderate, e.g. unable to compile the software
Scope:2 - Specific to an operating system
Subsystem:X11
Summary:1.3 very slow on X11 with Unicode locale
Version:1.3-current
Created By:michaelbaeuerle
Assigned To:Unassigned
Fix Version:Unassigned
Update Notification:

Receive EMails Don't Receive EMails

Trouble Report Files:

Post File ]
Name/Time/Date Filename/Size top right image
 
#2 michaelbaeuerle
23:54 May 09, 2013
test_xim_slowness.cxx
1k
 
 
#3 michaelbaeuerle
23:55 May 09, 2013
str_2961_r9895_xim.patch
2k
 
bottom left image   bottom right image

Trouble Report Comments:

Post Text ]
Name/Time/Date Text top right image
 
#1 michaelbaeuerle
04:28 May 08, 2013
FLTK 1.3 runs an order of magnitude slower if I change my locale from ISO8859-1 to UTF-8. This is most annoying in the Fl_Menu_Bar widget.
Here is a description of the problem:

$ uname -s -r
NetBSD 5.1
$ cat /proc/cpuinfo | grep name
model name      : Intel(R) Pentium(R) 4 CPU 1500MHz
$ Xorg -version
X.Org X Server 1.6.5
[...]
$ fltk-config --version
1.3.0
$ fltk-config -g --compile x.cxx
$ export LC_CTYPE=en_US.ISO8859-1
$ time -p ./x
real 0.12
user 0.05
sys 0.02
$ export LC_CTYPE=en_US.UTF-8
$ time -p ./x
real 0.86
user 0.66
sys 0.02

With the UTF-8 locale it consumes approx. half a second more CPU time.
Tracing the program with ktruss and gdb showed, that the lengthy operation seems to be parsing one of the following files triggered via _XimParseStringFile() by XOpenIM():

/usr/pkg/share/X11/locale/iso8859-1/Compose
/usr/pkg/share/X11/locale/en_US.UTF-8/Compose

The two files are 20KByte vs 500KByte in size on my system, this explains the difference in time required to parse them. Up to this point: Who cares if this happens once at startup.

But it happen very often while the program is running. Every time when you hit a new menubar entry, both (the old and the new one) will disappear and a delay twice as long as the startup delay occur before
the new menu and both entries of the menubar reapprear.
Note: Moving the mouse over the entries inside a pulldown menu is not slow.
 
 
#3 michaelbaeuerle
00:17 May 10, 2013
In the current 1.3.x version the problem still exist the same way as in 1.3.0. I have attached a patch and a test program for it.

I can't attach this video to the STR, maybe it is too big (approx. 7MByte):
http://micha.freeshell.org/tmp/demo_xim_patch.ogv
The video is in Ogg Theora format (e.g. Firefox can play this out of
the box). It show the current behaviour in the left demo window (note that the performance impact with ISO 8859 locales on NetBSD is lower, but FLTK is still annoying slow). As you can see, with Unicode locale the program gets completely unusable if you click into the menu bar and consumes 100% CPU until it become ready again.

Note that the patch is no solution but only a workaround that currently only allows to disable IM completely via the new environment variable FLTK_IM_MODULE (only accepted value is "simple"). The name and value was inspired by QT_IM_MODULE:
https://wiki.archlinux.org/index.php/Internationalization
I think FLTK_IM_MODULE would be a consistent solution for users and it won't have to be changed in the future if somebody want to improve the IM system of FLTK to some equivalent behaviour as in GTK+ and Qt.
 
 
#4 AlbrechtS
01:40 May 10, 2013
ISTR that calling XOpenIM() more often was a change requested by an STR such that starting and stopping XIM was possible while a (FLTK) program was running, or *after* the FLTK program was started. IIRC Manolo was the dev who handled it. See below for a note about related STR's.

So we have two conflicting requests:

 - use XOpenIM only once at startup (or optional never) [this STR]
 - use XOpenIM more often to make sure it's synchronized [old STR]

My first idea would be to make the *old* behavior (calling XOpenIM only once at startup) the default and let the user select the current behavior (calling XOpenIM more often) via an environment variable, instead of switching IM off via environment, as suggested by the patch.

However, this can be discussed, and I'm open for all solutions. The current behavior seems to be a problem, as the OP states, and should be fixed one way or the other ...

Note: I found the old STR(s):

- http://www.fltk.org/str.php?L2578 (multiple warnings "XOpenIM() failed"). This was caused by the fix in STR #2474.

- http://www.fltk.org/str.php?L2474 (Fltk does not accept xim input when the input method starts later).

The latter seems to be the culprit, and the fix was in svn r 8097 on 2010-12-22.
 
 
#5 michaelbaeuerle
04:04 May 10, 2013
I have looked at STR #2474 again and this patch have changed something in fl_handle() that seems not the real reason for my problem. The problematic location is at the very beginning of fl_handle():
---------------------------------------------------------------------
  if (fl_xim_ic && xevent.type == DestroyNotify &&
        xid != xim_win && !fl_find(xid))
  {
    XIM xim_im;
    xim_im = XOpenIM(fl_display, NULL, NULL, NULL);
    ...
---------------------------------------------------------------------
On my machine *this* call to XOpenIM() is executed twice for every mouse movement from one menu entry to the next. Anything behind this point in fl_handle() - like the patch from STR #2474 - can only make it worse but not better any more.

To verify I have removed only this "if" block and now it is as fast as with my patch.

Looks like the problem is not at all (or at least not alone) the reinitialization via fl_init_xim() and we all together still don't really understand what is going on here ...
 
 
#6 greg.ercolano
10:28 May 10, 2013
I did a grep on the fltk.commit newsgroups for 'XOpenIM',
and found these hits; perhaps the history of changes is useful:

http://www.fltk.org/newsgroups.php?gfltk.commit+v3345
http://www.fltk.org/newsgroups.php?gfltk.commit+v3370
http://www.fltk.org/newsgroups.php?gfltk.commit+v4541
http://www.fltk.org/newsgroups.php?gfltk.commit+v4702
http://www.fltk.org/newsgroups.php?gfltk.commit+v4705
 
 
#7 greg.ercolano
12:19 May 10, 2013
OK, sorry, scratch that last message -- I just rebuilt the fltk.commit
database (from a backup Mike sent me which has more accurate
commit information), and this changed all the article numbers.

New links to commits that match XOpenIM, oldest to newest..
perhaps the log comments are useful if the code diffs aren't:
http://www.fltk.org/newsgroups.php?gfltk.commit+v4977
http://www.fltk.org/newsgroups.php?gfltk.commit+v6553
http://www.fltk.org/newsgroups.php?gfltk.commit+v7025
http://www.fltk.org/newsgroups.php?gfltk.commit+v7746
http://www.fltk.org/newsgroups.php?gfltk.commit+v7854
http://www.fltk.org/newsgroups.php?gfltk.commit+v8644
http://www.fltk.org/newsgroups.php?gfltk.commit+v8692
http://www.fltk.org/newsgroups.php?gfltk.commit+v8887
http://www.fltk.org/newsgroups.php?gfltk.commit+v8892
http://www.fltk.org/newsgroups.php?gfltk.commit+v9112
http://www.fltk.org/newsgroups.php?gfltk.commit+v9447
http://www.fltk.org/newsgroups.php?gfltk.commit+v9467
 
 
#8 AlbrechtS
17:56 May 10, 2013
Okay, I got a suspicion to follow: the code fragment shown by Michael clearly(?) indicates that we get a DestroyNotify event for a window (xid) that doesn't exist anymore in FLTK's structures (!fl_find(xid)). That's not surprising at all. I'd assume that we get this message after the corresponding X function (destroy window or whatever) has been called by FLTK. This still needs some investigation, I'm not an X expert.

So this means that we probably run into this part of the code always after we destroy any window. To verify this, I used not only Michael's test program, but also test/unittests. The idea was that this would also apply to tooltip windows - and yes, that's obviously true!
[Other windows wouldn't matter much, since they are not created and destroy so often, so that you wouldn't see the delay in normal cases.]

I got comparable times when testing test_xim_slowness.cxx (much lower values, but a factor of ~5-6 with my Ubuntu 12.04 VM).

Here's what I did with test/unittests:
- select schemes test
- position the mouse over the green (top left) button
- wait until the tooltip appears
- move mouse down one button (blue), wait for the tooltip
- and so on for all four buttons,
- then up again, button by button, always waiting for the toolip
- close the window.

That's 7 tooltip windows in total. Speed of mouse movement doesn't matter much, opening and closing the tooltip window(s) is what counts.

Here are my measured times (user time is the one to look at):

$ locale|grep LC_CTYPE
LC_CTYPE="de_DE.UTF-8"
$ time ./unittests

real    0m11.567s
user    0m0.240s   <<<
sys     0m0.020s

$ locale|grep LC_CTYPE
LC_CTYPE=C
$ time ./unittests

real    0m11.440s
user    0m0.044s   <<<
sys     0m0.020s

Michael, could you please do the same test on your system? What results and what "feeling" do you get with tooltip windows? What difference with locales? I should note that I can hardly see a difference on my system (about 0.034s per tooltip), but somehow you can "feel" the difference, once you know it's there.

Note that I may not have the time to look into this further before Sunday, but I wanted to document my results so far.

Since I don't know much of the X11 protocol and particularly XIM, I don't know why this code does what it does after closing a window at all. I assume that the XIM is somehow attached to a specific window, and after destroying a window it must be re-initialized somehow. Or similar...

I saw a hint in one of Bill's commits (thanks, Greg, for the list) that he tried to postpone the XIM initialization until it's really needed. This would typically not be necessary for menu windows, so this may be the way to solve the problem cleanly, but ... needs more investigation.
Note that I only looked at the diffs so far. Maybe I'm wrong with this, but anyway: it was in svn r7473 for FLTK 2, see:
http://www.fltk.org/newsgroups.php?gfltk.commit+v7854

Albrecht
 
 
#9 michaelbaeuerle
00:53 May 13, 2013
Albrechts "7 tooltip" test repeated on my system:
The tooltip windows appear instantly, but when it should disappear, I see a grey box instead and the application is blocked for several 100 miliseconds. Measured times:

$ export LC_CTYPE=de_DE.UTF-8
$ time ./unittests

real    0m19.383s
user    0m5.185s
sys     0m0.121s

$ export LC_CTYPE=C
$ time ./unittests

real    0m21.997s
user    0m0.281s
sys     0m0.071s

=> 18 times slower with Unicode locale. It feels as slow as the menubar.

BTW: If I open and close the "FLTK scheme" pull-down menu on the top, the effect is the same (opens instantly and closes very slowly showing a grey box while blocking the application).
 
 
#10 michaelbaeuerle
01:38 May 13, 2013
Albrecht wrote:
| I assume that the XIM is somehow attached to a specific window,
| and after destroying a window it must be re-initialized somehow.
| Or similar...

There is an official document for the XIM protocol:
http://www.x.org/releases/X11R7.6/doc/libX11/specs/XIM/xim.html

According to the chapter "Architecture" I understand it this way:
In general you have to open the IM only once (at least if you don't want to change the locale "on the fly") to create the programs bindings to the locale and the IM server => XOpenIM() and XCloseIM.
The things you have to create and destroy while the program is running are only the input contexts (ICs) => XCreateIC() and XDestroyIC.

IMHO:
Because (with the client/server model) the IM server can be an external program running on a different machine (with a network redirected display it would run on the opposite side of the FLTK library), the XOpenIM() operation should be considered as inherently expensive and slow even if the locale stuff would not be an additional problem.

Micha
 
 
#11 AlbrechtS
02:35 May 13, 2013
Michael, thanks for the test results and the additional info regarding the pull-down menu. This confirms the fact that the problem lies in re-initializing XIM for every window close operation.

My assumption "that the XIM is somehow attached to a specific window" resulted from reading the code, but you're probably right that this is not the case and that it is unnecessary and causing the delay you can see.

I started looking for and reading X11 docs yesterday, thanks for the pointer. However, there's a newer version of this xim document here, although I suspect that there's not much changed:
http://www.x.org/releases/X11R7.7/doc/libX11/XIM/xim.html
http://www.x.org/releases/X11R7.7/doc/libX11/XIM/xim.pdf

I also found the Xlib docs at:
http://www.x.org/releases/X11R7.7/doc/libX11/libX11/libX11.html
http://www.x.org/releases/X11R7.7/doc/libX11/libX11/libX11.pdf

I had to start with some basic reading to understand that "language" the documentation is written in, and then I went to chapter 13. There's a part about "Input Methods" (p. 278 ff. in the pdf document). I'll have to read this...

The problem we have to solve is to remove unnecessary re-initializations, but still keep XIM working for special purposes like Asian input methods... More to follow later.
 
 
#12 michaelbaeuerle
07:35 May 13, 2013
If somebody with a system that is not affected want to "feel" how things behave on a system with expensive XOpenIM(), simply add a delay at the position I have cited above. Something like this:
---------------------------------------------------------------------
  if (fl_xim_ic && xevent.type == DestroyNotify &&
        xid != xim_win && !fl_find(xid))
  {
    XIM xim_im;
    usleep(700000);
    xim_im = XOpenIM(fl_display, NULL, NULL, NULL);
    ...
---------------------------------------------------------------------

Or within the wrapper of my original patch so that it applies to every XOpenIM() call. Then you get the startup delay too.

Micha
 
 
#13 michaelbaeuerle
03:34 Sep 02, 2013
Albrecht, you have voted for the variant to make "IM not initialized all
the time" the default. But for this we need a third option between "as
is" and "off" that we currently don't have. I still prefer the variant
from my patch "IM must be explicitly disabled" for the following
reasons:

1) Initialize IM only once at startup seems not enough for some users
   I assume this is the reason for the current implementation.
   In this case the switching options should be "as is" and "off".

2) On most systems the performance impact seems to be low/moderate
   This means "as is" doesn't really hurt as default in many cases.
   Even if the assumption from 1) is not true, it's maybe not worth
   to implement the third option "only once" with the problem in mind
   that we can't prove whether this works for all users and IMs.

3) The new default implementation should not surprise users
   Think about FLTK as shared library that is updated by admin/system.
   Using "as is" as default makes sure this is the case.
 
bottom left image   bottom right image

Return to Bugs & Features | Post Text | Post File ]

 
 

Comments are owned by the poster. All other content is copyright 1998-2020 by Bill Spitzak and others. This project is hosted by The FLTK Team. Please report site problems to 'erco@seriss.com'.