[ Return to Bugs & Features | Roadmap 1.3 | Post Text | Post File | Prev | Next ]

STR #2961

Trouble Report Files:

	Name/Time/Date	Filename/Size
	#2  michaelbaeuerle 23:54 May 09, 2013	test_xim_slowness.cxx 1k
	#3  michaelbaeuerle 23:55 May 09, 2013	str_2961_r9895_xim.patch 2k

Trouble Report Comments:

	Name/Time/Date	Text
	#1  michaelbaeuerle 04:28 May 08, 2013	FLTK 1.3 runs an order of magnitude slower if I change my locale from ISO8859-1 to UTF-8. This is most annoying in the Fl_Menu_Bar widget. Here is a description of the problem: $ uname -s -r NetBSD 5.1 $ cat /proc/cpuinfo | grep name model name      : Intel(R) Pentium(R) 4 CPU 1500MHz $ Xorg -version X.Org X Server 1.6.5 [...] $ fltk-config --version 1.3.0 $ fltk-config -g --compile x.cxx $ export LC_CTYPE=en_US.ISO8859-1 $ time -p ./x real 0.12 user 0.05 sys 0.02 $ export LC_CTYPE=en_US.UTF-8 $ time -p ./x real 0.86 user 0.66 sys 0.02 With the UTF-8 locale it consumes approx. half a second more CPU time. Tracing the program with ktruss and gdb showed, that the lengthy operation seems to be parsing one of the following files triggered via _XimParseStringFile() by XOpenIM(): /usr/pkg/share/X11/locale/iso8859-1/Compose /usr/pkg/share/X11/locale/en_US.UTF-8/Compose The two files are 20KByte vs 500KByte in size on my system, this explains the difference in time required to parse them. Up to this point: Who cares if this happens once at startup. But it happen very often while the program is running. Every time when you hit a new menubar entry, both (the old and the new one) will disappear and a delay twice as long as the startup delay occur before the new menu and both entries of the menubar reapprear. Note: Moving the mouse over the entries inside a pulldown menu is not slow.
	#3  michaelbaeuerle 00:17 May 10, 2013	In the current 1.3.x version the problem still exist the same way as in 1.3.0. I have attached a patch and a test program for it. I can't attach this video to the STR, maybe it is too big (approx. 7MByte): http://micha.freeshell.org/tmp/demo_xim_patch.ogv The video is in Ogg Theora format (e.g. Firefox can play this out of the box). It show the current behaviour in the left demo window (note that the performance impact with ISO 8859 locales on NetBSD is lower, but FLTK is still annoying slow). As you can see, with Unicode locale the program gets completely unusable if you click into the menu bar and consumes 100% CPU until it become ready again. Note that the patch is no solution but only a workaround that currently only allows to disable IM completely via the new environment variable FLTK_IM_MODULE (only accepted value is "simple"). The name and value was inspired by QT_IM_MODULE: https://wiki.archlinux.org/index.php/Internationalization I think FLTK_IM_MODULE would be a consistent solution for users and it won't have to be changed in the future if somebody want to improve the IM system of FLTK to some equivalent behaviour as in GTK+ and Qt.
	#4  AlbrechtS 01:40 May 10, 2013	ISTR that calling XOpenIM() more often was a change requested by an STR such that starting and stopping XIM was possible while a (FLTK) program was running, or *after* the FLTK program was started. IIRC Manolo was the dev who handled it. See below for a note about related STR's. So we have two conflicting requests:  - use XOpenIM only once at startup (or optional never) [this STR]  - use XOpenIM more often to make sure it's synchronized [old STR] My first idea would be to make the *old* behavior (calling XOpenIM only once at startup) the default and let the user select the current behavior (calling XOpenIM more often) via an environment variable, instead of switching IM off via environment, as suggested by the patch. However, this can be discussed, and I'm open for all solutions. The current behavior seems to be a problem, as the OP states, and should be fixed one way or the other ... Note: I found the old STR(s): - http://www.fltk.org/str.php?L2578 (multiple warnings "XOpenIM() failed"). This was caused by the fix in STR #2474. - http://www.fltk.org/str.php?L2474 (Fltk does not accept xim input when the input method starts later). The latter seems to be the culprit, and the fix was in svn r 8097 on 2010-12-22.
	#5  michaelbaeuerle 04:04 May 10, 2013	I have looked at STR #2474 again and this patch have changed something in fl_handle() that seems not the real reason for my problem. The problematic location is at the very beginning of fl_handle(): ---------------------------------------------------------------------   if (fl_xim_ic && xevent.type == DestroyNotify &&         xid != xim_win && !fl_find(xid))   {     XIM xim_im;     xim_im = XOpenIM(fl_display, NULL, NULL, NULL);     ... --------------------------------------------------------------------- On my machine *this* call to XOpenIM() is executed twice for every mouse movement from one menu entry to the next. Anything behind this point in fl_handle() - like the patch from STR #2474 - can only make it worse but not better any more. To verify I have removed only this "if" block and now it is as fast as with my patch. Looks like the problem is not at all (or at least not alone) the reinitialization via fl_init_xim() and we all together still don't really understand what is going on here ...
	#6  greg.ercolano 10:28 May 10, 2013	I did a grep on the fltk.commit newsgroups for 'XOpenIM', and found these hits; perhaps the history of changes is useful: http://www.fltk.org/newsgroups.php?gfltk.commit+v3345 http://www.fltk.org/newsgroups.php?gfltk.commit+v3370 http://www.fltk.org/newsgroups.php?gfltk.commit+v4541 http://www.fltk.org/newsgroups.php?gfltk.commit+v4702 http://www.fltk.org/newsgroups.php?gfltk.commit+v4705
	#7  greg.ercolano 12:19 May 10, 2013	OK, sorry, scratch that last message -- I just rebuilt the fltk.commit database (from a backup Mike sent me which has more accurate commit information), and this changed all the article numbers. New links to commits that match XOpenIM, oldest to newest.. perhaps the log comments are useful if the code diffs aren't: http://www.fltk.org/newsgroups.php?gfltk.commit+v4977 http://www.fltk.org/newsgroups.php?gfltk.commit+v6553 http://www.fltk.org/newsgroups.php?gfltk.commit+v7025 http://www.fltk.org/newsgroups.php?gfltk.commit+v7746 http://www.fltk.org/newsgroups.php?gfltk.commit+v7854 http://www.fltk.org/newsgroups.php?gfltk.commit+v8644 http://www.fltk.org/newsgroups.php?gfltk.commit+v8692 http://www.fltk.org/newsgroups.php?gfltk.commit+v8887 http://www.fltk.org/newsgroups.php?gfltk.commit+v8892 http://www.fltk.org/newsgroups.php?gfltk.commit+v9112 http://www.fltk.org/newsgroups.php?gfltk.commit+v9447 http://www.fltk.org/newsgroups.php?gfltk.commit+v9467
	#8  AlbrechtS 17:56 May 10, 2013	Okay, I got a suspicion to follow: the code fragment shown by Michael clearly(?) indicates that we get a DestroyNotify event for a window (xid) that doesn't exist anymore in FLTK's structures (!fl_find(xid)). That's not surprising at all. I'd assume that we get this message after the corresponding X function (destroy window or whatever) has been called by FLTK. This still needs some investigation, I'm not an X expert. So this means that we probably run into this part of the code always after we destroy any window. To verify this, I used not only Michael's test program, but also test/unittests. The idea was that this would also apply to tooltip windows - and yes, that's obviously true! [Other windows wouldn't matter much, since they are not created and destroy so often, so that you wouldn't see the delay in normal cases.] I got comparable times when testing test_xim_slowness.cxx (much lower values, but a factor of ~5-6 with my Ubuntu 12.04 VM). Here's what I did with test/unittests: - select schemes test - position the mouse over the green (top left) button - wait until the tooltip appears - move mouse down one button (blue), wait for the tooltip - and so on for all four buttons, - then up again, button by button, always waiting for the toolip - close the window. That's 7 tooltip windows in total. Speed of mouse movement doesn't matter much, opening and closing the tooltip window(s) is what counts. Here are my measured times (user time is the one to look at): $ locale|grep LC_CTYPE LC_CTYPE="de_DE.UTF-8" $ time ./unittests real    0m11.567s user    0m0.240s   <<< sys     0m0.020s $ locale|grep LC_CTYPE LC_CTYPE=C $ time ./unittests real    0m11.440s user    0m0.044s   <<< sys     0m0.020s Michael, could you please do the same test on your system? What results and what "feeling" do you get with tooltip windows? What difference with locales? I should note that I can hardly see a difference on my system (about 0.034s per tooltip), but somehow you can "feel" the difference, once you know it's there. Note that I may not have the time to look into this further before Sunday, but I wanted to document my results so far. Since I don't know much of the X11 protocol and particularly XIM, I don't know why this code does what it does after closing a window at all. I assume that the XIM is somehow attached to a specific window, and after destroying a window it must be re-initialized somehow. Or similar... I saw a hint in one of Bill's commits (thanks, Greg, for the list) that he tried to postpone the XIM initialization until it's really needed. This would typically not be necessary for menu windows, so this may be the way to solve the problem cleanly, but ... needs more investigation. Note that I only looked at the diffs so far. Maybe I'm wrong with this, but anyway: it was in svn r7473 for FLTK 2, see: http://www.fltk.org/newsgroups.php?gfltk.commit+v7854 Albrecht
	#9  michaelbaeuerle 00:53 May 13, 2013	Albrechts "7 tooltip" test repeated on my system: The tooltip windows appear instantly, but when it should disappear, I see a grey box instead and the application is blocked for several 100 miliseconds. Measured times: $ export LC_CTYPE=de_DE.UTF-8 $ time ./unittests real    0m19.383s user    0m5.185s sys     0m0.121s $ export LC_CTYPE=C $ time ./unittests real    0m21.997s user    0m0.281s sys     0m0.071s => 18 times slower with Unicode locale. It feels as slow as the menubar. BTW: If I open and close the "FLTK scheme" pull-down menu on the top, the effect is the same (opens instantly and closes very slowly showing a grey box while blocking the application).
	#10  michaelbaeuerle 01:38 May 13, 2013	Albrecht wrote: | I assume that the XIM is somehow attached to a specific window, | and after destroying a window it must be re-initialized somehow. | Or similar... There is an official document for the XIM protocol: http://www.x.org/releases/X11R7.6/doc/libX11/specs/XIM/xim.html According to the chapter "Architecture" I understand it this way: In general you have to open the IM only once (at least if you don't want to change the locale "on the fly") to create the programs bindings to the locale and the IM server => XOpenIM() and XCloseIM. The things you have to create and destroy while the program is running are only the input contexts (ICs) => XCreateIC() and XDestroyIC. IMHO: Because (with the client/server model) the IM server can be an external program running on a different machine (with a network redirected display it would run on the opposite side of the FLTK library), the XOpenIM() operation should be considered as inherently expensive and slow even if the locale stuff would not be an additional problem. Micha
	#11  AlbrechtS 02:35 May 13, 2013	Michael, thanks for the test results and the additional info regarding the pull-down menu. This confirms the fact that the problem lies in re-initializing XIM for every window close operation. My assumption "that the XIM is somehow attached to a specific window" resulted from reading the code, but you're probably right that this is not the case and that it is unnecessary and causing the delay you can see. I started looking for and reading X11 docs yesterday, thanks for the pointer. However, there's a newer version of this xim document here, although I suspect that there's not much changed: http://www.x.org/releases/X11R7.7/doc/libX11/XIM/xim.html http://www.x.org/releases/X11R7.7/doc/libX11/XIM/xim.pdf I also found the Xlib docs at: http://www.x.org/releases/X11R7.7/doc/libX11/libX11/libX11.html http://www.x.org/releases/X11R7.7/doc/libX11/libX11/libX11.pdf I had to start with some basic reading to understand that "language" the documentation is written in, and then I went to chapter 13. There's a part about "Input Methods" (p. 278 ff. in the pdf document). I'll have to read this... The problem we have to solve is to remove unnecessary re-initializations, but still keep XIM working for special purposes like Asian input methods... More to follow later.
	#12  michaelbaeuerle 07:35 May 13, 2013	If somebody with a system that is not affected want to "feel" how things behave on a system with expensive XOpenIM(), simply add a delay at the position I have cited above. Something like this: ---------------------------------------------------------------------   if (fl_xim_ic && xevent.type == DestroyNotify &&         xid != xim_win && !fl_find(xid))   {     XIM xim_im;     usleep(700000);     xim_im = XOpenIM(fl_display, NULL, NULL, NULL);     ... --------------------------------------------------------------------- Or within the wrapper of my original patch so that it applies to every XOpenIM() call. Then you get the startup delay too. Micha
	#13  michaelbaeuerle 03:34 Sep 02, 2013	Albrecht, you have voted for the variant to make "IM not initialized all the time" the default. But for this we need a third option between "as is" and "off" that we currently don't have. I still prefer the variant from my patch "IM must be explicitly disabled" for the following reasons: 1) Initialize IM only once at startup seems not enough for some users    I assume this is the reason for the current implementation.    In this case the switching options should be "as is" and "off". 2) On most systems the performance impact seems to be low/moderate    This means "as is" doesn't really hurt as default in many cases.    Even if the assumption from 1) is not true, it's maybe not worth    to implement the third option "only once" with the problem in mind    that we can't prove whether this works for all users and IMs. 3) The new default implementation should not surprise users    Think about FLTK as shared library that is updated by admin/system.    Using "as is" as default makes sure this is the case.

[ Return to Bugs & Features | Post Text | Post File ]