Re: [patch] python locking fixes

[Marko Kreen <marko@l-t.ee>]

On Fri, Oct 03, 2003 at 03:07:00PM +0100, Gustavo J. A. M. Carneiro wrote:
> A Sex, 2003-10-03 ās 12:55, Marko Kreen escreveu:
> > I checked pygtk 2.0.0 sources: it seems to have similar ideas
> > about locking as patched python.c - dialog.run() drops python
> > lock before calling gtk_dialog_run.  So no deadlock.
> 
>   If you look at _wrap_gdk_threads_init() in gdk.override, you'll notice
> that python locking acquire and release only happens if all the
> following are true:
> 	1. pygtk was built with threading (--enable-thread), which is enabled
> by default if the python thread module is found;
> 	2. The python code calls gtk.threads_init().
>   If 1 and 2 are not true, and if I'm reading the code correctly, then
> pyg_unblock_threads and pyg_block_threads do effectively nothing.
>   I have just realized this now, and that leads me to the conclusion
> that *probably* the deadlock problems disappear if I call
> gtk.threads_init().  I also found this FAQ:
> 
>   http://www.async.com.br/faq/pygtk/index.py?req=show&file=faq20.006.htp

Yes, gtk.threads_init() seems pretty important.  Another variant
is to compile python.c with '#undef WITH_THREAD' - that should
give also a guaranteed problem-free setup.

Basically, ATM I am not worried about Python locking -> both
Python and pygtk cooperate, the PyGILState API may make things
bit nicer, but everything should work OK already.

But now I am worried about xchat lock:

> >     xchat(1)->python->gtk_dialog_run(drops lock)->xchat(2)	[1]

> > Hmm.  I wanted to continue with "that messes xchat_lock
> > handling up", but now I wondering if its correct even now,
> > in light of [1].  Need to think a bit.  But I still dont
> > see anything wrong in my handling of Python lock.

I cant find a good solution to following scenario, when xchat
main C code is not cooperating:

1) xchat_lock released
2a) some python thread calls xchat.*, python.c sees unlocked xchat_lock,
    grabs it and calls xchat C code
2b) another python thread calls gtk.mainloop, which awakes xchat C code

The problem is that thanks to gtk.mainloop/dialog.run there are
other paths from python code to xchat C code which are not under
control of python.c.  That makes having xchat_lock in python.c
pointless, it should be managed by xchat main code.  (As
previously discussed, then it could be shared by other plugins)

Solution #1: set 2 requirements to python code:
1) call xchat.* only from main thread.
   [can be forced by python.c]
2) call gtk.mainloop/dialog.run also only from main thread
   [cannot be forced]

Solution #2: remove threading from python.c, and when xchat
itself becomes thread-aware, put it back...

#1 keeps generic threading working, only specifically xchat.*
and pygtk are affected.  But #2 also has good points ;)

>   Ok, in light of these new discoveries, I see nothing wrong with your
> patch.  However, I don't think things will ever work smoothly until both
> pygtk and xchat-python migrate to the Python GIL API.  The great thing
> about the GIL API is that the underlying mutex is recursive.  I don't
> mind waiting though.  In the mean time I can use my forked python plugin
> for my own plugins, and later move to xchat's own python plugin when it
> works for me.

About the idea we discussed in private: having one interpreter
state but loading scripts into different modules - I've got it
working, its not yet presentable tho', but idea seems fine.
Well, I hoped to get it to work with threading but now I am
starting to lose enthusiasm towards it, as without main xchat
cooperating, there will be several requirements not in standard
interpreter.

And without threads, the resulting code is basically same as
your code, only some changes in script loading...  Is it good
or bad sign? :)


-- 
marko

--
XChat-discuss: mailing list for XChat users
Archive:       http://mail.nl.linux.org/xchat-discuss/