[pygtk] PyGTK 2.0 threading broken? Includes (hacky) workaround.

William R Sowerbutts will@sowerbutts.com
Wed, 9 Oct 2002 14:00:41 +0100

Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline


I've been writing an app using PyGTK 1.99.13 as packaged by Debian. I've had
to build my own version with threading enabled, as their package is built with
it disabled. Without this change, threads other than the one which calls
gtk.mainloop() seem to spend most of their time blocked and only run very
rarely (typically coinciding with UI interaction or the application exiting).

My app is very simple; basically it has a UI which the user interacts with,
and a background thread which receives data from a network protocol stack
(written in threaded python) and uses it to update part of the UI.

My previous version of this app, using the GTK 1.2 bindings, worked just fine
with threads. All I had to do with ensure that I did the little
gtk.threads_enter() and gtk.threads_leave() dance at the appropriate points.
However, when using the GTK 2.0 bindings, this doesn't work -- I get
"unexpected async reply" errors from Xlib, assertion failures, etc.

My conclusion was that the threading support in the GTK 2 bindings is
incomplete. Is there a lot of work involved in fixing this? It's rather

I ended up using an ugly workaround, of which I'm not particularly proud, but
I'll share it in case anyone else has this problem. As I said, the threads
only have to do minor UI updates, so the problem is simply one of getting the
main thread to execute the GTK calls rather than the networking threads. I
package the data relating to the UI update into a dictionary and append it to
a list, and then write a byte to a pipe (as returned by os.pipe()). I've
previously called (from the main thread) gtk.input_add_full() to add a
callback whenever the pipe becomes readable, and I use this callback to drain
the data from the pipe and any UI updates from the list.

A variation of this workaround should allow any thread to execute gtk calls
directly, by first "parking" the main thread. Some ugly example code is
attached. It's not a good solution, and it's not very well tested, but it
works for me (so far). It uses a global lock and pair of pipes for
synchronisation, which probably isn't very robust. The first pipe is used to
force GTK to call my callback function to park the main thread. The second
pipe is used to remove the race to re-acquire the global lock. I had problems
with more than one thread waiting to acquire the lock at once, so I added a
second lock to serialise execution of this code. Like I said, it's ugly and
exceptionally inefficient. The real fix is of course to fix the PyGTK code,
but this at least allows me to develop threaded code until the PyGTK bindings
are fixed.

I'd be interested to know if the attached code is usable as a workaround in a
larger threaded application.



William R Sowerbutts                                  will@sowerbutts.com
Coder / Guru / Nrrrd                                http://sowerbutts.com
         main(){char*s=">#=0> ^#X@#@^7=",c=0,m;for(;c<15;c++)for

Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename=demo-workaround

#!/usr/bin/env python
# PyGTK 1.99.13 threading bug workaround
# (c) 2002 William R Sowerbutts <will@sowerbutts.com>

import pygtk

import gtk
import os
import fcntl
import threading, thread
import time
import random

# pygtk threading bug workaround:
def my_threads_enter():
    os.write(gtk.workaround_acquire_pipe_write, "!")

def my_threads_leave():
    os.write(gtk.workaround_release_pipe_write, "?")

def my_pipe_callback(fd, lock):
    os.read(gtk.workaround_acquire_pipe_read, 1)
    os.read(gtk.workaround_release_pipe_read, 1)
gtk.threads_init() # be sure to only call this once
gtk.workaround_work_lock = threading.Lock()
gtk.workaround_wait_lock = threading.Lock()
gtk.workaround_acquire_pipe_read,gtk.workaround_acquire_pipe_write = os.pipe()
gtk.workaround_release_pipe_read,gtk.workaround_release_pipe_write = os.pipe()
gtk.threads_enter = my_threads_enter
gtk.threads_leave = my_threads_leave
gtk.input_add_full(gtk.workaround_acquire_pipe_read, gtk.gdk.INPUT_READ, my_pipe_callback)
# end of workaround code

# An overly simple test app:

def update_worker_thread(my_label):
    iter = 0
    while 1:
        time.sleep(random.random() / 20)
        my_label.set_text("%d" % iter)

# A simple app to test it out:
window = gtk.Window()
vbox = gtk.VBox(0)
vbox.set_size_request(200, 400)
labels = []

for i in range(0, 30):
    l = gtk.Label("")
    thread.start_new_thread(update_worker_thread, (l,))

window.connect("destroy", gtk.mainquit)