[pygtk] Garbage collection prematurely clears cyclical objects referenced from GTK callbacks

Mark Seaborn mseaborn at cmedresearch.com
Wed Aug 13 19:41:15 WST 2008


On Wed, 2008-08-13 at 11:29 +0200, Hrvoje Nikšić wrote:

> I don't quite understand how this bug occurs, and why it fails to appear
> when the cyclical reference is removed.  After all, if the object is not
> correctly increffed from GTK, it should die as soon as Buggy() finishes,
> but it doesn't happen.

Something I don't understand is why the bug doesn't cause a crash.  I
would expect breaking the cycle to cause the refcount of the closure to
drop to zero, causing its memory to be freed and potentially reused
before the closure is called by Gtk.

I can think of two explanations:

* Something is incrementing the refcount of the closure after Python
attempts to break the cycle.
* Luck: the closure's memory is not reused and Python does not zero out
any other fields when freeing the block or check that the refcount is
sane when using the closure.  This should be testable with valgrind.


> If someone understands the mechanism of PyGTK's memory management that
> could provoke this bug, any insight would be appreciated.

Here's some history of the changes that have been made which I have
pieced together from Bugzilla and SVN:

2002: The original complaint was that cycles between GObjects and signal
handlers would never get freed because the cycles occurred in C land and
could not be tracked by Python:
http://bugzilla.gnome.org/show_bug.cgi?id=71435
"cyclic garbage collection and signal handlers"
This was addressed in 2002 in r152:
http://svn.gnome.org/viewvc/pygobject?view=revision&revision=152
This introduced a scheme whereby refcounts to Python callback functions
are not held in C land by GClosures, but by the Python wrapper for
GObjects.  The problem with this is that the GObject wrapper and the
Python callbacks form a cycle that appears to Python to be unreferenced,
even if the GObject is referenced from C land.  So you could get unsafe
behaviour: deleting cycles that are still live.

2003: The next bug complained about this unsafe behaviour:
http://bugzilla.gnome.org/show_bug.cgi?id=92955
"gc.collect() destroys __dict__?"
This was marked as fixed in 2003 in r201:
http://svn.gnome.org/viewvc/pygobject?view=revision&revision=201
This changed things so that "when setting up a wrapper for a GObject,
the GObject gets a ref to the PyObject and vice versa".  This introduces
a cycle for every GObject wrapper, but the cycle is declared to Python
via an extra case in pygobject_traverse(), which inspects the GObject's
refcount.  (This replaced some other code that would inspect the
GObject's refcount.)  This had the side effect that GObjects and their
wrappers could only get freed through Python's cycle GC, so there would
be a delay before GObjects are freed.

2005: The next bug complained about that delay:
http://bugzilla.gnome.org/show_bug.cgi?id=320428
"Break PyGObject<->GObject reference cycle"
A large number of patches were proposed to fix this bug.
The change that was committed, in April 2007, was r642:
http://svn.gnome.org/viewvc/pygobject?view=revision&revision=642
This introduced a use of gobject's "toggle refs" mechanism, which is
described in:
http://library.gnome.org/devel/gobject/stable/gobject-The-Base-Object-Type.html#g-object-add-toggle-ref
Use of toggle refs replaces direct inspection of the GObject's ref
count, which is now only checked by assertions and debugging code.
However, toggle refs are not *always* used.  They are only used when the
PyGObject's __dict__ is accessed (either directly or by assigning to
attributes).  I believe this change introduced the current bug, which
should only occur when toggle refs are not used.

Gustavo Carneiro's proposed fix for the current bug
(http://bugzilla.gnome.org/show_bug.cgi?id=546802) reintroduces code
that checks the refcount of the GObject.  That sounds about right: I
don't think the code can be correct if it doesn't check the GObject's
refcount, unless it is using toggle refs (which check the refcount for
you).

-- 
Mark Seaborn
Software Engineer

Cmed Technology Ltd.
Registered in England and Wales No. 3869835
Registered Office and Address for Communication:
Holmwood, Broadlands Business Campus,
Langhurstwood Road, Horsham, RH12 4QP, United Kingdom

E mseaborn at cmedresearch.com
W www.cmedresearch.com



More information about the pygtk mailing list