Yesterday I've stumbled upon a very surprising bug in some Python 2 code,
related to the use of the
__del__ method in a vendor library we employ at work.
Here is some minimal code that reproduces the issue I met:
class MyClass: def __init__(self): raise RuntimeError('Woops') def __del__(self): try: raise_recursion_error() except RuntimeError: print('Recursion error caught') def raise_recursion_error(): raise_recursion_error() MyClass()
What do you think will happen ? Take a moment to make a guess.
If you execute this code, you will witness a never-ending repetition of the
Recursion error caught message in your terminal.
Interestingly, if we replace the recursion error by a simple
RuntimeError, the problem vanishes !
I don't quite understand why this recursion error, despite being caught, causes this strange behaviour.
Somehow I think the
catch does not "entirely" suppress the exception,
and that it still bubbles up "enough" to cause the Python interpreter to call
If you remove the
raise_recursion_error(), you'll get the same behaviour, with a different message:
Exception RuntimeError: 'maximum recursion depth exceeded' in <bound method MyClass.__del__ of <__main__.MyClass instance at 0x6ffffebd758>> ignored
This kind of "exception ignored" messages disappeared in Python 3, which handles this case very smoothly, by halting immediately and displaying this:
Traceback (most recent call last): File "del_recurse_infinite_loop.py", line 36, in <module> MyClass() File "del_recurse_infinite_loop.py", line 24, in __init__ raise RuntimeError('Woops') RuntimeError: Woops Recursion error caught
In the real world code where I saw this behaviour, the recursion error was due to a
__getattr__ method calling itself.
The following class presents the same behaviour:
class MyClass: def __init__(self): raise RuntimeError('Woops') def __del__(self): try: self.foo() except RuntimeError: print('Recursion error caught') def __getattr__(self, name): return self.uninitialized_attribute
Not also that in its original form, I wasn't even able to stop the process with CTRL+C !
KeyboardInterrupt exceptions were ignored.
My key takeaways from this painful deep dive:
- Python 3 is safer than Python 2
- recursion error are a special breed of
RuntimeError, and are sometimes handled differently in Python 2 (even if they are not identified by a named subclass)
- a typo in the code of a
__getattr__method can lead to infinite recursion
- stay away from
I'm not the first to warn about the
__del__ method by the way:
- Python Gotchas 1: del is not the opposite of init
- Python destructor drawbacks
- An Interesting Fact About The Python Garbage Collector
This last one even mentions that it can generate memory leaks.
Let me know in the comments section if ever you have more information to explain this strange quirk !