如何正确清理Python对象?

问题:如何正确清理Python对象?

class Package:
    def __init__(self):
        self.files = []

    # ...

    def __del__(self):
        for file in self.files:
            os.unlink(file)

__del__(self)上面的失败,并带有AttributeError异常。我了解Python__del__()调用时不保证存在“全局变量”(在这种情况下是成员数据吗?)。如果是这种情况,并且这是导致异常的原因,那么如何确保对象正确销毁?

class Package:
    def __init__(self):
        self.files = []

    # ...

    def __del__(self):
        for file in self.files:
            os.unlink(file)

__del__(self) above fails with an AttributeError exception. I understand Python doesn’t guarantee the existence of “global variables” (member data in this context?) when __del__() is invoked. If that is the case and this is the reason for the exception, how do I make sure the object destructs properly?


回答 0

我建议使用Python的with语句来管理需要清理的资源。使用显式close()语句的问题在于,您必须担心人们会忘记完全调用它,或者忘记将其放在finally块中以防止发生异常时发生资源泄漏。

要使用该with语句,请使用以下方法创建一个类:

  def __enter__(self)
  def __exit__(self, exc_type, exc_value, traceback)

在上面的示例中,您将使用

class Package:
    def __init__(self):
        self.files = []

    def __enter__(self):
        return self

    # ...

    def __exit__(self, exc_type, exc_value, traceback):
        for file in self.files:
            os.unlink(file)

然后,当有人想使用您的类时,他们将执行以下操作:

with Package() as package_obj:
    # use package_obj

变量package_obj将是Package类型的实例(它是__enter__方法返回的值)。__exit__无论是否发生异常,都会自动调用其方法。

您甚至可以进一步采用这种方法。在上面的示例中,仍然可以使用其构造函数实例化Package而无需使用该with子句。您不希望这种情况发生。您可以通过创建定义__enter____exit__方法的PackageResource类来解决此问题。然后,将严格在__enter__方法内部定义Package类并返回。这样,调用者永远无法在不使用with语句的情况下实例化Package类:

class PackageResource:
    def __enter__(self):
        class Package:
            ...
        self.package_obj = Package()
        return self.package_obj

    def __exit__(self, exc_type, exc_value, traceback):
        self.package_obj.cleanup()

您将按以下方式使用它:

with PackageResource() as package_obj:
    # use package_obj

I’d recommend using Python’s with statement for managing resources that need to be cleaned up. The problem with using an explicit close() statement is that you have to worry about people forgetting to call it at all or forgetting to place it in a finally block to prevent a resource leak when an exception occurs.

To use the with statement, create a class with the following methods:

  def __enter__(self)
  def __exit__(self, exc_type, exc_value, traceback)

In your example above, you’d use

class Package:
    def __init__(self):
        self.files = []

    def __enter__(self):
        return self

    # ...

    def __exit__(self, exc_type, exc_value, traceback):
        for file in self.files:
            os.unlink(file)

Then, when someone wanted to use your class, they’d do the following:

with Package() as package_obj:
    # use package_obj

The variable package_obj will be an instance of type Package (it’s the value returned by the __enter__ method). Its __exit__ method will automatically be called, regardless of whether or not an exception occurs.

You could even take this approach a step further. In the example above, someone could still instantiate Package using its constructor without using the with clause. You don’t want that to happen. You can fix this by creating a PackageResource class that defines the __enter__ and __exit__ methods. Then, the Package class would be defined strictly inside the __enter__ method and returned. That way, the caller never could instantiate the Package class without using a with statement:

class PackageResource:
    def __enter__(self):
        class Package:
            ...
        self.package_obj = Package()
        return self.package_obj

    def __exit__(self, exc_type, exc_value, traceback):
        self.package_obj.cleanup()

You’d use this as follows:

with PackageResource() as package_obj:
    # use package_obj

回答 1

标准方法是使用atexit.register

# package.py
import atexit
import os

class Package:
    def __init__(self):
        self.files = []
        atexit.register(self.cleanup)

    def cleanup(self):
        print("Running cleanup...")
        for file in self.files:
            print("Unlinking file: {}".format(file))
            # os.unlink(file)

但是您应该记住,这将持久化所有创建的实例,Package直到终止Python。

使用上面的代码的演示保存为package.py

$ python
>>> from package import *
>>> p = Package()
>>> q = Package()
>>> q.files = ['a', 'b', 'c']
>>> quit()
Running cleanup...
Unlinking file: a
Unlinking file: b
Unlinking file: c
Running cleanup...

The standard way is to use atexit.register:

# package.py
import atexit
import os

class Package:
    def __init__(self):
        self.files = []
        atexit.register(self.cleanup)

    def cleanup(self):
        print("Running cleanup...")
        for file in self.files:
            print("Unlinking file: {}".format(file))
            # os.unlink(file)

But you should keep in mind that this will persist all created instances of Package until Python is terminated.

Demo using the code above saved as package.py:

$ python
>>> from package import *
>>> p = Package()
>>> q = Package()
>>> q.files = ['a', 'b', 'c']
>>> quit()
Running cleanup...
Unlinking file: a
Unlinking file: b
Unlinking file: c
Running cleanup...

回答 2

作为克林特答案的附录,您可以简化PackageResource使用contextlib.contextmanager

@contextlib.contextmanager
def packageResource():
    class Package:
        ...
    package = Package()
    yield package
    package.cleanup()

另外,尽管可能不如Pythonic,但您可以重写Package.__new__

class Package(object):
    def __new__(cls, *args, **kwargs):
        @contextlib.contextmanager
        def packageResource():
            # adapt arguments if superclass takes some!
            package = super(Package, cls).__new__(cls)
            package.__init__(*args, **kwargs)
            yield package
            package.cleanup()

    def __init__(self, *args, **kwargs):
        ...

并简单地使用with Package(...) as package

为了使事情更短,请命名清理函数close并使用contextlib.closing,在这种情况下,您可以通过使用未修改的Package类,with contextlib.closing(Package(...))或者将其改写__new__为更简单的类。

class Package(object):
    def __new__(cls, *args, **kwargs):
        package = super(Package, cls).__new__(cls)
        package.__init__(*args, **kwargs)
        return contextlib.closing(package)

而且此构造函数是继承的,因此您可以简单地继承,例如

class SubPackage(Package):
    def close(self):
        pass

As an appendix to Clint’s answer, you can simplify PackageResource using contextlib.contextmanager:

@contextlib.contextmanager
def packageResource():
    class Package:
        ...
    package = Package()
    yield package
    package.cleanup()

Alternatively, though probably not as Pythonic, you can override Package.__new__:

class Package(object):
    def __new__(cls, *args, **kwargs):
        @contextlib.contextmanager
        def packageResource():
            # adapt arguments if superclass takes some!
            package = super(Package, cls).__new__(cls)
            package.__init__(*args, **kwargs)
            yield package
            package.cleanup()

    def __init__(self, *args, **kwargs):
        ...

and simply use with Package(...) as package.

To get things shorter, name your cleanup function close and use contextlib.closing, in which case you can either use the unmodified Package class via with contextlib.closing(Package(...)) or override its __new__ to the simpler

class Package(object):
    def __new__(cls, *args, **kwargs):
        package = super(Package, cls).__new__(cls)
        package.__init__(*args, **kwargs)
        return contextlib.closing(package)

And this constructor is inherited, so you can simply inherit, e.g.

class SubPackage(Package):
    def close(self):
        pass

回答 3

我不认为实例成员可能在__del__调用之前被删除。我的猜测是您特定AttributeError的原因是其他原因(也许您在其他地方错误地删除了self.file)。

但是,正如其他人指出的那样,您应该避免使用__del__。这样做的主要原因是__del__不会对的实例进行垃圾回收(只有当其引用计数达到0时,它们才会被释放)。因此,如果您的实例涉及循环引用,则它们将在应用程序运行期间一直存在于内存中。(尽管我可能对所有这些都弄错了,我不得不再次阅读gc文档,但是我相当确定它的工作原理是这样的)。

I don’t think that it’s possible for instance members to be removed before __del__ is called. My guess would be that the reason for your particular AttributeError is somewhere else (maybe you mistakenly remove self.file elsewhere).

However, as the others pointed out, you should avoid using __del__. The main reason for this is that instances with __del__ will not be garbage collected (they will only be freed when their refcount reaches 0). Therefore, if your instances are involved in circular references, they will live in memory for as long as the application run. (I may be mistaken about all this though, I’d have to read the gc docs again, but I’m rather sure it works like this).


回答 4

更好的选择是使用weakref.finalize。请参见“ 终结器对象”和“ 将终结器与__del __()方法进行比较”中的示例。


回答 5

我认为问题可能出在__init__如果代码比显示的更多?

__del__即使__init__未正确执行或引发异常也会被调用。

资源

I think the problem could be in __init__ if there is more code than shown?

__del__ will be called even when __init__ has not been executed properly or threw an exception.

Source


回答 6

这是一个最小的工作框架:

class SkeletonFixture:

    def __init__(self):
        pass

    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        pass

    def method(self):
        pass


with SkeletonFixture() as fixture:
    fixture.method()

重要提示:自我回报


如果您像我一样,并且忽略了return self一部分(克林特·米勒的正确答案),那么您将盯着这个废话:

Traceback (most recent call last):
  File "tests/simplestpossible.py", line 17, in <module>                                                                                                                                                          
    fixture.method()                                                                                                                                                                                              
AttributeError: 'NoneType' object has no attribute 'method'

希望它能帮助下一个人。

Here is a minimal working skeleton:

class SkeletonFixture:

    def __init__(self):
        pass

    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        pass

    def method(self):
        pass


with SkeletonFixture() as fixture:
    fixture.method()

Important: return self


If you’re like me, and overlook the return self part (of Clint Miller’s correct answer), you will be staring at this nonsense:

Traceback (most recent call last):
  File "tests/simplestpossible.py", line 17, in <module>                                                                                                                                                          
    fixture.method()                                                                                                                                                                                              
AttributeError: 'NoneType' object has no attribute 'method'

Hope it helps the next person.


回答 7

只需用try / except语句包装您的析构函数,如果您的全局变量已被处理,它将不会引发异常。

编辑

尝试这个:

from weakref import proxy

class MyList(list): pass

class Package:
    def __init__(self):
        self.__del__.im_func.files = MyList([1,2,3,4])
        self.files = proxy(self.__del__.im_func.files)

    def __del__(self):
        print self.__del__.im_func.files

它将把文件列表填充到保证在调用时存在的del函数中。弱引用代理是为了防止Python或您自己以某种方式删除self.files变量(如果删除了它,则不会影响原始文件列表)。即使存在更多对该变量的引用,如果不是不是要删除此变量,则可以删除代理封装。

Just wrap your destructor with a try/except statement and it will not throw an exception if your globals are already disposed of.

Edit

Try this:

from weakref import proxy

class MyList(list): pass

class Package:
    def __init__(self):
        self.__del__.im_func.files = MyList([1,2,3,4])
        self.files = proxy(self.__del__.im_func.files)

    def __del__(self):
        print self.__del__.im_func.files

It will stuff the file list in the del function that is guaranteed to exist at the time of call. The weakref proxy is to prevent Python, or yourself from deleting the self.files variable somehow (if it is deleted, then it will not affect the original file list). If it is not the case that this is being deleted even though there are more references to the variable, then you can remove the proxy encapsulation.


回答 8

似乎惯用的方法是提供一个close()方法(或类似方法),并明确地调用它。

It seems that the idiomatic way to do this is to provide a close() method (or similar), and call it explicitely.