如何实现__getattribute__而没有无限递归错误?

问题:如何实现__getattribute__而没有无限递归错误?

我想覆盖对类中一个变量的访问,但通常返回所有其他变量。我该怎么做__getattribute__呢?

我尝试了以下操作(它也应说明我要执行的操作),但是出现了递归错误:

class D(object):
    def __init__(self):
        self.test=20
        self.test2=21
    def __getattribute__(self,name):
        if name=='test':
            return 0.
        else:
            return self.__dict__[name]

>>> print D().test
0.0
>>> print D().test2
...
RuntimeError: maximum recursion depth exceeded in cmp

I want to override access to one variable in a class, but return all others normally. How do I accomplish this with __getattribute__?

I tried the following (which should also illustrate what I’m trying to do) but I get a recursion error:

class D(object):
    def __init__(self):
        self.test=20
        self.test2=21
    def __getattribute__(self,name):
        if name=='test':
            return 0.
        else:
            return self.__dict__[name]

>>> print D().test
0.0
>>> print D().test2
...
RuntimeError: maximum recursion depth exceeded in cmp

回答 0

您收到递归错误,因为您尝试访问其中的self.__dict__属性会再次__getattribute__调用您__getattribute__。如果你使用object__getattribute__不是,它的工作原理:

class D(object):
    def __init__(self):
        self.test=20
        self.test2=21
    def __getattribute__(self,name):
        if name=='test':
            return 0.
        else:
            return object.__getattribute__(self, name)

之所以可行,是因为object(在此示例中)是基类。通过调用您的基本版本,__getattribute__可以避免您以前遇到的递归地狱。

IPython的输出与foo.py中的代码:

In [1]: from foo import *

In [2]: d = D()

In [3]: d.test
Out[3]: 0.0

In [4]: d.test2
Out[4]: 21

更新:

在当前文档中,标题为“ 针对新样式类的更多属性访问 ”的部分中有一些内容,他们建议完全这样做以避免无限递归。

You get a recursion error because your attempt to access the self.__dict__ attribute inside __getattribute__ invokes your __getattribute__ again. If you use object‘s __getattribute__ instead, it works:

class D(object):
    def __init__(self):
        self.test=20
        self.test2=21
    def __getattribute__(self,name):
        if name=='test':
            return 0.
        else:
            return object.__getattribute__(self, name)

This works because object (in this example) is the base class. By calling the base version of __getattribute__ you avoid the recursive hell you were in before.

Ipython output with code in foo.py:

In [1]: from foo import *

In [2]: d = D()

In [3]: d.test
Out[3]: 0.0

In [4]: d.test2
Out[4]: 21

Update:

There’s something in the section titled More attribute access for new-style classes in the current documentation, where they recommend doing exactly this to avoid the infinite recursion.


回答 1

实际上,我相信您想改用__getattr__特殊方法。

引用Python文档:

__getattr__( self, name)

当在常规位置未找到属性时调用该属性(即,它不是实例属性,也不是在自身的类树中找到该属性)。name是属性名称。此方法应返回(计算出的)属性值或引发AttributeError异常。
请注意,如果通过常规机制找到该属性,__getattr__()则不会调用该属性。(这是__getattr__()和之间的故意不对称__setattr__()。)这样做是出于效率方面的考虑,并且因为否则__setattr__()将无法访问实例的其他属性。请注意,至少对于实例变量,您可以通过不在实例属性字典中插入任何值(而是将其插入另一个对象中)来伪造总体控制。见__getattribute__() 方法,以实际获得新样式类中的总控制权。

注:对于这项工作,该实例应该不会有一个test属性,因此行self.test=20应该被删除。

Actually, I believe you want to use the __getattr__ special method instead.

Quote from the Python docs:

__getattr__( self, name)

Called when an attribute lookup has not found the attribute in the usual places (i.e. it is not an instance attribute nor is it found in the class tree for self). name is the attribute name. This method should return the (computed) attribute value or raise an AttributeError exception.
Note that if the attribute is found through the normal mechanism, __getattr__() is not called. (This is an intentional asymmetry between __getattr__() and __setattr__().) This is done both for efficiency reasons and because otherwise __setattr__() would have no way to access other attributes of the instance. Note that at least for instance variables, you can fake total control by not inserting any values in the instance attribute dictionary (but instead inserting them in another object). See the __getattribute__() method below for a way to actually get total control in new-style classes.

Note: for this to work, the instance should not have a test attribute, so the line self.test=20 should be removed.


回答 2

Python语言参考:

为了避免此方法的无限递归,其实现应始终调用具有相同名称的基类方法以访问其所需的任何属性,例如 object.__getattribute__(self, name)

含义:

def __getattribute__(self,name):
    ...
        return self.__dict__[name]

您正在调用名为的属性__dict__。由于它是一个属性,因此__getattribute__会在搜索__dict__中调用__getattribute__哪个调用而被调用… yada yada yada

return  object.__getattribute__(self, name)

使用基类__getattribute__有助于查找真实属性。

Python language reference:

In order to avoid infinite recursion in this method, its implementation should always call the base class method with the same name to access any attributes it needs, for example, object.__getattribute__(self, name).

Meaning:

def __getattribute__(self,name):
    ...
        return self.__dict__[name]

You’re calling for an attribute called __dict__. Because it’s an attribute, __getattribute__ gets called in search for __dict__ which calls __getattribute__ which calls … yada yada yada

return  object.__getattribute__(self, name)

Using the base classes __getattribute__ helps finding the real attribute.


回答 3

确定要使用__getattribute__吗?您实际上想实现什么?

最简单的方法是:

class D(object):
    def __init__(self):
        self.test = 20
        self.test2 = 21

    test = 0

要么:

class D(object):
    def __init__(self):
        self.test = 20
        self.test2 = 21

    @property
    def test(self):
        return 0

编辑:请注意,的实例在每种情况下D将具有不同的值test。在第一种情况下d.test为20,在第二种情况下为0。我将由您自己确定原因。

Edit2:Greg指出示例2将失败,因为该属性是只读属性,并且该__init__方法尝试将其设置为20。对此的更完整示例为:

class D(object):
    def __init__(self):
        self.test = 20
        self.test2 = 21

    _test = 0

    def get_test(self):
        return self._test

    def set_test(self, value):
        self._test = value

    test = property(get_test, set_test)

显然,作为一门课,这几乎是毫无用处的,但它为您提供了继续学习的想法。

Are you sure you want to use __getattribute__? What are you actually trying to achieve?

The easiest way to do what you ask is:

class D(object):
    def __init__(self):
        self.test = 20
        self.test2 = 21

    test = 0

or:

class D(object):
    def __init__(self):
        self.test = 20
        self.test2 = 21

    @property
    def test(self):
        return 0

Edit: Note that an instance of D would have different values of test in each case. In the first case d.test would be 20, in the second it would be 0. I’ll leave it to you to work out why.

Edit2: Greg pointed out that example 2 will fail because the property is read only and the __init__ method tried to set it to 20. A more complete example for that would be:

class D(object):
    def __init__(self):
        self.test = 20
        self.test2 = 21

    _test = 0

    def get_test(self):
        return self._test

    def set_test(self, value):
        self._test = value

    test = property(get_test, set_test)

Obviously, as a class this is almost entirely useless, but it gives you an idea to move on from.


回答 4

这是一个更可靠的版本:

class D(object):
    def __init__(self):
        self.test = 20
        self.test2 = 21
    def __getattribute__(self, name):
        if name == 'test':
            return 0.
        else:
            return super(D, self).__getattribute__(name)

它从父类调用__ getattribute __方法,最终退回到对象。__ getattribute __方法,如果其他祖先没有覆盖它。

Here is a more reliable version:

class D(object):
    def __init__(self):
        self.test = 20
        self.test2 = 21
    def __getattribute__(self, name):
        if name == 'test':
            return 0.
        else:
            return super(D, self).__getattribute__(name)

It calls __getattribute__ method from parent class, eventually falling back to object.__getattribute__ method if other ancestors don’t override it.


回答 5

如何__getattribute__使用该方法?

在普通的点分查找之前调用它。如果涨了AttributeError,我们打电话__getattr__

这种方法很少使用。标准库中只有两个定义:

$ grep -Erl  "def __getattribute__\(self" cpython/Lib | grep -v "/test/"
cpython/Lib/_threading_local.py
cpython/Lib/importlib/util.py

最佳实践

以编程方式控制对单个属性的访问的正确方法是使用property。类的D编写应如下所示(可以使用setter和Deleter来复制明显的预期行为):

class D(object):
    def __init__(self):
        self.test2=21

    @property
    def test(self):
        return 0.

    @test.setter
    def test(self, value):
        '''dummy function to avoid AttributeError on setting property'''

    @test.deleter
    def test(self):
        '''dummy function to avoid AttributeError on deleting property'''

和用法:

>>> o = D()
>>> o.test
0.0
>>> o.test = 'foo'
>>> o.test
0.0
>>> del o.test
>>> o.test
0.0

属性是数据描述符,因此它是常规点分查找算法中要查找的第一件事。

的选项 __getattribute__

如果您绝对需要通过来为每个属性实现查找,则有几种选择__getattribute__

  • 提高AttributeError,导致__getattr__被调用(如果已实现)
  • 从中退还东西
    • 通过super调用父类的(可能object的)执行
    • 呼唤 __getattr__
    • 以某种方式实现您自己的虚线查找算法

例如:

class NoisyAttributes(object):
    def __init__(self):
        self.test=20
        self.test2=21
    def __getattribute__(self, name):
        print('getting: ' + name)
        try:
            return super(NoisyAttributes, self).__getattribute__(name)
        except AttributeError:
            print('oh no, AttributeError caught and reraising')
            raise
    def __getattr__(self, name):
        """Called if __getattribute__ raises AttributeError"""
        return 'close but no ' + name    


>>> n = NoisyAttributes()
>>> nfoo = n.foo
getting: foo
oh no, AttributeError caught and reraising
>>> nfoo
'close but no foo'
>>> n.test
getting: test
20

您最初想要的。

此示例说明了如何执行您最初想要的操作:

class D(object):
    def __init__(self):
        self.test=20
        self.test2=21
    def __getattribute__(self,name):
        if name=='test':
            return 0.
        else:
            return super(D, self).__getattribute__(name)

并且会像这样:

>>> o = D()
>>> o.test = 'foo'
>>> o.test
0.0
>>> del o.test
>>> o.test
0.0
>>> del o.test

Traceback (most recent call last):
  File "<pyshell#216>", line 1, in <module>
    del o.test
AttributeError: test

代码审查

您的代码带注释。您在中对自己进行了点查询__getattribute__。这就是为什么您会得到递归错误的原因。您可以检查名称是否可用,"__dict__"并使用它super来解决,但这并不覆盖__slots__。我将其留给读者练习。

class D(object):
    def __init__(self):
        self.test=20
        self.test2=21
    def __getattribute__(self,name):
        if name=='test':
            return 0.
        else:      #   v--- Dotted lookup on self in __getattribute__
            return self.__dict__[name]

>>> print D().test
0.0
>>> print D().test2
...
RuntimeError: maximum recursion depth exceeded in cmp

How is the __getattribute__ method used?

It is called before the normal dotted lookup. If it raises AttributeError, then we call __getattr__.

Use of this method is rather rare. There are only two definitions in the standard library:

$ grep -Erl  "def __getattribute__\(self" cpython/Lib | grep -v "/test/"
cpython/Lib/_threading_local.py
cpython/Lib/importlib/util.py

Best Practice

The proper way to programmatically control access to a single attribute is with property. Class D should be written as follows (with the setter and deleter optionally to replicate apparent intended behavior):

class D(object):
    def __init__(self):
        self.test2=21

    @property
    def test(self):
        return 0.

    @test.setter
    def test(self, value):
        '''dummy function to avoid AttributeError on setting property'''

    @test.deleter
    def test(self):
        '''dummy function to avoid AttributeError on deleting property'''

And usage:

>>> o = D()
>>> o.test
0.0
>>> o.test = 'foo'
>>> o.test
0.0
>>> del o.test
>>> o.test
0.0

A property is a data descriptor, thus it is the first thing looked for in the normal dotted lookup algorithm.

Options for __getattribute__

You several options if you absolutely need to implement lookup for every attribute via __getattribute__.

  • raise AttributeError, causing __getattr__ to be called (if implemented)
  • return something from it by
    • using super to call the parent (probably object‘s) implementation
    • calling __getattr__
    • implementing your own dotted lookup algorithm somehow

For example:

class NoisyAttributes(object):
    def __init__(self):
        self.test=20
        self.test2=21
    def __getattribute__(self, name):
        print('getting: ' + name)
        try:
            return super(NoisyAttributes, self).__getattribute__(name)
        except AttributeError:
            print('oh no, AttributeError caught and reraising')
            raise
    def __getattr__(self, name):
        """Called if __getattribute__ raises AttributeError"""
        return 'close but no ' + name    


>>> n = NoisyAttributes()
>>> nfoo = n.foo
getting: foo
oh no, AttributeError caught and reraising
>>> nfoo
'close but no foo'
>>> n.test
getting: test
20

What you originally wanted.

And this example shows how you might do what you originally wanted:

class D(object):
    def __init__(self):
        self.test=20
        self.test2=21
    def __getattribute__(self,name):
        if name=='test':
            return 0.
        else:
            return super(D, self).__getattribute__(name)

And will behave like this:

>>> o = D()
>>> o.test = 'foo'
>>> o.test
0.0
>>> del o.test
>>> o.test
0.0
>>> del o.test

Traceback (most recent call last):
  File "<pyshell#216>", line 1, in <module>
    del o.test
AttributeError: test

Code review

Your code with comments. You have a dotted lookup on self in __getattribute__. This is why you get a recursion error. You could check if name is "__dict__" and use super to workaround, but that doesn’t cover __slots__. I’ll leave that as an exercise to the reader.

class D(object):
    def __init__(self):
        self.test=20
        self.test2=21
    def __getattribute__(self,name):
        if name=='test':
            return 0.
        else:      #   v--- Dotted lookup on self in __getattribute__
            return self.__dict__[name]

>>> print D().test
0.0
>>> print D().test2
...
RuntimeError: maximum recursion depth exceeded in cmp