列表理解甚至在理解范围之后也会重新绑定名称。这是正确的吗?

问题:列表理解甚至在理解范围之后也会重新绑定名称。这是正确的吗?

理解与范围界定存在一些意外的相互作用。这是预期的行为吗?

我有一个方法:

def leave_room(self, uid):
  u = self.user_by_id(uid)
  r = self.rooms[u.rid]

  other_uids = [ouid for ouid in r.users_by_id.keys() if ouid != u.uid]
  other_us = [self.user_by_id(uid) for uid in other_uids]

  r.remove_user(uid) # OOPS! uid has been re-bound by the list comprehension above

  # Interestingly, it's rebound to the last uid in the list, so the error only shows
  # up when len > 1

冒着抱怨的危险,这是错误的残酷来源。在编写新代码时,偶尔会由于重新绑定而发现非常奇怪的错误-即使现在我知道这是一个问题。我需要制定一条规则,例如“始终在下划线的列表理解中使用temp vars开头”,但是即使这样也不是万无一失的。

这种随机定时炸弹等待的事实否定了列表推导的所有“易用性”。

Comprehensions are having some unexpected interactions with scoping. Is this the expected behavior?

I’ve got a method:

def leave_room(self, uid):
  u = self.user_by_id(uid)
  r = self.rooms[u.rid]

  other_uids = [ouid for ouid in r.users_by_id.keys() if ouid != u.uid]
  other_us = [self.user_by_id(uid) for uid in other_uids]

  r.remove_user(uid) # OOPS! uid has been re-bound by the list comprehension above

  # Interestingly, it's rebound to the last uid in the list, so the error only shows
  # up when len > 1

At the risk of whining, this is a brutal source of errors. As I write new code, I just occasionally find very weird errors due to rebinding — even now that I know it’s a problem. I need to make a rule like “always preface temp vars in list comprehensions with underscore”, but even that’s not fool-proof.

The fact that there’s this random time-bomb waiting kind of negates all the nice “ease of use” of list comprehensions.


回答 0

列表推导泄漏了Python 2中的循环控制变量,但没有泄漏到Python 3中。这里是Guido van Rossum(Python的创建者)解释了其背后的历史:

我们还对Python 3进行了另一处更改,以改善列表理解与生成器表达式之间的等效性。在Python 2中,列表理解将“循环”控制变量“泄漏”到周围的范围内:

x = 'before'
a = [x for x in 1, 2, 3]
print x # this prints '3', not 'before'

这是列表理解的原始实现的产物。多年来,它一直是Python的“肮脏的小秘密”之一。它起初是一种有意的折衷,目的是使列表理解迅速变得盲目,虽然对于初学者来说这不是一个常见的陷阱,但它肯定偶尔会刺痛人们。对于生成器表达式,我们无法执行此操作。生成器表达式是使用生成器实现的,生成器的执行需要单独的执行框架。因此,生成器表达式(特别是如果它们在短序列上进行迭代)比列表理解的效率低。

但是,在Python 3中,我们决定通过使用与生成器表达式相同的实现策略来修复列表理解的“肮脏的小秘密”。因此,在Python 3中,上述示例(修改为使用print(x):-之后)将打印“ before”,证明列表理解中的“ x”会暂时遮盖阴影,但不会覆盖周围的“ x”范围。

List comprehensions leak the loop control variable in Python 2 but not in Python 3. Here’s Guido van Rossum (creator of Python) explaining the history behind this:

We also made another change in Python 3, to improve equivalence between list comprehensions and generator expressions. In Python 2, the list comprehension “leaks” the loop control variable into the surrounding scope:

x = 'before'
a = [x for x in 1, 2, 3]
print x # this prints '3', not 'before'

This was an artifact of the original implementation of list comprehensions; it was one of Python’s “dirty little secrets” for years. It started out as an intentional compromise to make list comprehensions blindingly fast, and while it was not a common pitfall for beginners, it definitely stung people occasionally. For generator expressions we could not do this. Generator expressions are implemented using generators, whose execution requires a separate execution frame. Thus, generator expressions (especially if they iterate over a short sequence) were less efficient than list comprehensions.

However, in Python 3, we decided to fix the “dirty little secret” of list comprehensions by using the same implementation strategy as for generator expressions. Thus, in Python 3, the above example (after modification to use print(x) :-) will print ‘before’, proving that the ‘x’ in the list comprehension temporarily shadows but does not override the ‘x’ in the surrounding scope.


回答 1

是的,列表理解在Python 2.x中“泄漏”其变量,就像for循环一样。

回想起来,这被认为是错误的,生成器表达式可以避免这种情况。编辑:正如 Matt B.指出的那样,从Python 3向后移植set和dictionary comprehension语法时,也避免了这种情况。

列表推导的行为必须保留在Python 2中,但在Python 3中已完全修复。

这意味着:

list(x for x in a if x>32)
set(x//4 for x in a if x>32)         # just another generator exp.
dict((x, x//16) for x in a if x>32)  # yet another generator exp.
{x//4 for x in a if x>32}            # 2.7+ syntax
{x: x//16 for x in a if x>32}        # 2.7+ syntax

这些x始终是表达式的局部变量,而这些条件是:

[x for x in a if x>32]
set([x//4 for x in a if x>32])         # just another list comp.
dict([(x, x//16) for x in a if x>32])  # yet another list comp.

在Python 2.x中,所有x变量都会泄漏到周围的范围内。


Python 3.8(?)的更新PEP 572将引入:=赋值运算符,该运算符故意泄漏出理解力和生成器表达式!它主要是由两个用例驱动的:从早期终止的功能(如any()和)中捕获“见证” all()

if any((comment := line).startswith('#') for line in lines):
    print("First comment:", comment)
else:
    print("There are no comments")

并更新可变状态:

total = 0
partial_sums = [total := total + v for v in values]

有关准确的作用域,请参见附录B。除非在函数中声明了或,否则变量会在最接近的def或中分配。lambdanonlocalglobal

Yes, list comprehensions “leak” their variable in Python 2.x, just like for loops.

In retrospect, this was recognized to be a mistake, and it was avoided with generator expressions. EDIT: As Matt B. notes it was also avoided when set and dictionary comprehension syntaxes were backported from Python 3.

List comprehensions’ behavior had to be left as it is in Python 2, but it’s fully fixed in Python 3.

This means that in all of:

list(x for x in a if x>32)
set(x//4 for x in a if x>32)         # just another generator exp.
dict((x, x//16) for x in a if x>32)  # yet another generator exp.
{x//4 for x in a if x>32}            # 2.7+ syntax
{x: x//16 for x in a if x>32}        # 2.7+ syntax

the x is always local to the expression while these:

[x for x in a if x>32]
set([x//4 for x in a if x>32])         # just another list comp.
dict([(x, x//16) for x in a if x>32])  # yet another list comp.

in Python 2.x all leak the x variable to the surrounding scope.


UPDATE for Python 3.8(?): PEP 572 will introduce := assignment operator that deliberately leaks out of comprehensions and generator expressions! It’s motivated by essentially 2 use cases: capturing a “witness” from early-terminating functions like any() and all():

if any((comment := line).startswith('#') for line in lines):
    print("First comment:", comment)
else:
    print("There are no comments")

and updating mutable state:

total = 0
partial_sums = [total := total + v for v in values]

See Appendix B for exact scoping. The variable is assigned in closest surrounding def or lambda, unless that function declares it nonlocal or global.


回答 2

是的,分配就在那里发生,就像for循环一样。没有新的作用域被创建。

这绝对是预期的行为:在每个循环中,该值都绑定到您指定的名称。例如,

>>> x=0
>>> a=[1,54,4,2,32,234,5234,]
>>> [x for x in a if x>32]
[54, 234, 5234]
>>> x
5234

一旦意识到这一点,似乎很容易避免:不要在理解范围内使用现有名称作为变量。

Yes, assignment occurs there, just like it would in a for loop. No new scope is being created.

This is definitely the expected behavior: on each cycle, the value is bound to the name you specify. For instance,

>>> x=0
>>> a=[1,54,4,2,32,234,5234,]
>>> [x for x in a if x>32]
[54, 234, 5234]
>>> x
5234

Once that’s recognized, it seems easy enough to avoid: don’t use existing names for the variables within comprehensions.


回答 3

有趣的是,这不会影响字典或设置理解力。

>>> [x for x in range(1, 10)]
[1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> x
9
>>> {x for x in range(1, 5)}
set([1, 2, 3, 4])
>>> x
9
>>> {x:x for x in range(1, 100)}
{1: 1, 2: 2, 3: 3, 4: 4, 5: 5, 6: 6, 7: 7, 8: 8, 9: 9, 10: 10, 11: 11, 12: 12, 13: 13, 14: 14, 15: 15, 16: 16, 17: 17, 18: 18, 19: 19, 20: 20, 21: 21, 22: 22, 23: 23, 24: 24, 25: 25, 26: 26, 27: 27, 28: 28, 29: 29, 30: 30, 31: 31, 32: 32, 33: 33, 34: 34, 35: 35, 36: 36, 37: 37, 38: 38, 39: 39, 40: 40, 41: 41, 42: 42, 43: 43, 44: 44, 45: 45, 46: 46, 47: 47, 48: 48, 49: 49, 50: 50, 51: 51, 52: 52, 53: 53, 54: 54, 55: 55, 56: 56, 57: 57, 58: 58, 59: 59, 60: 60, 61: 61, 62: 62, 63: 63, 64: 64, 65: 65, 66: 66, 67: 67, 68: 68, 69: 69, 70: 70, 71: 71, 72: 72, 73: 73, 74: 74, 75: 75, 76: 76, 77: 77, 78: 78, 79: 79, 80: 80, 81: 81, 82: 82, 83: 83, 84: 84, 85: 85, 86: 86, 87: 87, 88: 88, 89: 89, 90: 90, 91: 91, 92: 92, 93: 93, 94: 94, 95: 95, 96: 96, 97: 97, 98: 98, 99: 99}
>>> x
9

但是,如上所述,已将其固定为3。

Interestingly this doesn’t affect dictionary or set comprehensions.

>>> [x for x in range(1, 10)]
[1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> x
9
>>> {x for x in range(1, 5)}
set([1, 2, 3, 4])
>>> x
9
>>> {x:x for x in range(1, 100)}
{1: 1, 2: 2, 3: 3, 4: 4, 5: 5, 6: 6, 7: 7, 8: 8, 9: 9, 10: 10, 11: 11, 12: 12, 13: 13, 14: 14, 15: 15, 16: 16, 17: 17, 18: 18, 19: 19, 20: 20, 21: 21, 22: 22, 23: 23, 24: 24, 25: 25, 26: 26, 27: 27, 28: 28, 29: 29, 30: 30, 31: 31, 32: 32, 33: 33, 34: 34, 35: 35, 36: 36, 37: 37, 38: 38, 39: 39, 40: 40, 41: 41, 42: 42, 43: 43, 44: 44, 45: 45, 46: 46, 47: 47, 48: 48, 49: 49, 50: 50, 51: 51, 52: 52, 53: 53, 54: 54, 55: 55, 56: 56, 57: 57, 58: 58, 59: 59, 60: 60, 61: 61, 62: 62, 63: 63, 64: 64, 65: 65, 66: 66, 67: 67, 68: 68, 69: 69, 70: 70, 71: 71, 72: 72, 73: 73, 74: 74, 75: 75, 76: 76, 77: 77, 78: 78, 79: 79, 80: 80, 81: 81, 82: 82, 83: 83, 84: 84, 85: 85, 86: 86, 87: 87, 88: 88, 89: 89, 90: 90, 91: 91, 92: 92, 93: 93, 94: 94, 95: 95, 96: 96, 97: 97, 98: 98, 99: 99}
>>> x
9

However it has been fixed in 3 as noted above.


回答 4

不适用于python 2.6的一些解决方法

# python
Python 2.6.6 (r266:84292, Aug  9 2016, 06:11:56)
Type "help", "copyright", "credits" or "license" for more information.
>>> x=0
>>> a=list(x for x in xrange(9))
>>> x
0
>>> a=[x for x in xrange(9)]
>>> x
8

some workaround, for python 2.6, when this behaviour is not desirable

# python
Python 2.6.6 (r266:84292, Aug  9 2016, 06:11:56)
Type "help", "copyright", "credits" or "license" for more information.
>>> x=0
>>> a=list(x for x in xrange(9))
>>> x
0
>>> a=[x for x in xrange(9)]
>>> x
8

回答 5

在python3中,当在列表理解中时,变量的作用域超出范围后并不会发生变化,但是当我们使用简单的for循环时,变量会被重新分配到作用域之外。

i = 1 print(i)print([i in range(5)])print(i)i的值将仅保留1。

现在,仅使用for循环即可重新分配i的值。

In python3 while in list comprehension the variable is not getting change after it’s scope over but when we use simple for-loop the variable is getting reassigned out of scope.

i = 1 print(i) print([i in range(5)]) print(i) Value of i will remain 1 only.

Now just use simply for loop the value of i will be reassigned.