词汇闭包如何工作?

问题:词汇闭包如何工作?

当我研究Java代码中的词法闭包问题时,我遇到了Python中的这个问题:

flist = []

for i in xrange(3):
    def func(x): return x * i
    flist.append(func)

for f in flist:
    print f(2)

请注意,此示例应避免使用lambda。它打印“ 4 4 4”,这是令人惊讶的。我希望“ 0 2 4”。

等效的Perl代码可以正确执行此操作:

my @flist = ();

foreach my $i (0 .. 2)
{
    push(@flist, sub {$i * $_[0]});
}

foreach my $f (@flist)
{
    print $f->(2), "\n";
}

打印“ 0 2 4”。

你能解释一下区别吗?


更新:

这个问题是不是i是全球性的。这显示相同的行为:

flist = []

def outer():
    for i in xrange(3):
        def inner(x): return x * i
        flist.append(inner)

outer()
#~ print i   # commented because it causes an error

for f in flist:
    print f(2)

如注释行所示,i在这一点上未知。仍然打印“ 4 4 4”。

While I was investigating a problem I had with lexical closures in Javascript code, I came along this problem in Python:

flist = []

for i in xrange(3):
    def func(x): return x * i
    flist.append(func)

for f in flist:
    print f(2)

Note that this example mindfully avoids lambda. It prints “4 4 4”, which is surprising. I’d expect “0 2 4”.

This equivalent Perl code does it right:

my @flist = ();

foreach my $i (0 .. 2)
{
    push(@flist, sub {$i * $_[0]});
}

foreach my $f (@flist)
{
    print $f->(2), "\n";
}

“0 2 4” is printed.

Can you please explain the difference ?


Update:

The problem is not with i being global. This displays the same behavior:

flist = []

def outer():
    for i in xrange(3):
        def inner(x): return x * i
        flist.append(inner)

outer()
#~ print i   # commented because it causes an error

for f in flist:
    print f(2)

As the commented line shows, i is unknown at that point. Still, it prints “4 4 4”.


回答 0

实际上,Python的行为符合定义。创建了三个单独的函数,但是每个函数封闭了定义它们的环境 -在这种情况下,是全局环境(如果将循环放在另一个函数内部,则为外部函数的环境)。不过,这恰恰是问题所在-在这种环境下,i发生了变异,并且所有闭包都引用相同的i

这是我能想到的最佳解决方案-创建一个函数创建器,然后调用。这将为所创建的每个函数强制使用不同的环境,每个函数具有不同的i

flist = []

for i in xrange(3):
    def funcC(j):
        def func(x): return x * j
        return func
    flist.append(funcC(i))

for f in flist:
    print f(2)

当您混合副作用和功能编程时,就会发生这种情况。

Python is actually behaving as defined. Three separate functions are created, but they each have the closure of the environment they’re defined in – in this case, the global environment (or the outer function’s environment if the loop is placed inside another function). This is exactly the problem, though – in this environment, i is mutated, and the closures all refer to the same i.

Here is the best solution I can come up with – create a function creater and invoke that instead. This will force different environments for each of the functions created, with a different i in each one.

flist = []

for i in xrange(3):
    def funcC(j):
        def func(x): return x * j
        return func
    flist.append(funcC(i))

for f in flist:
    print f(2)

This is what happens when you mix side effects and functional programming.


回答 1

循环中定义的函数在i其值更改时继续访问相同的变量。在循环的最后,所有函数都指向同一个变量,该变量在循环中保存着最后一个值:效果就是示例中报告的结果。

为了评估i和使用其值,一种常见的模式是将其设置为参数默认值:在def执行语句时评估参数默认值,因此冻结了循环变量的值。

预期的工作如下:

flist = []

for i in xrange(3):
    def func(x, i=i): # the *value* of i is copied in func() environment
        return x * i
    flist.append(func)

for f in flist:
    print f(2)

The functions defined in the loop keep accessing the same variable i while its value changes. At the end of the loop, all the functions point to the same variable, which is holding the last value in the loop: the effect is what reported in the example.

In order to evaluate i and use its value, a common pattern is to set it as a parameter default: parameter defaults are evaluated when the def statement is executed, and thus the value of the loop variable is frozen.

The following works as expected:

flist = []

for i in xrange(3):
    def func(x, i=i): # the *value* of i is copied in func() environment
        return x * i
    flist.append(func)

for f in flist:
    print f(2)

回答 2

使用functools库的方法如下(提出问题时我不确定该库是否可用)。

from functools import partial

flist = []

def func(i, x): return x * i

for i in xrange(3):
    flist.append(partial(func, i))

for f in flist:
    print f(2)

输出0 2 4,如预期的那样。

Here’s how you do it using the functools library (which I’m not sure was available at the time the question was posed).

from functools import partial

flist = []

def func(i, x): return x * i

for i in xrange(3):
    flist.append(partial(func, i))

for f in flist:
    print f(2)

Outputs 0 2 4, as expected.


回答 3

看这个:

for f in flist:
    print f.func_closure


(<cell at 0x00C980B0: int object at 0x009864B4>,)
(<cell at 0x00C980B0: int object at 0x009864B4>,)
(<cell at 0x00C980B0: int object at 0x009864B4>,)

这意味着它们都指向同一个i变量实例,循环结束后其值将为2。

可读的解决方案:

for i in xrange(3):
        def ffunc(i):
            def func(x): return x * i
            return func
        flist.append(ffunc(i))

look at this:

for f in flist:
    print f.func_closure


(<cell at 0x00C980B0: int object at 0x009864B4>,)
(<cell at 0x00C980B0: int object at 0x009864B4>,)
(<cell at 0x00C980B0: int object at 0x009864B4>,)

It means they all point to the same i variable instance, which will have a value of 2 once the loop is over.

A readable solution:

for i in xrange(3):
        def ffunc(i):
            def func(x): return x * i
            return func
        flist.append(ffunc(i))

回答 4

发生的情况是捕获了变量i,并且函数正在返回它在被调用时绑定的值。在函数式语言中,这种情况永远不会出现,因为我不会反弹。但是,对于python以及您在lisp中所看到的,这不再是事实。

您的方案示例的不同之处在于do循环的语义。Scheme每次都在循环中有效地创建了一个新的i变量,而不是像其他语言一样重用现有的i绑定。如果您使用在循环外部创建的另一个变量并对它进行突变,则您会在方案中看到相同的行为。尝试将循环替换为:

(let ((ii 1)) (
  (do ((i 1 (+ 1 i)))
      ((>= i 4))
    (set! flist 
      (cons (lambda (x) (* ii x)) flist))
    (set! ii i))
))

看看这里为一些这方面的进一步讨论。

[描述]可能更好的描述方法是将do循环视为执行以下步骤的宏:

  1. 定义一个带单个参数(i)的lambda,其主体由循环的主体定义,
  2. 以i的适当值作为参数立即调用该lambda。

即。等效于以下python:

flist = []

def loop_body(i):      # extract body of the for loop to function
    def func(x): return x*i
    flist.append(func)

map(loop_body, xrange(3))  # for i in xrange(3): body

i不再是父作用域中的那个,而是它自己作用域中的一个全新变量(即lambda的参数),因此您可以观察到行为。Python没有这个隐式的新作用域,因此for循环的主体仅共享i变量。

What is happening is that the variable i is captured, and the functions are returning the value it is bound to at the time it is called. In functional languages this kind of situation never arises, as i wouldn’t be rebound. However with python, and also as you’ve seen with lisp, this is no longer true.

The difference with your scheme example is to do with the semantics of the do loop. Scheme is effectively creating a new i variable each time through the loop, rather than reusing an existing i binding as with the other languages. If you use a different variable created external to the loop and mutate it, you’ll see the same behaviour in scheme. Try replacing your loop with:

(let ((ii 1)) (
  (do ((i 1 (+ 1 i)))
      ((>= i 4))
    (set! flist 
      (cons (lambda (x) (* ii x)) flist))
    (set! ii i))
))

Take a look here for some further discussion of this.

[Edit] Possibly a better way to describe it is to think of the do loop as a macro which performs the following steps:

  1. Define a lambda taking a single parameter (i), with a body defined by the body of the loop,
  2. An immediate call of that lambda with appropriate values of i as its parameter.

ie. the equivalent to the below python:

flist = []

def loop_body(i):      # extract body of the for loop to function
    def func(x): return x*i
    flist.append(func)

map(loop_body, xrange(3))  # for i in xrange(3): body

The i is no longer the one from the parent scope but a brand new variable in its own scope (ie. the parameter to the lambda) and so you get the behaviour you observe. Python doesn’t have this implicit new scope, so the body of the for loop just shares the i variable.


回答 5

我仍然不完全相信为什么在某些语言中这会以一种方式在另一种方式下起作用。在Common Lisp中,就像Python:

(defvar *flist* '())

(dotimes (i 3 t)
  (setf *flist* 
    (cons (lambda (x) (* x i)) *flist*)))

(dolist (f *flist*)  
  (format t "~a~%" (funcall f 2)))

打印“ 6 6 6”(请注意,这里的列表是从1到3,并且是反向构建的。)。在Scheme中,它的作用类似于在Perl中:

(define flist '())

(do ((i 1 (+ 1 i)))
    ((>= i 4))
  (set! flist 
    (cons (lambda (x) (* i x)) flist)))

(map 
  (lambda (f)
    (printf "~a~%" (f 2)))
  flist)

打印“ 6 4 2”

正如我已经提到的那样,JavaScript处于Python / CL阵营中。似乎这里有一个实施决策,即不同的语言采用不同的方式进行处理。我很想知道到底是什么决定。

I’m still not entirely convinced why in some languages this works one way, and in some another way. In Common Lisp it’s like Python:

(defvar *flist* '())

(dotimes (i 3 t)
  (setf *flist* 
    (cons (lambda (x) (* x i)) *flist*)))

(dolist (f *flist*)  
  (format t "~a~%" (funcall f 2)))

Prints “6 6 6″ (note that here the list is from 1 to 3, and built in reverse”). While in Scheme it works like in Perl:

(define flist '())

(do ((i 1 (+ 1 i)))
    ((>= i 4))
  (set! flist 
    (cons (lambda (x) (* i x)) flist)))

(map 
  (lambda (f)
    (printf "~a~%" (f 2)))
  flist)

Prints “6 4 2”

And as I’ve mentioned already, Javascript is in the Python/CL camp. It appears there is an implementation decision here, which different languages approach in distinct ways. I would love to understand what is the decision, exactly.


回答 6

问题在于所有本地函数都绑定到相同的环境,因此绑定到相同的i变量。解决方案(解决方法)是为每个函数(或lambda)创建单独的环境(堆栈框架):

t = [ (lambda x: lambda y : x*y)(x) for x in range(5)]

>>> t[1](2)
2
>>> t[2](2)
4

The problem is that all of the local functions bind to the same environment and thus to the same i variable. The solution (workaround) is to create separate environments (stack frames) for each function (or lambda):

t = [ (lambda x: lambda y : x*y)(x) for x in range(5)]

>>> t[1](2)
2
>>> t[2](2)
4

回答 7

该变量i是全局变量,每次f调用该函数时其值为2 。

我倾向于实现以下行为:

>>> class f:
...  def __init__(self, multiplier): self.multiplier = multiplier
...  def __call__(self, multiplicand): return self.multiplier*multiplicand
... 
>>> flist = [f(i) for i in range(3)]
>>> [g(2) for g in flist]
[0, 2, 4]

对您的更新的响应:导致此问题的原因不是i 本身的全局性,而是事实是它是来自封闭范围的变量,在调用f时,该变量具有固定值。在第二个示例中,的值i取自kkk函数的范围,当您在上调用函数时,没有任何改变flist

The variable i is a global, whose value is 2 at each time the function f is called.

I would be inclined to implement the behavior you’re after as follows:

>>> class f:
...  def __init__(self, multiplier): self.multiplier = multiplier
...  def __call__(self, multiplicand): return self.multiplier*multiplicand
... 
>>> flist = [f(i) for i in range(3)]
>>> [g(2) for g in flist]
[0, 2, 4]

Response to your update: It’s not the globalness of i per se which is causing this behavior, it’s the fact that it’s a variable from an enclosing scope which has a fixed value over the times when f is called. In your second example, the value of i is taken from the scope of the kkk function, and nothing is changing that when you call the functions on flist.


回答 8

已经解释了该行为背后的原因,并发布了多种解决方案,但是我认为这是最pythonic的(请记住,Python中的所有对象都是对象!):

flist = []

for i in xrange(3):
    def func(x): return x * func.i
    func.i=i
    flist.append(func)

for f in flist:
    print f(2)

Claudiu的答案很不错,使用了一个函数生成器,但是老实说piro的答案是hack,因为它使我成为具有默认值的“隐藏”参数(可以正常工作,但不是“ pythonic”) 。

The reasoning behind the behavior has already been explained, and multiple solutions have been posted, but I think this is the most pythonic (remember, everything in Python is an object!):

flist = []

for i in xrange(3):
    def func(x): return x * func.i
    func.i=i
    flist.append(func)

for f in flist:
    print f(2)

Claudiu’s answer is pretty good, using a function generator, but piro’s answer is a hack, to be honest, as it’s making i into a “hidden” argument with a default value (it’ll work fine, but it’s not “pythonic”).