python生成器“发送”功能的目的?

问题:python生成器“发送”功能的目的?

谁能给我一个为什么与Python生成器函数关联的“发送”函数存在的示例?我完全了解yield函数。但是,发送功能使我感到困惑。有关此方法的文档非常复杂:

generator.send(value)

恢复执行并将值“发送”到生成器函数。value参数成为当前yield表达式的结果。send()方法返回生成器产生的下一个值,如果生成器退出而不产生另一个值,则返回StopIteration。

那是什么意思?我以为价值是功能的输入?短语“ send()方法返回生成器产生的下一个值”似乎也是yield函数的确切目的;yield返回生成器产生的下一个值…

有人可以给我一个使用send的生成器的示例,该生成完成了某些无法完成的工作吗?

Can someone give me an example of why the “send” function associated with Python generator function exists? I fully understand the yield function. However, the send function is confusing to me. The documentation on this method is convoluted:

generator.send(value)

Resumes the execution and “sends” a value into the generator function. The value argument becomes the result of the current yield expression. The send() method returns the next value yielded by the generator, or raises StopIteration if the generator exits without yielding another value.

What does that mean? I thought value was the input to the function? The phrase “The send() method returns the next value yielded by the generator” seems to be also the exact purpose of the yield function; yield returns the next value yielded by the generator…

Can someone give me an example of a generator utilizing send that accomplishes something yield cannot?


回答 0

它用于将值发送到刚产生的生成器中。这是一个人为的(无用的)解释性示例:

>>> def double_inputs():
...     while True:
...         x = yield
...         yield x * 2
...
>>> gen = double_inputs()
>>> next(gen)       # run up to the first yield
>>> gen.send(10)    # goes into 'x' variable
20
>>> next(gen)       # run up to the next yield
>>> gen.send(6)     # goes into 'x' again
12
>>> next(gen)       # run up to the next yield
>>> gen.send(94.3)  # goes into 'x' again
188.5999999999999

您不能只通过做到这一点yield

至于为什么有用,我见过的最好的用例之一是Twisted的@defer.inlineCallbacks。从本质上讲,它允许您编写如下函数:

@defer.inlineCallbacks
def doStuff():
    result = yield takesTwoSeconds()
    nextResult = yield takesTenSeconds(result * 10)
    defer.returnValue(nextResult / 10)

发生的情况是takesTwoSeconds()返回a Deferred,这是一个保证将在以后计算的值。Twisted可以在另一个线程中运行计算。计算完成后,将其传递给延迟的对象,然后将值发送回doStuff()函数。因此doStuff(),除了可能执行各种计算和回调等操作外,最终可能看起来或多或少像普通的过程函数。此功能之前的替代方法是:

def doStuff():
    returnDeferred = defer.Deferred()
    def gotNextResult(nextResult):
        returnDeferred.callback(nextResult / 10)
    def gotResult(result):
        takesTenSeconds(result * 10).addCallback(gotNextResult)
    takesTwoSeconds().addCallback(gotResult)
    return returnDeferred

这更加令人费解和笨拙。

It’s used to send values into a generator that just yielded. Here is an artificial (non-useful) explanatory example:

>>> def double_inputs():
...     while True:
...         x = yield
...         yield x * 2
...
>>> gen = double_inputs()
>>> next(gen)       # run up to the first yield
>>> gen.send(10)    # goes into 'x' variable
20
>>> next(gen)       # run up to the next yield
>>> gen.send(6)     # goes into 'x' again
12
>>> next(gen)       # run up to the next yield
>>> gen.send(94.3)  # goes into 'x' again
188.5999999999999

You can’t do this just with yield.

As to why it’s useful, one of the best use cases I’ve seen is Twisted’s @defer.inlineCallbacks. Essentially it allows you to write a function like this:

@defer.inlineCallbacks
def doStuff():
    result = yield takesTwoSeconds()
    nextResult = yield takesTenSeconds(result * 10)
    defer.returnValue(nextResult / 10)

What happens is that takesTwoSeconds() returns a Deferred, which is a value promising a value will be computed later. Twisted can run the computation in another thread. When the computation is done, it passes it into the deferred, and the value then gets sent back to the doStuff() function. Thus the doStuff() can end up looking more or less like a normal procedural function, except it can be doing all sorts of computations & callbacks etc. The alternative before this functionality would be to do something like:

def doStuff():
    returnDeferred = defer.Deferred()
    def gotNextResult(nextResult):
        returnDeferred.callback(nextResult / 10)
    def gotResult(result):
        takesTenSeconds(result * 10).addCallback(gotNextResult)
    takesTwoSeconds().addCallback(gotResult)
    return returnDeferred

It’s a lot more convoluted and unwieldy.


回答 1

该功能是编写协程

def coroutine():
    for i in range(1, 10):
        print("From generator {}".format((yield i)))
c = coroutine()
c.send(None)
try:
    while True:
        print("From user {}".format(c.send(1)))
except StopIteration: pass

版画

From generator 1
From user 2
From generator 1
From user 3
From generator 1
From user 4
...

看看控件是如何来回传递的?那些是协程。它们可以用于各种很酷的东西,例如异步IO和类似的东西。

这样想吧,没有生成器,只有一条路

==========       yield      ========
Generator |   ------------> | User |
==========                  ========

但是随着发送,它变成了一条双向路

==========       yield       ========
Generator |   ------------>  | User |
==========    <------------  ========
                  send

这开启了大门,用户自定义生成行为上飞和生成器响应用户。

This function is to write coroutines

def coroutine():
    for i in range(1, 10):
        print("From generator {}".format((yield i)))
c = coroutine()
c.send(None)
try:
    while True:
        print("From user {}".format(c.send(1)))
except StopIteration: pass

prints

From generator 1
From user 2
From generator 1
From user 3
From generator 1
From user 4
...

See how the control is being passed back and forth? Those are coroutines. They can be used for all kinds of cool things like asynch IO and similar.

Think of it like this, with a generator and no send, it’s a one way street

==========       yield      ========
Generator |   ------------> | User |
==========                  ========

But with send, it becomes a two way street

==========       yield       ========
Generator |   ------------>  | User |
==========    <------------  ========
                  send

Which opens up the door to the user customizing the generators behavior on the fly and the generator responding to the user.


回答 2

这可能会帮助某人。这是不受发送功能影响的生成器。它在实例化时接受number参数,并且不受send的影响:

>>> def double_number(number):
...     while True:
...         number *=2 
...         yield number
... 
>>> c = double_number(4)
>>> c.send(None)
8
>>> c.next()
16
>>> c.next()
32
>>> c.send(8)
64
>>> c.send(8)
128
>>> c.send(8)
256

现在,这是使用send执行相同类型的函数的方式,因此在每次迭代中都可以更改number的值:

def double_number(number):
    while True:
        number *= 2
        number = yield number

如下所示,您可以看到发送数字新值会改变结果:

>>> def double_number(number):
...     while True:
...         number *= 2
...         number = yield number
...
>>> c = double_number(4)
>>> 
>>> c.send(None)
8
>>> c.send(5) #10
10
>>> c.send(1500) #3000
3000
>>> c.send(3) #6
6

您也可以将其放在for循环中,如下所示:

for x in range(10):
    n = c.send(n)
    print n

要获得更多帮助,请查看此出色的教程

This may help someone. Here is a generator that is unaffected by send function. It takes in the number parameter on instantiation and is unaffected by send:

>>> def double_number(number):
...     while True:
...         number *=2 
...         yield number
... 
>>> c = double_number(4)
>>> c.send(None)
8
>>> c.next()
16
>>> c.next()
32
>>> c.send(8)
64
>>> c.send(8)
128
>>> c.send(8)
256

Now here is how you would do the same type of function using send, so on each iteration you can change the value of number:

def double_number(number):
    while True:
        number *= 2
        number = yield number

Here is what that looks like, as you can see sending a new value for number changes the outcome:

>>> def double_number(number):
...     while True:
...         number *= 2
...         number = yield number
...
>>> c = double_number(4)
>>> 
>>> c.send(None)
8
>>> c.send(5) #10
10
>>> c.send(1500) #3000
3000
>>> c.send(3) #6
6

You can also put this in a for loop as such:

for x in range(10):
    n = c.send(n)
    print n

For more help check out this great tutorial.


回答 3

一些使用generator和 send()

具有send()allow的生成器:

  • 记住执行的内部状态
    • 我们正在走哪一步
    • 我们数据的当前状态是什么
  • 返回值序列
  • 接收输入序列

以下是一些用例:

观看尝试遵循食谱的尝试

让我们有一个菜谱,它期望以某种顺序预定义一组输入。

我们可能:

  • watched_attempt从配方创建实例
  • 让它得到一些输入
  • 每个输入返回有关锅中当前内容的信息
  • 每次输入检查时,请确保输入是预期的输入(如果不是,则输入失败)

    def recipe():
        pot = []
        action = yield pot
        assert action == ("add", "water")
        pot.append(action[1])
    
        action = yield pot
        assert action == ("add", "salt")
        pot.append(action[1])
    
        action = yield pot
        assert action == ("boil", "water")
    
        action = yield pot
        assert action == ("add", "pasta")
        pot.append(action[1])
    
        action = yield pot
        assert action == ("decant", "water")
        pot.remove("water")
    
        action = yield pot
        assert action == ("serve")
        pot = []
        yield pot

要使用它,首先创建watched_attempt实例:

>>> watched_attempt = recipe()                                                                         
>>> watched_attempt.next()                                                                                     
[]                                                                                                     

调用.next()是开始执行生成器所必需的。

返回值显示,我们的底池当前为空。

现在,按照食谱的期望执行一些操作:

>>> watched_attempt.send(("add", "water"))                                                                     
['water']                                                                                              
>>> watched_attempt.send(("add", "salt"))                                                                      
['water', 'salt']                                                                                      
>>> watched_attempt.send(("boil", "water"))                                                                    
['water', 'salt']                                                                                      
>>> watched_attempt.send(("add", "pasta"))                                                                     
['water', 'salt', 'pasta']                                                                             
>>> watched_attempt.send(("decant", "water"))                                                                  
['salt', 'pasta']                                                                                      
>>> watched_attempt.send(("serve"))                                                                            
[] 

如我们所见,锅终于空了。

万一有人不遵循食谱,那它就会失败(观看尝试煮东西的理想结果-只是知道我们在给出指示时没有给予足够的重视。

>>> watched_attempt = running.recipe()                                                                         
>>> watched_attempt.next()                                                                                     
[]                                                                                                     
>>> watched_attempt.send(("add", "water"))                                                                     
['water']                                                                                              
>>> watched_attempt.send(("add", "pasta")) 

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-21-facdf014fe8e> in <module>()
----> 1 watched_attempt.send(("add", "pasta"))

/home/javl/sandbox/stack/send/running.py in recipe()
     29
     30     action = yield pot
---> 31     assert action == ("add", "salt")
     32     pot.append(action[1])
     33

AssertionError:

注意,

  • 有预期步骤的线性顺序
  • 步骤可能有所不同(有些正在移除,有些正在添加到锅中)
  • 我们设法通过函数/生成器来完成所有这些操作-无需使用复杂的类或类似的结构。

运行总计

我们可以使用生成器来跟踪发送给它的值的运行总计。

每当我们添加一个数字时,都会返回输入计数和总和(对于将先前的输入发送到该时刻有效)。

from collections import namedtuple

RunningTotal = namedtuple("RunningTotal", ["n", "total"])


def runningtotals(n=0, total=0):
    while True:
        delta = yield RunningTotal(n, total)
        if delta:
            n += 1
            total += delta


if __name__ == "__main__":
    nums = [9, 8, None, 3, 4, 2, 1]

    bookeeper = runningtotals()
    print bookeeper.next()
    for num in nums:
        print num, bookeeper.send(num)

输出如下:

RunningTotal(n=0, total=0)
9 RunningTotal(n=1, total=9)
8 RunningTotal(n=2, total=17)
None RunningTotal(n=2, total=17)
3 RunningTotal(n=3, total=20)
4 RunningTotal(n=4, total=24)
2 RunningTotal(n=5, total=26)
1 RunningTotal(n=6, total=27)

Some use cases for using generator and send()

Generators with send() allow:

  • remembering internal state of the execution
    • what step we are at
    • what is current status of our data
  • returning sequence of values
  • receiving sequence of inputs

Here are some use cases:

Watched attempt to follow a recipe

Let us have a recipe, which expects predefined set of inputs in some order.

We may:

  • create a watched_attempt instance from the recipe
  • let it get some inputs
  • with each input return information about what is currently in the pot
  • with each input check, that the input is the expected one (and fail if it is not)

    def recipe():
        pot = []
        action = yield pot
        assert action == ("add", "water")
        pot.append(action[1])
    
        action = yield pot
        assert action == ("add", "salt")
        pot.append(action[1])
    
        action = yield pot
        assert action == ("boil", "water")
    
        action = yield pot
        assert action == ("add", "pasta")
        pot.append(action[1])
    
        action = yield pot
        assert action == ("decant", "water")
        pot.remove("water")
    
        action = yield pot
        assert action == ("serve")
        pot = []
        yield pot
    

To use it, first create the watched_attempt instance:

>>> watched_attempt = recipe()                                                                         
>>> watched_attempt.next()                                                                                     
[]                                                                                                     

The call to .next() is necessary to start execution of the generator.

Returned value shows, our pot is currently empty.

Now do few actions following what the recipe expects:

>>> watched_attempt.send(("add", "water"))                                                                     
['water']                                                                                              
>>> watched_attempt.send(("add", "salt"))                                                                      
['water', 'salt']                                                                                      
>>> watched_attempt.send(("boil", "water"))                                                                    
['water', 'salt']                                                                                      
>>> watched_attempt.send(("add", "pasta"))                                                                     
['water', 'salt', 'pasta']                                                                             
>>> watched_attempt.send(("decant", "water"))                                                                  
['salt', 'pasta']                                                                                      
>>> watched_attempt.send(("serve"))                                                                            
[] 

As we see, the pot is finally empty.

In case, one would not follow the recipe, it would fail (what could be desired outcome of watched attempt to cook something – just learning we did not pay enough attention when given instructions.

>>> watched_attempt = running.recipe()                                                                         
>>> watched_attempt.next()                                                                                     
[]                                                                                                     
>>> watched_attempt.send(("add", "water"))                                                                     
['water']                                                                                              
>>> watched_attempt.send(("add", "pasta")) 

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-21-facdf014fe8e> in <module>()
----> 1 watched_attempt.send(("add", "pasta"))

/home/javl/sandbox/stack/send/running.py in recipe()
     29
     30     action = yield pot
---> 31     assert action == ("add", "salt")
     32     pot.append(action[1])
     33

AssertionError:

Notice, that:

  • there is linear sequence of expected steps
  • the steps may differ (some are removing, some are adding to the pot)
  • we manage to do all that by a functions/generator – no need to use complex class or similar strutures.

Running totals

We may use the generator to keep track of running total of values sent to it.

Any time we add a number, count of inputs and total sum is returned (valid for the moment previous input was send into it).

from collections import namedtuple

RunningTotal = namedtuple("RunningTotal", ["n", "total"])


def runningtotals(n=0, total=0):
    while True:
        delta = yield RunningTotal(n, total)
        if delta:
            n += 1
            total += delta


if __name__ == "__main__":
    nums = [9, 8, None, 3, 4, 2, 1]

    bookeeper = runningtotals()
    print bookeeper.next()
    for num in nums:
        print num, bookeeper.send(num)

The output would look like:

RunningTotal(n=0, total=0)
9 RunningTotal(n=1, total=9)
8 RunningTotal(n=2, total=17)
None RunningTotal(n=2, total=17)
3 RunningTotal(n=3, total=20)
4 RunningTotal(n=4, total=24)
2 RunningTotal(n=5, total=26)
1 RunningTotal(n=6, total=27)

回答 4

send()方法控制yield表达式左侧的值。

为了了解yield的不同以及它保持什么值,我们首先快速刷新评估python代码的顺序。

第6.15节评估顺序

Python从左到右计算表达式。请注意,在评估分配时,右侧的评估先于左侧。

因此,a = b首先评估右侧的表达式。

如下所示,a[p('left')] = p('right')首先评估了右侧。

>>> def p(side):
...     print(side)
...     return 0
... 
>>> a[p('left')] = p('right')
right
left
>>> 
>>> 
>>> [p('left'), p('right')]
left
right
[0, 0]

yield的作用是什么?yield,暂停函数的执行并返回到调用方,并在暂停之前停止的地方恢复执行。

究竟在哪里暂停执行?您可能已经猜到了…… 执行被挂在yield表达式的左侧和右侧之间。因此new_val = yield old_val,执行将在=符号处停止,右侧的值(在挂起之前,并且也是返回给调用方的值)可能与左侧的值(在恢复后分配的值)有所不同执行)。

yield 产生2个值,一个在右边,另一个在左边。

如何控制收益率表达式左侧的值?通过该.send()方法。

6.2.9。收益表达

恢复后的yield表达式的值取决于恢复执行的方法。如果__next__()使用if (通常通过for或next()内置),则结果为None。否则,如果send()使用,则结果将是传递给该方法的值。

The send() method controls what the value to the left of the yield expression will be.

To understand how yield differs and what value it holds, lets first quickly refresh on the order python code is evaluated.

Section 6.15 Evaluation order

Python evaluates expressions from left to right. Notice that while evaluating an assignment, the right-hand side is evaluated before the left-hand side.

So an expression a = b the right hand side is evaluated first.

As the following demonstrates that a[p('left')] = p('right') the right hand side is evaluated first.

>>> def p(side):
...     print(side)
...     return 0
... 
>>> a[p('left')] = p('right')
right
left
>>> 
>>> 
>>> [p('left'), p('right')]
left
right
[0, 0]

What does yield do?, yield, suspends execution of the function and returns to the caller, and resumes execution at the same place it left off prior to suspending.

Where exactly is execution suspended? You might have guessed it already… the execution is suspended between the right and left side of the yield expression. So new_val = yield old_val the execution is halted at the = sign, and the value on the right (which is before suspending, and is also the value returned to the caller) may be something different then the value on the left (which is the value being assigned after resuming execution).

yield yields 2 values, one to the right and another to the left.

How do you control the value to the left hand side of the yield expression? via the .send() method.

6.2.9. Yield expressions

The value of the yield expression after resuming depends on the method which resumed the execution. If __next__() is used (typically via either a for or the next() builtin) then the result is None. Otherwise, if send() is used, then the result will be the value passed in to that method.


回答 5

send方法实现协程

如果您还没有遇到过协程,那么它们很难缠住您的头,因为它们会改变程序的运行方式。您可以阅读良好的教程以了解更多详细信息。

The send method implements coroutines.

If you haven’t encountered Coroutines they are tricky to wrap your head around because they change the way a program flows. You can read a good tutorial for more details.


回答 6

“屈服”一词有两个含义:生产某种东西(例如,生产玉米),以及停下来让某人/其他东西继续(例如,汽车向行人屈服)。两种定义都适用于Python的yield关键字;生成器函数之所以与众不同,是与常规函数不同的是,可以将值“返回”给调用方,而只是暂停而不终止生成器函数。

最简单的将生成器想象为双向管道的一端,该管道的一端为“左”,另一端为“右”。该管道是在生成器本身与生成器函数的主体之间发送值的媒介。管道的每一端都有两个操作:push,它发送一个值并阻塞直到管道的另一端提取该值,并且什么都不返回;和pull,它会阻塞直到管道的另一端推送一个值,然后返回推送的值。在运行时,执行在管道任一侧的上下文之间来回跳动-每一侧都运行,直到将值发送到另一侧为止,此时它停止,让另一侧运行,并等待返回,这时另一边停止并继续。换句话说,管道的每个端部从接收值的那一刻到发送值的那一刻都在运行。

管道在功能上是对称的,但是-按照惯例,我在这个答案中定义-左端仅在生成器函数的主体内可用,并且可以通过yield关键字访问,而右端生成器,并且可以通过生成器的send功能。作为单数接口到它们各自的管道的端部,yieldsend从它们的管的端部,它们既各推拉值/:完成双重任务yield推向右和拉动而向左send则相反。这种双重职责是围绕诸如之类的语句语义混乱的症结所在x = yield y。打破yieldsend分为两个明确的推/拉措施将使它们的语义更加清晰:

  1. 假设g是生成器。g.send通过管道的右端向左推一个值。
  2. g暂停的上下文中执行,允许生成器函数的主体运行。
  3. 推入的值g.send向左拉,yield并在管道的左端接收。在中x = yield yx分配给上拉值。
  4. 在生成器函数的主体内继续执行,直到yield到达包含下一行的行为止。
  5. yield通过管道的左端向右推动一个值,返回到g.send。在中x = yield yy通过管道向右推动。
  6. 生成器函数的主体内的执行暂停,从而使外部示波器可以从其中断处继续执行。
  7. g.send 恢复并提取值并将其返回给用户。
  8. g.send一次调用,回到第1步。

虽然有周期性的,这个过程确实有一个开始:当g.send(None)-这是什么next(g)是短期的-首先被调用(这是非法通过其他的东西比None在第一send调用)。它可能会有一个结局:当yield在生成器函数的主体中没有更多的语句要到达时。

您看到什么使该yield语句(或更准确地说,生成器)如此特别吗?与measly return关键字不同,yield它可以将值传递给其调用方并从其调用方接收值,而无需终止其所驻留的功能!(当然,如果您确实希望终止函数或生成器,也可以使用return关键字。)yield遇到语句时,生成器函数只是暂停,然后在其左移的位置重新选择在发送另一个值时关闭。并且send仅仅是从外部与生成器函数内部进行通信的接口。

如果我们真的要下来,只要我们可以打破这种推/拉/管类比,我们结束了下面的伪代码,真正开车回家的是,除了步骤1-5,yield并且send是相同的,双方硬币管:

  1. right_end.push(None) # the first half of g.send; sending None is what starts a generator
  2. right_end.pause()
  3. left_end.start()
  4. initial_value = left_end.pull()
  5. if initial_value is not None: raise TypeError("can't send non-None value to a just-started generator")
  6. left_end.do_stuff()
  7. left_end.push(y) # the first half of yield
  8. left_end.pause()
  9. right_end.resume()
  10. value1 = right_end.pull() # the second half of g.send
  11. right_end.do_stuff()
  12. right_end.push(value2) # the first half of g.send (again, but with a different value)
  13. right_end.pause()
  14. left_end.resume()
  15. x = left_end.pull() # the second half of yield
  16. goto 6

关键的转变是我们必须拆分x = yield yvalue1 = g.send(value2)各自分成两个语句:left_end.push(y)x = left_end.pull(); 和value1 = right_end.pull()right_end.push(value2)yield关键字有两种特殊情况:x = yieldyield y。它们分别是x = yield None和的语法糖_ = yield y # discarding value

有关通过管道发送值的精确顺序的特定详细信息,请参见下文。


接下来是上述的相当长的具体模型。首先,首先应注意,对于任何生成器gnext(g)都等效于g.send(None)。考虑到这一点,我们只能专注于send工作原理,而只谈论使用来推动生成器send

假设我们有

def f(y):  # This is the "generator function" referenced above
    while True:
        x = yield y
        y = x
g = f(1)
g.send(None)  # yields 1
g.send(2)     # yields 2

现在,对f以下普通(非生成器)函数进行粗略的定义:

def f(y):
    bidirectional_pipe = BidirectionalPipe()
    left_end = bidirectional_pipe.left_end
    right_end = bidirectional_pipe.right_end

    def impl():
        initial_value = left_end.pull()
        if initial_value is not None:
            raise TypeError(
                "can't send non-None value to a just-started generator"
            )

        while True:
            left_end.push(y)
            x = left_end.pull()
            y = x

    def send(value):
        right_end.push(value)
        return right_end.pull()

    right_end.send = send

    # This isn't real Python; normally, returning exits the function. But
    # pretend that it's possible to return a value from a function and then
    # continue execution -- this is exactly the problem that generators were
    # designed to solve!
    return right_end
    impl()

在以下转换中发生了以下情况f

  1. 我们已经将实现移到了嵌套函数中。
  2. 我们创建了一个双向管道,该管道left_end将由嵌套函数right_end访问,并且将由外部范围返回和访问,这right_end就是我们所知道的生成器对象。
  3. 在嵌套函数中,我们要做的第一件事是检查是否left_end.pull()None,在过程中消耗推入值。
  4. 在嵌套函数中,该语句x = yield y已替换为两行:left_end.push(y)x = left_end.pull()
  5. 我们已经为定义了send函数right_end,这x = yield y与在上一步中替换语句的两行相对应。

在这个幻想的世界中,函数可以在返回后继续运行,它g被分配right_end然后impl()被调用。因此,在上面的示例中,如果我们逐行执行,将发生的情况大致如下:

left_end = bidirectional_pipe.left_end
right_end = bidirectional_pipe.right_end

y = 1  # from g = f(1)

# None pushed by first half of g.send(None)
right_end.push(None)
# The above push blocks, so the outer scope halts and lets `f` run until
# *it* blocks

# Receive the pushed value, None
initial_value = left_end.pull()

if initial_value is not None:  # ok, `g` sent None
    raise TypeError(
        "can't send non-None value to a just-started generator"
    )

left_end.push(y)
# The above line blocks, so `f` pauses and g.send picks up where it left off

# y, aka 1, is pulled by right_end and returned by `g.send(None)`
right_end.pull()

# Rinse and repeat
# 2 pushed by first half of g.send(2)
right_end.push(2)
# Once again the above blocks, so g.send (the outer scope) halts and `f` resumes

# Receive the pushed value, 2
x = left_end.pull()
y = x  # y == x == 2

left_end.push(y)
# The above line blocks, so `f` pauses and g.send(2) picks up where it left off

# y, aka 2, is pulled by right_end and returned to the outer scope
right_end.pull()

x = left_end.pull()
# blocks until the next call to g.send

这正好映射到上面的16步伪代码。

还有其他一些细节,例如错误的传播方式以及到达发生器末端(管道已关闭)时会发生什么,但这应该清楚使用基本控制流的工作send方式。

使用这些相同的删除规则,让我们看两个特殊情况:

def f1(x):
    while True:
        x = yield x

def f2():  # No parameter
    while True:
        x = yield x

在大多数情况下,它们与的解糖方式相同f,唯一的区别是yield语句的转换方式:

def f1(x):
    # ... set up pipe

    def impl():
        # ... check that initial sent value is None

        while True:
            left_end.push(x)
            x = left_end.pull()

    # ... set up right_end


def f2():
    # ... set up pipe

    def impl():
        # ... check that initial sent value is None

        while True:
            left_end.push(x)
            x = left_end.pull()

    # ... set up right_end

首先,传递给的值首先f1被推送(屈服),然后所有被拉(发送)的值都被推回(屈服)。在第二个中,x第一次到达时尚无值(尚未)push,因此UnboundLocalError引发。

The word “yield” has two meanings: to produce something (e.g., to yield corn), and to halt to let someone/thing else continue (e.g., cars yielding to pedestrians). Both definitions apply to Python’s yield keyword; what makes generator functions special is that unlike in regular functions, values can be “returned” to the caller while merely pausing, not terminating, a generator function.

It is easiest to imagine a generator as one end of a bidirectional pipe with a “left” end and a “right” end; this pipe is the medium over which values are sent between the generator itself and the generator function’s body. Each end of the pipe has two operations: push, which sends a value and blocks until the other end of the pipe pulls the value, and returns nothing; and pull, which blocks until the other end of the pipe pushes a value, and returns the pushed value. At runtime, execution bounces back and forth between the contexts on either side of the pipe — each side runs until it sends a value to the other side, at which point it halts, lets the other side run, and waits for a value in return, at which point the other side halts and it resumes. In other words, each end of the pipe runs from the moment it receives a value to the moment it sends a value.

The pipe is functionally symmetric, but — by convention I’m defining in this answer — the left end is only available inside the generator function’s body and is accessible via the yield keyword, while the right end is the generator and is accessible via the generator’s send function. As singular interfaces to their respective ends of the pipe, yield and send do double duty: they each both push and pull values to/from their ends of the pipe, yield pushing rightward and pulling leftward while send does the opposite. This double duty is the crux of the confusion surrounding the semantics of statements like x = yield y. Breaking yield and send down into two explicit push/pull steps will make their semantics much more clear:

  1. Suppose g is the generator. g.send pushes a value leftward through the right end of the pipe.
  2. Execution within the context of g pauses, allowing the generator function’s body to run.
  3. The value pushed by g.send is pulled leftward by yield and received on the left end of the pipe. In x = yield y, x is assigned to the pulled value.
  4. Execution continues within the generator function’s body until the next line containing yield is reached.
  5. yield pushes a value rightward through the left end of the pipe, back up to g.send. In x = yield y, y is pushed rightward through the pipe.
  6. Execution within the generator function’s body pauses, allowing the outer scope to continue where it left off.
  7. g.send resumes and pulls the value and returns it to the user.
  8. When g.send is next called, go back to Step 1.

While cyclical, this procedure does have a beginning: when g.send(None) — which is what next(g) is short for — is first called (it is illegal to pass something other than None to the first send call). And it may have an end: when there are no more yield statements to be reached in the generator function’s body.

Do you see what makes the yield statement (or more accurately, generators) so special? Unlike the measly return keyword, yield is able to pass values to its caller and receive values from its caller all without terminating the function it lives in! (Of course, if you do wish to terminate a function — or a generator — it’s handy to have the return keyword as well.) When a yield statement is encountered, the generator function merely pauses, and then picks back up right where it left off upon being sent another value. And send is just the interface for communicating with the inside of a generator function from outside it.

If we really want to break this push/pull/pipe analogy down as far as we can, we end up with the following pseudocode that really drives home that, aside from steps 1-5, yield and send are two sides of the same coin pipe:

  1. right_end.push(None) # the first half of g.send; sending None is what starts a generator
  2. right_end.pause()
  3. left_end.start()
  4. initial_value = left_end.pull()
  5. if initial_value is not None: raise TypeError("can't send non-None value to a just-started generator")
  6. left_end.do_stuff()
  7. left_end.push(y) # the first half of yield
  8. left_end.pause()
  9. right_end.resume()
  10. value1 = right_end.pull() # the second half of g.send
  11. right_end.do_stuff()
  12. right_end.push(value2) # the first half of g.send (again, but with a different value)
  13. right_end.pause()
  14. left_end.resume()
  15. x = left_end.pull() # the second half of yield
  16. goto 6

The key transformation is that we have split x = yield y and value1 = g.send(value2) each into two statements: left_end.push(y) and x = left_end.pull(); and value1 = right_end.pull() and right_end.push(value2). There are two special cases of the yield keyword: x = yield and yield y. These are syntactic sugar, respectively, for x = yield None and _ = yield y # discarding value.

For specific details regarding the precise order in which values are sent through the pipe, see below.


What follows is a rather long concrete model of the above. First, it should first be noted that for any generator g, next(g) is exactly equivalent to g.send(None). With this in mind we can focus only on how send works and talk only about advancing the generator with send.

Suppose we have

def f(y):  # This is the "generator function" referenced above
    while True:
        x = yield y
        y = x
g = f(1)
g.send(None)  # yields 1
g.send(2)     # yields 2

Now, the definition of f roughly desugars to the following ordinary (non-generator) function:

def f(y):
    bidirectional_pipe = BidirectionalPipe()
    left_end = bidirectional_pipe.left_end
    right_end = bidirectional_pipe.right_end

    def impl():
        initial_value = left_end.pull()
        if initial_value is not None:
            raise TypeError(
                "can't send non-None value to a just-started generator"
            )

        while True:
            left_end.push(y)
            x = left_end.pull()
            y = x

    def send(value):
        right_end.push(value)
        return right_end.pull()

    right_end.send = send

    # This isn't real Python; normally, returning exits the function. But
    # pretend that it's possible to return a value from a function and then
    # continue execution -- this is exactly the problem that generators were
    # designed to solve!
    return right_end
    impl()

The following has happened in this transformation of f:

  1. We’ve moved the implementation into a nested function.
  2. We’ve created a bidirectional pipe whose left_end will be accessed by the nested function and whose right_end will be returned and accessed by the outer scope — right_end is what we know as the generator object.
  3. Within the nested function, the very first thing we do is check that left_end.pull() is None, consuming a pushed value in the process.
  4. Within the nested function, the statement x = yield y has been replaced by two lines: left_end.push(y) and x = left_end.pull().
  5. We’ve defined the send function for right_end, which is the counterpart to the two lines we replaced the x = yield y statement with in the previous step.

In this fantasy world where functions can continue after returning, g is assigned right_end and then impl() is called. So in our example above, were we to follow execution line by line, what would happen is roughly the following:

left_end = bidirectional_pipe.left_end
right_end = bidirectional_pipe.right_end

y = 1  # from g = f(1)

# None pushed by first half of g.send(None)
right_end.push(None)
# The above push blocks, so the outer scope halts and lets `f` run until
# *it* blocks

# Receive the pushed value, None
initial_value = left_end.pull()

if initial_value is not None:  # ok, `g` sent None
    raise TypeError(
        "can't send non-None value to a just-started generator"
    )

left_end.push(y)
# The above line blocks, so `f` pauses and g.send picks up where it left off

# y, aka 1, is pulled by right_end and returned by `g.send(None)`
right_end.pull()

# Rinse and repeat
# 2 pushed by first half of g.send(2)
right_end.push(2)
# Once again the above blocks, so g.send (the outer scope) halts and `f` resumes

# Receive the pushed value, 2
x = left_end.pull()
y = x  # y == x == 2

left_end.push(y)
# The above line blocks, so `f` pauses and g.send(2) picks up where it left off

# y, aka 2, is pulled by right_end and returned to the outer scope
right_end.pull()

x = left_end.pull()
# blocks until the next call to g.send

This maps exactly to the 16-step pseudocode above.

There are some other details, like how errors are propagated and what happens when you reach the end of the generator (the pipe is closed), but this should make clear how the basic control flow works when send is used.

Using these same desugaring rules, let’s look at two special cases:

def f1(x):
    while True:
        x = yield x

def f2():  # No parameter
    while True:
        x = yield x

For the most part they desugar the same way as f, the only differences are how the yield statements are transformed:

def f1(x):
    # ... set up pipe

    def impl():
        # ... check that initial sent value is None

        while True:
            left_end.push(x)
            x = left_end.pull()

    # ... set up right_end


def f2():
    # ... set up pipe

    def impl():
        # ... check that initial sent value is None

        while True:
            left_end.push(x)
            x = left_end.pull()

    # ... set up right_end

In the first, the value passed to f1 is pushed (yielded) initially, and then all values pulled (sent) are pushed (yielded) right back. In the second, x has no value (yet) when it first come times to push, so an UnboundLocalError is raised.


回答 7

这些也让我感到困惑。这是我尝试设置生成器的一个示例,该生成器以交替的顺序产生(屈服,接受,屈服,接受)。

def echo_sound():

    thing_to_say = '<Sound of wind on cliffs>'
    while True:
        thing_to_say = (yield thing_to_say)
        thing_to_say = '...'.join([thing_to_say]+[thing_to_say[-6:]]*2)
        yield None  # This is the return value of send.

gen = echo_sound()

print 'You are lost in the wilderness, calling for help.'

print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Hello!'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)

print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Is anybody out there?'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)

print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Help!'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)

输出为:

You are lost in the wilderness, calling for help.
------
You hear: "<Sound of wind on cliffs>"
You yell "Hello!"
------
You hear: "Hello!...Hello!...Hello!"
You yell "Is anybody out there?"
------
You hear: "Is anybody out there?...there?...there?"
You yell "Help!"

These confused me too. Here is an example I made when trying to set up a generator which yields and accepts signals in alternating order (yield, accept, yield, accept)…

def echo_sound():

    thing_to_say = '<Sound of wind on cliffs>'
    while True:
        thing_to_say = (yield thing_to_say)
        thing_to_say = '...'.join([thing_to_say]+[thing_to_say[-6:]]*2)
        yield None  # This is the return value of send.

gen = echo_sound()

print 'You are lost in the wilderness, calling for help.'

print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Hello!'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)

print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Is anybody out there?'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)

print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Help!'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)

The output is:

You are lost in the wilderness, calling for help.
------
You hear: "<Sound of wind on cliffs>"
You yell "Hello!"
------
You hear: "Hello!...Hello!...Hello!"
You yell "Is anybody out there?"
------
You hear: "Is anybody out there?...there?...there?"
You yell "Help!"