functools部分如何做?

问题:functools部分如何做?

我无法了解部分功能在functools中的工作方式。我从这里有以下代码:

>>> sum = lambda x, y : x + y
>>> sum(1, 2)
3
>>> incr = lambda y : sum(1, y)
>>> incr(2)
3
>>> def sum2(x, y):
    return x + y

>>> incr2 = functools.partial(sum2, 1)
>>> incr2(4)
5

现在排队

incr = lambda y : sum(1, y)

我知道我传递给incr它的任何参数都将传递ylambda哪个参数,sum(1, y)即返回1 + y

我明白那个。但是我不明白incr2(4)

如何在部分函数中4传递获取x?对我来说,4应该更换sum2x和之间是什么关系4

I am not able to get my head on how the partial works in functools. I have the following code from here:

>>> sum = lambda x, y : x + y
>>> sum(1, 2)
3
>>> incr = lambda y : sum(1, y)
>>> incr(2)
3
>>> def sum2(x, y):
    return x + y

>>> incr2 = functools.partial(sum2, 1)
>>> incr2(4)
5

Now in the line

incr = lambda y : sum(1, y)

I get that whatever argument I pass to incr it will be passed as y to lambda which will return sum(1, y) i.e 1 + y.

I understand that. But I didn’t understand this incr2(4).

How does the 4 gets passed as x in partial function? To me, 4 should replace the sum2. What is the relation between x and 4?


回答 0

大致地,partial做这样的事情(除了关键字args支持等):

def partial(func, *part_args):
    def wrapper(*extra_args):
        args = list(part_args)
        args.extend(extra_args)
        return func(*args)

    return wrapper

因此,通过调用partial(sum2, 4)您可以创建一个行为类似于的新函数(准确地说是一个可调用的函数)sum2,但位置参数要少一个。缺少的参数总是由代替4,因此partial(sum2, 4)(2) == sum2(4, 2)

至于为什么需要它,有很多情况。仅举一个例子,假设您必须在某个有两个参数的地方传递一个函数:

class EventNotifier(object):
    def __init__(self):
        self._listeners = []

    def add_listener(self, callback):
        ''' callback should accept two positional arguments, event and params '''
        self._listeners.append(callback)
        # ...

    def notify(self, event, *params):
        for f in self._listeners:
            f(event, params)

但是您已经拥有的功能需要访问某些第三context对象才能完成其工作:

def log_event(context, event, params):
    context.log_event("Something happened %s, %s", event, params)

因此,有几种解决方案:

自定义对象:

class Listener(object):
   def __init__(self, context):
       self._context = context

   def __call__(self, event, params):
       self._context.log_event("Something happened %s, %s", event, params)


 notifier.add_listener(Listener(context))

Lambda:

log_listener = lambda event, params: log_event(context, event, params)
notifier.add_listener(log_listener)

带有局部:

context = get_context()  # whatever
notifier.add_listener(partial(log_event, context))

在这三个中,partial最短和最快。(对于更复杂的情况,您可能需要自定义对象)。

Roughly, partial does something like this (apart from keyword args support etc):

def partial(func, *part_args):
    def wrapper(*extra_args):
        args = list(part_args)
        args.extend(extra_args)
        return func(*args)

    return wrapper

So, by calling partial(sum2, 4) you create a new function (a callable, to be precise) that behaves like sum2, but has one positional argument less. That missing argument is always substituted by 4, so that partial(sum2, 4)(2) == sum2(4, 2)

As for why it’s needed, there’s a variety of cases. Just for one, suppose you have to pass a function somewhere where it’s expected to have 2 arguments:

class EventNotifier(object):
    def __init__(self):
        self._listeners = []

    def add_listener(self, callback):
        ''' callback should accept two positional arguments, event and params '''
        self._listeners.append(callback)
        # ...

    def notify(self, event, *params):
        for f in self._listeners:
            f(event, params)

But a function you already have needs access to some third context object to do its job:

def log_event(context, event, params):
    context.log_event("Something happened %s, %s", event, params)

So, there are several solutions:

A custom object:

class Listener(object):
   def __init__(self, context):
       self._context = context

   def __call__(self, event, params):
       self._context.log_event("Something happened %s, %s", event, params)


 notifier.add_listener(Listener(context))

Lambda:

log_listener = lambda event, params: log_event(context, event, params)
notifier.add_listener(log_listener)

With partials:

context = get_context()  # whatever
notifier.add_listener(partial(log_event, context))

Of those three, partial is the shortest and the fastest. (For a more complex case you might want a custom object though).


回答 1

局部函数非常有用。

例如,在“管线式”函数调用序列中(其中一个函数的返回值是传递给下一个函数的参数)。

有时,此类管道中的函数需要单个参数,但是紧接其上游的函数将返回两个值

在这种情况下,functools.partial可能允许您保持此功能管道完整。

这是一个特定的隔离示例:假设您想按每个数据点与目标之间的距离对一些数据进行排序:

# create some data
import random as RND
fnx = lambda: RND.randint(0, 10)
data = [ (fnx(), fnx()) for c in range(10) ]
target = (2, 4)

import math
def euclid_dist(v1, v2):
    x1, y1 = v1
    x2, y2 = v2
    return math.sqrt((x2 - x1)**2 + (y2 - y1)**2)

要按距目标的距离对数据进行排序,您当然要做的是:

data.sort(key=euclid_dist)

但你不可阻挡-的排序方法的关键参数,只接受拍摄功能单一的参数。

因此,请改写euclid_dist为带有单个参数的函数:

from functools import partial

p_euclid_dist = partial(euclid_dist, target)

p_euclid_dist 现在接受一个参数,

>>> p_euclid_dist((3, 3))
  1.4142135623730951

因此,现在您可以通过传递sort方法的key参数的局部函数来对数据进行排序:

data.sort(key=p_euclid_dist)

# verify that it works:
for p in data:
    print(round(p_euclid_dist(p), 3))

    1.0
    2.236
    2.236
    3.606
    4.243
    5.0
    5.831
    6.325
    7.071
    8.602

又例如,函数的参数之一在外循环中更改,但在内循环迭代期间是固定的。通过使用部分函数,​​您无需在内部循环的迭代过程中传递其他参数,因为修改后的(部分函数)不需要此参数。

>>> from functools import partial

>>> def fnx(a, b, c):
      return a + b + c

>>> fnx(3, 4, 5)
      12

创建一个局部函数(使用关键字arg)

>>> pfnx = partial(fnx, a=12)

>>> pfnx(b=4, c=5)
     21

您还可以使用位置参数创建部分函数

>>> pfnx = partial(fnx, 12)

>>> pfnx(4, 5)
      21

但这会抛出(例如,创建带有关键字参数的partial,然后使用位置参数调用)

>>> pfnx = partial(fnx, a=12)

>>> pfnx(4, 5)
      Traceback (most recent call last):
      File "<pyshell#80>", line 1, in <module>
      pfnx(4, 5)
      TypeError: fnx() got multiple values for keyword argument 'a'

另一个用例:使用python的multiprocessing库编写分布式代码。使用Pool方法创建一个进程池:

>>> import multiprocessing as MP

>>> # create a process pool:
>>> ppool = MP.Pool()

Pool 有一个map方法,但是它只需要一个可迭代的方法,因此,如果您需要传入带有较长参数列表的函数,请将该函数重新定义为局部函数,以修复除一个以外的所有函数:

>>> ppool.map(pfnx, [4, 6, 7, 8])

partials are incredibly useful.

For instance, in a ‘pipe-lined’ sequence of function calls (in which the returned value from one function is the argument passed to the next).

Sometimes a function in such a pipeline requires a single argument, but the function immediately upstream from it returns two values.

In this scenario, functools.partial might allow you to keep this function pipeline intact.

Here’s a specific, isolated example: suppose you want to sort some data by each data point’s distance from some target:

# create some data
import random as RND
fnx = lambda: RND.randint(0, 10)
data = [ (fnx(), fnx()) for c in range(10) ]
target = (2, 4)

import math
def euclid_dist(v1, v2):
    x1, y1 = v1
    x2, y2 = v2
    return math.sqrt((x2 - x1)**2 + (y2 - y1)**2)

To sort this data by distance from the target, what you would like to do of course is this:

data.sort(key=euclid_dist)

but you can’t–the sort method’s key parameter only accepts functions that take a single argument.

so re-write euclid_dist as a function taking a single parameter:

from functools import partial

p_euclid_dist = partial(euclid_dist, target)

p_euclid_dist now accepts a single argument,

>>> p_euclid_dist((3, 3))
  1.4142135623730951

so now you can sort your data by passing in the partial function for the sort method’s key argument:

data.sort(key=p_euclid_dist)

# verify that it works:
for p in data:
    print(round(p_euclid_dist(p), 3))

    1.0
    2.236
    2.236
    3.606
    4.243
    5.0
    5.831
    6.325
    7.071
    8.602

Or for instance, one of the function’s arguments changes in an outer loop but is fixed during iteration in the inner loop. By using a partial, you don’t have to pass in the additional parameter during iteration of the inner loop, because the modified (partial) function doesn’t require it.

>>> from functools import partial

>>> def fnx(a, b, c):
      return a + b + c

>>> fnx(3, 4, 5)
      12

create a partial function (using keyword arg)

>>> pfnx = partial(fnx, a=12)

>>> pfnx(b=4, c=5)
     21

you can also create a partial function with a positional argument

>>> pfnx = partial(fnx, 12)

>>> pfnx(4, 5)
      21

but this will throw (e.g., creating partial with keyword argument then calling using positional arguments)

>>> pfnx = partial(fnx, a=12)

>>> pfnx(4, 5)
      Traceback (most recent call last):
      File "<pyshell#80>", line 1, in <module>
      pfnx(4, 5)
      TypeError: fnx() got multiple values for keyword argument 'a'

another use case: writing distributed code using python’s multiprocessing library. A pool of processes is created using the Pool method:

>>> import multiprocessing as MP

>>> # create a process pool:
>>> ppool = MP.Pool()

Pool has a map method, but it only takes a single iterable, so if you need to pass in a function with a longer parameter list, re-define the function as a partial, to fix all but one:

>>> ppool.map(pfnx, [4, 6, 7, 8])

回答 2

简短的答案,partial为函数的参数提供默认值,否则将没有默认值。

from functools import partial

def foo(a,b):
    return a+b

bar = partial(foo, a=1) # equivalent to: foo(a=1, b)
bar(b=10)
#11 = 1+10
bar(a=101, b=10)
#111=101+10

short answer, partial gives default values to the parameters of a function that would otherwise not have default values.

from functools import partial

def foo(a,b):
    return a+b

bar = partial(foo, a=1) # equivalent to: foo(a=1, b)
bar(b=10)
#11 = 1+10
bar(a=101, b=10)
#111=101+10

回答 3

可以使用部分函数来创建新的派生函数,这些函数具有预先分配的一些输入参数

要了解部分用法在现实世界中的用法,请参阅此非常好的博客文章:http :
//chriskiehl.com/article/Cleaner-coding-through-partially-applied-functions/

博客中的一个简单但简洁的示例,介绍了如何使用partialre.search使代码更具可读性。 re.search方法的签名是:

search(pattern, string, flags=0) 

通过应用,partial我们可以创建多个版本的正则表达式search来满足我们的要求,例如:

is_spaced_apart = partial(re.search, '[a-zA-Z]\s\=')
is_grouped_together = partial(re.search, '[a-zA-Z]\=')

现在is_spaced_apartis_grouped_together是从中派生的两个新函数re.search,它们pattern应用了自变量(因为它patternre.search方法签名中的第一个自变量)。

这两个新函数(可调用)的签名为:

is_spaced_apart(string, flags=0)     # pattern '[a-zA-Z]\s\=' applied
is_grouped_together(string, flags=0) # pattern '[a-zA-Z]\=' applied

这样便可以在某些文本上使用这些部分函数:

for text in lines:
    if is_grouped_together(text):
        some_action(text)
    elif is_spaced_apart(text):
        some_other_action(text)
    else:
        some_default_action()

您可以参考上面的链接,以更深入地了解该主题,因为它涵盖了此特定示例以及更多内容。

Partials can be used to make new derived functions that have some input parameters pre-assigned

To see some real world usage of partials, refer to this really good blog post:
http://chriskiehl.com/article/Cleaner-coding-through-partially-applied-functions/

A simple but neat beginner’s example from the blog, covers how one might use partial on re.search to make code more readable. re.search method’s signature is:

search(pattern, string, flags=0) 

By applying partial we can create multiple versions of the regular expression search to suit our requirements, so for example:

is_spaced_apart = partial(re.search, '[a-zA-Z]\s\=')
is_grouped_together = partial(re.search, '[a-zA-Z]\=')

Now is_spaced_apart and is_grouped_together are two new functions derived from re.search that have the pattern argument applied(since pattern is the first argument in the re.search method’s signature).

The signature of these two new functions(callable) is:

is_spaced_apart(string, flags=0)     # pattern '[a-zA-Z]\s\=' applied
is_grouped_together(string, flags=0) # pattern '[a-zA-Z]\=' applied

This is how you could then use these partial functions on some text:

for text in lines:
    if is_grouped_together(text):
        some_action(text)
    elif is_spaced_apart(text):
        some_other_action(text)
    else:
        some_default_action()

You can refer the link above to get a more in depth understanding of the subject, as it covers this specific example and much more..


回答 4

我认为,这是在python中实现currying的一种方式。

from functools import partial
def add(a,b):
    return a + b

def add2number(x,y,z):
    return x + y + z

if __name__ == "__main__":
    add2 = partial(add,2)
    print("result of add2 ",add2(1))
    add3 = partial(partial(add2number,1),2)
    print("result of add3",add3(1))

结果是3和4。

In my opinion, it’s a way to implement currying in python.

from functools import partial
def add(a,b):
    return a + b

def add2number(x,y,z):
    return x + y + z

if __name__ == "__main__":
    add2 = partial(add,2)
    print("result of add2 ",add2(1))
    add3 = partial(partial(add2number,1),2)
    print("result of add3",add3(1))

The result is 3 and 4.


回答 5

还值得一提的是,当部分函数传递了另一个我们要“硬编码”某些参数的函数时,该参数应该是最右边的参数。

def func(a,b):
    return a*b
prt = partial(func, b=7)
    print(prt(4))
#return 28

但是,如果我们执行相同的操作,而是改为更改参数

def func(a,b):
    return a*b
 prt = partial(func, a=7)
    print(prt(4))

它将引发错误,“ TypeError:func()为参数’a’获得了多个值”

Also worth to mention, that when partial function passed another function where we want to “hard code” some parameters, that should be rightmost parameter

def func(a,b):
    return a*b
prt = partial(func, b=7)
    print(prt(4))
#return 28

but if we do the same, but changing a parameter instead

def func(a,b):
    return a*b
 prt = partial(func, a=7)
    print(prt(4))

it will throw error, “TypeError: func() got multiple values for argument ‘a'”


回答 6

这个答案更多是示例代码。上面的所有答案都很好地解释了为什么应该部分使用。我将给出我的观察和有关局部的用例。

from functools import partial
 def adder(a,b,c):
    print('a:{},b:{},c:{}'.format(a,b,c))
    ans = a+b+c
    print(ans)
partial_adder = partial(adder,1,2)
partial_adder(3)  ## now partial_adder is a callable that can take only one argument

以上代码的输出应为:

a:1,b:2,c:3
6

注意,在上面的示例中,返回了一个新的callable,它将参数(c)作为其参数。请注意,它也是函数的最后一个参数。

args = [1,2]
partial_adder = partial(adder,*args)
partial_adder(3)

上面代码的输出也是:

a:1,b:2,c:3
6

请注意,*用于解压缩非关键字参数,而返回的callable可以接受的参数与上面相同。

另一个观察结果是: 下面的示例演示了partial返回一个callable,它将以未声明的参数(a)作为参数。

def adder(a,b=1,c=2,d=3,e=4):
    print('a:{},b:{},c:{},d:{},e:{}'.format(a,b,c,d,e))
    ans = a+b+c+d+e
    print(ans)
partial_adder = partial(adder,b=10,c=2)
partial_adder(20)

以上代码的输出应为:

a:20,b:10,c:2,d:3,e:4
39

同样,

kwargs = {'b':10,'c':2}
partial_adder = partial(adder,**kwargs)
partial_adder(20)

以上代码打印

a:20,b:10,c:2,d:3,e:4
39

当我使用模块中的Pool.map_async方法时,我不得不使用它multiprocessing。您只能将一个参数传递给worker函数,因此我不得不使用它partial来使我的worker函数看起来像只有一个输入参数的可调用对象,但实际上我的worker函数具有多个输入参数。

This answer is more of an example code. All the above answers give good explanations regarding why one should use partial. I will give my observations and use cases about partial.

from functools import partial
 def adder(a,b,c):
    print('a:{},b:{},c:{}'.format(a,b,c))
    ans = a+b+c
    print(ans)
partial_adder = partial(adder,1,2)
partial_adder(3)  ## now partial_adder is a callable that can take only one argument

Output of the above code should be:

a:1,b:2,c:3
6

Notice that in the above example a new callable was returned that will take parameter (c) as it’s argument. Note that it is also the last argument to the function.

args = [1,2]
partial_adder = partial(adder,*args)
partial_adder(3)

Output of the above code is also:

a:1,b:2,c:3
6

Notice that * was used to unpack the non-keyword arguments and the callable returned in terms of which argument it can take is same as above.

Another observation is: Below example demonstrates that partial returns a callable which will take the undeclared parameter (a) as an argument.

def adder(a,b=1,c=2,d=3,e=4):
    print('a:{},b:{},c:{},d:{},e:{}'.format(a,b,c,d,e))
    ans = a+b+c+d+e
    print(ans)
partial_adder = partial(adder,b=10,c=2)
partial_adder(20)

Output of the above code should be:

a:20,b:10,c:2,d:3,e:4
39

Similarly,

kwargs = {'b':10,'c':2}
partial_adder = partial(adder,**kwargs)
partial_adder(20)

Above code prints

a:20,b:10,c:2,d:3,e:4
39

I had to use it when I was using Pool.map_async method from multiprocessing module. You can pass only one argument to the worker function so I had to use partial to make my worker function look like a callable with only one input argument but in reality my worker function had multiple input arguments.