标签归档:functional-programming

使用python map和其他功能工具

问题:使用python map和其他功能工具

这相当不错,但是我正在尝试学习/理解python中的函数式编程。如下代码:

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest(foo, bar):
    print foo, bar

map(maptest, foos, bars)

生成:

1.0 1
2.0 2
3.0 3
4.0 None
5.0 None

问:有没有一种方法可以使用map或python中的任何其他功能工具来产生以下内容而没有循环等。

1.0 [1,2,3]
2.0 [1,2,3]
3.0 [1,2,3]
4.0 [1,2,3]
5.0 [1,2,3]

顺便提一下,如果foo和bar之间存在依赖关系,则实现将如何更改。例如

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3,4,5]

并打印:

1.0 [2,3,4,5]
2.0 [1,3,4,5]
3.0 [1,2,4,5]
...

PS:我知道如何使用if,循环和/或生成器天真地做到这一点,但是我想学习如何使用功能性工具来实现这一点。是否只是在maptest中添加if语句或在maptest内部将其他过滤器映射应用于bar的情况?

This is quite n00bish, but I’m trying to learn/understand functional programming in python. The following code:

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest(foo, bar):
    print foo, bar

map(maptest, foos, bars)

produces:

1.0 1
2.0 2
3.0 3
4.0 None
5.0 None

Q. Is there a way to use map or any other functional tools in python to produce the following without loops etc.

1.0 [1,2,3]
2.0 [1,2,3]
3.0 [1,2,3]
4.0 [1,2,3]
5.0 [1,2,3]

Just as a side note how would the implementation change if there is a dependency between foo and bar. e.g.

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3,4,5]

and print:

1.0 [2,3,4,5]
2.0 [1,3,4,5]
3.0 [1,2,4,5]
...

P.S: I know how to do it naively using if, loops and/or generators, but I’d like to learn how to achieve the same using functional tools. Is it just a case of adding an if statement to maptest or apply another filter map to bars internally within maptest?


回答 0

最简单的方法是不bars通过不同的功能,而直接从maptest以下位置访问它:

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest(foo):
    print foo, bars

map(maptest, foos)

使用原始maptest函数,您还可以在map以下代码中使用lambda函数:

map((lambda foo: maptest(foo, bars)), foos)

The easiest way would be not to pass bars through the different functions, but to access it directly from maptest:

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest(foo):
    print foo, bars

map(maptest, foos)

With your original maptest function you could also use a lambda function in map:

map((lambda foo: maptest(foo, bars)), foos)

回答 1

您是否熟悉其他功能语言?即,您是要学习python如何进行函数编程,还是要学习有关函数编程并使用python作为工具?

另外,您了解列表理解吗?

map(f, sequence)

与(*)直接等效:

[f(x) for x in sequence]

实际上,我认为map()曾经打算从python 3.0中删除它是多余的(那没有发生)。

map(f, sequence1, sequence2)

大致等于:

[f(x1, x2) for x1, x2 in zip(sequence1, sequence2)]

(在处理序列长度不同的情况时,它的处理方式有所不同。如您所见,map()当其中一个序列用完时,填入None,而zip()当最短序列停止时,则填满)

因此,为了解决您的特定问题,您尝试产生结果:

foos[0], bars
foos[1], bars
foos[2], bars
# etc.

您可以通过编写一个带有单个参数并打印它的函数,然后加上杠来做到这一点:

def maptest(x):
     print x, bars
map(maptest, foos)

或者,您可以创建一个如下所示的列表:

[bars, bars, bars, ] # etc.

并使用原始的maptest:

def maptest(x, y):
    print x, y

一种方法是事先显式构建列表:

barses = [bars] * len(foos)
map(maptest, foos, barses)

或者,您可以拉入itertools模块。 itertools包含许多巧妙的功能,可帮助您在python中进行功能风格的延迟评估编程。在这种情况下,我们需要itertools.repeat,当您对其进行迭代时,它将无限期地输出其参数。最后一个事实意味着,如果您这样做:

map(maptest, foos, itertools.repeat(bars))

map()只要参数之一仍在产生输出,您就会得到无穷的输出,因为它一直持续下去。但是,itertools.imap就像map(),但最短的可迭代停止就停止。

itertools.imap(maptest, foos, itertools.repeat(bars))

希望这可以帮助 :-)

(*)在python 3.0中有些不同。在那里,map()本质上返回一个生成器表达式。

Are you familiar with other functional languages? i.e. are you trying to learn how python does functional programming, or are you trying to learn about functional programming and using python as the vehicle?

Also, do you understand list comprehensions?

map(f, sequence)

is directly equivalent (*) to:

[f(x) for x in sequence]

In fact, I think map() was once slated for removal from python 3.0 as being redundant (that didn’t happen).

map(f, sequence1, sequence2)

is mostly equivalent to:

[f(x1, x2) for x1, x2 in zip(sequence1, sequence2)]

(there is a difference in how it handles the case where the sequences are of different length. As you saw, map() fills in None when one of the sequences runs out, whereas zip() stops when the shortest sequence stops)

So, to address your specific question, you’re trying to produce the result:

foos[0], bars
foos[1], bars
foos[2], bars
# etc.

You could do this by writing a function that takes a single argument and prints it, followed by bars:

def maptest(x):
     print x, bars
map(maptest, foos)

Alternatively, you could create a list that looks like this:

[bars, bars, bars, ] # etc.

and use your original maptest:

def maptest(x, y):
    print x, y

One way to do this would be to explicitely build the list beforehand:

barses = [bars] * len(foos)
map(maptest, foos, barses)

Alternatively, you could pull in the itertools module. itertools contains many clever functions that help you do functional-style lazy-evaluation programming in python. In this case, we want itertools.repeat, which will output its argument indefinitely as you iterate over it. This last fact means that if you do:

map(maptest, foos, itertools.repeat(bars))

you will get endless output, since map() keeps going as long as one of the arguments is still producing output. However, itertools.imap is just like map(), but stops as soon as the shortest iterable stops.

itertools.imap(maptest, foos, itertools.repeat(bars))

Hope this helps 🙂

(*) It’s a little different in python 3.0. There, map() essentially returns a generator expression.


回答 2

这是您要寻找的解决方案:

>>> foos = [1.0, 2.0, 3.0, 4.0, 5.0]
>>> bars = [1, 2, 3]
>>> [(x, bars) for x in foos]
[(1.0, [1, 2, 3]), (2.0, [1, 2, 3]), (3.0, [1, 2, 3]), (4.0, [1, 2, 3]), (5.0, [
1, 2, 3])]

我建议使用列表理解([(x, bars) for x in foos]部分)而不是使用地图,因为它避免了每次迭代时函数调用的开销(这可能非常重要)。如果只打算在for循环中使用它,则可以通过使用生成器理解来获得更好的速度:

>>> y = ((x, bars) for x in foos)
>>> for z in y:
...     print z
...
(1.0, [1, 2, 3])
(2.0, [1, 2, 3])
(3.0, [1, 2, 3])
(4.0, [1, 2, 3])
(5.0, [1, 2, 3])

区别在于生成器理解迟缓地加载

更新 针对此评论:

当然,您知道您不复制栏,所有条目都是相同的栏列表。因此,如果您修改其中的任何一个(包括原始条),那么您将修改所有的它们。

我想这是一个正确的观点。我可以想到两种解决方案。最有效的可能是这样的:

tbars = tuple(bars)
[(x, tbars) for x in foos]

由于元组是不可变的,因此这将防止通过此列表理解的结果(或通过该路线生成器理解)的结果来修改钢筋。如果确实需要修改每个结果,则可以执行以下操作:

from copy import copy
[(x, copy(bars)) for x in foos]

但是,这在内存使用和速度方面都可能会有些昂贵,因此我建议您不要这样做,除非您确实需要添加每个内存。

Here’s the solution you’re looking for:

>>> foos = [1.0, 2.0, 3.0, 4.0, 5.0]
>>> bars = [1, 2, 3]
>>> [(x, bars) for x in foos]
[(1.0, [1, 2, 3]), (2.0, [1, 2, 3]), (3.0, [1, 2, 3]), (4.0, [1, 2, 3]), (5.0, [
1, 2, 3])]

I’d recommend using a list comprehension (the [(x, bars) for x in foos] part) over using map as it avoids the overhead of a function call on every iteration (which can be very significant). If you’re just going to use it in a for loop, you’ll get better speeds by using a generator comprehension:

>>> y = ((x, bars) for x in foos)
>>> for z in y:
...     print z
...
(1.0, [1, 2, 3])
(2.0, [1, 2, 3])
(3.0, [1, 2, 3])
(4.0, [1, 2, 3])
(5.0, [1, 2, 3])

The difference is that the generator comprehension is lazily loaded.

UPDATE In response to this comment:

Of course you know, that you don’t copy bars, all entries are the same bars list. So if you modify any one of them (including original bars), you modify all of them.

I suppose this is a valid point. There are two solutions to this that I can think of. The most efficient is probably something like this:

tbars = tuple(bars)
[(x, tbars) for x in foos]

Since tuples are immutable, this will prevent bars from being modified through the results of this list comprehension (or generator comprehension if you go that route). If you really need to modify each and every one of the results, you can do this:

from copy import copy
[(x, copy(bars)) for x in foos]

However, this can be a bit expensive both in terms of memory usage and in speed, so I’d recommend against it unless you really need to add to each one of them.


回答 3

函数式编程是关于创建无副作用的代码。

map是功能列表转换的抽象。您可以使用它来获取一系列序列并将其转换为其他序列。

您正在尝试将其用作迭代器。不要那样做 🙂

这是一个示例,说明如何使用地图构建所需的列表。有较短的解决方案(我只是使用理解力),但这将帮助您了解哪种地图效果更好:

def my_transform_function(input):
    return [input, [1, 2, 3]]

new_list = map(my_transform, input_list)

请注意,此时您仅完成了数据操作。现在您可以打印它:

for n,l in new_list:
    print n, ll

-我不确定“没有循环”是什么意思。fp并不是要避免循环(您无法访问列表中的每个项目都无法对其进行检查)。这是关于避免副作用,从而减少错误。

Functional programming is about creating side-effect-free code.

map is a functional list transformation abstraction. You use it to take a sequence of something and turn it into a sequence of something else.

You are trying to use it as an iterator. Don’t do that. 🙂

Here is an example of how you might use map to build the list you want. There are shorter solutions (I’d just use comprehensions), but this will help you understand what map does a bit better:

def my_transform_function(input):
    return [input, [1, 2, 3]]

new_list = map(my_transform, input_list)

Notice at this point, you’ve only done a data manipulation. Now you can print it:

for n,l in new_list:
    print n, ll

— I’m not sure what you mean by ‘without loops.’ fp isn’t about avoiding loops (you can’t examine every item in a list without visiting each one). It’s about avoiding side-effects, thus writing fewer bugs.


回答 4

>>> from itertools import repeat
>>> for foo, bars in zip(foos, repeat(bars)):
...     print foo, bars
... 
1.0 [1, 2, 3]
2.0 [1, 2, 3]
3.0 [1, 2, 3]
4.0 [1, 2, 3]
5.0 [1, 2, 3]
>>> from itertools import repeat
>>> for foo, bars in zip(foos, repeat(bars)):
...     print foo, bars
... 
1.0 [1, 2, 3]
2.0 [1, 2, 3]
3.0 [1, 2, 3]
4.0 [1, 2, 3]
5.0 [1, 2, 3]

回答 5

import itertools

foos=[1.0, 2.0, 3.0, 4.0, 5.0]
bars=[1, 2, 3]

print zip(foos, itertools.cycle([bars]))
import itertools

foos=[1.0, 2.0, 3.0, 4.0, 5.0]
bars=[1, 2, 3]

print zip(foos, itertools.cycle([bars]))

回答 6

以下是该map(function, *sequences)函数的参数概述:

  • function 是函数的名称。
  • sequences是任意数量的序列,通常是列表或元组。 map同时迭代它们并将当前值提供给function。这就是为什么序列数应等于函数的参数数的原因。

听起来您正在尝试迭代某些function参数,但保持其他参数不变,但是不幸的map是不支持该参数。我发现一个向Python添加此类功能的旧建议,但是map构造是如此干净且完善,以至于我怀疑这样的东西是否会实现。

像其他人建议的那样,使用诸如全局变量或列表理解之类的解决方法。

Here’s an overview of the parameters to the map(function, *sequences) function:

  • function is the name of your function.
  • sequences is any number of sequences, which are usually lists or tuples. map will iterate over them simultaneously and give the current values to function. That’s why the number of sequences should equal the number of parameters to your function.

It sounds like you’re trying to iterate for some of function‘s parameters but keep others constant, and unfortunately map doesn’t support that. I found an old proposal to add such a feature to Python, but the map construct is so clean and well-established that I doubt something like that will ever be implemented.

Use a workaround like global variables or list comprehensions, as others have suggested.


回答 7

这样可以吗?

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest2(bar):
  print bar

def maptest(foo):
  print foo
  map(maptest2, bars)

map(maptest, foos)

Would this do it?

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest2(bar):
  print bar

def maptest(foo):
  print foo
  map(maptest2, bars)

map(maptest, foos)

回答 8

这个怎么样:

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest(foo, bar):
    print foo, bar

map(maptest, foos, [bars]*len(foos))

How about this:

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest(foo, bar):
    print foo, bar

map(maptest, foos, [bars]*len(foos))

使用reduce()的有用代码?[关闭]

问题:使用reduce()的有用代码?[关闭]

这里有没有人有任何有用的代码在python中使用reduce()函数?除了示例中常见的+和*之外,是否还有其他代码?

通过GvR 引用Python 3000中的reduce()的命运

Does anyone here have any useful code which uses reduce() function in python? Is there any code other than the usual + and * that we see in the examples?

Refer Fate of reduce() in Python 3000 by GvR


回答 0

除+和*外,我为它找到的其他用途是与和和或,但现在我们有anyall来替换这些情况。

foldl并且foldr确实在Scheme中出现了很多…

这是一些可爱的用法:

整理清单

目标:[[1, 2, 3], [4, 5], [6, 7, 8]]变成[1, 2, 3, 4, 5, 6, 7, 8]

reduce(list.__add__, [[1, 2, 3], [4, 5], [6, 7, 8]], [])

数字列表到一个数字

目标:[1, 2, 3, 4, 5, 6, 7, 8]变成12345678

丑陋,缓慢的方式:

int("".join(map(str, [1,2,3,4,5,6,7,8])))

漂亮的reduce方式:

reduce(lambda a,d: 10*a+d, [1,2,3,4,5,6,7,8], 0)

The other uses I’ve found for it besides + and * were with and and or, but now we have any and all to replace those cases.

foldl and foldr do come up in Scheme a lot…

Here’s some cute usages:

Flatten a list

Goal: turn [[1, 2, 3], [4, 5], [6, 7, 8]] into [1, 2, 3, 4, 5, 6, 7, 8].

reduce(list.__add__, [[1, 2, 3], [4, 5], [6, 7, 8]], [])

List of digits to a number

Goal: turn [1, 2, 3, 4, 5, 6, 7, 8] into 12345678.

Ugly, slow way:

int("".join(map(str, [1,2,3,4,5,6,7,8])))

Pretty reduce way:

reduce(lambda a,d: 10*a+d, [1,2,3,4,5,6,7,8], 0)

回答 1

reduce()可用于查找3个或更多数字的最小公倍数

#!/usr/bin/env python
from fractions import gcd
from functools import reduce

def lcm(*args):
    return reduce(lambda a,b: a * b // gcd(a, b), args)

例:

>>> lcm(100, 23, 98)
112700
>>> lcm(*range(1, 20))
232792560

reduce() can be used to find Least common multiple for 3 or more numbers:

#!/usr/bin/env python
from fractions import gcd
from functools import reduce

def lcm(*args):
    return reduce(lambda a,b: a * b // gcd(a, b), args)

Example:

>>> lcm(100, 23, 98)
112700
>>> lcm(*range(1, 20))
232792560

回答 2

reduce()可以用来解析点名(eval()太不安全了,无法使用):

>>> import __main__
>>> reduce(getattr, "os.path.abspath".split('.'), __main__)
<function abspath at 0x009AB530>

reduce() could be used to resolve dotted names (where eval() is too unsafe to use):

>>> import __main__
>>> reduce(getattr, "os.path.abspath".split('.'), __main__)
<function abspath at 0x009AB530>

回答 3

找到N个给定列表的交集:

input_list = [[1, 2, 3, 4, 5], [2, 3, 4, 5, 6], [3, 4, 5, 6, 7]]

result = reduce(set.intersection, map(set, input_list))

返回:

result = set([3, 4, 5])

通过:Python-两个列表的交集

Find the intersection of N given lists:

input_list = [[1, 2, 3, 4, 5], [2, 3, 4, 5, 6], [3, 4, 5, 6, 7]]

result = reduce(set.intersection, map(set, input_list))

returns:

result = set([3, 4, 5])

via: Python – Intersection of two lists


回答 4

我认为reduce是一个愚蠢的命令。因此:

reduce(lambda hold,next:hold+chr(((ord(next.upper())-65)+13)%26+65),'znlorabggbbhfrshy','')

I think reduce is a silly command. Hence:

reduce(lambda hold,next:hold+chr(((ord(next.upper())-65)+13)%26+65),'znlorabggbbhfrshy','')

回答 5

reduce我在代码中发现的用法涉及以下情况:我具有一些用于逻辑表达式的类结构,因此需要将这些表达式对象的列表转换为表达式的并集。我已经有了一个make_and给定两个表达式的连接词创建函数,因此我写了reduce(make_and,l)。(我知道列表不为空;否则它将是类似reduce(make_and,l,make_true)。)

这正是(某些)函数式程序员喜欢reduce(或折叠函数,通常称为此类函数)的原因。经常有已经有很多二元函数喜欢+*minmax,级联和,在我的情况,make_andmake_or。有一个reduce它,将这些操作提升到列表(或一般来说是折叠功能的树或其他任何东西)变得微不足道。

当然,如果sum经常使用某些实例化(例如),那么您就不想继续写作reduce。但是,除了sum使用某些for循环定义之外,您还可以使用轻松定义它reduce

正如其他人提到的,可读性确实是一个问题。但是,您可能会争辩说,人们发现reduce“清晰”较少的唯一原因是因为它不是许多人知道和/或使用的功能。

The usage of reduce that I found in my code involved the situation where I had some class structure for logic expression and I needed to convert a list of these expression objects to a conjunction of the expressions. I already had a function make_and to create a conjunction given two expressions, so I wrote reduce(make_and,l). (I knew the list wasn’t empty; otherwise it would have been something like reduce(make_and,l,make_true).)

This is exactly the reason that (some) functional programmers like reduce (or fold functions, as such functions are typically called). There are often already many binary functions like +, *, min, max, concatenation and, in my case, make_and and make_or. Having a reduce makes it trivial to lift these operations to lists (or trees or whatever you got, for fold functions in general).

Of course, if certain instantiations (such as sum) are often used, then you don’t want to keep writing reduce. However, instead of defining the sum with some for-loop, you can just as easily define it with reduce.

Readability, as mentioned by others, is indeed an issue. You could argue, however, that only reason why people find reduce less “clear” is because it is not a function that many people know and/or use.


回答 6

函数组成:如果您已经有了要连续应用的函数列表,例如:

color = lambda x: x.replace('brown', 'blue')
speed = lambda x: x.replace('quick', 'slow')
work = lambda x: x.replace('lazy', 'industrious')
fs = [str.lower, color, speed, work, str.title]

然后,您可以使用以下命令连续应用它们:

>>> call = lambda s, func: func(s)
>>> s = "The Quick Brown Fox Jumps Over the Lazy Dog"
>>> reduce(call, fs, s)
'The Slow Blue Fox Jumps Over The Industrious Dog'

在这种情况下,方法链接可能更具可读性。但是有时这是不可能的,并且这种组合可能比f1(f2(f3(f4(x))))某种语法更易读和可维护。

Function composition: If you already have a list of functions that you’d like to apply in succession, such as:

color = lambda x: x.replace('brown', 'blue')
speed = lambda x: x.replace('quick', 'slow')
work = lambda x: x.replace('lazy', 'industrious')
fs = [str.lower, color, speed, work, str.title]

Then you can apply them all consecutively with:

>>> call = lambda s, func: func(s)
>>> s = "The Quick Brown Fox Jumps Over the Lazy Dog"
>>> reduce(call, fs, s)
'The Slow Blue Fox Jumps Over The Industrious Dog'

In this case, method chaining may be more readable. But sometimes it isn’t possible, and this kind of composition may be more readable and maintainable than a f1(f2(f3(f4(x)))) kind of syntax.


回答 7

您可以替换value = json_obj['a']['b']['c']['d']['e']为:

value = reduce(dict.__getitem__, 'abcde', json_obj)

如果您已经将路径a/b/c/..作为列表。例如,使用list中的项目更改嵌套字典的dict中的值

You could replace value = json_obj['a']['b']['c']['d']['e'] with:

value = reduce(dict.__getitem__, 'abcde', json_obj)

If you already have the path a/b/c/.. as a list. For example, Change values in dict of nested dicts using items in a list.


回答 8

@Blair Conrad:您也可以使用sum来实现glob / reduce,如下所示:

files = sum([glob.glob(f) for f in args], [])

这比您的两个示例中的任何一个都不那么冗长,完全是Python风格的,并且仍然只是一行代码。

因此,为了回答最初的问题,我个人尝试避免使用reduce,因为它从来没有真正需要过,而且我发现它比其他方法不太清楚。但是,有些人习惯于减少并开始喜欢它来列出理解力(尤其是Haskell程序员)。但是,如果您还没有考虑过reduce的问题,那么您可能不必担心使用它。

@Blair Conrad: You could also implement your glob/reduce using sum, like so:

files = sum([glob.glob(f) for f in args], [])

This is less verbose than either of your two examples, is perfectly Pythonic, and is still only one line of code.

So to answer the original question, I personally try to avoid using reduce because it’s never really necessary and I find it to be less clear than other approaches. However, some people get used to reduce and come to prefer it to list comprehensions (especially Haskell programmers). But if you’re not already thinking about a problem in terms of reduce, you probably don’t need to worry about using it.


回答 9

reduce 可用于支持链式属性查找:

reduce(getattr, ('request', 'user', 'email'), self)

当然,这相当于

self.request.user.email

但是在代码需要接受任意属性列表时很有用。

(在处理Django模型时,任意长度的链接属性是常见的。)

reduce can be used to support chained attribute lookups:

reduce(getattr, ('request', 'user', 'email'), self)

Of course, this is equivalent to

self.request.user.email

but it’s useful when your code needs to accept an arbitrary list of attributes.

(Chained attributes of arbitrary length are common when dealing with Django models.)


回答 10

reduce当您需要查找类似set对象序列的并集或交集时,此功能很有用。

>>> reduce(operator.or_, ({1}, {1, 2}, {1, 3}))  # union
{1, 2, 3}
>>> reduce(operator.and_, ({1}, {1, 2}, {1, 3}))  # intersection
{1}

(除了实际set的,其中的一个示例是Django的Q对象。)

另一方面,如果要处理bools,则应使用anyall

>>> any((True, False, True))
True

reduce is useful when you need to find the union or intersection of a sequence of set-like objects.

>>> reduce(operator.or_, ({1}, {1, 2}, {1, 3}))  # union
{1, 2, 3}
>>> reduce(operator.and_, ({1}, {1, 2}, {1, 3}))  # intersection
{1}

(Apart from actual sets, an example of these are Django’s Q objects.)

On the other hand, if you’re dealing with bools, you should use any and all:

>>> any((True, False, True))
True

回答 11

重复我的代码后,看来我使用过的reduce唯一要做的就是计算阶乘:

reduce(operator.mul, xrange(1, x+1) or (1,))

After grepping my code, it seems the only thing I’ve used reduce for is calculating the factorial:

reduce(operator.mul, xrange(1, x+1) or (1,))

回答 12

我正在为一种语言编写一个compose函数,因此我使用reduce和我的apply运算符来构造该组合函数。

简而言之,compose将一系列函数组合成一个函数。如果我有一个分阶段应用的复杂操作,那么我希望将其全部组合起来,如下所示:

complexop = compose(stage4, stage3, stage2, stage1)

这样,我便可以将其应用于这样的表达式:

complexop(expression)

我希望它等同于:

stage4(stage3(stage2(stage1(expression))))

现在,要构建内部对象,我希望它说:

Lambda([Symbol('x')], Apply(stage4, Apply(stage3, Apply(stage2, Apply(stage1, Symbol('x'))))))

(Lambda类构建用户定义的函数,Apply构建函数应用程序。)

现在,不幸的是,reduce折叠的方向错误,所以我大致使用了:

reduce(lambda x,y: Apply(y, x), reversed(args + [Symbol('x')]))

要弄清楚reduce产生了什么,请在REPL中尝试以下方法:

reduce(lambda x, y: (x, y), range(1, 11))
reduce(lambda x, y: (y, x), reversed(range(1, 11)))

I’m writing a compose function for a language, so I construct the composed function using reduce along with my apply operator.

In a nutshell, compose takes a list of functions to compose into a single function. If I have a complex operation that is applied in stages, I want to put it all together like so:

complexop = compose(stage4, stage3, stage2, stage1)

This way, I can then apply it to an expression like so:

complexop(expression)

And I want it to be equivalent to:

stage4(stage3(stage2(stage1(expression))))

Now, to build my internal objects, I want it to say:

Lambda([Symbol('x')], Apply(stage4, Apply(stage3, Apply(stage2, Apply(stage1, Symbol('x'))))))

(The Lambda class builds a user-defined function, and Apply builds a function application.)

Now, reduce, unfortunately, folds the wrong way, so I wound up using, roughly:

reduce(lambda x,y: Apply(y, x), reversed(args + [Symbol('x')]))

To figure out what reduce produces, try these in the REPL:

reduce(lambda x, y: (x, y), range(1, 11))
reduce(lambda x, y: (y, x), reversed(range(1, 11)))

回答 13

reduce可以用来获取第n个元素最大的列表

reduce(lambda x,y: x if x[2] > y[2] else y,[[1,2,3,4],[5,2,5,7],[1,6,0,2]])

将返回[5、2、5、7],因为它是具有最大3rd元素的列表+

reduce can be used to get the list with the maximum nth element

reduce(lambda x,y: x if x[2] > y[2] else y,[[1,2,3,4],[5,2,5,7],[1,6,0,2]])

would return [5, 2, 5, 7] as it is the list with max 3rd element +


回答 14

Reduce不仅限于标量运算;它也可以用于将事物分类到存储桶中。(这是我最常使用的减少方法)。

想象一下,如果您有一个对象列表,并且想根据对象中平面存储的属性按层次结构对其进行重新组织。在以下示例中,我使用该articles功能生成了与XML编码报纸中的文章相关的元数据对象列表。articles生成一个XML元素列表,然后一个一个地映射它们,生成包含一些有趣信息的对象。在前端,我要让用户按节/小节/标题浏览文章。因此,我通常使用reduce文章列表,并返回一个反映章节/小节/文章层次结构的字典。

from lxml import etree
from Reader import Reader

class IssueReader(Reader):
    def articles(self):
        arts = self.q('//div3')  # inherited ... runs an xpath query against the issue
        subsection = etree.XPath('./ancestor::div2/@type')
        section = etree.XPath('./ancestor::div1/@type')
        header_text = etree.XPath('./head//text()')
        return map(lambda art: {
            'text_id': self.id,
            'path': self.getpath(art)[0],
            'subsection': (subsection(art)[0] or '[none]'),
            'section': (section(art)[0] or '[none]'),
            'headline': (''.join(header_text(art)) or '[none]')
        }, arts)

    def by_section(self):
        arts = self.articles()

        def extract(acc, art):  # acc for accumulator
            section = acc.get(art['section'], False)
            if section:
                subsection = acc.get(art['subsection'], False)
                if subsection:
                    subsection.append(art)
                else:
                    section[art['subsection']] = [art]
            else:
                acc[art['section']] = {art['subsection']: [art]}
            return acc

        return reduce(extract, arts, {})

我在这里给出两个函数,因为我认为它显示了map和reduce在处理对象时如何很好地互补。使用for循环可以完成相同的事情,但是,……花一些严肃的时间使用函数式语言往往会使我在映射和归约方面进行思考。

顺便说一句,如果有人像我在中所做的那样设置属性的更好方法extract,而您要设置的属性的父级可能还不存在,请告诉我。

Reduce isn’t limited to scalar operations; it can also be used to sort things into buckets. (This is what I use reduce for most often).

Imagine a case in which you have a list of objects, and you want to re-organize it hierarchically based on properties stored flatly in the object. In the following example, I produce a list of metadata objects related to articles in an XML-encoded newspaper with the articles function. articles generates a list of XML elements, and then maps through them one by one, producing objects that hold some interesting info about them. On the front end, I’m going to want to let the user browse the articles by section/subsection/headline. So I use reduce to take the list of articles and return a single dictionary that reflects the section/subsection/article hierarchy.

from lxml import etree
from Reader import Reader

class IssueReader(Reader):
    def articles(self):
        arts = self.q('//div3')  # inherited ... runs an xpath query against the issue
        subsection = etree.XPath('./ancestor::div2/@type')
        section = etree.XPath('./ancestor::div1/@type')
        header_text = etree.XPath('./head//text()')
        return map(lambda art: {
            'text_id': self.id,
            'path': self.getpath(art)[0],
            'subsection': (subsection(art)[0] or '[none]'),
            'section': (section(art)[0] or '[none]'),
            'headline': (''.join(header_text(art)) or '[none]')
        }, arts)

    def by_section(self):
        arts = self.articles()

        def extract(acc, art):  # acc for accumulator
            section = acc.get(art['section'], False)
            if section:
                subsection = acc.get(art['subsection'], False)
                if subsection:
                    subsection.append(art)
                else:
                    section[art['subsection']] = [art]
            else:
                acc[art['section']] = {art['subsection']: [art]}
            return acc

        return reduce(extract, arts, {})

I give both functions here because I think it shows how map and reduce can complement each other nicely when dealing with objects. The same thing could have been accomplished with a for loop, … but spending some serious time with a functional language has tended to make me think in terms of map and reduce.

By the way, if anybody has a better way to set properties like I’m doing in extract, where the parents of the property you want to set might not exist yet, please let me know.


回答 15

不知道这是您要追求的,但是您可以在Google上搜索源代码

点击链接以搜索“ function:reduce()lang:python”在Google Code搜索上

乍看之下,以下项目使用 reduce()

  • MoinMoin
  • 佐佩
  • 数字
  • 科学Python

等),但由于它们是大型项目,因此这些不足为奇。

reduce的功能可以使用函数递归来完成,我想Guido认为它更明确。

更新:

由于Google的代码搜索已于2012年1月15日停产,因此除了恢复常规的Google搜索外,还有一个名为“代码片段集”的代码看起来很有希望。这个(封闭的)问题的答案中提到了许多其他资源。是否可以替换Google Code Search?

更新2(2017年5月29日):

Nullege搜索引擎是Python示例(使用开源代码)的一个很好的来源。

Not sure if this is what you are after but you can search source code on Google.

Follow the link for a search on ‘function:reduce() lang:python’ on Google Code search

At first glance the following projects use reduce()

  • MoinMoin
  • Zope
  • Numeric
  • ScientificPython

etc. etc. but then these are hardly surprising since they are huge projects.

The functionality of reduce can be done using function recursion which I guess Guido thought was more explicit.

Update:

Since Google’s Code Search was discontinued on 15-Jan-2012, besides reverting to regular Google searches, there’s something called Code Snippets Collection that looks promising. A number of other resources are mentioned in answers this (closed) question Replacement for Google Code Search?.

Update 2 (29-May-2017):

A good source for Python examples (in open-source code) is the Nullege search engine.


回答 16

import os

files = [
    # full filenames
    "var/log/apache/errors.log",
    "home/kane/images/avatars/crusader.png",
    "home/jane/documents/diary.txt",
    "home/kane/images/selfie.jpg",
    "var/log/abc.txt",
    "home/kane/.vimrc",
    "home/kane/images/avatars/paladin.png",
]

# unfolding of plain filiname list to file-tree
fs_tree = ({}, # dict of folders
           []) # list of files
for full_name in files:
    path, fn = os.path.split(full_name)
    reduce(
        # this fucction walks deep into path
        # and creates placeholders for subfolders
        lambda d, k: d[0].setdefault(k,         # walk deep
                                     ({}, [])), # or create subfolder storage
        path.split(os.path.sep),
        fs_tree
    )[1].append(fn)

print fs_tree
#({'home': (
#    {'jane': (
#        {'documents': (
#           {},
#           ['diary.txt']
#        )},
#        []
#    ),
#    'kane': (
#       {'images': (
#          {'avatars': (
#             {},
#             ['crusader.png',
#             'paladin.png']
#          )},
#          ['selfie.jpg']
#       )},
#       ['.vimrc']
#    )},
#    []
#  ),
#  'var': (
#     {'log': (
#         {'apache': (
#            {},
#            ['errors.log']
#         )},
#         ['abc.txt']
#     )},
#     [])
#},
#[])
import os

files = [
    # full filenames
    "var/log/apache/errors.log",
    "home/kane/images/avatars/crusader.png",
    "home/jane/documents/diary.txt",
    "home/kane/images/selfie.jpg",
    "var/log/abc.txt",
    "home/kane/.vimrc",
    "home/kane/images/avatars/paladin.png",
]

# unfolding of plain filiname list to file-tree
fs_tree = ({}, # dict of folders
           []) # list of files
for full_name in files:
    path, fn = os.path.split(full_name)
    reduce(
        # this fucction walks deep into path
        # and creates placeholders for subfolders
        lambda d, k: d[0].setdefault(k,         # walk deep
                                     ({}, [])), # or create subfolder storage
        path.split(os.path.sep),
        fs_tree
    )[1].append(fn)

print fs_tree
#({'home': (
#    {'jane': (
#        {'documents': (
#           {},
#           ['diary.txt']
#        )},
#        []
#    ),
#    'kane': (
#       {'images': (
#          {'avatars': (
#             {},
#             ['crusader.png',
#             'paladin.png']
#          )},
#          ['selfie.jpg']
#       )},
#       ['.vimrc']
#    )},
#    []
#  ),
#  'var': (
#     {'log': (
#         {'apache': (
#            {},
#            ['errors.log']
#         )},
#         ['abc.txt']
#     )},
#     [])
#},
#[])

回答 17

def dump(fname,iterable):
  with open(fname,'w') as f:
    reduce(lambda x, y: f.write(unicode(y,'utf-8')), iterable)
def dump(fname,iterable):
  with open(fname,'w') as f:
    reduce(lambda x, y: f.write(unicode(y,'utf-8')), iterable)

回答 18

我曾经用sqlalchemy-searchable中的运算符reduce 来连接PostgreSQL搜索向量列表||

vectors = (self.column_vector(getattr(self.table.c, column_name))
           for column_name in self.indexed_columns)
concatenated = reduce(lambda x, y: x.op('||')(y), vectors)
compiled = concatenated.compile(self.conn)

I used reduce to concatenate a list of PostgreSQL search vectors with the || operator in sqlalchemy-searchable:

vectors = (self.column_vector(getattr(self.table.c, column_name))
           for column_name in self.indexed_columns)
concatenated = reduce(lambda x, y: x.op('||')(y), vectors)
compiled = concatenated.compile(self.conn)

回答 19

我有一个老式的pipegrep Python实现,该实现使用reduce和glob模块来构建要处理的文件列表:

files = []
files.extend(reduce(lambda x, y: x + y, map(glob.glob, args)))

当时我觉得很方便,但实际上没有必要,因为类似的东西一样好,而且可读性更高

files = []
for f in args:
    files.extend(glob.glob(f))

I have an old Python implementation of pipegrep that uses reduce and the glob module to build a list of files to process:

files = []
files.extend(reduce(lambda x, y: x + y, map(glob.glob, args)))

I found it handy at the time, but it’s really not necessary, as something similar is just as good, and probably more readable

files = []
for f in args:
    files.extend(glob.glob(f))

回答 20

假设有一些年度统计数据存储在“计数器”列表中。我们想要查找不同年份中每个月的MIN / MAX值。例如,对于1月将是10。对于2月将是15。我们需要将结果存储在新的Counter中。

from collections import Counter

stat2011 = Counter({"January": 12, "February": 20, "March": 50, "April": 70, "May": 15,
           "June": 35, "July": 30, "August": 15, "September": 20, "October": 60,
           "November": 13, "December": 50})

stat2012 = Counter({"January": 36, "February": 15, "March": 50, "April": 10, "May": 90,
           "June": 25, "July": 35, "August": 15, "September": 20, "October": 30,
           "November": 10, "December": 25})

stat2013 = Counter({"January": 10, "February": 60, "March": 90, "April": 10, "May": 80,
           "June": 50, "July": 30, "August": 15, "September": 20, "October": 75,
           "November": 60, "December": 15})

stat_list = [stat2011, stat2012, stat2013]

print reduce(lambda x, y: x & y, stat_list)     # MIN
print reduce(lambda x, y: x | y, stat_list)     # MAX

Let say that there are some yearly statistic data stored a list of Counters. We want to find the MIN/MAX values in each month across the different years. For example, for January it would be 10. And for February it would be 15. We need to store the results in a new Counter.

from collections import Counter

stat2011 = Counter({"January": 12, "February": 20, "March": 50, "April": 70, "May": 15,
           "June": 35, "July": 30, "August": 15, "September": 20, "October": 60,
           "November": 13, "December": 50})

stat2012 = Counter({"January": 36, "February": 15, "March": 50, "April": 10, "May": 90,
           "June": 25, "July": 35, "August": 15, "September": 20, "October": 30,
           "November": 10, "December": 25})

stat2013 = Counter({"January": 10, "February": 60, "March": 90, "April": 10, "May": 80,
           "June": 50, "July": 30, "August": 15, "September": 20, "October": 75,
           "November": 60, "December": 15})

stat_list = [stat2011, stat2012, stat2013]

print reduce(lambda x, y: x & y, stat_list)     # MIN
print reduce(lambda x, y: x | y, stat_list)     # MAX

回答 21

我有代表某种重叠区间(基因组外显子)的对象,并使用__and__以下方法重新定义了它们的交集:

class Exon:
    def __init__(self):
        ...
    def __and__(self,other):
        ...
        length = self.length + other.length  # (e.g.)
        return self.__class__(...length,...)

然后,当我有它们的集合(例如,在同一个基因中)时,我使用

intersection = reduce(lambda x,y: x&y, exons)

I have objects representing some kind of overlapping intervals (genomic exons), and redefined their intersection using __and__:

class Exon:
    def __init__(self):
        ...
    def __and__(self,other):
        ...
        length = self.length + other.length  # (e.g.)
        return self.__class__(...length,...)

Then when I have a collection of them (for instance, in the same gene), I use

intersection = reduce(lambda x,y: x&y, exons)

回答 22

我只是发现了有用的用法reduce:在不删除定界符的情况下拆分字符串该代码完全来自“编程口语”博客。这是代码:

reduce(lambda acc, elem: acc[:-1] + [acc[-1] + elem] if elem == "\n" else acc + [elem], re.split("(\n)", "a\nb\nc\n"), [])

结果如下:

['a\n', 'b\n', 'c\n', '']

请注意,它处理的是在SO中无法普遍解决的极端情况。有关更深入的解释,我将您重定向到原始博客文章。

I just found useful usage of reduce: splitting string without removing the delimiter. The code is entirely from Programatically Speaking blog. Here’s the code:

reduce(lambda acc, elem: acc[:-1] + [acc[-1] + elem] if elem == "\n" else acc + [elem], re.split("(\n)", "a\nb\nc\n"), [])

Here’s the result:

['a\n', 'b\n', 'c\n', '']

Note that it handles edge cases that popular answer in SO doesn’t. For more in-depth explanation, I am redirecting you to original blog post.


回答 23

使用reduce()来确定日期列表是否连续:

from datetime import date, timedelta


def checked(d1, d2):
    """
    We assume the date list is sorted.
    If d2 & d1 are different by 1, everything up to d2 is consecutive, so d2
    can advance to the next reduction.
    If d2 & d1 are not different by 1, returning d1 - 1 for the next reduction
    will guarantee the result produced by reduce() to be something other than
    the last date in the sorted date list.

    Definition 1: 1/1/14, 1/2/14, 1/2/14, 1/3/14 is consider consecutive
    Definition 2: 1/1/14, 1/2/14, 1/2/14, 1/3/14 is consider not consecutive

    """
    #if (d2 - d1).days == 1 or (d2 - d1).days == 0:  # for Definition 1
    if (d2 - d1).days == 1:                          # for Definition 2
        return d2
    else:
        return d1 + timedelta(days=-1)

# datelist = [date(2014, 1, 1), date(2014, 1, 3),
#             date(2013, 12, 31), date(2013, 12, 30)]

# datelist = [date(2014, 2, 19), date(2014, 2, 19), date(2014, 2, 20),
#             date(2014, 2, 21), date(2014, 2, 22)]

datelist = [date(2014, 2, 19), date(2014, 2, 21),
            date(2014, 2, 22), date(2014, 2, 20)]

datelist.sort()

if datelist[-1] == reduce(checked, datelist):
    print "dates are consecutive"
else:
    print "dates are not consecutive"

Using reduce() to find out if a list of dates are consecutive:

from datetime import date, timedelta


def checked(d1, d2):
    """
    We assume the date list is sorted.
    If d2 & d1 are different by 1, everything up to d2 is consecutive, so d2
    can advance to the next reduction.
    If d2 & d1 are not different by 1, returning d1 - 1 for the next reduction
    will guarantee the result produced by reduce() to be something other than
    the last date in the sorted date list.

    Definition 1: 1/1/14, 1/2/14, 1/2/14, 1/3/14 is consider consecutive
    Definition 2: 1/1/14, 1/2/14, 1/2/14, 1/3/14 is consider not consecutive

    """
    #if (d2 - d1).days == 1 or (d2 - d1).days == 0:  # for Definition 1
    if (d2 - d1).days == 1:                          # for Definition 2
        return d2
    else:
        return d1 + timedelta(days=-1)

# datelist = [date(2014, 1, 1), date(2014, 1, 3),
#             date(2013, 12, 31), date(2013, 12, 30)]

# datelist = [date(2014, 2, 19), date(2014, 2, 19), date(2014, 2, 20),
#             date(2014, 2, 21), date(2014, 2, 22)]

datelist = [date(2014, 2, 19), date(2014, 2, 21),
            date(2014, 2, 22), date(2014, 2, 20)]

datelist.sort()

if datelist[-1] == reduce(checked, datelist):
    print "dates are consecutive"
else:
    print "dates are not consecutive"

函数式编程中的“ pythonic”等同于“ fold”函数是什么?

问题:函数式编程中的“ pythonic”等同于“ fold”函数是什么?

在Haskell中,实现以下目标的最惯用的方法是:

foldl (+) 0 [1,2,3,4,5]
--> 15

或等效的Ruby:

[1,2,3,4,5].inject(0) {|m,x| m + x}
#> 15

显然,Python提供了reduce与fold完全相同的功能,但实际上是如上所述的fold的实现,但是,有人告诉我,“ pythonic”编程方式是避免使用lambda术语和高阶函数,并尽可能使用列表理解。因此,有没有一种首选的方式来折叠列表或不是Python reduce函数的类似列表的结构,或者是reduce惯用的方式来实现此目的?

What is the most idiomatic way to achieve something like the following, in Haskell:

foldl (+) 0 [1,2,3,4,5]
--> 15

Or its equivalent in Ruby:

[1,2,3,4,5].inject(0) {|m,x| m + x}
#> 15

Obviously, Python provides the reduce function, which is an implementation of fold, exactly as above, however, I was told that the ‘pythonic’ way of programming was to avoid lambda terms and higher-order functions, preferring list-comprehensions where possible. Therefore, is there a preferred way of folding a list, or list-like structure in Python that isn’t the reduce function, or is reduce the idiomatic way of achieving this?


回答 0

用Python方式对数组求和的方法是使用sum。为了其他目的,有时可以使用reduce(来自functools模块)和operator模块的某种组合,例如:

def product(xs):
    return reduce(operator.mul, xs, 1)

请注意,reduce实际上这是一个foldl用Haskell表示的。没有执行折叠的特殊语法,没有内置函数foldr,实际上reduce与非关联运算符一起使用被认为是不良样式。

使用高阶函数是相当Python的;它很好地利用了Python的原理,即一切都是对象,包括函数和类。没错,lambda被某些Pythonista所反对,但这主要是因为它们变得复杂时往往不太可读。

The Pythonic way of summing an array is using sum. For other purposes, you can sometimes use some combination of reduce (from the functools module) and the operator module, e.g.:

def product(xs):
    return reduce(operator.mul, xs, 1)

Be aware that reduce is actually a foldl, in Haskell terms. There is no special syntax to perform folds, there’s no builtin foldr, and actually using reduce with non-associative operators is considered bad style.

Using higher-order functions is quite pythonic; it makes good use of Python’s principle that everything is an object, including functions and classes. You are right that lambdas are frowned upon by some Pythonistas, but mostly because they tend not to be very readable when they get complex.


回答 1

哈斯克尔

foldl (+) 0 [1,2,3,4,5]

Python

reduce(lambda a,b: a+b, [1,2,3,4,5], 0)

显然,这是一个说明问题的简单例子。在Python中,您只需要这样做sum([1,2,3,4,5]),甚至Haskell纯粹主义者通常也会更喜欢sum [1,2,3,4,5]

对于没有明显便利函数的非平凡场景,惯用的pythonic方法是显式写出for循环并使用可变变量分配,而不是使用reduceor fold

那根本不是功能样式,而是“ pythonic”方式。Python并非为功能纯正者设计。了解Python如何支持流控制的异常,以了解非功能性惯用python的情况。

Haskell

foldl (+) 0 [1,2,3,4,5]

Python

reduce(lambda a,b: a+b, [1,2,3,4,5], 0)

Obviously, that is a trivial example to illustrate a point. In Python you would just do sum([1,2,3,4,5]) and even Haskell purists would generally prefer sum [1,2,3,4,5].

For non-trivial scenarios when there is no obvious convenience function, the idiomatic pythonic approach is to explicitly write out the for loop and use mutable variable assignment instead of using reduce or a fold.

That is not at all the functional style, but that is the “pythonic” way. Python is not designed for functional purists. See how Python favors exceptions for flow control to see how non-functional idiomatic python is.


回答 2

在Python 3中,reduce已被删除:版本说明。不过,您可以使用functools模块

import operator, functools
def product(xs):
    return functools.reduce(operator.mul, xs, 1)

另一方面,文档表达了对for-loop而不是的偏好reduce,因此:

def product(xs):
    result = 1
    for i in xs:
        result *= i
    return result

In Python 3, the reduce has been removed: Release notes. Nevertheless you can use the functools module

import operator, functools
def product(xs):
    return functools.reduce(operator.mul, xs, 1)

On the other hand, the documentation expresses preference towards for-loop instead of reduce, hence:

def product(xs):
    result = 1
    for i in xs:
        result *= i
    return result

回答 3

您也可以重新发明轮子:

def fold(f, l, a):
    """
    f: the function to apply
    l: the list to fold
    a: the accumulator, who is also the 'zero' on the first call
    """ 
    return a if(len(l) == 0) else fold(f, l[1:], f(a, l[0]))

print "Sum:", fold(lambda x, y : x+y, [1,2,3,4,5], 0)

print "Any:", fold(lambda x, y : x or y, [False, True, False], False)

print "All:", fold(lambda x, y : x and y, [False, True, False], True)

# Prove that result can be of a different type of the list's elements
print "Count(x==True):", 
print fold(lambda x, y : x+1 if(y) else x, [False, True, True], 0)

You can reinvent the wheel as well:

def fold(f, l, a):
    """
    f: the function to apply
    l: the list to fold
    a: the accumulator, who is also the 'zero' on the first call
    """ 
    return a if(len(l) == 0) else fold(f, l[1:], f(a, l[0]))

print "Sum:", fold(lambda x, y : x+y, [1,2,3,4,5], 0)

print "Any:", fold(lambda x, y : x or y, [False, True, False], False)

print "All:", fold(lambda x, y : x and y, [False, True, False], True)

# Prove that result can be of a different type of the list's elements
print "Count(x==True):", 
print fold(lambda x, y : x+1 if(y) else x, [False, True, True], 0)

回答 4

并不是真正回答这个问题,而是折叠和折叠的一线:

a = [8,3,4]

## Foldl
reduce(lambda x,y: x**y, a)
#68719476736

## Foldr
reduce(lambda x,y: y**x, a[::-1])
#14134776518227074636666380005943348126619871175004951664972849610340958208L

Not really answer to the question, but one-liners for foldl and foldr:

a = [8,3,4]

## Foldl
reduce(lambda x,y: x**y, a)
#68719476736

## Foldr
reduce(lambda x,y: y**x, a[::-1])
#14134776518227074636666380005943348126619871175004951664972849610340958208L

回答 5

开始Python 3.8并引入赋值表达式(PEP 572):=运算符),这使您可以为表达式的结果命名,我们可以使用列表推导来复制其他语言称为fold / foldleft / reduce的操作:

给定一个列表,一个约简函数和一个累加器:

items = [1, 2, 3, 4, 5]
f = lambda acc, x: acc * x
accumulator = 1

我们可以折叠itemsf在为了获得所得accumulation

[accumulator := f(accumulator, x) for x in items]
# accumulator = 120

或以压缩形式形成:

acc = 1; [acc := acc * x for x in [1, 2, 3, 4, 5]]
# acc = 120

请注意,这实际上也是“ scanleft”操作,因为列表理解的结果表示每个步骤的累加状态:

acc = 1
scanned = [acc := acc * x for x in [1, 2, 3, 4, 5]]
# scanned = [1, 2, 6, 24, 120]
# acc = 120

Starting Python 3.8, and the introduction of assignment expressions (PEP 572) (:= operator), which gives the possibility to name the result of an expression, we can use a list comprehension to replicate what other languages call fold/foldleft/reduce operations:

Given a list, a reducing function and an accumulator:

items = [1, 2, 3, 4, 5]
f = lambda acc, x: acc * x
accumulator = 1

we can fold items with f in order to obtain the resulting accumulation:

[accumulator := f(accumulator, x) for x in items]
# accumulator = 120

or in a condensed formed:

acc = 1; [acc := acc * x for x in [1, 2, 3, 4, 5]]
# acc = 120

Note that this is actually also a “scanleft” operation as the result of the list comprehension represents the state of the accumulation at each step:

acc = 1
scanned = [acc := acc * x for x in [1, 2, 3, 4, 5]]
# scanned = [1, 2, 6, 24, 120]
# acc = 120

回答 6

对这个(减少)问题的实际答案是:只需使用一个循环!

initial_value = 0
for x in the_list:
    initial_value += x #or any function.

这将比reduce更快,而PyPy之类的东西可以优化循环。

顺便说一句,总和的情况应该用sum函数来解决

The actual answer to this (reduce) problem is: Just use a loop!

initial_value = 0
for x in the_list:
    initial_value += x #or any function.

This will be faster than a reduce and things like PyPy can optimize loops like that.

BTW, the sum case should be solved with the sum function


回答 7

我相信这个问题的一些回答者错过了该fold功能作为抽象工具的广泛含义。是的,sum可以对整数列表执行相同的操作,但这是不重要的情况。fold更通用。当您具有一系列形状各异的数据结构并想要清晰地表达聚合时,此功能很有用。因此,不必for每次都用一个聚合变量建立一个循环并手动重新计算它,fold函数(或reduce似乎对应的Python版本)允许程序员通过简单地提供以下内容来更清楚地表达聚合的意图:两件事情:

  • 聚合的默认起始值​​或“种子”值。
  • 该函数采用聚合的当前值(以“种子”开头)和列表中的下一个元素,并返回下一个聚合值。

I believe some of the respondents of this question have missed the broader implication of the fold function as an abstract tool. Yes, sum can do the same thing for a list of integers, but this is a trivial case. fold is more generic. It is useful when you have a sequence of data structures of varying shape and want to cleanly express an aggregation. So instead of having to build up a for loop with an aggregate variable and manually recompute it each time, a fold function (or the Python version, which reduce appears to correspond to) allows the programmer to express the intent of the aggregation much more plainly by simply providing two things:

  • A default starting or “seed” value for the aggregation.
  • A function that takes the current value of the aggregation (starting with the “seed”) and the next element in the list, and returns the next aggregation value.

回答 8

我参加聚会可能已经很晚了,但是我们可以foldr使用简单的lambda演算和curried函数创建自定义项。这是我在python中实现的foldr。

def foldr(func):
    def accumulator(acc):
        def listFunc(l):
            if l:
                x = l[0]
                xs = l[1:]
                return func(x)(foldr(func)(acc)(xs))
            else:
                return acc
        return listFunc
    return accumulator  


def curried_add(x):
    def inner(y):
        return x + y
    return inner

def curried_mult(x):
    def inner(y):
        return x * y
    return inner

print foldr(curried_add)(0)(range(1, 6))
print foldr(curried_mult)(1)(range(1, 6))

即使实现是递归的(可能很慢),它也会分别打印值15120

I may be quite late to the party, but we can create custom foldr using simple lambda calculus and curried function. Here is my implementation of foldr in python.

def foldr(func):
    def accumulator(acc):
        def listFunc(l):
            if l:
                x = l[0]
                xs = l[1:]
                return func(x)(foldr(func)(acc)(xs))
            else:
                return acc
        return listFunc
    return accumulator  


def curried_add(x):
    def inner(y):
        return x + y
    return inner

def curried_mult(x):
    def inner(y):
        return x * y
    return inner

print foldr(curried_add)(0)(range(1, 6))
print foldr(curried_mult)(1)(range(1, 6))

Even though the implementation is recursive (might be slow), it will print the values 15 and 120 respectively


Python:为什么functools.partial是必需的?

问题:Python:为什么functools.partial是必需的?

部分应用程序很酷。哪些功能functools.partial提供了lambda无法获得的功能?

>>> sum = lambda x, y : x + y
>>> sum(1, 2)
3
>>> incr = lambda y : sum(1, y)
>>> incr(2)
3
>>> def sum2(x, y):
    return x + y

>>> incr2 = functools.partial(sum2, 1)
>>> incr2(4)
5

functools某种方式更有效或更可读吗?

Partial application is cool. What functionality does functools.partial offer that you can’t get through lambdas?

>>> sum = lambda x, y : x + y
>>> sum(1, 2)
3
>>> incr = lambda y : sum(1, y)
>>> incr(2)
3
>>> def sum2(x, y):
    return x + y

>>> incr2 = functools.partial(sum2, 1)
>>> incr2(4)
5

Is functools somehow more efficient, or readable?


回答 0

哪些功能functools.partial提供了lambda无法获得的功能?

在额外功能方面并没有太多(但是,请参阅稍后)–旁观者眼中的可读性。
大多数熟悉函数式编程语言的人(尤其是Lisp / Scheme系列的人)看起来都lambda很好–我说“大多数”,绝对不是全部,因为Guido和我肯定是“熟悉”的人(等) 却被认为是lambdaPython中的一种令人眼花an乱的异常……
他为曾经接受过Python而打算将其从Python 3中删除(作为“ Python的小故障”之一)感到re悔。
我对此表示完全支持。(我喜欢lambda Scheme,但是它在Python中有局限性,而且它只是奇怪的方式而没有 与其他语言一起使用,让我的皮肤爬行)。

但是,对于成群的lambda恋人而言并非如此-他们在Python的历史中上演过最接近叛逆的事情之一,直到Guido回溯并决定离开lambda
一些可能的添加functools(以使函数返回常量,标识,等)没有发生(避免显式地复制的更多lambda功能),尽管partial当然仍然存在(这不是完全重复,也不是令人讨厌的)。

请记住,lambda身体仅限于表达,因此有其局限性。例如…:

>>> import functools
>>> f = functools.partial(int, base=2)
>>> f.args
()
>>> f.func
<type 'int'>
>>> f.keywords
{'base': 2}
>>> 

functools.partial返回的函数装饰有用于自省的属性-它包装的函数,以及其中固定的位置和命名参数。此外,可以直接改写命名的参数(在某种意义上,“固定”在某种意义上是默认设置):

>>> f('23', base=10)
23

因此,如您所见,它绝对不像lambda s: int(s, base=2)!-)那样简单

是的,您可以扭曲您的lambda来为您提供一些帮助-例如,对于关键字覆盖,

>>> f = lambda s, **k: int(s, **dict({'base': 2}, **k))

但我非常希望,即使是最热心的- lambda情人,也不要认为这种恐怖比partial通话更容易理解!-)。由于Python的“主体是单个表达式”的局限性,“属性设置”部分更加困难lambda(加上赋值永远不能成为Python表达式的一部分这一事实)……您最终会“伪造表达式中的赋值”通过将列表理解范围扩展到远远超出其设计限制…:

>>> f = [f for f in (lambda f: int(s, base=2),)
           if setattr(f, 'keywords', {'base': 2}) is None][0]

现在结合命名参数覆盖性,再加上三个属性的设置,到一个单一的表达,并告诉我是多么可读将是…!

What functionality does functools.partial offer that you can’t get through lambdas?

Not much in terms of extra functionality (but, see later) – and, readability is in the eye of the beholder.
Most people who are familiar with functional programming languages (those in the Lisp/Scheme families in particular) appear to like lambda just fine – I say “most”, definitely not all, because Guido and I assuredly are among those “familiar with” (etc) yet think of lambda as an eyesore anomaly in Python…
He was repentant of ever having accepted it into Python whereas planned to remove it from Python 3, as one of “Python’s glitches”.
I fully supported him in that. (I love lambda in Scheme… while its limitations in Python, and the weird way it just doesn’t fit in with the rest of the language, make my skin crawl).

Not so, however, for the hordes of lambda lovers — who staged one of the closest things to a rebellion ever seen in Python’s history, until Guido backtracked and decided to leave lambda in.
Several possible additions to functools (to make functions returning constants, identity, etc) didn’t happen (to avoid explicitly duplicating more of lambda‘s functionality), though partial did of course remain (it’s no total duplication, nor is it an eyesore).

Remember that lambda‘s body is limited to be an expression, so it’s got limitations. For example…:

>>> import functools
>>> f = functools.partial(int, base=2)
>>> f.args
()
>>> f.func
<type 'int'>
>>> f.keywords
{'base': 2}
>>> 

functools.partial‘s returned function is decorated with attributes useful for introspection — the function it’s wrapping, and what positional and named arguments it fixes therein. Further, the named arguments can be overridden right back (the “fixing” is rather, in a sense, the setting of defaults):

>>> f('23', base=10)
23

So, as you see, it’s definely not as simplistic as lambda s: int(s, base=2)!-)

Yes, you could contort your lambda to give you some of this – e.g., for the keyword-overriding,

>>> f = lambda s, **k: int(s, **dict({'base': 2}, **k))

but I dearly hope that even the most ardent lambda-lover doesn’t consider this horror more readable than the partial call!-). The “attribute setting” part is even harder, because of the “body’s a single expression” limitation of Python’s lambda (plus the fact that assignment can never be part of a Python expression)… you end up “faking assignments within an expression” by stretching list comprehension well beyond its design limits…:

>>> f = [f for f in (lambda f: int(s, base=2),)
           if setattr(f, 'keywords', {'base': 2}) is None][0]

Now combine the named-arguments overridability, plus the setting of three attributes, into a single expression, and tell me just how readable that is going to be…!


回答 1

好吧,这是一个显示差异的示例:

In [132]: sum = lambda x, y: x + y

In [133]: n = 5

In [134]: incr = lambda y: sum(n, y)

In [135]: incr2 = partial(sum, n)

In [136]: print incr(3), incr2(3)
8 8

In [137]: n = 9

In [138]: print incr(3), incr2(3)
12 8

Ivan Moore的这些帖子扩展了“ lambda的局限性”和python中的闭包:

Well, here’s an example that shows a difference:

In [132]: sum = lambda x, y: x + y

In [133]: n = 5

In [134]: incr = lambda y: sum(n, y)

In [135]: incr2 = partial(sum, n)

In [136]: print incr(3), incr2(3)
8 8

In [137]: n = 9

In [138]: print incr(3), incr2(3)
12 8

These posts by Ivan Moore expand on the “limitations of lambda” and closures in python:


回答 2

在最新版本的Python(> = 2.7)中,您可以pickle使用partial,但不能使用lambda

>>> pickle.dumps(partial(int))
'cfunctools\npartial\np0\n(c__builtin__\nint\np1\ntp2\nRp3\n(g1\n(tNNtp4\nb.'
>>> pickle.dumps(lambda x: int(x))
Traceback (most recent call last):
  File "<ipython-input-11-e32d5a050739>", line 1, in <module>
    pickle.dumps(lambda x: int(x))
  File "/usr/lib/python2.7/pickle.py", line 1374, in dumps
    Pickler(file, protocol).dump(obj)
  File "/usr/lib/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 748, in save_global
    (obj, module, name))
PicklingError: Can't pickle <function <lambda> at 0x1729aa0>: it's not found as __main__.<lambda>

In the latest versions of Python (>=2.7), you can pickle a partial, but not a lambda:

>>> pickle.dumps(partial(int))
'cfunctools\npartial\np0\n(c__builtin__\nint\np1\ntp2\nRp3\n(g1\n(tNNtp4\nb.'
>>> pickle.dumps(lambda x: int(x))
Traceback (most recent call last):
  File "<ipython-input-11-e32d5a050739>", line 1, in <module>
    pickle.dumps(lambda x: int(x))
  File "/usr/lib/python2.7/pickle.py", line 1374, in dumps
    Pickler(file, protocol).dump(obj)
  File "/usr/lib/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 748, in save_global
    (obj, module, name))
PicklingError: Can't pickle <function <lambda> at 0x1729aa0>: it's not found as __main__.<lambda>

回答 3

functools在某种程度上更有效吗?

作为对此的部分回答,我决定测试性能。这是我的示例:

from functools import partial
import time, math

def make_lambda():
    x = 1.3
    return lambda: math.sin(x)

def make_partial():
    x = 1.3
    return partial(math.sin, x)

Iter = 10**7

start = time.clock()
for i in range(0, Iter):
    l = make_lambda()
stop = time.clock()
print('lambda creation time {}'.format(stop - start))

start = time.clock()
for i in range(0, Iter):
    l()
stop = time.clock()
print('lambda execution time {}'.format(stop - start))

start = time.clock()
for i in range(0, Iter):
    p = make_partial()
stop = time.clock()
print('partial creation time {}'.format(stop - start))

start = time.clock()
for i in range(0, Iter):
    p()
stop = time.clock()
print('partial execution time {}'.format(stop - start))

在Python 3.3上,它提供了:

lambda creation time 3.1743163756961392
lambda execution time 3.040552701787919
partial creation time 3.514482823352731
partial execution time 1.7113973411608114

这意味着partial需要更多的时间来创建,但是执行的时间却要少得多。这很可能是ars的答案中讨论的早期绑定和后期绑定的效果。

Is functools somehow more efficient..?

As a partly answer to this I decided to test the performance. Here is my example:

from functools import partial
import time, math

def make_lambda():
    x = 1.3
    return lambda: math.sin(x)

def make_partial():
    x = 1.3
    return partial(math.sin, x)

Iter = 10**7

start = time.clock()
for i in range(0, Iter):
    l = make_lambda()
stop = time.clock()
print('lambda creation time {}'.format(stop - start))

start = time.clock()
for i in range(0, Iter):
    l()
stop = time.clock()
print('lambda execution time {}'.format(stop - start))

start = time.clock()
for i in range(0, Iter):
    p = make_partial()
stop = time.clock()
print('partial creation time {}'.format(stop - start))

start = time.clock()
for i in range(0, Iter):
    p()
stop = time.clock()
print('partial execution time {}'.format(stop - start))

on Python 3.3 it gives:

lambda creation time 3.1743163756961392
lambda execution time 3.040552701787919
partial creation time 3.514482823352731
partial execution time 1.7113973411608114

Which means that partial needs a bit more time for creation but considerably less time for execution. This can well be the effect of the early and late binding which are discussed in the answer from ars.


回答 4

除了Alex提到的额外功能之外,functools.partial的另一个优点是速度。使用partial,您可以避免构造(和破坏)另一个堆栈框架。

默认情况下,由partial和lambda生成的函数都没有文档字符串(尽管您可以通过以下方式为任何对象设置文档字符串 __doc__)。

您可以在此博客中找到更多详细信息:Python中的部分函数应用程序

Besides the extra functionality Alex mentioned, another advantage of functools.partial is speed. With partial you can avoid constructing (and destructing) another stack frame.

Neither the function generated by partial nor lambdas have docstrings by default (though you can set the doc string for any objects via __doc__).

You can find more details in this blog: Partial Function Application in Python


回答 5

我在第三个示例中最快地了解了意图。

当我解析lambda时,我期望比直接由标准库提供的复杂性/奇数更高。

另外,您会注意到,第三个示例是唯一一个不依赖于sum2; 的完整签名的示例。因此使其耦合松散一些。

I understand the intent quickest in the third example.

When I parse lambdas, I’m expecting more complexity/oddity than offered by the standard library directly.

Also, you’ll notice that the third example is the only one which doesn’t depend on the full signature of sum2; thus making it slightly more loosely coupled.


functools部分如何做?

问题:functools部分如何做?

我无法了解部分功能在functools中的工作方式。我从这里有以下代码:

>>> sum = lambda x, y : x + y
>>> sum(1, 2)
3
>>> incr = lambda y : sum(1, y)
>>> incr(2)
3
>>> def sum2(x, y):
    return x + y

>>> incr2 = functools.partial(sum2, 1)
>>> incr2(4)
5

现在排队

incr = lambda y : sum(1, y)

我知道我传递给incr它的任何参数都将传递ylambda哪个参数,sum(1, y)即返回1 + y

我明白那个。但是我不明白incr2(4)

如何在部分函数中4传递获取x?对我来说,4应该更换sum2x和之间是什么关系4

I am not able to get my head on how the partial works in functools. I have the following code from here:

>>> sum = lambda x, y : x + y
>>> sum(1, 2)
3
>>> incr = lambda y : sum(1, y)
>>> incr(2)
3
>>> def sum2(x, y):
    return x + y

>>> incr2 = functools.partial(sum2, 1)
>>> incr2(4)
5

Now in the line

incr = lambda y : sum(1, y)

I get that whatever argument I pass to incr it will be passed as y to lambda which will return sum(1, y) i.e 1 + y.

I understand that. But I didn’t understand this incr2(4).

How does the 4 gets passed as x in partial function? To me, 4 should replace the sum2. What is the relation between x and 4?


回答 0

大致地,partial做这样的事情(除了关键字args支持等):

def partial(func, *part_args):
    def wrapper(*extra_args):
        args = list(part_args)
        args.extend(extra_args)
        return func(*args)

    return wrapper

因此,通过调用partial(sum2, 4)您可以创建一个行为类似于的新函数(准确地说是一个可调用的函数)sum2,但位置参数要少一个。缺少的参数总是由代替4,因此partial(sum2, 4)(2) == sum2(4, 2)

至于为什么需要它,有很多情况。仅举一个例子,假设您必须在某个有两个参数的地方传递一个函数:

class EventNotifier(object):
    def __init__(self):
        self._listeners = []

    def add_listener(self, callback):
        ''' callback should accept two positional arguments, event and params '''
        self._listeners.append(callback)
        # ...

    def notify(self, event, *params):
        for f in self._listeners:
            f(event, params)

但是您已经拥有的功能需要访问某些第三context对象才能完成其工作:

def log_event(context, event, params):
    context.log_event("Something happened %s, %s", event, params)

因此,有几种解决方案:

自定义对象:

class Listener(object):
   def __init__(self, context):
       self._context = context

   def __call__(self, event, params):
       self._context.log_event("Something happened %s, %s", event, params)


 notifier.add_listener(Listener(context))

Lambda:

log_listener = lambda event, params: log_event(context, event, params)
notifier.add_listener(log_listener)

带有局部:

context = get_context()  # whatever
notifier.add_listener(partial(log_event, context))

在这三个中,partial最短和最快。(对于更复杂的情况,您可能需要自定义对象)。

Roughly, partial does something like this (apart from keyword args support etc):

def partial(func, *part_args):
    def wrapper(*extra_args):
        args = list(part_args)
        args.extend(extra_args)
        return func(*args)

    return wrapper

So, by calling partial(sum2, 4) you create a new function (a callable, to be precise) that behaves like sum2, but has one positional argument less. That missing argument is always substituted by 4, so that partial(sum2, 4)(2) == sum2(4, 2)

As for why it’s needed, there’s a variety of cases. Just for one, suppose you have to pass a function somewhere where it’s expected to have 2 arguments:

class EventNotifier(object):
    def __init__(self):
        self._listeners = []

    def add_listener(self, callback):
        ''' callback should accept two positional arguments, event and params '''
        self._listeners.append(callback)
        # ...

    def notify(self, event, *params):
        for f in self._listeners:
            f(event, params)

But a function you already have needs access to some third context object to do its job:

def log_event(context, event, params):
    context.log_event("Something happened %s, %s", event, params)

So, there are several solutions:

A custom object:

class Listener(object):
   def __init__(self, context):
       self._context = context

   def __call__(self, event, params):
       self._context.log_event("Something happened %s, %s", event, params)


 notifier.add_listener(Listener(context))

Lambda:

log_listener = lambda event, params: log_event(context, event, params)
notifier.add_listener(log_listener)

With partials:

context = get_context()  # whatever
notifier.add_listener(partial(log_event, context))

Of those three, partial is the shortest and the fastest. (For a more complex case you might want a custom object though).


回答 1

局部函数非常有用。

例如,在“管线式”函数调用序列中(其中一个函数的返回值是传递给下一个函数的参数)。

有时,此类管道中的函数需要单个参数,但是紧接其上游的函数将返回两个值

在这种情况下,functools.partial可能允许您保持此功能管道完整。

这是一个特定的隔离示例:假设您想按每个数据点与目标之间的距离对一些数据进行排序:

# create some data
import random as RND
fnx = lambda: RND.randint(0, 10)
data = [ (fnx(), fnx()) for c in range(10) ]
target = (2, 4)

import math
def euclid_dist(v1, v2):
    x1, y1 = v1
    x2, y2 = v2
    return math.sqrt((x2 - x1)**2 + (y2 - y1)**2)

要按距目标的距离对数据进行排序,您当然要做的是:

data.sort(key=euclid_dist)

但你不可阻挡-的排序方法的关键参数,只接受拍摄功能单一的参数。

因此,请改写euclid_dist为带有单个参数的函数:

from functools import partial

p_euclid_dist = partial(euclid_dist, target)

p_euclid_dist 现在接受一个参数,

>>> p_euclid_dist((3, 3))
  1.4142135623730951

因此,现在您可以通过传递sort方法的key参数的局部函数来对数据进行排序:

data.sort(key=p_euclid_dist)

# verify that it works:
for p in data:
    print(round(p_euclid_dist(p), 3))

    1.0
    2.236
    2.236
    3.606
    4.243
    5.0
    5.831
    6.325
    7.071
    8.602

又例如,函数的参数之一在外循环中更改,但在内循环迭代期间是固定的。通过使用部分函数,​​您无需在内部循环的迭代过程中传递其他参数,因为修改后的(部分函数)不需要此参数。

>>> from functools import partial

>>> def fnx(a, b, c):
      return a + b + c

>>> fnx(3, 4, 5)
      12

创建一个局部函数(使用关键字arg)

>>> pfnx = partial(fnx, a=12)

>>> pfnx(b=4, c=5)
     21

您还可以使用位置参数创建部分函数

>>> pfnx = partial(fnx, 12)

>>> pfnx(4, 5)
      21

但这会抛出(例如,创建带有关键字参数的partial,然后使用位置参数调用)

>>> pfnx = partial(fnx, a=12)

>>> pfnx(4, 5)
      Traceback (most recent call last):
      File "<pyshell#80>", line 1, in <module>
      pfnx(4, 5)
      TypeError: fnx() got multiple values for keyword argument 'a'

另一个用例:使用python的multiprocessing库编写分布式代码。使用Pool方法创建一个进程池:

>>> import multiprocessing as MP

>>> # create a process pool:
>>> ppool = MP.Pool()

Pool 有一个map方法,但是它只需要一个可迭代的方法,因此,如果您需要传入带有较长参数列表的函数,请将该函数重新定义为局部函数,以修复除一个以外的所有函数:

>>> ppool.map(pfnx, [4, 6, 7, 8])

partials are incredibly useful.

For instance, in a ‘pipe-lined’ sequence of function calls (in which the returned value from one function is the argument passed to the next).

Sometimes a function in such a pipeline requires a single argument, but the function immediately upstream from it returns two values.

In this scenario, functools.partial might allow you to keep this function pipeline intact.

Here’s a specific, isolated example: suppose you want to sort some data by each data point’s distance from some target:

# create some data
import random as RND
fnx = lambda: RND.randint(0, 10)
data = [ (fnx(), fnx()) for c in range(10) ]
target = (2, 4)

import math
def euclid_dist(v1, v2):
    x1, y1 = v1
    x2, y2 = v2
    return math.sqrt((x2 - x1)**2 + (y2 - y1)**2)

To sort this data by distance from the target, what you would like to do of course is this:

data.sort(key=euclid_dist)

but you can’t–the sort method’s key parameter only accepts functions that take a single argument.

so re-write euclid_dist as a function taking a single parameter:

from functools import partial

p_euclid_dist = partial(euclid_dist, target)

p_euclid_dist now accepts a single argument,

>>> p_euclid_dist((3, 3))
  1.4142135623730951

so now you can sort your data by passing in the partial function for the sort method’s key argument:

data.sort(key=p_euclid_dist)

# verify that it works:
for p in data:
    print(round(p_euclid_dist(p), 3))

    1.0
    2.236
    2.236
    3.606
    4.243
    5.0
    5.831
    6.325
    7.071
    8.602

Or for instance, one of the function’s arguments changes in an outer loop but is fixed during iteration in the inner loop. By using a partial, you don’t have to pass in the additional parameter during iteration of the inner loop, because the modified (partial) function doesn’t require it.

>>> from functools import partial

>>> def fnx(a, b, c):
      return a + b + c

>>> fnx(3, 4, 5)
      12

create a partial function (using keyword arg)

>>> pfnx = partial(fnx, a=12)

>>> pfnx(b=4, c=5)
     21

you can also create a partial function with a positional argument

>>> pfnx = partial(fnx, 12)

>>> pfnx(4, 5)
      21

but this will throw (e.g., creating partial with keyword argument then calling using positional arguments)

>>> pfnx = partial(fnx, a=12)

>>> pfnx(4, 5)
      Traceback (most recent call last):
      File "<pyshell#80>", line 1, in <module>
      pfnx(4, 5)
      TypeError: fnx() got multiple values for keyword argument 'a'

another use case: writing distributed code using python’s multiprocessing library. A pool of processes is created using the Pool method:

>>> import multiprocessing as MP

>>> # create a process pool:
>>> ppool = MP.Pool()

Pool has a map method, but it only takes a single iterable, so if you need to pass in a function with a longer parameter list, re-define the function as a partial, to fix all but one:

>>> ppool.map(pfnx, [4, 6, 7, 8])

回答 2

简短的答案,partial为函数的参数提供默认值,否则将没有默认值。

from functools import partial

def foo(a,b):
    return a+b

bar = partial(foo, a=1) # equivalent to: foo(a=1, b)
bar(b=10)
#11 = 1+10
bar(a=101, b=10)
#111=101+10

short answer, partial gives default values to the parameters of a function that would otherwise not have default values.

from functools import partial

def foo(a,b):
    return a+b

bar = partial(foo, a=1) # equivalent to: foo(a=1, b)
bar(b=10)
#11 = 1+10
bar(a=101, b=10)
#111=101+10

回答 3

可以使用部分函数来创建新的派生函数,这些函数具有预先分配的一些输入参数

要了解部分用法在现实世界中的用法,请参阅此非常好的博客文章:http :
//chriskiehl.com/article/Cleaner-coding-through-partially-applied-functions/

博客中的一个简单但简洁的示例,介绍了如何使用partialre.search使代码更具可读性。 re.search方法的签名是:

search(pattern, string, flags=0) 

通过应用,partial我们可以创建多个版本的正则表达式search来满足我们的要求,例如:

is_spaced_apart = partial(re.search, '[a-zA-Z]\s\=')
is_grouped_together = partial(re.search, '[a-zA-Z]\=')

现在is_spaced_apartis_grouped_together是从中派生的两个新函数re.search,它们pattern应用了自变量(因为它patternre.search方法签名中的第一个自变量)。

这两个新函数(可调用)的签名为:

is_spaced_apart(string, flags=0)     # pattern '[a-zA-Z]\s\=' applied
is_grouped_together(string, flags=0) # pattern '[a-zA-Z]\=' applied

这样便可以在某些文本上使用这些部分函数:

for text in lines:
    if is_grouped_together(text):
        some_action(text)
    elif is_spaced_apart(text):
        some_other_action(text)
    else:
        some_default_action()

您可以参考上面的链接,以更深入地了解该主题,因为它涵盖了此特定示例以及更多内容。

Partials can be used to make new derived functions that have some input parameters pre-assigned

To see some real world usage of partials, refer to this really good blog post:
http://chriskiehl.com/article/Cleaner-coding-through-partially-applied-functions/

A simple but neat beginner’s example from the blog, covers how one might use partial on re.search to make code more readable. re.search method’s signature is:

search(pattern, string, flags=0) 

By applying partial we can create multiple versions of the regular expression search to suit our requirements, so for example:

is_spaced_apart = partial(re.search, '[a-zA-Z]\s\=')
is_grouped_together = partial(re.search, '[a-zA-Z]\=')

Now is_spaced_apart and is_grouped_together are two new functions derived from re.search that have the pattern argument applied(since pattern is the first argument in the re.search method’s signature).

The signature of these two new functions(callable) is:

is_spaced_apart(string, flags=0)     # pattern '[a-zA-Z]\s\=' applied
is_grouped_together(string, flags=0) # pattern '[a-zA-Z]\=' applied

This is how you could then use these partial functions on some text:

for text in lines:
    if is_grouped_together(text):
        some_action(text)
    elif is_spaced_apart(text):
        some_other_action(text)
    else:
        some_default_action()

You can refer the link above to get a more in depth understanding of the subject, as it covers this specific example and much more..


回答 4

我认为,这是在python中实现currying的一种方式。

from functools import partial
def add(a,b):
    return a + b

def add2number(x,y,z):
    return x + y + z

if __name__ == "__main__":
    add2 = partial(add,2)
    print("result of add2 ",add2(1))
    add3 = partial(partial(add2number,1),2)
    print("result of add3",add3(1))

结果是3和4。

In my opinion, it’s a way to implement currying in python.

from functools import partial
def add(a,b):
    return a + b

def add2number(x,y,z):
    return x + y + z

if __name__ == "__main__":
    add2 = partial(add,2)
    print("result of add2 ",add2(1))
    add3 = partial(partial(add2number,1),2)
    print("result of add3",add3(1))

The result is 3 and 4.


回答 5

还值得一提的是,当部分函数传递了另一个我们要“硬编码”某些参数的函数时,该参数应该是最右边的参数。

def func(a,b):
    return a*b
prt = partial(func, b=7)
    print(prt(4))
#return 28

但是,如果我们执行相同的操作,而是改为更改参数

def func(a,b):
    return a*b
 prt = partial(func, a=7)
    print(prt(4))

它将引发错误,“ TypeError:func()为参数’a’获得了多个值”

Also worth to mention, that when partial function passed another function where we want to “hard code” some parameters, that should be rightmost parameter

def func(a,b):
    return a*b
prt = partial(func, b=7)
    print(prt(4))
#return 28

but if we do the same, but changing a parameter instead

def func(a,b):
    return a*b
 prt = partial(func, a=7)
    print(prt(4))

it will throw error, “TypeError: func() got multiple values for argument ‘a'”


回答 6

这个答案更多是示例代码。上面的所有答案都很好地解释了为什么应该部分使用。我将给出我的观察和有关局部的用例。

from functools import partial
 def adder(a,b,c):
    print('a:{},b:{},c:{}'.format(a,b,c))
    ans = a+b+c
    print(ans)
partial_adder = partial(adder,1,2)
partial_adder(3)  ## now partial_adder is a callable that can take only one argument

以上代码的输出应为:

a:1,b:2,c:3
6

注意,在上面的示例中,返回了一个新的callable,它将参数(c)作为其参数。请注意,它也是函数的最后一个参数。

args = [1,2]
partial_adder = partial(adder,*args)
partial_adder(3)

上面代码的输出也是:

a:1,b:2,c:3
6

请注意,*用于解压缩非关键字参数,而返回的callable可以接受的参数与上面相同。

另一个观察结果是: 下面的示例演示了partial返回一个callable,它将以未声明的参数(a)作为参数。

def adder(a,b=1,c=2,d=3,e=4):
    print('a:{},b:{},c:{},d:{},e:{}'.format(a,b,c,d,e))
    ans = a+b+c+d+e
    print(ans)
partial_adder = partial(adder,b=10,c=2)
partial_adder(20)

以上代码的输出应为:

a:20,b:10,c:2,d:3,e:4
39

同样,

kwargs = {'b':10,'c':2}
partial_adder = partial(adder,**kwargs)
partial_adder(20)

以上代码打印

a:20,b:10,c:2,d:3,e:4
39

当我使用模块中的Pool.map_async方法时,我不得不使用它multiprocessing。您只能将一个参数传递给worker函数,因此我不得不使用它partial来使我的worker函数看起来像只有一个输入参数的可调用对象,但实际上我的worker函数具有多个输入参数。

This answer is more of an example code. All the above answers give good explanations regarding why one should use partial. I will give my observations and use cases about partial.

from functools import partial
 def adder(a,b,c):
    print('a:{},b:{},c:{}'.format(a,b,c))
    ans = a+b+c
    print(ans)
partial_adder = partial(adder,1,2)
partial_adder(3)  ## now partial_adder is a callable that can take only one argument

Output of the above code should be:

a:1,b:2,c:3
6

Notice that in the above example a new callable was returned that will take parameter (c) as it’s argument. Note that it is also the last argument to the function.

args = [1,2]
partial_adder = partial(adder,*args)
partial_adder(3)

Output of the above code is also:

a:1,b:2,c:3
6

Notice that * was used to unpack the non-keyword arguments and the callable returned in terms of which argument it can take is same as above.

Another observation is: Below example demonstrates that partial returns a callable which will take the undeclared parameter (a) as an argument.

def adder(a,b=1,c=2,d=3,e=4):
    print('a:{},b:{},c:{},d:{},e:{}'.format(a,b,c,d,e))
    ans = a+b+c+d+e
    print(ans)
partial_adder = partial(adder,b=10,c=2)
partial_adder(20)

Output of the above code should be:

a:20,b:10,c:2,d:3,e:4
39

Similarly,

kwargs = {'b':10,'c':2}
partial_adder = partial(adder,**kwargs)
partial_adder(20)

Above code prints

a:20,b:10,c:2,d:3,e:4
39

I had to use it when I was using Pool.map_async method from multiprocessing module. You can pass only one argument to the worker function so I had to use partial to make my worker function look like a callable with only one input argument but in reality my worker function had multiple input arguments.


如何在Python 3中使用过滤,映射和归约

问题:如何在Python 3中使用过滤,映射和归约

filter,,map并且reduce可以在Python 2中完美运行。这是一个示例:

>>> def f(x):
        return x % 2 != 0 and x % 3 != 0
>>> filter(f, range(2, 25))
[5, 7, 11, 13, 17, 19, 23]

>>> def cube(x):
        return x*x*x
>>> map(cube, range(1, 11))
[1, 8, 27, 64, 125, 216, 343, 512, 729, 1000]

>>> def add(x,y):
        return x+y
>>> reduce(add, range(1, 11))
55

但是在Python 3中,我收到以下输出:

>>> filter(f, range(2, 25))
<filter object at 0x0000000002C14908>

>>> map(cube, range(1, 11))
<map object at 0x0000000002C82B70>

>>> reduce(add, range(1, 11))
Traceback (most recent call last):
  File "<pyshell#8>", line 1, in <module>
    reduce(add, range(1, 11))
NameError: name 'reduce' is not defined

如果有人可以向我解释为什么,我将不胜感激。

代码的屏幕截图,用于进一步说明:

filter, map, and reduce work perfectly in Python 2. Here is an example:

>>> def f(x):
        return x % 2 != 0 and x % 3 != 0
>>> filter(f, range(2, 25))
[5, 7, 11, 13, 17, 19, 23]

>>> def cube(x):
        return x*x*x
>>> map(cube, range(1, 11))
[1, 8, 27, 64, 125, 216, 343, 512, 729, 1000]

>>> def add(x,y):
        return x+y
>>> reduce(add, range(1, 11))
55

But in Python 3, I receive the following outputs:

>>> filter(f, range(2, 25))
<filter object at 0x0000000002C14908>

>>> map(cube, range(1, 11))
<map object at 0x0000000002C82B70>

>>> reduce(add, range(1, 11))
Traceback (most recent call last):
  File "<pyshell#8>", line 1, in <module>
    reduce(add, range(1, 11))
NameError: name 'reduce' is not defined

I would appreciate if someone could explain to me why this is.

Screenshot of code for further clarity:


回答 0

您可以阅读Python 3.0的新增功能中的更改。从2.x升级到3.x时,应该仔细阅读它,因为已经做了很多更改。

此处的完整答案是文档中的引号。

视图和迭代器而不是列表

一些著名的API不再返回列表:

  • […]
  • map()filter()返回迭代器。如果您确实需要列表,则可以使用快速解决方案,例如list(map(...)),但是更好的解决方案通常是使用列表理解(特别是当原始代码使用lambda时),或者重写代码以使其根本不需要列表。map()该函数的副作用特别棘手。正确的转换是使用常规for循环(因为创建列表将很浪费)。
  • […]

内建

  • […]
  • 已删除reduce()functools.reduce()如果确实需要,请使用;但是,在99%的时间里,显式for循环更易于阅读。
  • […]

You can read about the changes in What’s New In Python 3.0. You should read it thoroughly when you move from 2.x to 3.x since a lot has been changed.

The whole answer here are quotes from the documentation.

Views And Iterators Instead Of Lists

Some well-known APIs no longer return lists:

  • […]
  • map() and filter() return iterators. If you really need a list, a quick fix is e.g. list(map(...)), but a better fix is often to use a list comprehension (especially when the original code uses lambda), or rewriting the code so it doesn’t need a list at all. Particularly tricky is map() invoked for the side effects of the function; the correct transformation is to use a regular for loop (since creating a list would just be wasteful).
  • […]

Builtins

  • […]
  • Removed reduce(). Use functools.reduce() if you really need it; however, 99 percent of the time an explicit for loop is more readable.
  • […]

回答 1

的功能mapfilter被有意改为返回迭代器,并减少从被除去的内置和放置在functools.reduce

因此,对于filtermap,您可以将它们包装起来以list()像以前一样查看结果。

>>> def f(x): return x % 2 != 0 and x % 3 != 0
...
>>> list(filter(f, range(2, 25)))
[5, 7, 11, 13, 17, 19, 23]
>>> def cube(x): return x*x*x
...
>>> list(map(cube, range(1, 11)))
[1, 8, 27, 64, 125, 216, 343, 512, 729, 1000]
>>> import functools
>>> def add(x,y): return x+y
...
>>> functools.reduce(add, range(1, 11))
55
>>>

现在的建议是,用生成器表达式或列表推导替换map和filter的用法。例:

>>> def f(x): return x % 2 != 0 and x % 3 != 0
...
>>> [i for i in range(2, 25) if f(i)]
[5, 7, 11, 13, 17, 19, 23]
>>> def cube(x): return x*x*x
...
>>> [cube(i) for i in range(1, 11)]
[1, 8, 27, 64, 125, 216, 343, 512, 729, 1000]
>>>

他们说for循环在99%的时间里比减少容易阅读,但是我还是坚持functools.reduce

编辑:99%的数字直接从Guido van Rossum编写的“ Python 3.0的新功能”页面中提取。

The functionality of map and filter was intentionally changed to return iterators, and reduce was removed from being a built-in and placed in functools.reduce.

So, for filter and map, you can wrap them with list() to see the results like you did before.

>>> def f(x): return x % 2 != 0 and x % 3 != 0
...
>>> list(filter(f, range(2, 25)))
[5, 7, 11, 13, 17, 19, 23]
>>> def cube(x): return x*x*x
...
>>> list(map(cube, range(1, 11)))
[1, 8, 27, 64, 125, 216, 343, 512, 729, 1000]
>>> import functools
>>> def add(x,y): return x+y
...
>>> functools.reduce(add, range(1, 11))
55
>>>

The recommendation now is that you replace your usage of map and filter with generators expressions or list comprehensions. Example:

>>> def f(x): return x % 2 != 0 and x % 3 != 0
...
>>> [i for i in range(2, 25) if f(i)]
[5, 7, 11, 13, 17, 19, 23]
>>> def cube(x): return x*x*x
...
>>> [cube(i) for i in range(1, 11)]
[1, 8, 27, 64, 125, 216, 343, 512, 729, 1000]
>>>

They say that for loops are 99 percent of the time easier to read than reduce, but I’d just stick with functools.reduce.

Edit: The 99 percent figure is pulled directly from the What’s New In Python 3.0 page authored by Guido van Rossum.


回答 2

作为其他答案的补充,对于上下文管理器来说,这听起来像是一个很好的用例,它将重新将这些函数的名称映射为返回列表并引入reduce全局命名空间的函数。

快速实现可能如下所示:

from contextlib import contextmanager    

@contextmanager
def noiters(*funcs):
    if not funcs: 
        funcs = [map, filter, zip] # etc
    from functools import reduce
    globals()[reduce.__name__] = reduce
    for func in funcs:
        globals()[func.__name__] = lambda *ar, func = func, **kwar: list(func(*ar, **kwar))
    try:
        yield
    finally:
        del globals()[reduce.__name__]
        for func in funcs: globals()[func.__name__] = func

用法如下所示:

with noiters(map):
    from operator import add
    print(reduce(add, range(1, 20)))
    print(map(int, ['1', '2']))

哪些打印:

190
[1, 2]

只是我的2美分:-)

As an addendum to the other answers, this sounds like a fine use-case for a context manager that will re-map the names of these functions to ones which return a list and introduce reduce in the global namespace.

A quick implementation might look like this:

from contextlib import contextmanager    

@contextmanager
def noiters(*funcs):
    if not funcs: 
        funcs = [map, filter, zip] # etc
    from functools import reduce
    globals()[reduce.__name__] = reduce
    for func in funcs:
        globals()[func.__name__] = lambda *ar, func = func, **kwar: list(func(*ar, **kwar))
    try:
        yield
    finally:
        del globals()[reduce.__name__]
        for func in funcs: globals()[func.__name__] = func

With a usage that looks like this:

with noiters(map):
    from operator import add
    print(reduce(add, range(1, 20)))
    print(map(int, ['1', '2']))

Which prints:

190
[1, 2]

Just my 2 cents 🙂


回答 3

由于该reduce方法已从Python3的内置函数中删除,因此请不要忘记functools在您的代码中导入。请查看下面的代码段。

import functools
my_list = [10,15,20,25,35]
sum_numbers = functools.reduce(lambda x ,y : x+y , my_list)
print(sum_numbers)

Since the reduce method has been removed from the built in function from Python3, don’t forget to import the functools in your code. Please look at the code snippet below.

import functools
my_list = [10,15,20,25,35]
sum_numbers = functools.reduce(lambda x ,y : x+y , my_list)
print(sum_numbers)

回答 4

以下是Filter,map和reduce函数的示例。

数字= [10,11,12,22,34,43,54,34,67,87,88,98,99,87,44,66]

//过滤

奇数=列表(filter(lambda x:x%2!= 0,数字))

打印(奇数)

//地图

multipleOf2 = list(map(lambda x:x * 2,数字))

打印(multiplyOf2)

//降低

由于不常用reduce函数,因此已从Python 3的内置函数中删除了它。functools模块中仍提供了reduce函数,因此您可以执行以下操作:

从functools进口减少

sumOfNumbers = reduce(lambda x,y:x + y,数字)

打印(sumOfNumbers)

Here are the examples of Filter, map and reduce functions.

numbers = [10,11,12,22,34,43,54,34,67,87,88,98,99,87,44,66]

//Filter

oddNumbers = list(filter(lambda x: x%2 != 0, numbers))

print(oddNumbers)

//Map

multiplyOf2 = list(map(lambda x: x*2, numbers))

print(multiplyOf2)

//Reduce

The reduce function, since it is not commonly used, was removed from the built-in functions in Python 3. It is still available in the functools module, so you can do:

from functools import reduce

sumOfNumbers = reduce(lambda x,y: x+y, numbers)

print(sumOfNumbers)


回答 5

map,filter和reduce的优点之一是当您将它们“链接”在一起以进行复杂的操作时,它们变得清晰易读。但是,内置语法不清晰,全都是“向后的”。因此,我建议使用该PyFunctional软件包(https://pypi.org/project/PyFunctional/)。 这是两者的比较:

flight_destinations_dict = {'NY': {'London', 'Rome'}, 'Berlin': {'NY'}}

Py功能版本

非常清晰的语法。你可以说:

“我有一个飞行目的地序列。如果城市位于dict值中,我想从中获得dict键。最后,过滤掉我在流程中创建的空列表。”

from functional import seq  # PyFunctional package to allow easier syntax

def find_return_flights_PYFUNCTIONAL_SYNTAX(city, flight_destinations_dict):
    return seq(flight_destinations_dict.items()) \
        .map(lambda x: x[0] if city in x[1] else []) \
        .filter(lambda x: x != []) \

默认Python版本

都是倒退。您需要说:

“好的,所以有一个列表。我想从中过滤出空列表。为什么?因为如果城市位于dict值中,我首先得到了dict键。哦,我要执行的列表是flight_destinations_dict。 ”

def find_return_flights_DEFAULT_SYNTAX(city, flight_destinations_dict):
    return list(
        filter(lambda x: x != [],
               map(lambda x: x[0] if city in x[1] else [], flight_destinations_dict.items())
               )
    )

One of the advantages of map, filter and reduce is how legible they become when you “chain” them together to do something complex. However, the built-in syntax isn’t legible and is all “backwards”. So, I suggest using the PyFunctional package (https://pypi.org/project/PyFunctional/). Here’s a comparison of the two:

flight_destinations_dict = {'NY': {'London', 'Rome'}, 'Berlin': {'NY'}}

PyFunctional version

Very legible syntax. You can say:

“I have a sequence of flight destinations. Out of which I want to get the dict key if city is in the dict values. Finally, filter out the empty lists I created in the process.”

from functional import seq  # PyFunctional package to allow easier syntax

def find_return_flights_PYFUNCTIONAL_SYNTAX(city, flight_destinations_dict):
    return seq(flight_destinations_dict.items()) \
        .map(lambda x: x[0] if city in x[1] else []) \
        .filter(lambda x: x != []) \

Default Python version

It’s all backwards. You need to say:

“OK, so, there’s a list. I want to filter empty lists out of it. Why? Because I first got the dict key if the city was in the dict values. Oh, the list I’m doing this to is flight_destinations_dict.”

def find_return_flights_DEFAULT_SYNTAX(city, flight_destinations_dict):
    return list(
        filter(lambda x: x != [],
               map(lambda x: x[0] if city in x[1] else [], flight_destinations_dict.items())
               )
    )

为什么Python对于函数式编程不是很好?[关闭]

问题:为什么Python对于函数式编程不是很好?[关闭]

我一直认为函数式编程可以在Python中完成。因此,令我感到惊讶的是,Python在这个问题上没有得到太多提及,而当提及它时,通常不是很积极。但是,没有给出太多的原因(缺少模式匹配和代数数据类型)。所以我的问题是:为什么Python对于函数式编程不是很好?除了缺乏模式匹配和代数数据类型之外,还有其他原因吗?还是这些概念对函数式编程如此重要,以致于不支持它们的语言只能被归类为一流的函数式编程语言?(请记住,我在函数式编程方面的经验非常有限。)

I have always thought that functional programming can be done in Python. Thus, I was surprised that Python didn’t get much of a mention in this question, and when it was mentioned, it normally wasn’t very positive. However, not many reasons were given for this (lack of pattern matching and algebraic data types were mentioned). So my question is: why isn’t Python very good for functional programming? Are there more reasons than its lack of pattern matching and algebraic data types? Or are these concepts so important to functional programming that a language that doesn’t support them can only be classed as a second rate functional programming language? (Keep in mind that my experience with functional programming is quite limited.)


回答 0

您所参考的问题将询问哪些语言同时促进OO和功能编程。即使Python 运作良好,它也不会促进函数式编程。

反对 Python 中的函数式编程的最佳论据是Guido仔细考虑了命令式/ OO用例,而函数式编程用例则没有。当我编写命令式Python时,它是我所知道的最漂亮的语言之一。当我编写函数式Python时,它变得和没有BDFL的普通语言一样丑陋和令人不快。

这并不是说这很糟糕,只是您必须比您改用促进功能编程的语言或改用编写OO Python的语言更加努力。

以下是我在Python中缺少的功能:


  • 没有模式匹配,也没有尾递归,这意味着必须强制性地编写基本算法。递归在Python中很丑陋而且很慢。
  • 一个小的列表库,没有功能词典,这意味着您必须自己编写很多东西。
  • 没有用于currying或composition的语法,这意味着无点样式几乎像显式传递参数一样充满标点符号。
  • 迭代器而不是惰性列表意味着您必须知道是要效率还是持久性,并且list如果需要持久性,则需要分散调用。(迭代器只能使用一次)
  • Python的简单命令式语法及其简单的LL1解析器意味着,基本上不可能为if-expressions和lambda-expressions提供更好的语法。Guido喜欢这种方式,我认为他是对的。

The question you reference asks which languages promote both OO and functional programming. Python does not promote functional programming even though it works fairly well.

The best argument against functional programming in Python is that imperative/OO use cases are carefully considered by Guido, while functional programming use cases are not. When I write imperative Python, it’s one of the prettiest languages I know. When I write functional Python, it becomes as ugly and unpleasant as your average language that doesn’t have a BDFL.

Which is not to say that it’s bad, just that you have to work harder than you would if you switched to a language that promotes functional programming or switched to writing OO Python.

Here are the functional things I miss in Python:


  • No pattern matching and no tail recursion mean your basic algorithms have to be written imperatively. Recursion is ugly and slow in Python.
  • A small list library and no functional dictionaries mean that you have to write a lot of stuff yourself.
  • No syntax for currying or composition means that point-free style is about as full of punctuation as explicitly passing arguments.
  • Iterators instead of lazy lists means that you have to know whether you want efficiency or persistence, and to scatter calls to list around if you want persistence. (Iterators are use-once)
  • Python’s simple imperative syntax, along with its simple LL1 parser, mean that a better syntax for if-expressions and lambda-expressions is basically impossible. Guido likes it this way, and I think he’s right.

回答 1

Guido 在这里对此有很好的解释。这是最相关的部分:

我从来没有考虑过Python受功能语言的严重影响,无论人们怎么说。我对诸如C和Algol 68之类的命令式语言更加熟悉,尽管我使函数成为一流的对象,但我并不认为Python是一种函数式编程语言。但是,很早以前,很明显,用户希望对列表和函数做更多的事情。

还值得注意的是,即使我没有将Python设想为一种功能语言,闭包的引入对于许多其他高级编程功能的开发也很有用。例如,新型类,装饰器和其他现代功能的某些方面都依赖此功能。

最后,尽管这些年来已经引入了许多功能编程功能,但是Python仍然缺少“真实”功能编程语言中的某些功能。例如,Python不执行某些类型的优化(例如,尾部递归)。通常,由于Python具有极强的动态特性,因此无法进行从Haskell或ML之类的功能语言中已知的那种编译时优化。很好。

我从中得出两点:

  1. 该语言的创建者并没有真正将Python视为一种功能语言。因此,有可能看到“功能式”功能,但是您不太可能看到任何确定的功能。
  2. Python的动态特性抑制了您在其他功能语言中看到的某些优化。诚然,Lisp与Python一样动态(如果不是更动态的话),因此这只是部分解释。

Guido has a good explanation of this here. Here’s the most relevant part:

I have never considered Python to be heavily influenced by functional languages, no matter what people say or think. I was much more familiar with imperative languages such as C and Algol 68 and although I had made functions first-class objects, I didn’t view Python as a functional programming language. However, earlier on, it was clear that users wanted to do much more with lists and functions.

It is also worth noting that even though I didn’t envision Python as a functional language, the introduction of closures has been useful in the development of many other advanced programming features. For example, certain aspects of new-style classes, decorators, and other modern features rely upon this capability.

Lastly, even though a number of functional programming features have been introduced over the years, Python still lacks certain features found in “real” functional programming languages. For instance, Python does not perform certain kinds of optimizations (e.g., tail recursion). In general, because Python’s extremely dynamic nature, it is impossible to do the kind of compile-time optimization known from functional languages like Haskell or ML. And that’s fine.

I pull two things out of this:

  1. The language’s creator doesn’t really consider Python to be a functional language. Therefore, it’s possible to see “functional-esque” features, but you’re unlikely to see anything that is definitively functional.
  2. Python’s dynamic nature inhibits some of the optimizations you see in other functional languages. Granted, Lisp is just as dynamic (if not more dynamic) as Python, so this is only a partial explanation.

回答 2

Scheme没有代数数据类型或模式匹配,但肯定是一种功能语言。从功能编程的角度来看,有关Python的烦人的事情:

  1. 残缺的Lambdas。由于Lambda仅包含一个表达式,并且您无法在表达式上下文中轻松地进行所有操作,因此这意味着您可以“即时”定义的功能受到限制。

  2. 如果是语句,而不是表达式。这意味着,除其他事项外,您不能在其中带有If的lambda。(这在Python 2.5中由三元数修复,但看起来很丑。)

  3. 圭多威胁要删除地图,过滤器,并减少在每过一段时间

另一方面,python具有词汇闭包,Lambda和列表理解(无论Guido是否接受,这实际上都是“功能性”概念)。我在Python中进行了大量的“函数式”编程,但是我很难说它是理想的。

Scheme doesn’t have algebraic data types or pattern matching but it’s certainly a functional language. Annoying things about Python from a functional programming perspective:

  1. Crippled Lambdas. Since Lambdas can only contain an expression, and you can’t do everything as easily in an expression context, this means that the functions you can define “on the fly” are limited.

  2. Ifs are statements, not expressions. This means, among other things, you can’t have a lambda with an If inside it. (This is fixed by ternaries in Python 2.5, but it looks ugly.)

  3. Guido threatens to remove map, filter, and reduce every once in a while

On the other hand, python has lexical closures, Lambdas, and list comprehensions (which are really a “functional” concept whether or not Guido admits it). I do plenty of “functional-style” programming in Python, but I’d hardly say it’s ideal.


回答 3

我永远都不会将Python称为“功能性的”,但每当我使用Python进行编程时,代码总是以几乎纯粹的功能性结尾。

诚然,这主要是由于列表理解非常好。因此,我不一定会建议将Python作为一种功能编程语言,但我建议对使用Python的任何人进行功能编程。

I would never call Python “functional” but whenever I program in Python the code invariably ends up being almost purely functional.

Admittedly, that’s mainly due to the extremely nice list comprehension. So I wouldn’t necessarily suggest Python as a functional programming language but I would suggest functional programming for anyone using Python.


回答 4

让我用从SO上的“功能性” Python问题的答案中摘录的一段代码来演示

Python:

def grandKids(generation, kidsFunc, val):
  layer = [val]
  for i in xrange(generation):
    layer = itertools.chain.from_iterable(itertools.imap(kidsFunc, layer))
  return layer

Haskell:

grandKids generation kidsFunc val =
  iterate (concatMap kidsFunc) [val] !! generation

这里的主要区别是,Haskell的标准库中有函数式编程有用的功能:在这种情况下iterateconcat(!!)

Let me demonstrate with a piece of code taken from an answer to a “functional” Python question on SO

Python:

def grandKids(generation, kidsFunc, val):
  layer = [val]
  for i in xrange(generation):
    layer = itertools.chain.from_iterable(itertools.imap(kidsFunc, layer))
  return layer

Haskell:

grandKids generation kidsFunc val =
  iterate (concatMap kidsFunc) [val] !! generation

The main difference here is that Haskell’s standard library has useful functions for functional programming: in this case iterate, concat, and (!!)


回答 5

对于这个问题(和答案)而言,真正重要的一件事是:函数式编程到底是什么,它的最重要特性是什么。我将尽我的看法:

函数式编程很像在白板上写数学。在白板上写方程式时,您无需考虑执行顺序。(通常)没有突变。您不会在第二天回来查看它,并且当您再次进行计算时,您会得到不同的结果(或者,如果您喝了一些新鲜的咖啡,则可以:)。基本上,板子上有什么,当您开始写下内容时,答案已经在那里,您只是还没有意识到它到底是什么。

函数式编程非常像这样。您不会改变任何事情,只需评估方程式(在本例中为“程序”),然后找出答案是什么。该程序仍然存在,未经修改。与数据相同。

我将以下内容列为函数式编程的最重要特征:a)引用透明性-如果您在其他时间和地点评估相同的语句,但具有相同的变量值,则其含义仍相同。b)没有副作用-不管您凝视白板多长时间,另一个人看着另一个白板的方程式都不会意外改变。c)函数也是值。可以传递给其他变量或与其他变量一起应用。d)函数组合,您可以执行h = g·f,从而定义一个新函数h(..),该函数等效于调用g(f(..))。

此列表按我的优先顺序排列,因此参照透明性是最重要的,其次没有副作用。

现在,如果您通过python检查语言和库在这些方面的支持和保证程度,那么您就可以很好地回答自己的问题了。

One thing that is really important for this question (and the answers) is the following: What the hell is functional programming, and what are the most important properties of it. I’ll try to give my view of it:

Functional programming is a lot like writing math on a whiteboard. When you write equations on a whiteboard, you do not think about an execution order. There is (typically) no mutation. You don’t come back the day after and look at it, and when you make the calculations again, you get a different result (or you may, if you’ve had some fresh coffee :)). Basically, what is on the board is there, and the answer was already there when you started writing things down, you just haven’t realized what it is yet.

Functional programming is a lot like that; you don’t change things, you just evaluate the equation (or in this case, “program”) and figure out what the answer is. The program is still there, unmodified. The same with the data.

I would rank the following as the most important features of functional programming: a) referential transparency – if you evaluate the same statement at some other time and place, but with the same variable values, it will still mean the same. b) no side effect – no matter how long you stare at the whiteboard, the equation another guy is looking at at another whiteboard won’t accidentally change. c) functions are values too. which can be passed around and applied with, or to, other variables. d) function composition, you can do h=g·f and thus define a new function h(..) which is equivalent to calling g(f(..)).

This list is in my prioritized order, so referential transparency is the most important, followed by no side effects.

Now, if you go through python and check how well the language and libraries supports, and guarantees, these aspects – then you are well on the way to answer your own question.


回答 6

Python几乎是一种功能语言。这是“功能精简版”。

它具有额外的功能,因此对于某些人来说还不够纯粹。

它还缺少一些功能,因此对于某些功能来说还不够完善。

缺少的功能相对容易编写。在Python中的FP上查看此类帖子。

Python is almost a functional language. It’s “functional lite”.

It has extra features, so it isn’t pure enough for some.

It also lacks some features, so it isn’t complete enough for some.

The missing features are relatively easy to write. Check out posts like this on FP in Python.


回答 7

上面未提及的另一个原因是,许多内置类型的内置函数和方法会修改对象,但不会返回修改后的对象。如果返回那些修改后的对象,那将使功能代码更简洁。例如,如果some_list.append(some_object)返回添加了some_object的some_list。

Another reason not mentioned above is that many built-in functions and methods of built-in types modify an object but do not return the modified object. If those modified objects were returned, that would make functional code cleaner and more concise. For example, if some_list.append(some_object) returned some_list with some_object appended.


回答 8

除了其他答案外,Python和大多数其他多范式语言不太适合真正的函数式编程的一个原因是,因为它们的编译器/虚拟机/运行时不支持函数优化。通过编译器理解数学规则可以实现这种优化。例如,许多编程语言都支持map函数或方法。这是一个相当标准的函数,该函数将一个函数作为一个参数,将一个Iterable作为第二个参数,然后将该函数应用于Iterable中的每个元素。

无论如何,结果map( foo() , x ) * map( foo(), y )与相同map( foo(), x * y )。后者的情况实际上比前者快,因为前者执行两份副本,后者执行一份。

更好的功能语言会识别这些基于数学的关系并自动执行优化。非专用于功能范例的语言可能不会进行优化。

In addition to other answers, one reason Python and most other multi-paradigm languages are not well suited for true functional programming is because their compilers / virtual machines / run-times do not support functional optimization. This sort of optimization is achieved by the compiler understanding mathematical rules. For example, many programming languages support a map function or method. This is a fairly standard function that takes a function as one argument and a iterable as the second argument then applies that function to each element in the iterable.

Anyways it turns out that map( foo() , x ) * map( foo(), y ) is the same as map( foo(), x * y ). The latter case is actually faster than the former because the former performs two copies where the latter performs one.

Better functional languages recognize these mathematically based relationships and automatically perform the optimization. Languages that aren’t dedicated to the functional paradigm will likely not optimize.


列表理解与lambda +过滤器

问题:列表理解与lambda +过滤器

我碰巧发现自己有一个基本的过滤需求:我有一个列表,并且必须按项目的属性对其进行过滤。

我的代码如下所示:

my_list = [x for x in my_list if x.attribute == value]

但是后来我想,这样写会更好吗?

my_list = filter(lambda x: x.attribute == value, my_list)

它更具可读性,并且如果需要性能,可以将lambda取出以获取收益。

问题是:使用第二种方法是否有警告?有任何性能差异吗?我是否完全想念Pythonic Way™,应该以另一种方式来做到这一点(例如,使用itemgetter而不是lambda)吗?

I happened to find myself having a basic filtering need: I have a list and I have to filter it by an attribute of the items.

My code looked like this:

my_list = [x for x in my_list if x.attribute == value]

But then I thought, wouldn’t it be better to write it like this?

my_list = filter(lambda x: x.attribute == value, my_list)

It’s more readable, and if needed for performance the lambda could be taken out to gain something.

Question is: are there any caveats in using the second way? Any performance difference? Am I missing the Pythonic Way™ entirely and should do it in yet another way (such as using itemgetter instead of the lambda)?


回答 0

奇怪的是,不同的人有多少美丽。我发现列表理解比filter+ 清晰得多lambda,但是请使用任何您更容易理解的列表。

有两件事可能会减慢您对的使用filter

第一个是函数调用开销:使用Python函数(无论是由def还是创建的lambda)后,过滤器的运行速度可能会比列表理解慢。几乎可以肯定,这还不够重要,并且在对代码进行计时并发现它是瓶颈之前,您不应该对性能进行太多的考虑,但是区别仍然存在。

可能适用的其他开销是,lambda被强制访问作用域变量(value)。这比访问局部变量要慢,并且在Python 2.x中,列表推导仅访问局部变量。如果您使用的是Python 3.x,则列表推导是在单独的函数中运行的,因此它也将value通过闭包进行访问,这种区别将不适用。

要考虑的另一个选项是使用生成器而不是列表推导:

def filterbyvalue(seq, value):
   for el in seq:
       if el.attribute==value: yield el

然后,在您的主要代码(这才是真正的可读性)中,您已经用有希望的有意义的函数名称替换了列表理解和过滤器。

It is strange how much beauty varies for different people. I find the list comprehension much clearer than filter+lambda, but use whichever you find easier.

There are two things that may slow down your use of filter.

The first is the function call overhead: as soon as you use a Python function (whether created by def or lambda) it is likely that filter will be slower than the list comprehension. It almost certainly is not enough to matter, and you shouldn’t think much about performance until you’ve timed your code and found it to be a bottleneck, but the difference will be there.

The other overhead that might apply is that the lambda is being forced to access a scoped variable (value). That is slower than accessing a local variable and in Python 2.x the list comprehension only accesses local variables. If you are using Python 3.x the list comprehension runs in a separate function so it will also be accessing value through a closure and this difference won’t apply.

The other option to consider is to use a generator instead of a list comprehension:

def filterbyvalue(seq, value):
   for el in seq:
       if el.attribute==value: yield el

Then in your main code (which is where readability really matters) you’ve replaced both list comprehension and filter with a hopefully meaningful function name.


回答 1

在Python中,这是一个有点宗教性的问题。即使Guido考虑从Python 3中删除它mapfilter并且reduce存在足够的反弹,最终只reduce从内置函数转移到functools.reduce

我个人认为列表理解更容易阅读。[i for i in list if i.attribute == value]由于所有行为都在表面上而不是在过滤器函数内部,因此表达式中发生的事情更加明确。

我不会太担心这两种方法之间的性能差异,因为这是微不足道的。如果确实证明这是您应用程序中的瓶颈(不太可能),我真的只会对其进行优化。

另外,由于BDFL希望filter摆脱这种语言,因此可以肯定地自动使列表理解更具Pythonic 😉

This is a somewhat religious issue in Python. Even though Guido considered removing map, filter and reduce from Python 3, there was enough of a backlash that in the end only reduce was moved from built-ins to functools.reduce.

Personally I find list comprehensions easier to read. It is more explicit what is happening from the expression [i for i in list if i.attribute == value] as all the behaviour is on the surface not inside the filter function.

I would not worry too much about the performance difference between the two approaches as it is marginal. I would really only optimise this if it proved to be the bottleneck in your application which is unlikely.

Also since the BDFL wanted filter gone from the language then surely that automatically makes list comprehensions more Pythonic 😉


回答 2

由于任何速度差都将是微不足道的,因此使用过滤器还是列表理解都取决于品味。总的来说,我倾向于使用理解(这里似乎与大多数其他答案一致),但是在某些情况下,我更喜欢使用filter

一个非常常见的用例是抽取某些可迭代的X的值作为谓词P(x):

[x for x in X if P(x)]

但有时您想先将某些函数应用于这些值:

[f(x) for x in X if P(f(x))]


作为一个具体的例子,考虑

primes_cubed = [x*x*x for x in range(1000) if prime(x)]

我认为这看起来比使用略好filter。但是现在考虑

prime_cubes = [x*x*x for x in range(1000) if prime(x*x*x)]

在这种情况下,我们要filter反对后计算值。除了两次计算多维数据集的问题(想象一个更昂贵的计算)外,还有两次写入表达式的问题,这违背了DRY的审美观。在这种情况下,我倾向于使用

prime_cubes = filter(prime, [x*x*x for x in range(1000)])

Since any speed difference is bound to be miniscule, whether to use filters or list comprehensions comes down to a matter of taste. In general I’m inclined to use comprehensions (which seems to agree with most other answers here), but there is one case where I prefer filter.

A very frequent use case is pulling out the values of some iterable X subject to a predicate P(x):

[x for x in X if P(x)]

but sometimes you want to apply some function to the values first:

[f(x) for x in X if P(f(x))]


As a specific example, consider

primes_cubed = [x*x*x for x in range(1000) if prime(x)]

I think this looks slightly better than using filter. But now consider

prime_cubes = [x*x*x for x in range(1000) if prime(x*x*x)]

In this case we want to filter against the post-computed value. Besides the issue of computing the cube twice (imagine a more expensive calculation), there is the issue of writing the expression twice, violating the DRY aesthetic. In this case I’d be apt to use

prime_cubes = filter(prime, [x*x*x for x in range(1000)])

回答 3

尽管filter可能是“更快的方法”,但“ Python方式”将不在乎这些事情,除非性能绝对至关重要(在这种情况下,您将不会使用Python!)。

Although filter may be the “faster way”, the “Pythonic way” would be not to care about such things unless performance is absolutely critical (in which case you wouldn’t be using Python!).


回答 4

我以为我会在python 3中添加,filter()实际上是一个迭代器对象,因此您必须将filter方法调用传递给list()才能构建过滤后的列表。所以在python 2:

lst_a = range(25) #arbitrary list
lst_b = [num for num in lst_a if num % 2 == 0]
lst_c = filter(lambda num: num % 2 == 0, lst_a)

列表b和c具有相同的值,并且大约在相同的时间内完成,因为filter()等效[如果在z中,则x表示y中的x]。但是,在3中,相同的代码将使列表c包含过滤器对象,而不是过滤后的列表。要在3中产生相同的值:

lst_a = range(25) #arbitrary list
lst_b = [num for num in lst_a if num % 2 == 0]
lst_c = list(filter(lambda num: num %2 == 0, lst_a))

问题在于list()接受一个可迭代的参数,并从该参数创建一个新列表。结果是,在python 3中以这种方式使用filter所花费的时间是[x for x in y if z]中方法的两倍,因为您必须遍历filter()的输出以及原始列表。

I thought I’d just add that in python 3, filter() is actually an iterator object, so you’d have to pass your filter method call to list() in order to build the filtered list. So in python 2:

lst_a = range(25) #arbitrary list
lst_b = [num for num in lst_a if num % 2 == 0]
lst_c = filter(lambda num: num % 2 == 0, lst_a)

lists b and c have the same values, and were completed in about the same time as filter() was equivalent [x for x in y if z]. However, in 3, this same code would leave list c containing a filter object, not a filtered list. To produce the same values in 3:

lst_a = range(25) #arbitrary list
lst_b = [num for num in lst_a if num % 2 == 0]
lst_c = list(filter(lambda num: num %2 == 0, lst_a))

The problem is that list() takes an iterable as it’s argument, and creates a new list from that argument. The result is that using filter in this way in python 3 takes up to twice as long as the [x for x in y if z] method because you have to iterate over the output from filter() as well as the original list.


回答 5

一个重要的区别是列表理解将list在过滤器返回a 的同时返回filter,而您不能像a那样操作list(即:对其进行调用len,但不能与的返回一起使用filter)。

我自己的自学使我遇到了一些类似的问题。

话虽这么说,如果有一种方法可以list从a中获得结果filter,就像您在.NET中所做的那样lst.Where(i => i.something()).ToList(),我很想知道。

编辑:这是Python 3而不是2的情况(请参阅注释中的讨论)。

An important difference is that list comprehension will return a list while the filter returns a filter, which you cannot manipulate like a list (ie: call len on it, which does not work with the return of filter).

My own self-learning brought me to some similar issue.

That being said, if there is a way to have the resulting list from a filter, a bit like you would do in .NET when you do lst.Where(i => i.something()).ToList(), I am curious to know it.

EDIT: This is the case for Python 3, not 2 (see discussion in comments).


回答 6

我发现第二种方法更具可读性。它确切地告诉您意图是什么:过滤列表。
PS:请勿将“列表”用作变量名

I find the second way more readable. It tells you exactly what the intention is: filter the list.
PS: do not use ‘list’ as a variable name


回答 7

filter如果使用内置函数,通常会稍快一些。

我希望列表理解在您的情况下会更快

generally filter is slightly faster if using a builtin function.

I would expect the list comprehension to be slightly faster in your case


回答 8

过滤器就是这样。它过滤出列表的元素。您可以看到定义中提到的内容相同(在我之前提到的官方文档链接中)。然而,列表理解是什么,作用于后产生一个新的列表的东西前面的列表上。(两个过滤器和列表理解创造了新的名单,并取代旧的名单无法执行操作。这里一个新的名单是像一个列表(例如,一种全新的数据类型。例如将整数转换为字符串等)

在您的示例中,按照定义,使用过滤器比使用列表理解更好。但是,如果您想从列表元素中说other_attribute,在您的示例中要作为新列表进行检索,则可以使用列表理解。

return [item.other_attribute for item in my_list if item.attribute==value]

这就是我实际上记得有关过滤器和列表理解的方式。删除列表中的一些内容并保持其他元素不变,请使用过滤器。在元素上自行使用一些逻辑,并创建适合于某些目的的缩减列表,使用列表理解。

Filter is just that. It filters out the elements of a list. You can see the definition mentions the same(in the official docs link I mentioned before). Whereas, list comprehension is something that produces a new list after acting upon something on the previous list.(Both filter and list comprehension creates new list and not perform operation in place of the older list. A new list here is something like a list with, say, an entirely new data type. Like converting integers to string ,etc)

In your example, it is better to use filter than list comprehension, as per the definition. However, if you want, say other_attribute from the list elements, in your example is to be retrieved as a new list, then you can use list comprehension.

return [item.other_attribute for item in my_list if item.attribute==value]

This is how I actually remember about filter and list comprehension. Remove a few things within a list and keep the other elements intact, use filter. Use some logic on your own at the elements and create a watered down list suitable for some purpose, use list comprehension.


回答 9

这是我需要在列表理解进行筛选时使用的一小段内容。只是过滤器,lambda和列表(也称为猫的忠诚度和狗的清洁度)的组合。

在这种情况下,我正在读取文件,删除空白行,注释掉行,以及对行进行注释后的所有内容:

# Throw out blank lines and comments
with open('file.txt', 'r') as lines:        
    # From the inside out:
    #    [s.partition('#')[0].strip() for s in lines]... Throws out comments
    #   filter(lambda x: x!= '', [s.part... Filters out blank lines
    #  y for y in filter... Converts filter object to list
    file_contents = [y for y in filter(lambda x: x != '', [s.partition('#')[0].strip() for s in lines])]

Here’s a short piece I use when I need to filter on something after the list comprehension. Just a combination of filter, lambda, and lists (otherwise known as the loyalty of a cat and the cleanliness of a dog).

In this case I’m reading a file, stripping out blank lines, commented out lines, and anything after a comment on a line:

# Throw out blank lines and comments
with open('file.txt', 'r') as lines:        
    # From the inside out:
    #    [s.partition('#')[0].strip() for s in lines]... Throws out comments
    #   filter(lambda x: x!= '', [s.part... Filters out blank lines
    #  y for y in filter... Converts filter object to list
    file_contents = [y for y in filter(lambda x: x != '', [s.partition('#')[0].strip() for s in lines])]

回答 10

除了可接受的答案外,还有一个极端的情况,您应该使用过滤器而不是列表推导。如果列表不可散列,则无法直接使用列表推导处理它。一个真实的例子是,如果您用来pyodbc从数据库中读取结果。在fetchAll()从结果cursor是unhashable列表。在这种情况下,要直接处理返回的结果,应使用过滤器:

cursor.execute("SELECT * FROM TABLE1;")
data_from_db = cursor.fetchall()
processed_data = filter(lambda s: 'abc' in s.field1 or s.StartTime >= start_date_time, data_from_db) 

如果您在此处使用列表理解,则会出现错误:

TypeError:无法散列的类型:“列表”

In addition to the accepted answer, there is a corner case when you should use filter instead of a list comprehension. If the list is unhashable you cannot directly process it with a list comprehension. A real world example is if you use pyodbc to read results from a database. The fetchAll() results from cursor is an unhashable list. In this situation, to directly manipulating on the returned results, filter should be used:

cursor.execute("SELECT * FROM TABLE1;")
data_from_db = cursor.fetchall()
processed_data = filter(lambda s: 'abc' in s.field1 or s.StartTime >= start_date_time, data_from_db) 

If you use list comprehension here you will get the error:

TypeError: unhashable type: ‘list’


回答 11

我花了一些时间熟悉higher order functions filterand map。因此,我习惯了它们,实际上我很喜欢filter,因为很明显它通过保留真实内容来进行过滤,而且我知道一些functional programming术语也很酷。

然后,我读了这段文章(Fluent Python书):

map和filter函数仍是Python 3中的内置函数,但是由于引入了列表理解和生成器表达式,因此它们并不那么重要。listcomp或genexp可将地图和过滤器组合在一起,但可读性更高。

现在,我想,如果您可以使用已经很广泛的成语(例如列表理解)来实现filter/ 的概念,那又何必困扰 map呢?而且mapsfilters是种功能。在这种情况下,我更喜欢使用Anonymous functionslambda。

最后,仅出于测试目的,我对这两种方法(maplistComp)都进行了计时,但没有看到任何相关的速度差异来证明对此进行论证的合理性。

from timeit import Timer

timeMap = Timer(lambda: list(map(lambda x: x*x, range(10**7))))
print(timeMap.timeit(number=100))

timeListComp = Timer(lambda:[(lambda x: x*x) for x in range(10**7)])
print(timeListComp.timeit(number=100))

#Map:                 166.95695265199174
#List Comprehension   177.97208347299602

It took me some time to get familiarized with the higher order functions filter and map. So i got used to them and i actually liked filter as it was explicit that it filters by keeping whatever is truthy and I’ve felt cool that I knew some functional programming terms.

Then I read this passage (Fluent Python Book):

The map and filter functions are still builtins in Python 3, but since the introduction of list comprehensions and generator ex‐ pressions, they are not as important. A listcomp or a genexp does the job of map and filter combined, but is more readable.

And now I think, why bother with the concept of filter / map if you can achieve it with already widely spread idioms like list comprehensions. Furthermore maps and filters are kind of functions. In this case I prefer using Anonymous functions lambdas.

Finally, just for the sake of having it tested, I’ve timed both methods (map and listComp) and I didn’t see any relevant speed difference that would justify making arguments about it.

from timeit import Timer

timeMap = Timer(lambda: list(map(lambda x: x*x, range(10**7))))
print(timeMap.timeit(number=100))

timeListComp = Timer(lambda:[(lambda x: x*x) for x in range(10**7)])
print(timeListComp.timeit(number=100))

#Map:                 166.95695265199174
#List Comprehension   177.97208347299602

回答 12

奇怪的是,在Python 3上,我看到过滤器的执行速度快于列表推导。

我一直认为列表理解会更有效。类似于:[如果名称不是None,则在brand_names_db中使用名称命名]生成的字节码要好一些。

>>> def f1(seq):
...     return list(filter(None, seq))
>>> def f2(seq):
...     return [i for i in seq if i is not None]
>>> disassemble(f1.__code__)
2         0 LOAD_GLOBAL              0 (list)
          2 LOAD_GLOBAL              1 (filter)
          4 LOAD_CONST               0 (None)
          6 LOAD_FAST                0 (seq)
          8 CALL_FUNCTION            2
         10 CALL_FUNCTION            1
         12 RETURN_VALUE
>>> disassemble(f2.__code__)
2           0 LOAD_CONST               1 (<code object <listcomp> at 0x10cfcaa50, file "<stdin>", line 2>)
          2 LOAD_CONST               2 ('f2.<locals>.<listcomp>')
          4 MAKE_FUNCTION            0
          6 LOAD_FAST                0 (seq)
          8 GET_ITER
         10 CALL_FUNCTION            1
         12 RETURN_VALUE

但是它们实际上要慢一些:

   >>> timeit(stmt="f1(range(1000))", setup="from __main__ import f1,f2")
   21.177661532000116
   >>> timeit(stmt="f2(range(1000))", setup="from __main__ import f1,f2")
   42.233950221000214

Curiously on Python 3, I see filter performing faster than list comprehensions.

I always thought that the list comprehensions would be more performant. Something like: [name for name in brand_names_db if name is not None] The bytecode generated is a bit better.

>>> def f1(seq):
...     return list(filter(None, seq))
>>> def f2(seq):
...     return [i for i in seq if i is not None]
>>> disassemble(f1.__code__)
2         0 LOAD_GLOBAL              0 (list)
          2 LOAD_GLOBAL              1 (filter)
          4 LOAD_CONST               0 (None)
          6 LOAD_FAST                0 (seq)
          8 CALL_FUNCTION            2
         10 CALL_FUNCTION            1
         12 RETURN_VALUE
>>> disassemble(f2.__code__)
2           0 LOAD_CONST               1 (<code object <listcomp> at 0x10cfcaa50, file "<stdin>", line 2>)
          2 LOAD_CONST               2 ('f2.<locals>.<listcomp>')
          4 MAKE_FUNCTION            0
          6 LOAD_FAST                0 (seq)
          8 GET_ITER
         10 CALL_FUNCTION            1
         12 RETURN_VALUE

But they are actually slower:

   >>> timeit(stmt="f1(range(1000))", setup="from __main__ import f1,f2")
   21.177661532000116
   >>> timeit(stmt="f2(range(1000))", setup="from __main__ import f1,f2")
   42.233950221000214

回答 13

我拿

def filter_list(list, key, value, limit=None):
    return [i for i in list if i[key] == value][:limit]

My take

def filter_list(list, key, value, limit=None):
    return [i for i in list if i[key] == value][:limit]