分类目录归档:知识问答

Python日期字符串到日期对象

问题:Python日期字符串到日期对象

如何在python中将字符串转换为日期对象?

该字符串是:"24052010"(对应于格式:"%d%m%Y"

我不想要datetime.datetime对象,而是想要datetime.date。

How do I convert a string to a date object in python?

The string would be: "24052010" (corresponding to the format: "%d%m%Y")

I don’t want a datetime.datetime object, but rather a datetime.date.


回答 0

您可以strptimedatetimePython包中使用:

>>> import datetime
>>> datetime.datetime.strptime('24052010', "%d%m%Y").date()
datetime.date(2010, 5, 24)

You can use strptime in the datetime package of Python:

>>> import datetime
>>> datetime.datetime.strptime('24052010', "%d%m%Y").date()
datetime.date(2010, 5, 24)

回答 1

import datetime
datetime.datetime.strptime('24052010', '%d%m%Y').date()
import datetime
datetime.datetime.strptime('24052010', '%d%m%Y').date()

回答 2

直接相关的问题:

如果有的话

datetime.datetime.strptime("2015-02-24T13:00:00-08:00", "%Y-%B-%dT%H:%M:%S-%H:%M").date()

你会得到:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/_strptime.py", line 308, in _strptime
    format_regex = _TimeRE_cache.compile(format)
  File "/usr/local/lib/python2.7/_strptime.py", line 265, in compile
    return re_compile(self.pattern(format), IGNORECASE)
  File "/usr/local/lib/python2.7/re.py", line 194, in compile
    return _compile(pattern, flags)
  File "/usr/local/lib/python2.7/re.py", line 251, in _compile
    raise error, v # invalid expression
sre_constants.error: redefinition of group name 'H' as group 7; was group 4

并且您尝试了:

<-24T13:00:00-08:00", "%Y-%B-%dT%HH:%MM:%SS-%HH:%MM").date()

但是您仍然可以得到上面的追溯。

回答:

>>> from dateutil.parser import parse
>>> from datetime import datetime
>>> parse("2015-02-24T13:00:00-08:00")
datetime.datetime(2015, 2, 24, 13, 0, tzinfo=tzoffset(None, -28800))

Directly related question:

What if you have

datetime.datetime.strptime("2015-02-24T13:00:00-08:00", "%Y-%B-%dT%H:%M:%S-%H:%M").date()

and you get:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/_strptime.py", line 308, in _strptime
    format_regex = _TimeRE_cache.compile(format)
  File "/usr/local/lib/python2.7/_strptime.py", line 265, in compile
    return re_compile(self.pattern(format), IGNORECASE)
  File "/usr/local/lib/python2.7/re.py", line 194, in compile
    return _compile(pattern, flags)
  File "/usr/local/lib/python2.7/re.py", line 251, in _compile
    raise error, v # invalid expression
sre_constants.error: redefinition of group name 'H' as group 7; was group 4

and you tried:

<-24T13:00:00-08:00", "%Y-%B-%dT%HH:%MM:%SS-%HH:%MM").date()

but you still get the traceback above.

Answer:

>>> from dateutil.parser import parse
>>> from datetime import datetime
>>> parse("2015-02-24T13:00:00-08:00")
datetime.datetime(2015, 2, 24, 13, 0, tzinfo=tzoffset(None, -28800))

回答 3

如果您懒惰并且不想与字符串文字打架,则可以使用该parser模块。

from dateutil import parser
dt = parser.parse("Jun 1 2005  1:33PM")
print(dt.year, dt.month, dt.day,dt.hour, dt.minute, dt.second)
>2005 6 1 13 33 0

只是一点说明,我们在尝试匹配any字符串表示形式时,它的速度比10倍慢strptime

If you are lazy and don’t want to fight with string literals, you can just go with the parser module.

from dateutil import parser
dt = parser.parse("Jun 1 2005  1:33PM")
print(dt.year, dt.month, dt.day,dt.hour, dt.minute, dt.second)
>2005 6 1 13 33 0

Just a side note, as we are trying to match any string representation, it is 10x slower than strptime


回答 4

您有一个像这样的日期字符串“ 24052010”,并且您想要这个日期对象,

from datetime import datetime
cus_date = datetime.strptime("24052010", "%d%m%Y").date()

此cus_date将为您提供日期对象。

您可以使用此方法从日期对象中检索日期字符串,

cus_date.strftime("%d%m%Y")

you have a date string like this, “24052010” and you want date object for this,

from datetime import datetime
cus_date = datetime.strptime("24052010", "%d%m%Y").date()

this cus_date will give you date object.

you can retrieve date string from your date object using this,

cus_date.strftime("%d%m%Y")

回答 5

还有一个叫做arrowpython的很棒的库,可以对python日期进行操作。

import arrow
import datetime

a = arrow.get('24052010', 'DMYYYY').date()
print(isinstance(a, datetime.date)) # True

There is another library called arrow really great to make manipulation on python date.

import arrow
import datetime

a = arrow.get('24052010', 'DMYYYY').date()
print(isinstance(a, datetime.date)) # True

回答 6

使用时间模块转换数据。

程式码片段:

import time 
tring='20150103040500'
var = int(time.mktime(time.strptime(tring, '%Y%m%d%H%M%S')))
print var

Use time module to convert data.

Code snippet:

import time 
tring='20150103040500'
var = int(time.mktime(time.strptime(tring, '%Y%m%d%H%M%S')))
print var

查找字符串中最后出现的子字符串的索引

问题:查找字符串中最后出现的子字符串的索引

我想在给定的输入string中找到某个子字符串最后一次出现的位置(或索引)str

例如,假设输入字符串为str = 'hello',子字符串为target = 'l',则它应输出3。

我怎样才能做到这一点?

I want to find the position (or index) of the last occurrence of a certain substring in given input string str.

For example, suppose the input string is str = 'hello' and the substring is target = 'l', then it should output 3.

How can I do this?


回答 0

用途.rfind()

>>> s = 'hello'
>>> s.rfind('l')
3

另外,请勿将其str用作变量名,否则将使内置的阴影变暗str()

Use .rfind():

>>> s = 'hello'
>>> s.rfind('l')
3

Also don’t use str as variable name or you’ll shadow the built-in str().


回答 1

您可以使用rfind()Python2链接:rindex()
rfind() rindex()

>>> s = 'Hello StackOverflow Hi everybody'

>>> print( s.rfind('H') )
20

>>> print( s.rindex('H') )
20

>>> print( s.rfind('other') )
-1

>>> print( s.rindex('other') )
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: substring not found

区别在于未找到子字符串时rfind()返回,-1rindex()引发异常ValueError(Python2链接:)ValueError

如果您不想检查rfind()返回码-1,则可能会希望rindex()提供一个可理解的错误消息。否则,您可能会搜索分钟,其中意外值-1来自您的代码…


示例:搜索最后一个换行符

>>> txt = '''first line
... second line
... third line'''

>>> txt.rfind('\n')
22

>>> txt.rindex('\n')
22

You can use rfind() or rindex()
Python2 links: rfind() rindex()

>>> s = 'Hello StackOverflow Hi everybody'

>>> print( s.rfind('H') )
20

>>> print( s.rindex('H') )
20

>>> print( s.rfind('other') )
-1

>>> print( s.rindex('other') )
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: substring not found

The difference is when the substring is not found, rfind() returns -1 while rindex() raises an exception ValueError (Python2 link: ValueError).

If you do not want to check the rfind() return code -1, you may prefer rindex() that will provide an understandable error message. Else you may search for minutes where the unexpected value -1 is coming from within your code…


Example: Search of last newline character

>>> txt = '''first line
... second line
... third line'''

>>> txt.rfind('\n')
22

>>> txt.rindex('\n')
22

回答 2

使用str.rindex方法。

>>> 'hello'.rindex('l')
3
>>> 'hello'.index('l')
2

Use the str.rindex method.

>>> 'hello'.rindex('l')
3
>>> 'hello'.index('l')
2

回答 3

尝试这个:

s = 'hello plombier pantin'
print (s.find('p'))
6
print (s.index('p'))
6
print (s.rindex('p'))
15
print (s.rfind('p'))

Try this:

s = 'hello plombier pantin'
print (s.find('p'))
6
print (s.index('p'))
6
print (s.rindex('p'))
15
print (s.rfind('p'))

回答 4

more_itertools库提供了用于查找所有字符或所有子字符串的索引的工具

给定

import more_itertools as mit


s = "hello"
pred = lambda x: x == "l"

性格

现在有rlocate可用的工具:

next(mit.rlocate(s, pred))
# 3

补充工具是locate

list(mit.locate(s, pred))[-1]
# 3

mit.last(mit.locate(s, pred))
# 3

子串

还有一个window_size参数可用于查找多个项目的前导项目:

s = "How much wood would a woodchuck chuck if a woodchuck could chuck wood?"
substring = "chuck"
pred = lambda *args: args == tuple(substring)

next(mit.rlocate(s, pred=pred, window_size=len(substring)))
# 59

The more_itertools library offers tools for finding indices of all characters or all substrings.

Given

import more_itertools as mit


s = "hello"
pred = lambda x: x == "l"

Code

Characters

Now there is the rlocate tool available:

next(mit.rlocate(s, pred))
# 3

A complementary tool is locate:

list(mit.locate(s, pred))[-1]
# 3

mit.last(mit.locate(s, pred))
# 3

Substrings

There is also a window_size parameter available for locating the leading item of several items:

s = "How much wood would a woodchuck chuck if a woodchuck could chuck wood?"
substring = "chuck"
pred = lambda *args: args == tuple(substring)

next(mit.rlocate(s, pred=pred, window_size=len(substring)))
# 59

回答 5

尚未尝试恢复无效的帖子,但是由于尚未发布…

(这是我在发现此问题之前的做法)

s = "hello"
target = "l"
last_pos = len(s) - 1 - s[::-1].index(target)

说明:当您搜索最后一个匹配项时,实际上是在搜索反向字符串中的第一个匹配项。知道了这一点,我做了s[::-1](返回一个反向字符串),然后target从那里索引了。然后我这样做len(s) - 1 - the index found是因为我们希望索引不被反转(即原始)字符串中建立。

不过要当心!如果target超过一个字符,则可能无法在反向字符串中找到它。要解决此问题,请使用last_pos = len(s) - 1 - s[::-1].index(target[::-1]),它会搜索的反向版本target

Not trying to resurrect an inactive post, but since this hasn’t been posted yet…

(This is how I did it before finding this question)

s = "hello"
target = "l"
last_pos = len(s) - 1 - s[::-1].index(target)

Explanation: When you’re searching for the last occurrence, really you’re searching for the first occurrence in the reversed string. Knowing this, I did s[::-1] (which returns a reversed string), and then indexed the target from there. Then I did len(s) - 1 - the index found because we want the index in the unreversed (i.e. original) string.

Watch out, though! If target is more than one character, you probably won’t find it in the reversed string. To fix this, use last_pos = len(s) - 1 - s[::-1].index(target[::-1]), which searches for a reversed version of target.


回答 6

如果您不想使用rfind,则可以解决问题/

def find_last(s, t):
    last_pos = -1
    while True:
        pos = s.find(t, last_pos + 1)
        if pos == -1:
            return last_pos
        else:
            last_pos = pos

If you don’t wanna use rfind then this will do the trick/

def find_last(s, t):
    last_pos = -1
    while True:
        pos = s.find(t, last_pos + 1)
        if pos == -1:
            return last_pos
        else:
            last_pos = pos

回答 7

您可以使用rindex()函数获取字符串中字符的最后一次出现

s="hellloooloo"
b='l'
print(s.rindex(b))

you can use rindex() function to get the last occurrence of a character in string

s="hellloooloo"
b='l'
print(s.rindex(b))

是否有标准化的方法可以在Python中交换两个变量?

问题:是否有标准化的方法可以在Python中交换两个变量?

在Python中,我已经看到使用此语法交换了两个变量值:

left, right = right, left

这是否被认为是交换两个变量值的标准方法,或者是否有其他一些方式可以按照惯例最通常地交换两个变量?

In Python, I’ve seen two variable values swapped using this syntax:

left, right = right, left

Is this considered the standard way to swap two variable values or is there some other means by which two variables are by convention most usually swapped?


回答 0

Python从左到右计算表达式。请注意,在评估分配时,右侧的评估先于左侧。

http://docs.python.org/3/reference/expressions.html#evaluation-order

这意味着该表达式的以下内容a,b = b,a

  • 对右侧b,a进行求值,即在内存中创建两个元素的元组。这两个元素是由标识符b和所指定的对象,这些对象a在程序执行期间对指令进行加密之前就已存在
  • 创建该元组后,仍未分配该元组对象,但这没关系,Python内部知道它在哪里
  • 然后,评估左侧,即将元组分配给左侧
  • 由于左侧是由两个标识符组成的,因此将元组解压缩,以便将第一个标识符a分配给元组的第一个元素(这是交换之前为b的对象,因为它具有名称b
    ,并且第二个标识符b分配给元组的第二个元素(该对象以前是交换之前的a,因为其标识符为a

该机制有效地交换了分配给标识符的对象,a并且b

因此,回答您的问题:是的,这是在两个对象上交换两个标识符的标准方法。
顺便说一下,对象不是变量,而是对象。

Python evaluates expressions from left to right. Notice that while evaluating an assignment, the right-hand side is evaluated before the left-hand side.

http://docs.python.org/3/reference/expressions.html#evaluation-order

That means the following for the expression a,b = b,a :

  • the right-hand side b,a is evaluated, that is to say a tuple of two elements is created in the memory. The two element are the objects designated by the identifiers b and a, that were existing before the instruction is encoutered during an execution of program
  • just after the creation of this tuple, no assignement of this tuple object have still been made, but it doesn’t matter, Python internally knows where it is
  • then, the left-hand side is evaluated, that is to say the tuple is assigned to the left-hand side
  • as the left-hand side is composed of two identifiers, the tuple is unpacked in order that the first identifier a be assigned to the first element of the tuple (which is the object that was formely b before the swap because it had name b)
    and the second identifier b is assigned to the second element of the tuple (which is the object that was formerly a before the swap because its identifiers was a)

This mechanism has effectively swapped the objects assigned to the identifiers a and b

So, to answer your question: YES, it’s the standard way to swap two identifiers on two objects.
By the way, the objects are not variables, they are objects.


回答 1

这是交换两个变量的标准方法,是的。

That is the standard way to swap two variables, yes.


回答 2

我知道三种交换变量的方法,但是a, b = b, a最简单。有

XOR(整数)

x = x ^ y
y = y ^ x
x = x ^ y

或简而言之,

x ^= y
y ^= x
x ^= y

临时变量

w = x
x = y
y = w
del w

元组交换

x, y = y, x

I know three ways to swap variables, but a, b = b, a is the simplest. There is

XOR (for integers)

x = x ^ y
y = y ^ x
x = x ^ y

Or concisely,

x ^= y
y ^= x
x ^= y

Temporary variable

w = x
x = y
y = w
del w

Tuple swap

x, y = y, x

回答 3

我不会说这是一种标准的交换方式,因为它将导致一些意外错误。

nums[i], nums[nums[i] - 1] = nums[nums[i] - 1], nums[i]

nums[i]将首先被修改,然后影响第二个变量nums[nums[i] - 1]

I would not say it is a standard way to swap because it will cause some unexpected errors.

nums[i], nums[nums[i] - 1] = nums[nums[i] - 1], nums[i]

nums[i] will be modified first and then affect the second variable nums[nums[i] - 1].


回答 4

不适用于多维数组,因为此处使用了引用。

import numpy as np

# swaps
data = np.random.random(2)
print(data)
data[0], data[1] = data[1], data[0]
print(data)

# does not swap
data = np.random.random((2, 2))
print(data)
data[0], data[1] = data[1], data[0]
print(data)

另请参见交换Numpy数组的切片

Does not work for multidimensional arrays, because references are used here.

import numpy as np

# swaps
data = np.random.random(2)
print(data)
data[0], data[1] = data[1], data[0]
print(data)

# does not swap
data = np.random.random((2, 2))
print(data)
data[0], data[1] = data[1], data[0]
print(data)

See also Swap slices of Numpy arrays


回答 5

为了解决eyquem解释的问题,您可以使用copy模块通过一个函数返回一个包含(反向)值副本的元组:

from copy import copy

def swapper(x, y):
  return (copy(y), copy(x))

与的功能相同lambda

swapper = lambda x, y: (copy(y), copy(x))

然后,将它们分配给所需的名称,如下所示:

x, y = swapper(y, x)

注意:如果需要,可以导入/使用deepcopy而不是copy

To get around the problems explained by eyquem, you could use the copy module to return a tuple containing (reversed) copies of the values, via a function:

from copy import copy

def swapper(x, y):
  return (copy(y), copy(x))

Same function as a lambda:

swapper = lambda x, y: (copy(y), copy(x))

Then, assign those to the desired names, like this:

x, y = swapper(y, x)

NOTE: if you wanted to you could import/use deepcopy instead of copy.


回答 6

您可以组合元组XOR交换:x,y = x ^ x ^ y,x ^ y ^ y

x, y = 10, 20

print('Before swapping: x = %s, y = %s '%(x,y))

x, y = x ^ x ^ y, x ^ y ^ y

print('After swapping: x = %s, y = %s '%(x,y))

要么

x, y = 10, 20

print('Before swapping: x = %s, y = %s '%(x,y))

print('After swapping: x = %s, y = %s '%(x ^ x ^ y, x ^ y ^ y))

使用lambda

x, y = 10, 20

print('Before swapping: x = %s, y = %s' % (x, y))

swapper = lambda x, y : ((x ^ x ^ y), (x ^ y ^ y))

print('After swapping: x = %s, y = %s ' % swapper(x, y))

输出:

Before swapping: x =  10 , y =  20
After swapping: x =  20 , y =  10

You can combine tuple and XOR swaps: x, y = x ^ x ^ y, x ^ y ^ y

x, y = 10, 20

print('Before swapping: x = %s, y = %s '%(x,y))

x, y = x ^ x ^ y, x ^ y ^ y

print('After swapping: x = %s, y = %s '%(x,y))

or

x, y = 10, 20

print('Before swapping: x = %s, y = %s '%(x,y))

print('After swapping: x = %s, y = %s '%(x ^ x ^ y, x ^ y ^ y))

Using lambda:

x, y = 10, 20

print('Before swapping: x = %s, y = %s' % (x, y))

swapper = lambda x, y : ((x ^ x ^ y), (x ^ y ^ y))

print('After swapping: x = %s, y = %s ' % swapper(x, y))

Output:

Before swapping: x =  10 , y =  20
After swapping: x =  20 , y =  10

禁止InsecureRequestWarning:在Python2.6中发出未经验证的HTTPS请求

问题:禁止InsecureRequestWarning:在Python2.6中发出未经验证的HTTPS请求

我正在使用pyVmomi并使用一种连接方法在Python2.6中编写脚本:

service_instance = connect.SmartConnect(host=args.ip,
                                        user=args.user,
                                        pwd=args.password)

我收到以下警告:

/usr/lib/python2.6/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)

有趣的是,我没有随pip一起安装urllib3(但在/usr/lib/python2.6/site-packages/requests/packages/urllib3/中)。

我已经尝试按照这里的建议

import urllib3
...
urllib3.disable_warnings()

但这并没有改变任何东西。

I am writing scripts in Python2.6 with use of pyVmomi and while using one of the connection methods:

service_instance = connect.SmartConnect(host=args.ip,
                                        user=args.user,
                                        pwd=args.password)

I get the following warning:

/usr/lib/python2.6/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)

What’s interesting is that I do not have urllib3 installed with pip (but it’s there in /usr/lib/python2.6/site-packages/requests/packages/urllib3/).

I have tried as suggested here

import urllib3
...
urllib3.disable_warnings()

but that didn’t change anything.


回答 0

您可以通过PYTHONWARNINGS环境变量禁用任何Python警告。在这种情况下,您需要:

export PYTHONWARNINGS="ignore:Unverified HTTPS request"

要禁用使用Python代码(requests >= 2.16.0):

import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

对于requests < 2.16.0,请参见下面的原始答案。

原始答案

这样urllib3.disable_warnings()做对您不起作用的原因是,您似乎正在使用请求中提供的urllib3的单独实例。

我根据这里的路径收集此信息: /usr/lib/python2.6/site-packages/requests/packages/urllib3/connectionpool.py

要在请求的供应商urllib3中禁用警告,您需要导入模块的特定实例:

import requests
from requests.packages.urllib3.exceptions import InsecureRequestWarning

requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

You can disable any Python warnings via the PYTHONWARNINGS environment variable. In this case, you want:

export PYTHONWARNINGS="ignore:Unverified HTTPS request"

To disable using Python code (requests >= 2.16.0):

import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

For requests < 2.16.0, see original answer below.

Original answer

The reason doing urllib3.disable_warnings() didn’t work for you is because it looks like you’re using a separate instance of urllib3 vendored inside of requests.

I gather this based on the path here: /usr/lib/python2.6/site-packages/requests/packages/urllib3/connectionpool.py

To disable warnings in requests’ vendored urllib3, you’ll need to import that specific instance of the module:

import requests
from requests.packages.urllib3.exceptions import InsecureRequestWarning

requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

回答 1

这是在2017年的答案urllib3不是的一部分requests

import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

This is the answer in 2017. urllib3 not a part of requests anymore

import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

回答 2

根据这个github评论,可以urllib3通过requests1-liner 禁用请求警告:

requests.packages.urllib3.disable_warnings()

但这不仅会抑制所有警告InsecureRequest(即也会抑制InsecurePlatform等)。如果我们只是想让东西起作用,我会觉得简洁很方便。

Per this github comment, one can disable urllib3 request warnings via requests in a 1-liner:

requests.packages.urllib3.disable_warnings()

This will suppress all warnings though, not just InsecureRequest (ie it will also suppress InsecurePlatform etc). In cases where we just want stuff to work, I find the conciseness handy.


回答 3

正确的方法是阅读所提供链接上的相关部分,并按照说明进行操作。根据CA证书-高级用法-要求2.8.1文档的特定方式requests(捆绑有其自己的副本urllib3):

  • requests 带有自己的证书捆绑包(但只能与模块一起更新)
  • 它将使用(因为requests v2.4.0certifi,而不是如果已安装它

HTTPS证书验证安全性措施不能轻易丢弃。它阻止了中间人攻击,从而保护了您免受第三方的侵害,例如在其中感染病毒,篡改或窃取数据。

如今,借助政府支持的全球黑客攻击活动(如量身定制的访问操作针对中国网络基础架构的中国防火墙),您比您想象的更有可能。

The correct way is to read the relevant section on the provided link and do as it says. The way specific for requests (which bundles with its own copy of urllib3), as per CA Certificates — Advanced Usage — Requests 2.8.1 documentation:

  • requests ships with its own certificate bundle (but it can only be updated together with the module)
  • it will use (since requests v2.4.0) the certifi package instead if it’s installed

The HTTPS certificate verification security measure isn’t something to be discarded light-heartedly. The Man-in-the-middle attack that it prevents safeguards you from a third party e.g. sipping a virus in or tampering with or stealing your data.

Which, with today’s government-backed global hacking operations like Tailored Access Operations and the Great Firewall of China that target network infrastructure, is more probable than you think.


回答 4

不耐烦的是,禁用python未经验证的HTTPS警告的快速方法:

export PYTHONWARNINGS="ignore:Unverified HTTPS request"

For impatient, a quick way to disable python unverified HTTPS warning:

export PYTHONWARNINGS="ignore:Unverified HTTPS request"

回答 5

如果某些软件包供应商拥有自己的urllib3副本,则可接受的答案不起作用,在这种情况下,它仍然有效:

import warnings

warnings.filterwarnings('ignore', message='Unverified HTTPS request')

The accepted answer doesn’t work if some package vendors it’s own copy of urllib3, in which case this will still work:

import warnings

warnings.filterwarnings('ignore', message='Unverified HTTPS request')

回答 6

我的PyVmomi Client也有类似的问题。使用Python版本2.7.9,我用以下代码行解决了这个问题:

default_sslContext = ssl._create_unverified_context()
self.client = \
                Client(<vcenterip>, username=<username>, password=<passwd>,
                       sslContext=default_sslContext )

请注意,要使其正常工作,您至少需要Python 2.7.9。

I had a similar issue with PyVmomi Client. With Python Version 2.7.9, I have solved this issue with the following line of code:

default_sslContext = ssl._create_unverified_context()
self.client = \
                Client(<vcenterip>, username=<username>, password=<passwd>,
                       sslContext=default_sslContext )

Note that, for this to work, you need Python 2.7.9 atleast.


回答 7

为什么不使用pyvmomi 原始功能 SmartConnectNoSSL。他们添加了此函数June 14, 2016并为其命名ConnectNoSSL将名称更改为一天后SmartConnectNoSSL,使用它而不是通过项目中不必要的代码行传递警告?

提供一种无需SSL验证即可连接到指定服务器的标准方法。在使用自签名证书连接到服务器或希望完全忽略SSL时很有用

service_instance = connect.SmartConnectNoSSL(host=args.ip,
                                             user=args.user,
                                             pwd=args.password)

Why not using pyvmomi original function SmartConnectNoSSL. They added this function on June 14, 2016 and named it ConnectNoSSL, one day after they changed the name to SmartConnectNoSSL, use that instead of by passing the warning with unnecessary lines of code in your project?

Provides a standard method for connecting to a specified server without SSL verification. Useful when connecting to servers with self-signed certificates or when you wish to ignore SSL altogether

service_instance = connect.SmartConnectNoSSL(host=args.ip,
                                             user=args.user,
                                             pwd=args.password)

回答 8

对于Python 2.7

将环境变量PYTHONWARNINGS作为键添加,并忽略相应的值,例如:

os.environ['PYTHONWARNINGS']="ignore:Unverified HTTPS request"

For Python 2.7

Add the environment variable PYTHONWARNINGS as key and the corresponding value to be ignored like:

os.environ['PYTHONWARNINGS']="ignore:Unverified HTTPS request"


将字典转换为JSON

问题:将字典转换为JSON

r = {'is_claimed': 'True', 'rating': 3.5}
r = json.dumps(r)
file.write(str(r['rating']))

我无法访问JSON中的数据。我究竟做错了什么?

TypeError: string indices must be integers, not str
r = {'is_claimed': 'True', 'rating': 3.5}
r = json.dumps(r)
file.write(str(r['rating']))

I am not able to access my data in the JSON. What am I doing wrong?

TypeError: string indices must be integers, not str

回答 0

json.dumps()将字典转换为str对象,而不是json(dict)对象!因此,您必须使用方法将其加载strdictjson.loads()

请参阅json.dumps()作为保存方法和json.loads()检索方法。

这是代码示例,可以帮助您进一步了解它:

import json

r = {'is_claimed': 'True', 'rating': 3.5}
r = json.dumps(r)
loaded_r = json.loads(r)
loaded_r['rating'] #Output 3.5
type(r) #Output str
type(loaded_r) #Output dict

json.dumps() converts a dictionary to str object, not a json(dict) object! So you have to load your str into a dict to use it by using json.loads() method

See json.dumps() as a save method and json.loads() as a retrieve method.

This is the code sample which might help you understand it more:

import json

r = {'is_claimed': 'True', 'rating': 3.5}
r = json.dumps(r)
loaded_r = json.loads(r)
loaded_r['rating'] #Output 3.5
type(r) #Output str
type(loaded_r) #Output dict

回答 1

json.dumps()返回python字典的JSON字符串表示形式。查看文件

您不能这样做,r['rating']因为r是字符串,不再是dict

也许你的意思是

r = {'is_claimed': 'True', 'rating': 3.5}
json = json.dumps(r) # note i gave it a different name
file.write(str(r['rating']))

json.dumps() returns the JSON string representation of the python dict. See the docs

You can’t do r['rating'] because r is a string, not a dict anymore

Perhaps you meant something like

r = {'is_claimed': 'True', 'rating': 3.5}
json = json.dumps(r) # note i gave it a different name
file.write(str(r['rating']))

回答 2

无需通过使用将其转换为字符串 json.dumps()

r = {'is_claimed': 'True', 'rating': 3.5}
file.write(r['is_claimed'])
file.write(str(r['rating']))

您可以直接从dict对象获取值。

No need to convert it in a string by using json.dumps()

r = {'is_claimed': 'True', 'rating': 3.5}
file.write(r['is_claimed'])
file.write(str(r['rating']))

You can get the values directly from the dict object.


回答 3

将r定义为字典应该可以解决这个问题:

>>> r: dict = {'is_claimed': 'True', 'rating': 3.5}
>>> print(r['rating'])
3.5
>>> type(r)
<class 'dict'>

Defining r as a dictionary should do the trick:

>>> r: dict = {'is_claimed': 'True', 'rating': 3.5}
>>> print(r['rating'])
3.5
>>> type(r)
<class 'dict'>

将字典作为关键字参数传递给函数

问题:将字典作为关键字参数传递给函数

我想使用字典在python中调用一个函数。

这是一些代码:

d = dict(param='test')

def f(param):
    print(param)

f(d)

这可以打印,{'param': 'test'}但我希望只打印test

我希望它可以类似地工作以获取更多参数:

d = dict(p1=1, p2=2)
def f2(p1, p2):
    print(p1, p2)
f2(d)

这可能吗?

I’d like to call a function in python using a dictionary.

Here is some code:

d = dict(param='test')

def f(param):
    print(param)

f(d)

This prints {'param': 'test'} but I’d like it to just print test.

I’d like it to work similarly for more parameters:

d = dict(p1=1, p2=2)
def f2(p1, p2):
    print(p1, p2)
f2(d)

Is this possible?


回答 0

最后自己解决了。很简单,我只是缺少**运算符来解开字典

因此,我的示例变为:

d = dict(p1=1, p2=2)
def f2(p1,p2):
    print p1, p2
f2(**d)

Figured it out for myself in the end. It is simple, I was just missing the ** operator to unpack the dictionary

So my example becomes:

d = dict(p1=1, p2=2)
def f2(p1,p2):
    print p1, p2
f2(**d)

回答 1

In[1]: def myfunc(a=1, b=2):
In[2]:    print(a, b)

In[3]: mydict = {'a': 100, 'b': 200}

In[4]: myfunc(**mydict)
100 200

一些可能有助于了解的其他详细信息(阅读本文并经过测试后遇到的问题):

  1. 该函数可以具有字典中包含的参数
  2. 不能覆盖字典中已经存在的参数
  3. 字典不能包含函数中没有的参数。

例子:

数字1:该函数可以具有字典中未包含的参数

In[5]: mydict = {'a': 100}
In[6]: myfunc(**mydict)
100 2

数字2:您不能覆盖字典中已经存在的参数

In[7]: mydict = {'a': 100, 'b': 200}
In[8]: myfunc(a=3, **mydict)

TypeError: myfunc() got multiple values for keyword argument 'a'

数字3:字典中不能包含函数中没有的参数。

In[9]:  mydict = {'a': 100, 'b': 200, 'c': 300}
In[10]: myfunc(**mydict)

TypeError: myfunc() got an unexpected keyword argument 'c'

根据注释中的要求,第3号的解决方案是根据函数中可用的关键字参数来过滤字典:

In[11]: import inspect
In[12]: mydict = {'a': 100, 'b': 200, 'c': 300}
In[13]: filtered_mydict = {k: v for k, v in mydict.items() if k in [p.name for p in inspect.signature(myfunc).parameters.values()]}
In[14]: myfunc(**filtered_mydict)
100 200

另一种选择是在函数中接受(并忽略)其他kwarg:

In[15]: def myfunc2(a=None, **kwargs):
In[16]:    print(a)

In[17]: mydict = {'a': 100, 'b': 200, 'c': 300}

In[18]: myfunc2(**mydict)
100

请注意,除了可以有效地使用位置参数和列表或元组之外,还可以使用与kwargs相同的方式,这是一个更高级的示例,其中同时包含位置和关键字args:

In[19]: def myfunc3(a, *posargs, b=2, **kwargs):
In[20]:    print(a, b)
In[21]:    print(posargs)
In[22]:    print(kwargs)

In[23]: mylist = [10, 20, 30]
In[24]: mydict = {'b': 200, 'c': 300}

In[25]: myfunc3(*mylist, **mydict)
10 200
(20, 30)
{'c': 300}
In[1]: def myfunc(a=1, b=2):
In[2]:    print(a, b)

In[3]: mydict = {'a': 100, 'b': 200}

In[4]: myfunc(**mydict)
100 200

A few extra details that might be helpful to know (questions I had after reading this and went and tested):

  1. The function can have parameters that are not included in the dictionary
  2. You can not override a parameter that is already in the dictionary
  3. The dictionary can not have parameters that aren’t in the function.

Examples:

Number 1: The function can have parameters that are not included in the dictionary

In[5]: mydict = {'a': 100}
In[6]: myfunc(**mydict)
100 2

Number 2: You can not override a parameter that is already in the dictionary

In[7]: mydict = {'a': 100, 'b': 200}
In[8]: myfunc(a=3, **mydict)

TypeError: myfunc() got multiple values for keyword argument 'a'

Number 3: The dictionary can not have parameters that aren’t in the function.

In[9]:  mydict = {'a': 100, 'b': 200, 'c': 300}
In[10]: myfunc(**mydict)

TypeError: myfunc() got an unexpected keyword argument 'c'

As requested in comments, a solution to Number 3 is to filter the dictionary based on the keyword arguments available in the function:

In[11]: import inspect
In[12]: mydict = {'a': 100, 'b': 200, 'c': 300}
In[13]: filtered_mydict = {k: v for k, v in mydict.items() if k in [p.name for p in inspect.signature(myfunc).parameters.values()]}
In[14]: myfunc(**filtered_mydict)
100 200

Another option is to accept (and ignore) additional kwargs in your function:

In[15]: def myfunc2(a=None, **kwargs):
In[16]:    print(a)

In[17]: mydict = {'a': 100, 'b': 200, 'c': 300}

In[18]: myfunc2(**mydict)
100

Notice further than you can use positional arguments and lists or tuples in effectively the same way as kwargs, here’s a more advanced example incorporating both positional and keyword args:

In[19]: def myfunc3(a, *posargs, b=2, **kwargs):
In[20]:    print(a, b)
In[21]:    print(posargs)
In[22]:    print(kwargs)

In[23]: mylist = [10, 20, 30]
In[24]: mydict = {'b': 200, 'c': 300}

In[25]: myfunc3(*mylist, **mydict)
10 200
(20, 30)
{'c': 300}

回答 2

在python中,这称为“拆包”,您​​可以在本教程中找到有关它的信息。我同意,它的文档很烂,尤其是因为它是如此的有用。

In python, this is called “unpacking”, and you can find a bit about it in the tutorial. The documentation of it sucks, I agree, especially because of how fantasically useful it is.


回答 3

在这里-可以进行其他任何迭代:

d = {'param' : 'test'}

def f(dictionary):
    for key in dictionary:
        print key

f(d)

Here ya go – works just any other iterable:

d = {'param' : 'test'}

def f(dictionary):
    for key in dictionary:
        print key

f(d)

是否有一个与Ruby的字符串插值等效的Python?

问题:是否有一个与Ruby的字符串插值等效的Python?

Ruby示例:

name = "Spongebob Squarepants"
puts "Who lives in a Pineapple under the sea? \n#{name}."

对我而言,成功的Python字符串连接似乎很冗长。

Ruby example:

name = "Spongebob Squarepants"
puts "Who lives in a Pineapple under the sea? \n#{name}."

The successful Python string concatenation is seemingly verbose to me.


回答 0

Python 3.6将添加与Ruby的字符串插值类似的文字字符串插值。从该版本的Python(计划于2016年底发布)开始,您将能够在“ f-strings”中包含表达式,例如

name = "Spongebob Squarepants"
print(f"Who lives in a Pineapple under the sea? {name}.")

在3.6之前的版本中,最接近的是

name = "Spongebob Squarepants"
print("Who lives in a Pineapple under the sea? %(name)s." % locals())

%运算符可用于Python中的字符串插值。第一个操作数是要内插的字符串,第二个操作数可以具有不同的类型,包括“映射”,将字段名称映射到要内插的值。在这里,我使用了局部变量字典locals()来映射字段名称name为它的值作为局部变量。

使用.format()最新Python版本的方法的相同代码如下所示:

name = "Spongebob Squarepants"
print("Who lives in a Pineapple under the sea? {name!s}.".format(**locals()))

还有一个string.Template类:

tmpl = string.Template("Who lives in a Pineapple under the sea? $name.")
print(tmpl.substitute(name="Spongebob Squarepants"))

Python 3.6 will add literal string interpolation similar to Ruby’s string interpolation. Starting with that version of Python (which is scheduled to be released by the end of 2016), you will be able to include expressions in “f-strings”, e.g.

name = "Spongebob Squarepants"
print(f"Who lives in a Pineapple under the sea? {name}.")

Prior to 3.6, the closest you can get to this is

name = "Spongebob Squarepants"
print("Who lives in a Pineapple under the sea? %(name)s." % locals())

The % operator can be used for string interpolation in Python. The first operand is the string to be interpolated, the second can have different types including a “mapping”, mapping field names to the values to be interpolated. Here I used the dictionary of local variables locals() to map the field name name to its value as a local variable.

The same code using the .format() method of recent Python versions would look like this:

name = "Spongebob Squarepants"
print("Who lives in a Pineapple under the sea? {name!s}.".format(**locals()))

There is also the string.Template class:

tmpl = string.Template("Who lives in a Pineapple under the sea? $name.")
print(tmpl.substitute(name="Spongebob Squarepants"))

回答 1

从Python 2.6.X开始,您可能要使用:

"my {0} string: {1}".format("cool", "Hello there!")

Since Python 2.6.X you might want to use:

"my {0} string: {1}".format("cool", "Hello there!")

回答 2

我开发了interpy软件包,该软件包可在Python启用字符串插值

只需通过安装即可pip install interpy。然后,# coding: interpy在文件开头添加该行!

例:

#!/usr/bin/env python
# coding: interpy

name = "Spongebob Squarepants"
print "Who lives in a Pineapple under the sea? \n#{name}."

I’ve developed the interpy package, that enables string interpolation in Python.

Just install it via pip install interpy. And then, add the line # coding: interpy at the beginning of your files!

Example:

#!/usr/bin/env python
# coding: interpy

name = "Spongebob Squarepants"
print "Who lives in a Pineapple under the sea? \n#{name}."

回答 3

Python的字符串插值类似于C的printf()

如果你试试:

name = "SpongeBob Squarepants"
print "Who lives in a Pineapple under the sea? %s" % name

标签%s将被替换为name变量。您应该看一下打印功能标签:http : //docs.python.org/library/functions.html

Python’s string interpolation is similar to C’s printf()

If you try:

name = "SpongeBob Squarepants"
print "Who lives in a Pineapple under the sea? %s" % name

The tag %s will be replaced with the name variable. You should take a look to the print function tags: http://docs.python.org/library/functions.html


回答 4

按照PEP 498的规定,Python 3.6包含字符串插值。您将可以执行以下操作:

name = 'Spongebob Squarepants'
print(f'Who lives in a Pineapple under the sea? \n{name}')

请注意,我讨厌海绵宝宝,所以写这篇文章有点痛苦。:)

String interpolation is going to be included with Python 3.6 as specified in PEP 498. You will be able to do this:

name = 'Spongebob Squarepants'
print(f'Who lives in a Pineapple under the sea? \n{name}')

Note that I hate Spongebob, so writing this was slightly painful. :)


回答 5

你也可以有这个

name = "Spongebob Squarepants"
print "Who lives in a Pineapple under the sea? \n{name}.".format(name=name)

http://docs.python.org/2/library/string.html#formatstrings

You can also have this

name = "Spongebob Squarepants"
print "Who lives in a Pineapple under the sea? \n{name}.".format(name=name)

http://docs.python.org/2/library/string.html#formatstrings


回答 6

import inspect
def s(template, **kwargs):
    "Usage: s(string, **locals())"
    if not kwargs:
        frame = inspect.currentframe()
        try:
            kwargs = frame.f_back.f_locals
        finally:
            del frame
        if not kwargs:
            kwargs = globals()
    return template.format(**kwargs)

用法:

a = 123
s('{a}', locals()) # print '123'
s('{a}') # it is equal to the above statement: print '123'
s('{b}') # raise an KeyError: b variable not found

PS:性能可能有问题。这对于本地脚本很有用,而不对生产日志有用。

重复的:

import inspect
def s(template, **kwargs):
    "Usage: s(string, **locals())"
    if not kwargs:
        frame = inspect.currentframe()
        try:
            kwargs = frame.f_back.f_locals
        finally:
            del frame
        if not kwargs:
            kwargs = globals()
    return template.format(**kwargs)

Usage:

a = 123
s('{a}', locals()) # print '123'
s('{a}') # it is equal to the above statement: print '123'
s('{b}') # raise an KeyError: b variable not found

PS: performance may be a problem. This is useful for local scripts, not for production logs.

Duplicated:


回答 7

对于旧的Python(在2.4上测试),最佳解决方案指明了方向。你可以这样做:

import string

def try_interp():
    d = 1
    f = 1.1
    s = "s"
    print string.Template("d: $d f: $f s: $s").substitute(**locals())

try_interp()

你得到

d: 1 f: 1.1 s: s

For old Python (tested on 2.4) the top solution points the way. You can do this:

import string

def try_interp():
    d = 1
    f = 1.1
    s = "s"
    print string.Template("d: $d f: $f s: $s").substitute(**locals())

try_interp()

And you get

d: 1 f: 1.1 s: s

回答 8

Python 3.6和更高版本具有使用f字符串的文字字符串插值

name='world'
print(f"Hello {name}!")

Python 3.6 and newer have literal string interpolation using f-strings:

name='world'
print(f"Hello {name}!")

在Python中管道输出标准输出时设置正确的编码

问题:在Python中管道输出标准输出时设置正确的编码

当传递Python程序的输出的管道时,Python解释器会对编码感到困惑,并将其设置为None。这意味着这样的程序:

# -*- coding: utf-8 -*-
print u"åäö"

正常运行时可以正常工作,但失败:

UnicodeEncodeError:’ascii’编解码器无法在位置0编码字符u’\ xa0’:序数不在范围内(128)

以管道顺序使用时。

使管道工作的最佳方法是什么?我能告诉它使用外壳程序/文件系统/正在使用的任何编码吗?

到目前为止,我所看到的建议是直接修改site.py,或使用此hack硬编码defaultencoding:

# -*- coding: utf-8 -*-
import sys
reload(sys)
sys.setdefaultencoding('utf-8')
print u"åäö"

有没有更好的方法可以使管道工作?

When piping the output of a Python program, the Python interpreter gets confused about encoding and sets it to None. This means a program like this:

# -*- coding: utf-8 -*-
print u"åäö"

will work fine when run normally, but fail with:

UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\xa0′ in position 0: ordinal not in range(128)

when used in a pipe sequence.

What is the best way to make this work when piping? Can I just tell it to use whatever encoding the shell/filesystem/whatever is using?

The suggestions I have seen thus far is to modify your site.py directly, or hardcoding the defaultencoding using this hack:

# -*- coding: utf-8 -*-
import sys
reload(sys)
sys.setdefaultencoding('utf-8')
print u"åäö"

Is there a better way to make piping work?


回答 0

您的代码在脚本中运行时有效,因为Python将输出编码为您的终端应用程序正在使用的任何编码。如果要进行管道传输,则必须自己对其进行编码。

经验法则是:始终在内部使用Unicode。解码收到的内容,并对发送的内容进行编码。

# -*- coding: utf-8 -*-
print u"åäö".encode('utf-8')

另一个教学示例是一个Python程序,用于在ISO-8859-1和UTF-8之间进行转换,从而使两者之间的所有内容均大写。

import sys
for line in sys.stdin:
    # Decode what you receive:
    line = line.decode('iso8859-1')

    # Work with Unicode internally:
    line = line.upper()

    # Encode what you send:
    line = line.encode('utf-8')
    sys.stdout.write(line)

设置系统默认编码不是一个好主意,因为您使用的某些模块和库可能依赖于它是ASCII的事实。不要这样

Your code works when run in an script because Python encodes the output to whatever encoding your terminal application is using. If you are piping you must encode it yourself.

A rule of thumb is: Always use Unicode internally. Decode what you receive, and encode what you send.

# -*- coding: utf-8 -*-
print u"åäö".encode('utf-8')

Another didactic example is a Python program to convert between ISO-8859-1 and UTF-8, making everything uppercase in between.

import sys
for line in sys.stdin:
    # Decode what you receive:
    line = line.decode('iso8859-1')

    # Work with Unicode internally:
    line = line.upper()

    # Encode what you send:
    line = line.encode('utf-8')
    sys.stdout.write(line)

Setting the system default encoding is a bad idea, because some modules and libraries you use can rely on the fact it is ASCII. Don’t do it.


回答 1

首先,关于此解决方案:

# -*- coding: utf-8 -*-
print u"åäö".encode('utf-8')

每次都使用给定的编码显式打印是不实际的。那将是重复的并且容易出错。

更好的解决方案是sys.stdout在程序开始时进行更改,以使用选定的编码进行编码。这是我在Python上找到的一种解决方案:如何选择sys.stdout.encoding?,特别是“ toka”的评论:

import sys
import codecs
sys.stdout = codecs.getwriter('utf8')(sys.stdout)

First, regarding this solution:

# -*- coding: utf-8 -*-
print u"åäö".encode('utf-8')

It’s not practical to explicitly print with a given encoding every time. That would be repetitive and error-prone.

A better solution is to change sys.stdout at the start of your program, to encode with a selected encoding. Here is one solution I found on Python: How is sys.stdout.encoding chosen?, in particular a comment by “toka”:

import sys
import codecs
sys.stdout = codecs.getwriter('utf8')(sys.stdout)

回答 2

您可能需要尝试将环境变量“ PYTHONIOENCODING”更改为“ utf_8”。我写了一篇关于这个问题的磨难页面

博客文章的Tl; dr:

import sys, locale, os
print(sys.stdout.encoding)
print(sys.stdout.isatty())
print(locale.getpreferredencoding())
print(sys.getfilesystemencoding())
print(os.environ["PYTHONIOENCODING"])
print(chr(246), chr(9786), chr(9787))

给你

utf_8
False
ANSI_X3.4-1968
ascii
utf_8
ö ☺ ☻

You may want to try changing the environment variable “PYTHONIOENCODING” to “utf_8”. I have written a page on my ordeal with this problem.

Tl;dr of the blog post:

import sys, locale, os
print(sys.stdout.encoding)
print(sys.stdout.isatty())
print(locale.getpreferredencoding())
print(sys.getfilesystemencoding())
print(os.environ["PYTHONIOENCODING"])
print(chr(246), chr(9786), chr(9787))

gives you

utf_8
False
ANSI_X3.4-1968
ascii
utf_8
ö ☺ ☻

回答 3

export PYTHONIOENCODING=utf-8

做这项工作,但不能在python本身上设置它…

我们可以做的是验证是否未设置,并在调用脚本之前通过以下命令告诉用户进行设置:

if __name__ == '__main__':
    if (sys.stdout.encoding is None):
        print >> sys.stderr, "please set python env PYTHONIOENCODING=UTF-8, example: export PYTHONIOENCODING=UTF-8, when write to stdout."
        exit(1)

更新以回复评论:该问题仅在传递到stdout时存在。我在Fedora 25 Python 2.7.13中进行了测试

python --version
Python 2.7.13

猫b.py

#!/usr/bin/env python
#-*- coding: utf-8 -*-
import sys

print sys.stdout.encoding

运行./b.py

UTF-8

运行./b.py | 减

None
export PYTHONIOENCODING=utf-8

do the job, but can’t set it on python itself …

what we can do is verify if isn’t setting and tell the user to set it before call script with :

if __name__ == '__main__':
    if (sys.stdout.encoding is None):
        print >> sys.stderr, "please set python env PYTHONIOENCODING=UTF-8, example: export PYTHONIOENCODING=UTF-8, when write to stdout."
        exit(1)

Update to reply to the comment: the problem just exist when piping to stdout . I tested in Fedora 25 Python 2.7.13

python --version
Python 2.7.13

cat b.py

#!/usr/bin/env python
#-*- coding: utf-8 -*-
import sys

print sys.stdout.encoding

running ./b.py

UTF-8

running ./b.py | less

None

回答 4

上周有一个类似的问题。在我的IDE(PyCharm)中很容易修复。

这是我的解决方法:

从PyCharm菜单栏开始:文件->设置…->编辑器->文件编码,然后将:“ IDE编码”,“项目编码”和“属性文件的默认编码”全部设置为UTF-8,她现在可以工作了像个魅力。

希望这可以帮助!

I had a similar issue last week. It was easy to fix in my IDE (PyCharm).

Here was my fix:

Starting from PyCharm menu bar: File -> Settings… -> Editor -> File Encodings, then set: “IDE Encoding”, “Project Encoding” and “Default encoding for properties files” ALL to UTF-8 and she now works like a charm.

Hope this helps!


回答 5

克雷格·麦昆(Craig McQueen)的答案可能是经过消毒的版本。

import sys, codecs
class EncodedOut:
    def __init__(self, enc):
        self.enc = enc
        self.stdout = sys.stdout
    def __enter__(self):
        if sys.stdout.encoding is None:
            w = codecs.getwriter(self.enc)
            sys.stdout = w(sys.stdout)
    def __exit__(self, exc_ty, exc_val, tb):
        sys.stdout = self.stdout

用法:

with EncodedOut('utf-8'):
    print u'ÅÄÖåäö'

An arguable sanitized version of Craig McQueen’s answer.

import sys, codecs
class EncodedOut:
    def __init__(self, enc):
        self.enc = enc
        self.stdout = sys.stdout
    def __enter__(self):
        if sys.stdout.encoding is None:
            w = codecs.getwriter(self.enc)
            sys.stdout = w(sys.stdout)
    def __exit__(self, exc_ty, exc_val, tb):
        sys.stdout = self.stdout

Usage:

with EncodedOut('utf-8'):
    print u'ÅÄÖåäö'

回答 6

我可以通过以下方式“自动化”它:

def __fix_io_encoding(last_resort_default='UTF-8'):
  import sys
  if [x for x in (sys.stdin,sys.stdout,sys.stderr) if x.encoding is None] :
      import os
      defEnc = None
      if defEnc is None :
        try:
          import locale
          defEnc = locale.getpreferredencoding()
        except: pass
      if defEnc is None :
        try: defEnc = sys.getfilesystemencoding()
        except: pass
      if defEnc is None :
        try: defEnc = sys.stdin.encoding
        except: pass
      if defEnc is None :
        defEnc = last_resort_default
      os.environ['PYTHONIOENCODING'] = os.environ.get("PYTHONIOENCODING",defEnc)
      os.execvpe(sys.argv[0],sys.argv,os.environ)
__fix_io_encoding() ; del __fix_io_encoding

是的,如果此“ setenv”失败,则有可能在此处获得无限循环。

I could “automate” it with a call to:

def __fix_io_encoding(last_resort_default='UTF-8'):
  import sys
  if [x for x in (sys.stdin,sys.stdout,sys.stderr) if x.encoding is None] :
      import os
      defEnc = None
      if defEnc is None :
        try:
          import locale
          defEnc = locale.getpreferredencoding()
        except: pass
      if defEnc is None :
        try: defEnc = sys.getfilesystemencoding()
        except: pass
      if defEnc is None :
        try: defEnc = sys.stdin.encoding
        except: pass
      if defEnc is None :
        defEnc = last_resort_default
      os.environ['PYTHONIOENCODING'] = os.environ.get("PYTHONIOENCODING",defEnc)
      os.execvpe(sys.argv[0],sys.argv,os.environ)
__fix_io_encoding() ; del __fix_io_encoding

Yes, it’s possible to get an infinite loop here if this “setenv” fails.


回答 7

我只是以为我在这里提到了一些东西,在我最终意识到发生了什么之前,我不得不花很长时间进行试验。对于这里的每个人来说,这可能是如此明显,以至于他们都没有理会它。但是如果他们有的话,这对我会有所帮助,所以按照这个原则…!

注意:我专门使用的是Jython 2.7版,所以可能这可能不适用于CPython

NB2:我的.py文件的前两行是:

# -*- coding: utf-8 -*-
from __future__ import print_function

“%”(也称为“插值运算符”)字符串构造机制也会引起其他问题……如果“环境”的默认编码为ASCII,则尝试执行类似的操作

print( "bonjour, %s" % "fréd" )  # Call this "print A"

您将在Eclipse中运行没有困难…在Windows CLI(DOS窗口)中,您会发现编码是代码页850(我的Windows 7 OS)或类似的东西,至少可以处理欧洲带有重音符号的字符,因此它会工作的。

print( u"bonjour, %s" % "fréd" ) # Call this "print B"

也可以。

如果是OTOH,您从CLI定向到文件,则stdout编码将为None,它将默认设置为ASCII(无论如何在我的OS上),它将无法处理以上任何打印…(可怕的编码)错误)。

因此,您可能会考虑使用来重定向您的标准输出

sys.stdout = codecs.getwriter('utf8')(sys.stdout)

并尝试在CLI管道中运行到文件…很奇怪,上面的打印A可以工作…但是上面的打印B将抛出编码错误!但是,以下内容可以正常运行:

print( u"bonjour, " + "fréd" ) # Call this "print C"

我得出的结论(临时)是,如果将使用“ u”前缀指定为Unicode字符串的字符串提交给%-handling机制,则似乎涉及使用默认环境编码,无论是否已将stdout设置为重定向!

人们如何处理这是一个选择问题。我欢迎Unicode专家说出为什么会发生这种情况,我是否以某种方式出错了,对此的首选解决方案,是否也适用于CPython,它是否发生在Python 3中,等等。

I just thought I’d mention something here which I had to spent a long time experimenting with before I finally realised what was going on. This may be so obvious to everyone here that they haven’t bothered mentioning it. But it would’ve helped me if they had, so on that principle…!

NB: I am using Jython specifically, v 2.7, so just possibly this may not apply to CPython

NB2: the first two lines of my .py file here are:

# -*- coding: utf-8 -*-
from __future__ import print_function

The “%” (AKA “interpolation operator”) string construction mechanism causes ADDITIONAL problems too… If the default encoding of the “environment” is ASCII and you try to do something like

print( "bonjour, %s" % "fréd" )  # Call this "print A"

You will have no difficulty running in Eclipse… In a Windows CLI (DOS window) you will find that the encoding is code page 850 (my Windows 7 OS) or something similar, which can handle European accented characters at least, so it’ll work.

print( u"bonjour, %s" % "fréd" ) # Call this "print B"

will also work.

If, OTOH, you direct to a file from the CLI, the stdout encoding will be None, which will default to ASCII (on my OS anyway), which will not be able to handle either of the above prints… (dreaded encoding error).

So then you might think of redirecting your stdout by using

sys.stdout = codecs.getwriter('utf8')(sys.stdout)

and try running in the CLI piping to a file… Very oddly, print A above will work… But print B above will throw the encoding error! The following will however work OK:

print( u"bonjour, " + "fréd" ) # Call this "print C"

The conclusion I have come to (provisionally) is that if a string which is specified to be a Unicode string using the “u” prefix is submitted to the %-handling mechanism it appears to involve the use of the default environment encoding, regardless of whether you have set stdout to redirect!

How people deal with this is a matter of choice. I would welcome a Unicode expert to say why this happens, whether I’ve got it wrong in some way, what the preferred solution to this, whether it also applies to CPython, whether it happens in Python 3, etc., etc.


回答 8

我在旧版应用程序中遇到了这个问题,很难确定打印的内容。我帮助自己解决了这个问题:

# encoding_utf8.py
import codecs
import builtins


def print_utf8(text, **kwargs):
    print(str(text).encode('utf-8'), **kwargs)


def print_utf8(fn):
    def print_fn(*args, **kwargs):
        return fn(str(*args).encode('utf-8'), **kwargs)
    return print_fn


builtins.print = print_utf8(print)

在我的脚本之上,test.py:

import encoding_utf8
string = 'Axwell Λ Ingrosso'
print(string)

请注意,这会将所有调用更改为使用编码进行打印,因此您的控制台将打印以下内容:

$ python test.py
b'Axwell \xce\x9b Ingrosso'

I ran into this problem in a legacy application, and it was difficult to identify where what was printed. I helped myself with this hack:

# encoding_utf8.py
import codecs
import builtins


def print_utf8(text, **kwargs):
    print(str(text).encode('utf-8'), **kwargs)


def print_utf8(fn):
    def print_fn(*args, **kwargs):
        return fn(str(*args).encode('utf-8'), **kwargs)
    return print_fn


builtins.print = print_utf8(print)

On top of my script, test.py:

import encoding_utf8
string = 'Axwell Λ Ingrosso'
print(string)

Note that this changes ALL calls to print to use an encoding, so your console will print this:

$ python test.py
b'Axwell \xce\x9b Ingrosso'

回答 9

在Windows上,当从编辑器(例如Sublime Text)运行Python代码时,我经常遇到此问题,但没有从命令行运行它时。

在这种情况下,请检查编辑器的参数。对于SublimeText,这Python.sublime-build解决了它:

{
  "cmd": ["python", "-u", "$file"],
  "file_regex": "^[ ]*File \"(...*?)\", line ([0-9]*)",
  "selector": "source.python",
  "encoding": "utf8",
  "env": {"PYTHONIOENCODING": "utf-8", "LANG": "en_US.UTF-8"}
}

On Windows, I had this problem very often when running a Python code from an editor (like Sublime Text), but not if running it from command-line.

In this case, check your editor’s parameters. In the case of SublimeText, this Python.sublime-build solved it:

{
  "cmd": ["python", "-u", "$file"],
  "file_regex": "^[ ]*File \"(...*?)\", line ([0-9]*)",
  "selector": "source.python",
  "encoding": "utf8",
  "env": {"PYTHONIOENCODING": "utf-8", "LANG": "en_US.UTF-8"}
}

如何从数据框的单元格获取值?

问题:如何从数据框的单元格获取值?

我构造了一个条件,可以从我的数据帧中准确提取一行:

d2 = df[(df['l_ext']==l_ext) & (df['item']==item) & (df['wn']==wn) & (df['wd']==1)]

现在,我想从特定列中获取一个值:

val = d2['col_name']

但是结果是我得到一个包含一行一列(一个单元格)的数据帧。这不是我所需要的。我需要一个值(一个浮点数)。我该如何在熊猫中做到这一点?

I have constructed a condition that extract exactly one row from my data frame:

d2 = df[(df['l_ext']==l_ext) & (df['item']==item) & (df['wn']==wn) & (df['wd']==1)]

Now I would like to take a value from a particular column:

val = d2['col_name']

But as a result I get a data frame that contains one row and one column (i.e. one cell). It is not what I need. I need one value (one float number). How can I do it in pandas?


回答 0

如果您的DataFrame仅包含一行,则使用iloc,以Series的形式访问第一(唯一)行,然后使用列名访问值:

In [3]: sub_df
Out[3]:
          A         B
2 -0.133653 -0.030854

In [4]: sub_df.iloc[0]
Out[4]:
A   -0.133653
B   -0.030854
Name: 2, dtype: float64

In [5]: sub_df.iloc[0]['A']
Out[5]: -0.13365288513107493

If you have a DataFrame with only one row, then access the first (only) row as a Series using iloc, and then the value using the column name:

In [3]: sub_df
Out[3]:
          A         B
2 -0.133653 -0.030854

In [4]: sub_df.iloc[0]
Out[4]:
A   -0.133653
B   -0.030854
Name: 2, dtype: float64

In [5]: sub_df.iloc[0]['A']
Out[5]: -0.13365288513107493

回答 1

这些是标量的快速访问

In [15]: df = pandas.DataFrame(numpy.random.randn(5,3),columns=list('ABC'))

In [16]: df
Out[16]: 
          A         B         C
0 -0.074172 -0.090626  0.038272
1 -0.128545  0.762088 -0.714816
2  0.201498 -0.734963  0.558397
3  1.563307 -1.186415  0.848246
4  0.205171  0.962514  0.037709

In [17]: df.iat[0,0]
Out[17]: -0.074171888537611502

In [18]: df.at[0,'A']
Out[18]: -0.074171888537611502

These are fast access for scalars

In [15]: df = pandas.DataFrame(numpy.random.randn(5,3),columns=list('ABC'))

In [16]: df
Out[16]: 
          A         B         C
0 -0.074172 -0.090626  0.038272
1 -0.128545  0.762088 -0.714816
2  0.201498 -0.734963  0.558397
3  1.563307 -1.186415  0.848246
4  0.205171  0.962514  0.037709

In [17]: df.iat[0,0]
Out[17]: -0.074171888537611502

In [18]: df.at[0,'A']
Out[18]: -0.074171888537611502

回答 2

您可以将1×1数据帧转换为numpy数组,然后访问该数组的第一个也是唯一的值:

val = d2['col_name'].values[0]

You can turn your 1×1 dataframe into a numpy array, then access the first and only value of that array:

val = d2['col_name'].values[0]

回答 3

多数答案都在使用iloc,这对于按位置选择非常有用。

如果需要按标签选择 loc会更方便。

用于显式获取值(等于不推荐使用的df.get_value(’a’,’A’))

# this is also equivalent to df1.at['a','A']
In [55]: df1.loc['a', 'A'] 
Out[55]: 0.13200317033032932

Most answers are using iloc which is good for selection by position.

If you need selection-by-label loc would be more convenient.

For getting a value explicitly (equiv to deprecated df.get_value(‘a’,’A’))

# this is also equivalent to df1.at['a','A']
In [55]: df1.loc['a', 'A'] 
Out[55]: 0.13200317033032932

回答 4

我需要一个由列和索引名称选择的单元格的值。此解决方案为我工作:

original_conversion_frequency.loc[1,:].values[0]

I needed the value of one cell, selected by column and index names. This solution worked for me:

original_conversion_frequency.loc[1,:].values[0]


回答 5

熊猫10.1 / 13.1之后看起来像变化

在iloc不可用之前,我从10.1升级到13.1。

现在有了13.1,iloc[0]['label']将获得单个值数组而不是标量。

像这样:

lastprice=stock.iloc[-1]['Close']

输出:

date
2014-02-26 118.2
name:Close, dtype: float64

It looks like changes after pandas 10.1/13.1

I upgraded from 10.1 to 13.1, before iloc is not available.

Now with 13.1, iloc[0]['label'] gets a single value array rather than a scalar.

Like this:

lastprice=stock.iloc[-1]['Close']

Output:

date
2014-02-26 118.2
name:Close, dtype: float64

回答 6

我找到的最快/最简单的选项如下。501表示行索引。

df.at[501,'column_name']
df.get_value(501,'column_name')

The quickest/easiest options I have found are the following. 501 represents the row index.

df.at[501,'column_name']
df.get_value(501,'column_name')

回答 7

对于0.10的大熊猫,在iloc不可取的地方,过滤a DF并获取列的第一行数据VALUE

df_filt = df[df['C1'] == C1val & df['C2'] == C2val]
result = df_filt.get_value(df_filt.index[0],'VALUE')

如果过滤的行数超过1,则获取第一行的值。如果过滤器导致数据帧为空,则会出现异常。

For pandas 0.10, where iloc is unavalable, filter a DF and get the first row data for the column VALUE:

df_filt = df[df['C1'] == C1val & df['C2'] == C2val]
result = df_filt.get_value(df_filt.index[0],'VALUE')

if there is more then 1 row filtered, obtain the first row value. There will be an exception if the filter result in empty data frame.


回答 8

不知道这是否是一个好习惯,但是我注意到我也可以通过将序列强制转换为来获得值float

例如

rate

3 0.042679

名称:Unemployment_rate,dtype:float64

float(rate)

0.0426789

Not sure if this is a good practice, but I noticed I can also get just the value by casting the series as float.

e.g.

rate

3 0.042679

Name: Unemployment_rate, dtype: float64

float(rate)

0.0426789


回答 9

它不需要很复杂:

val = df.loc[df.wd==1, 'col_name'].values[0]

It doesn’t need to be complicated:

val = df.loc[df.wd==1, 'col_name'].values[0]

回答 10

df_gdp.columns

索引([u’Country’,u’Country Code’,u’Indicator Name’,u’Indicator Code’,u’1960’,u’1961’,u’1962’,u’1963’,u’1964′ ,u’1965’,u’1966’,u’1967’,u’1968’,u’1969’,u’1970’,u’1971’,u’1972’,u’1973’,u’1974′ ,u’1975’,u’1976’,u’1977’,u’1978’,u’1979’,u’1980’,u’1981’,u’1982’,u’1983’,u’1984′ ,u’1985’,u’1986’,u’1987’,u’1988’,u’1989’,u’1990’,u’1991’,u’1992’,u’1993’,u’1994′ ,u’1995’,u’1996’,u’1997’,u’1998’,u’1999’,u’2000’,u’2001’,u’2002’,u’2003’,u’2004’,u’2005’,u’2006’,u’2007’,u’2008’,u’2009’,u’2010’, u’2011’,u’2012’,u’2013’,u’2014’,u’2015’,u’2016′],dtype =’object’)

df_gdp[df_gdp["Country Code"] == "USA"]["1996"].values[0]

8100000000000.0

df_gdp.columns

Index([u’Country’, u’Country Code’, u’Indicator Name’, u’Indicator Code’, u’1960′, u’1961′, u’1962′, u’1963′, u’1964′, u’1965′, u’1966′, u’1967′, u’1968′, u’1969′, u’1970′, u’1971′, u’1972′, u’1973′, u’1974′, u’1975′, u’1976′, u’1977′, u’1978′, u’1979′, u’1980′, u’1981′, u’1982′, u’1983′, u’1984′, u’1985′, u’1986′, u’1987′, u’1988′, u’1989′, u’1990′, u’1991′, u’1992′, u’1993′, u’1994′, u’1995′, u’1996′, u’1997′, u’1998′, u’1999′, u’2000′, u’2001′, u’2002′, u’2003′, u’2004′, u’2005′, u’2006′, u’2007′, u’2008′, u’2009′, u’2010′, u’2011′, u’2012′, u’2013′, u’2014′, u’2015′, u’2016′], dtype=’object’)

df_gdp[df_gdp["Country Code"] == "USA"]["1996"].values[0]

8100000000000.0


如何从python中的线程获取返回值?

问题:如何从python中的线程获取返回值?

foo下面的函数返回一个字符串'foo'。如何获取'foo'从线程目标返回的值?

from threading import Thread

def foo(bar):
    print('hello {}'.format(bar))
    return 'foo'

thread = Thread(target=foo, args=('world!',))
thread.start()
return_value = thread.join()

上面显示的“一种显而易见的方法”不起作用:thread.join()return None

The function foo below returns a string 'foo'. How can I get the value 'foo' which is returned from the thread’s target?

from threading import Thread

def foo(bar):
    print('hello {}'.format(bar))
    return 'foo'

thread = Thread(target=foo, args=('world!',))
thread.start()
return_value = thread.join()

The “one obvious way to do it”, shown above, doesn’t work: thread.join() returned None.


回答 0

在Python 3.2+中,stdlib concurrent.futures模块向提供了更高级别的API threading,包括将返回值或异常从工作线程传递回主线程:

import concurrent.futures

def foo(bar):
    print('hello {}'.format(bar))
    return 'foo'

with concurrent.futures.ThreadPoolExecutor() as executor:
    future = executor.submit(foo, 'world!')
    return_value = future.result()
    print(return_value)

In Python 3.2+, stdlib concurrent.futures module provides a higher level API to threading, including passing return values or exceptions from a worker thread back to the main thread:

import concurrent.futures

def foo(bar):
    print('hello {}'.format(bar))
    return 'foo'

with concurrent.futures.ThreadPoolExecutor() as executor:
    future = executor.submit(foo, 'world!')
    return_value = future.result()
    print(return_value)

回答 1

FWIW,该multiprocessing模块为此类提供了一个不错的接口Pool。而且,如果您要坚持使用线程而不是进程,则可以只使用multiprocessing.pool.ThreadPool该类作为替代品。

def foo(bar, baz):
  print 'hello {0}'.format(bar)
  return 'foo' + baz

from multiprocessing.pool import ThreadPool
pool = ThreadPool(processes=1)

async_result = pool.apply_async(foo, ('world', 'foo')) # tuple of args for foo

# do some other stuff in the main process

return_val = async_result.get()  # get the return value from your function.

FWIW, the multiprocessing module has a nice interface for this using the Pool class. And if you want to stick with threads rather than processes, you can just use the multiprocessing.pool.ThreadPool class as a drop-in replacement.

def foo(bar, baz):
  print 'hello {0}'.format(bar)
  return 'foo' + baz

from multiprocessing.pool import ThreadPool
pool = ThreadPool(processes=1)

async_result = pool.apply_async(foo, ('world', 'foo')) # tuple of args for foo

# do some other stuff in the main process

return_val = async_result.get()  # get the return value from your function.

回答 2

我见过的一种方法是将可变对象(例如列表或字典)与索引或某种其他标识符一起传递给线程的构造函数。然后,线程可以将其结果存储在该对象的专用插槽中。例如:

def foo(bar, result, index):
    print 'hello {0}'.format(bar)
    result[index] = "foo"

from threading import Thread

threads = [None] * 10
results = [None] * 10

for i in range(len(threads)):
    threads[i] = Thread(target=foo, args=('world!', results, i))
    threads[i].start()

# do some other stuff

for i in range(len(threads)):
    threads[i].join()

print " ".join(results)  # what sound does a metasyntactic locomotive make?

如果您确实想join()返回被调用函数的返回值,则可以使用如下所示的Thread子类来实现:

from threading import Thread

def foo(bar):
    print 'hello {0}'.format(bar)
    return "foo"

class ThreadWithReturnValue(Thread):
    def __init__(self, group=None, target=None, name=None,
                 args=(), kwargs={}, Verbose=None):
        Thread.__init__(self, group, target, name, args, kwargs, Verbose)
        self._return = None
    def run(self):
        if self._Thread__target is not None:
            self._return = self._Thread__target(*self._Thread__args,
                                                **self._Thread__kwargs)
    def join(self):
        Thread.join(self)
        return self._return

twrv = ThreadWithReturnValue(target=foo, args=('world!',))

twrv.start()
print twrv.join()   # prints foo

由于名称修改,这有点麻烦,并且它访问特定于Thread实现的“私有”数据结构…但是它可以工作。

对于python3

class ThreadWithReturnValue(Thread):
    def __init__(self, group=None, target=None, name=None,
                 args=(), kwargs={}, Verbose=None):
        Thread.__init__(self, group, target, name, args, kwargs)
        self._return = None
    def run(self):
        print(type(self._target))
        if self._target is not None:
            self._return = self._target(*self._args,
                                                **self._kwargs)
    def join(self, *args):
        Thread.join(self, *args)
        return self._return

One way I’ve seen is to pass a mutable object, such as a list or a dictionary, to the thread’s constructor, along with a an index or other identifier of some sort. The thread can then store its results in its dedicated slot in that object. For example:

def foo(bar, result, index):
    print 'hello {0}'.format(bar)
    result[index] = "foo"

from threading import Thread

threads = [None] * 10
results = [None] * 10

for i in range(len(threads)):
    threads[i] = Thread(target=foo, args=('world!', results, i))
    threads[i].start()

# do some other stuff

for i in range(len(threads)):
    threads[i].join()

print " ".join(results)  # what sound does a metasyntactic locomotive make?

If you really want join() to return the return value of the called function, you can do this with a Thread subclass like the following:

from threading import Thread

def foo(bar):
    print 'hello {0}'.format(bar)
    return "foo"

class ThreadWithReturnValue(Thread):
    def __init__(self, group=None, target=None, name=None,
                 args=(), kwargs={}, Verbose=None):
        Thread.__init__(self, group, target, name, args, kwargs, Verbose)
        self._return = None
    def run(self):
        if self._Thread__target is not None:
            self._return = self._Thread__target(*self._Thread__args,
                                                **self._Thread__kwargs)
    def join(self):
        Thread.join(self)
        return self._return

twrv = ThreadWithReturnValue(target=foo, args=('world!',))

twrv.start()
print twrv.join()   # prints foo

That gets a little hairy because of some name mangling, and it accesses “private” data structures that are specific to Thread implementation… but it works.

For python3

class ThreadWithReturnValue(Thread):
    def __init__(self, group=None, target=None, name=None,
                 args=(), kwargs={}, Verbose=None):
        Thread.__init__(self, group, target, name, args, kwargs)
        self._return = None
    def run(self):
        print(type(self._target))
        if self._target is not None:
            self._return = self._target(*self._args,
                                                **self._kwargs)
    def join(self, *args):
        Thread.join(self, *args)
        return self._return

回答 3

Jake的答案很好,但是如果您不想使用线程池(您不知道需要多少线程,而是根据需要创建它们),那么内置的一种在线程之间传输信息的好方法队列类,因为它提供线程安全性。

我创建了以下装饰器,以使其与线程池类似:

def threaded(f, daemon=False):
    import Queue

    def wrapped_f(q, *args, **kwargs):
        '''this function calls the decorated function and puts the 
        result in a queue'''
        ret = f(*args, **kwargs)
        q.put(ret)

    def wrap(*args, **kwargs):
        '''this is the function returned from the decorator. It fires off
        wrapped_f in a new thread and returns the thread object with
        the result queue attached'''

        q = Queue.Queue()

        t = threading.Thread(target=wrapped_f, args=(q,)+args, kwargs=kwargs)
        t.daemon = daemon
        t.start()
        t.result_queue = q        
        return t

    return wrap

然后,将其用作:

@threaded
def long_task(x):
    import time
    x = x + 5
    time.sleep(5)
    return x

# does not block, returns Thread object
y = long_task(10)
print y

# this blocks, waiting for the result
result = y.result_queue.get()
print result

装饰函数每次调用时都会创建一个新线程,并返回一个Thread对象,该对象包含将接收结果的队列。

更新

自从我发布这个答案已经有一段时间了,但是它仍然得到视图,所以我想我将对其进行更新以反映我在较新版本的Python中执行此操作的方式:

concurrent.futures模块中添加了Python 3.2,该模块为并行任务提供了高级接口。它提供ThreadPoolExecutorProcessPoolExecutor,因此您可以使用具有相同api的线程或进程池。

此API的一个好处是将任务提交给Executor返回值Future return会对象,该对象将以您提交的可调用对象的返回值完成。

这使得queue不需要附加对象,从而大大简化了装饰器:

_DEFAULT_POOL = ThreadPoolExecutor()

def threadpool(f, executor=None):
    @wraps(f)
    def wrap(*args, **kwargs):
        return (executor or _DEFAULT_POOL).submit(f, *args, **kwargs)

    return wrap

这将使用默认模块如果未传入,线程池执行程序。

用法与之前非常相似:

@threadpool
def long_task(x):
    import time
    x = x + 5
    time.sleep(5)
    return x

# does not block, returns Future object
y = long_task(10)
print y

# this blocks, waiting for the result
result = y.result()
print result

如果您使用的是Python 3.4+,则使用此方法(通常是Future对象)的一个非常不错的功能是可以包装返回的future并将其转换为asyncio.Futurewith asyncio.wrap_future。这使得它很容易与协程一起工作:

result = await asyncio.wrap_future(long_task(10))

如果不需要访问基础concurrent.Future对象,则可以在包装器中包含自动换行:

_DEFAULT_POOL = ThreadPoolExecutor()

def threadpool(f, executor=None):
    @wraps(f)
    def wrap(*args, **kwargs):
        return asyncio.wrap_future((executor or _DEFAULT_POOL).submit(f, *args, **kwargs))

    return wrap

然后,每当需要将cpu密集型代码或阻塞代码从事件循环线程中推出时,都可以将其放入经过修饰的函数中:

@threadpool
def some_long_calculation():
    ...

# this will suspend while the function is executed on a threadpool
result = await some_long_calculation()

Jake’s answer is good, but if you don’t want to use a threadpool (you don’t know how many threads you’ll need, but create them as needed) then a good way to transmit information between threads is the built-in Queue.Queue class, as it offers thread safety.

I created the following decorator to make it act in a similar fashion to the threadpool:

def threaded(f, daemon=False):
    import Queue

    def wrapped_f(q, *args, **kwargs):
        '''this function calls the decorated function and puts the 
        result in a queue'''
        ret = f(*args, **kwargs)
        q.put(ret)

    def wrap(*args, **kwargs):
        '''this is the function returned from the decorator. It fires off
        wrapped_f in a new thread and returns the thread object with
        the result queue attached'''

        q = Queue.Queue()

        t = threading.Thread(target=wrapped_f, args=(q,)+args, kwargs=kwargs)
        t.daemon = daemon
        t.start()
        t.result_queue = q        
        return t

    return wrap

Then you just use it as:

@threaded
def long_task(x):
    import time
    x = x + 5
    time.sleep(5)
    return x

# does not block, returns Thread object
y = long_task(10)
print y

# this blocks, waiting for the result
result = y.result_queue.get()
print result

The decorated function creates a new thread each time it’s called and returns a Thread object that contains the queue that will receive the result.

UPDATE

It’s been quite a while since I posted this answer, but it still gets views so I thought I would update it to reflect the way I do this in newer versions of Python:

Python 3.2 added in the concurrent.futures module which provides a high-level interface for parallel tasks. It provides ThreadPoolExecutor and ProcessPoolExecutor, so you can use a thread or process pool with the same api.

One benefit of this api is that submitting a task to an Executor returns a Future object, which will complete with the return value of the callable you submit.

This makes attaching a queue object unnecessary, which simplifies the decorator quite a bit:

_DEFAULT_POOL = ThreadPoolExecutor()

def threadpool(f, executor=None):
    @wraps(f)
    def wrap(*args, **kwargs):
        return (executor or _DEFAULT_POOL).submit(f, *args, **kwargs)

    return wrap

This will use a default module threadpool executor if one is not passed in.

The usage is very similar to before:

@threadpool
def long_task(x):
    import time
    x = x + 5
    time.sleep(5)
    return x

# does not block, returns Future object
y = long_task(10)
print y

# this blocks, waiting for the result
result = y.result()
print result

If you’re using Python 3.4+, one really nice feature of using this method (and Future objects in general) is that the returned future can be wrapped to turn it into an asyncio.Future with asyncio.wrap_future. This makes it work easily with coroutines:

result = await asyncio.wrap_future(long_task(10))

If you don’t need access to the underlying concurrent.Future object, you can include the wrap in the decorator:

_DEFAULT_POOL = ThreadPoolExecutor()

def threadpool(f, executor=None):
    @wraps(f)
    def wrap(*args, **kwargs):
        return asyncio.wrap_future((executor or _DEFAULT_POOL).submit(f, *args, **kwargs))

    return wrap

Then, whenever you need to push cpu intensive or blocking code off the event loop thread, you can put it in a decorated function:

@threadpool
def some_long_calculation():
    ...

# this will suspend while the function is executed on a threadpool
result = await some_long_calculation()

回答 4

另一个不需要更改现有代码的解决方案:

import Queue
from threading import Thread

def foo(bar):
    print 'hello {0}'.format(bar)
    return 'foo'

que = Queue.Queue()

t = Thread(target=lambda q, arg1: q.put(foo(arg1)), args=(que, 'world!'))
t.start()
t.join()
result = que.get()
print result

还可以轻松地将其调整为多线程环境:

import Queue
from threading import Thread

def foo(bar):
    print 'hello {0}'.format(bar)
    return 'foo'

que = Queue.Queue()
threads_list = list()

t = Thread(target=lambda q, arg1: q.put(foo(arg1)), args=(que, 'world!'))
t.start()
threads_list.append(t)

# Add more threads here
...
threads_list.append(t2)
...
threads_list.append(t3)
...

# Join all the threads
for t in threads_list:
    t.join()

# Check thread's return value
while not que.empty():
    result = que.get()
    print result

Another solution that doesn’t require changing your existing code:

import Queue
from threading import Thread

def foo(bar):
    print 'hello {0}'.format(bar)
    return 'foo'

que = Queue.Queue()

t = Thread(target=lambda q, arg1: q.put(foo(arg1)), args=(que, 'world!'))
t.start()
t.join()
result = que.get()
print result

It can be also easily adjusted to a multi-threaded environment:

import Queue
from threading import Thread

def foo(bar):
    print 'hello {0}'.format(bar)
    return 'foo'

que = Queue.Queue()
threads_list = list()

t = Thread(target=lambda q, arg1: q.put(foo(arg1)), args=(que, 'world!'))
t.start()
threads_list.append(t)

# Add more threads here
...
threads_list.append(t2)
...
threads_list.append(t3)
...

# Join all the threads
for t in threads_list:
    t.join()

# Check thread's return value
while not que.empty():
    result = que.get()
    print result

回答 5

Parris / kindall的答案 join / return移植到Python 3 的答案

from threading import Thread

def foo(bar):
    print('hello {0}'.format(bar))
    return "foo"

class ThreadWithReturnValue(Thread):
    def __init__(self, group=None, target=None, name=None, args=(), kwargs=None, *, daemon=None):
        Thread.__init__(self, group, target, name, args, kwargs, daemon=daemon)

        self._return = None

    def run(self):
        if self._target is not None:
            self._return = self._target(*self._args, **self._kwargs)

    def join(self):
        Thread.join(self)
        return self._return


twrv = ThreadWithReturnValue(target=foo, args=('world!',))

twrv.start()
print(twrv.join())   # prints foo

请注意,Thread该类在Python 3中的实现方式有所不同。

Parris / kindall’s answer join/return answer ported to Python 3:

from threading import Thread

def foo(bar):
    print('hello {0}'.format(bar))
    return "foo"

class ThreadWithReturnValue(Thread):
    def __init__(self, group=None, target=None, name=None, args=(), kwargs=None, *, daemon=None):
        Thread.__init__(self, group, target, name, args, kwargs, daemon=daemon)

        self._return = None

    def run(self):
        if self._target is not None:
            self._return = self._target(*self._args, **self._kwargs)

    def join(self):
        Thread.join(self)
        return self._return


twrv = ThreadWithReturnValue(target=foo, args=('world!',))

twrv.start()
print(twrv.join())   # prints foo

Note, the Thread class is implemented differently in Python 3.


回答 6

我偷了kindall的答案并整理了一下。

关键部分是将* args和** kwargs添加到join()中以处理超时

class threadWithReturn(Thread):
    def __init__(self, *args, **kwargs):
        super(threadWithReturn, self).__init__(*args, **kwargs)

        self._return = None

    def run(self):
        if self._Thread__target is not None:
            self._return = self._Thread__target(*self._Thread__args, **self._Thread__kwargs)

    def join(self, *args, **kwargs):
        super(threadWithReturn, self).join(*args, **kwargs)

        return self._return

下面的更新的答案

这是我最受欢迎的答案,因此我决定使用将同时在py2和py3上运行的代码进行更新。

另外,我看到这个问题的许多答案表明对Thread.join()缺乏理解。有些人完全无法处理timeoutarg。但是,当您拥有(1)可以返回的目标函数None并且(2)您还传递了(timeout arg给join()。请参阅“测试4”以了解这种极端情况。

与py2和py3一起使用的ThreadWithReturn类:

import sys
from threading import Thread
from builtins import super    # https://stackoverflow.com/a/30159479

if sys.version_info >= (3, 0):
    _thread_target_key = '_target'
    _thread_args_key = '_args'
    _thread_kwargs_key = '_kwargs'
else:
    _thread_target_key = '_Thread__target'
    _thread_args_key = '_Thread__args'
    _thread_kwargs_key = '_Thread__kwargs'

class ThreadWithReturn(Thread):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self._return = None

    def run(self):
        target = getattr(self, _thread_target_key)
        if not target is None:
            self._return = target(
                *getattr(self, _thread_args_key),
                **getattr(self, _thread_kwargs_key)
            )

    def join(self, *args, **kwargs):
        super().join(*args, **kwargs)
        return self._return

一些示例测试如下所示:

import time, random

# TEST TARGET FUNCTION
def giveMe(arg, seconds=None):
    if not seconds is None:
        time.sleep(seconds)
    return arg

# TEST 1
my_thread = ThreadWithReturn(target=giveMe, args=('stringy',))
my_thread.start()
returned = my_thread.join()
# (returned == 'stringy')

# TEST 2
my_thread = ThreadWithReturn(target=giveMe, args=(None,))
my_thread.start()
returned = my_thread.join()
# (returned is None)

# TEST 3
my_thread = ThreadWithReturn(target=giveMe, args=('stringy',), kwargs={'seconds': 5})
my_thread.start()
returned = my_thread.join(timeout=2)
# (returned is None) # because join() timed out before giveMe() finished

# TEST 4
my_thread = ThreadWithReturn(target=giveMe, args=(None,), kwargs={'seconds': 5})
my_thread.start()
returned = my_thread.join(timeout=random.randint(1, 10))

您能否确定在测试4中可能遇到的特殊情况?

问题在于,我们希望GiveMe()返回None(请参见测试2),但是我们也希望join()如果超时则返回None。

returned is None 意味着:

(1)这就是GiveMe()返回的结果,或者

(2)join()超时

这个例子很简单,因为我们知道GiveMe()将始终返回None。但是在实际情况下(目标可能合法返回None或其他),我们希望显式检查发生了什么。

以下是解决这种情况的方法:

# TEST 4
my_thread = ThreadWithReturn(target=giveMe, args=(None,), kwargs={'seconds': 5})
my_thread.start()
returned = my_thread.join(timeout=random.randint(1, 10))

if my_thread.isAlive():
    # returned is None because join() timed out
    # this also means that giveMe() is still running in the background
    pass
    # handle this based on your app's logic
else:
    # join() is finished, and so is giveMe()
    # BUT we could also be in a race condition, so we need to update returned, just in case
    returned = my_thread.join()

I stole kindall’s answer and cleaned it up just a little bit.

The key part is adding *args and **kwargs to join() in order to handle the timeout

class threadWithReturn(Thread):
    def __init__(self, *args, **kwargs):
        super(threadWithReturn, self).__init__(*args, **kwargs)

        self._return = None

    def run(self):
        if self._Thread__target is not None:
            self._return = self._Thread__target(*self._Thread__args, **self._Thread__kwargs)

    def join(self, *args, **kwargs):
        super(threadWithReturn, self).join(*args, **kwargs)

        return self._return

UPDATED ANSWER BELOW

This is my most popularly upvoted answer, so I decided to update with code that will run on both py2 and py3.

Additionally, I see many answers to this question that show a lack of comprehension regarding Thread.join(). Some completely fail to handle the timeout arg. But there is also a corner-case that you should be aware of regarding instances when you have (1) a target function that can return None and (2) you also pass the timeout arg to join(). Please see “TEST 4” to understand this corner case.

ThreadWithReturn class that works with py2 and py3:

import sys
from threading import Thread
from builtins import super    # https://stackoverflow.com/a/30159479

if sys.version_info >= (3, 0):
    _thread_target_key = '_target'
    _thread_args_key = '_args'
    _thread_kwargs_key = '_kwargs'
else:
    _thread_target_key = '_Thread__target'
    _thread_args_key = '_Thread__args'
    _thread_kwargs_key = '_Thread__kwargs'

class ThreadWithReturn(Thread):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self._return = None

    def run(self):
        target = getattr(self, _thread_target_key)
        if not target is None:
            self._return = target(
                *getattr(self, _thread_args_key),
                **getattr(self, _thread_kwargs_key)
            )

    def join(self, *args, **kwargs):
        super().join(*args, **kwargs)
        return self._return

Some sample tests are shown below:

import time, random

# TEST TARGET FUNCTION
def giveMe(arg, seconds=None):
    if not seconds is None:
        time.sleep(seconds)
    return arg

# TEST 1
my_thread = ThreadWithReturn(target=giveMe, args=('stringy',))
my_thread.start()
returned = my_thread.join()
# (returned == 'stringy')

# TEST 2
my_thread = ThreadWithReturn(target=giveMe, args=(None,))
my_thread.start()
returned = my_thread.join()
# (returned is None)

# TEST 3
my_thread = ThreadWithReturn(target=giveMe, args=('stringy',), kwargs={'seconds': 5})
my_thread.start()
returned = my_thread.join(timeout=2)
# (returned is None) # because join() timed out before giveMe() finished

# TEST 4
my_thread = ThreadWithReturn(target=giveMe, args=(None,), kwargs={'seconds': 5})
my_thread.start()
returned = my_thread.join(timeout=random.randint(1, 10))

Can you identify the corner-case that we may possibly encounter with TEST 4?

The problem is that we expect giveMe() to return None (see TEST 2), but we also expect join() to return None if it times out.

returned is None means either:

(1) that’s what giveMe() returned, or

(2) join() timed out

This example is trivial since we know that giveMe() will always return None. But in real-world instance (where the target may legitimately return None or something else) we’d want to explicitly check for what happened.

Below is how to address this corner-case:

# TEST 4
my_thread = ThreadWithReturn(target=giveMe, args=(None,), kwargs={'seconds': 5})
my_thread.start()
returned = my_thread.join(timeout=random.randint(1, 10))

if my_thread.isAlive():
    # returned is None because join() timed out
    # this also means that giveMe() is still running in the background
    pass
    # handle this based on your app's logic
else:
    # join() is finished, and so is giveMe()
    # BUT we could also be in a race condition, so we need to update returned, just in case
    returned = my_thread.join()

回答 7

使用队列:

import threading, queue

def calc_square(num, out_queue1):
  l = []
  for x in num:
    l.append(x*x)
  out_queue1.put(l)


arr = [1,2,3,4,5,6,7,8,9,10]
out_queue1=queue.Queue()
t1=threading.Thread(target=calc_square, args=(arr,out_queue1))
t1.start()
t1.join()
print (out_queue1.get())

Using Queue :

import threading, queue

def calc_square(num, out_queue1):
  l = []
  for x in num:
    l.append(x*x)
  out_queue1.put(l)


arr = [1,2,3,4,5,6,7,8,9,10]
out_queue1=queue.Queue()
t1=threading.Thread(target=calc_square, args=(arr,out_queue1))
t1.start()
t1.join()
print (out_queue1.get())

回答 8

我对这个问题的解决方案是将函数和线程包装在一个类中。不需要使用池,队列或c类型变量传递。这也是非阻塞的。您改为查看状态。请参阅代码末尾有关如何使用它的示例。

import threading

class ThreadWorker():
    '''
    The basic idea is given a function create an object.
    The object can then run the function in a thread.
    It provides a wrapper to start it,check its status,and get data out the function.
    '''
    def __init__(self,func):
        self.thread = None
        self.data = None
        self.func = self.save_data(func)

    def save_data(self,func):
        '''modify function to save its returned data'''
        def new_func(*args, **kwargs):
            self.data=func(*args, **kwargs)

        return new_func

    def start(self,params):
        self.data = None
        if self.thread is not None:
            if self.thread.isAlive():
                return 'running' #could raise exception here

        #unless thread exists and is alive start or restart it
        self.thread = threading.Thread(target=self.func,args=params)
        self.thread.start()
        return 'started'

    def status(self):
        if self.thread is None:
            return 'not_started'
        else:
            if self.thread.isAlive():
                return 'running'
            else:
                return 'finished'

    def get_results(self):
        if self.thread is None:
            return 'not_started' #could return exception
        else:
            if self.thread.isAlive():
                return 'running'
            else:
                return self.data

def add(x,y):
    return x +y

add_worker = ThreadWorker(add)
print add_worker.start((1,2,))
print add_worker.status()
print add_worker.get_results()

My solution to the problem is to wrap the function and thread in a class. Does not require using pools,queues, or c type variable passing. It is also non blocking. You check status instead. See example of how to use it at end of code.

import threading

class ThreadWorker():
    '''
    The basic idea is given a function create an object.
    The object can then run the function in a thread.
    It provides a wrapper to start it,check its status,and get data out the function.
    '''
    def __init__(self,func):
        self.thread = None
        self.data = None
        self.func = self.save_data(func)

    def save_data(self,func):
        '''modify function to save its returned data'''
        def new_func(*args, **kwargs):
            self.data=func(*args, **kwargs)

        return new_func

    def start(self,params):
        self.data = None
        if self.thread is not None:
            if self.thread.isAlive():
                return 'running' #could raise exception here

        #unless thread exists and is alive start or restart it
        self.thread = threading.Thread(target=self.func,args=params)
        self.thread.start()
        return 'started'

    def status(self):
        if self.thread is None:
            return 'not_started'
        else:
            if self.thread.isAlive():
                return 'running'
            else:
                return 'finished'

    def get_results(self):
        if self.thread is None:
            return 'not_started' #could return exception
        else:
            if self.thread.isAlive():
                return 'running'
            else:
                return self.data

def add(x,y):
    return x +y

add_worker = ThreadWorker(add)
print add_worker.start((1,2,))
print add_worker.status()
print add_worker.get_results()

回答 9

join总是返回None,我想你应该子类Thread来处理返回码等等。

join always return None, i think you should subclass Thread to handle return codes and so.


回答 10

考虑到@iman@JakeBiesinger答案的评论,我将其重新组成为具有多个线程:

from multiprocessing.pool import ThreadPool

def foo(bar, baz):
    print 'hello {0}'.format(bar)
    return 'foo' + baz

numOfThreads = 3 
results = []

pool = ThreadPool(numOfThreads)

for i in range(0, numOfThreads):
    results.append(pool.apply_async(foo, ('world', 'foo'))) # tuple of args for foo)

# do some other stuff in the main process
# ...
# ...

results = [r.get() for r in results]
print results

pool.close()
pool.join()

干杯,

伙计

Taking into consideration @iman comment on @JakeBiesinger answer I have recomposed it to have various number of threads:

from multiprocessing.pool import ThreadPool

def foo(bar, baz):
    print 'hello {0}'.format(bar)
    return 'foo' + baz

numOfThreads = 3 
results = []

pool = ThreadPool(numOfThreads)

for i in range(0, numOfThreads):
    results.append(pool.apply_async(foo, ('world', 'foo'))) # tuple of args for foo)

# do some other stuff in the main process
# ...
# ...

results = [r.get() for r in results]
print results

pool.close()
pool.join()

Cheers,

Guy.


回答 11

您可以在线程函数的作用域之上定义一个可变变量,并将结果添加到该变量中。(我也将代码修改为与python3兼容)

returns = {}
def foo(bar):
    print('hello {0}'.format(bar))
    returns[bar] = 'foo'

from threading import Thread
t = Thread(target=foo, args=('world!',))
t.start()
t.join()
print(returns)

这返回 {'world!': 'foo'}

如果使用函数输入作为结果字典的键,则保证每个唯一的输入都会在结果中给出一个条目

You can define a mutable above the scope of the threaded function, and add the result to that. (I also modified the code to be python3 compatible)

returns = {}
def foo(bar):
    print('hello {0}'.format(bar))
    returns[bar] = 'foo'

from threading import Thread
t = Thread(target=foo, args=('world!',))
t.start()
t.join()
print(returns)

This returns {'world!': 'foo'}

If you use the function input as the key to your results dict, every unique input is guaranteed to give an entry in the results


回答 12

我正在使用此包装器,该包装器可以轻松地打开任何函数以在其中运行Thread-照顾其返回值或异常。它不会增加Queue开销。

def threading_func(f):
    """Decorator for running a function in a thread and handling its return
    value or exception"""
    def start(*args, **kw):
        def run():
            try:
                th.ret = f(*args, **kw)
            except:
                th.exc = sys.exc_info()
        def get(timeout=None):
            th.join(timeout)
            if th.exc:
                raise th.exc[0], th.exc[1], th.exc[2] # py2
                ##raise th.exc[1] #py3                
            return th.ret
        th = threading.Thread(None, run)
        th.exc = None
        th.get = get
        th.start()
        return th
    return start

使用范例

def f(x):
    return 2.5 * x
th = threading_func(f)(4)
print("still running?:", th.is_alive())
print("result:", th.get(timeout=1.0))

@threading_func
def th_mul(a, b):
    return a * b
th = th_mul("text", 2.5)

try:
    print(th.get())
except TypeError:
    print("exception thrown ok.")

threading模块注意事项

线程函数的舒适的返回值和异常处理是“ Pythonic”的常见需求,并且确实应该已经由threading模块提供-可能直接在标准Thread类中提供。ThreadPool对于简单的任务来说有太多的开销-3个管理线程,很多官僚作风。不幸Thread的是,其布局最初是从Java复制的-例如,您仍然可以从仍然无效的1st(!)构造函数参数中看到该布局group

I’m using this wrapper, which comfortably turns any function for running in a Thread – taking care of its return value or exception. It doesn’t add Queue overhead.

def threading_func(f):
    """Decorator for running a function in a thread and handling its return
    value or exception"""
    def start(*args, **kw):
        def run():
            try:
                th.ret = f(*args, **kw)
            except:
                th.exc = sys.exc_info()
        def get(timeout=None):
            th.join(timeout)
            if th.exc:
                raise th.exc[0], th.exc[1], th.exc[2] # py2
                ##raise th.exc[1] #py3                
            return th.ret
        th = threading.Thread(None, run)
        th.exc = None
        th.get = get
        th.start()
        return th
    return start

Usage Examples

def f(x):
    return 2.5 * x
th = threading_func(f)(4)
print("still running?:", th.is_alive())
print("result:", th.get(timeout=1.0))

@threading_func
def th_mul(a, b):
    return a * b
th = th_mul("text", 2.5)

try:
    print(th.get())
except TypeError:
    print("exception thrown ok.")

Notes on threading module

Comfortable return value & exception handling of a threaded function is a frequent “Pythonic” need and should indeed already be offered by the threading module – possibly directly in the standard Thread class. ThreadPool has way too much overhead for simple tasks – 3 managing threads, lots of bureaucracy. Unfortunately Thread‘s layout was copied from Java originally – which you see e.g. from the still useless 1st (!) constructor parameter group.


回答 13

将目标定义为
1)接受参数q
2)将任何语句替换return fooq.put(foo); return

所以一个功能

def func(a):
    ans = a * a
    return ans

会成为

def func(a, q):
    ans = a * a
    q.put(ans)
    return

然后您将照此进行

from Queue import Queue
from threading import Thread

ans_q = Queue()
arg_tups = [(i, ans_q) for i in xrange(10)]

threads = [Thread(target=func, args=arg_tup) for arg_tup in arg_tups]
_ = [t.start() for t in threads]
_ = [t.join() for t in threads]
results = [q.get() for _ in xrange(len(threads))]

而且,您可以使用函数装饰器/包装器来制作它,这样就可以target不修改而使用现有功能,而是遵循此基本方案。

Define your target to
1) take an argument q
2) replace any statements return foo with q.put(foo); return

so a function

def func(a):
    ans = a * a
    return ans

would become

def func(a, q):
    ans = a * a
    q.put(ans)
    return

and then you would proceed as such

from Queue import Queue
from threading import Thread

ans_q = Queue()
arg_tups = [(i, ans_q) for i in xrange(10)]

threads = [Thread(target=func, args=arg_tup) for arg_tup in arg_tups]
_ = [t.start() for t in threads]
_ = [t.join() for t in threads]
results = [q.get() for _ in xrange(len(threads))]

And you can use function decorators/wrappers to make it so you can use your existing functions as target without modifying them, but follow this basic scheme.


回答 14

如上所述,多处理池比基本线程慢得多。使用一些答案中提出的队列是一种非常有效的选择。我将它与字典配合使用,以便能够运行许多小线程并通过将它们与字典结合来调理多个答案:

#!/usr/bin/env python3

import threading
# use Queue for python2
import queue
import random

LETTERS = 'abcdefghijklmnopqrstuvwxyz'
LETTERS = [ x for x in LETTERS ]

NUMBERS = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

def randoms(k, q):
    result = dict()
    result['letter'] = random.choice(LETTERS)
    result['number'] = random.choice(NUMBERS)
    q.put({k: result})

threads = list()
q = queue.Queue()
results = dict()

for name in ('alpha', 'oscar', 'yankee',):
    threads.append( threading.Thread(target=randoms, args=(name, q)) )
    threads[-1].start()
_ = [ t.join() for t in threads ]
while not q.empty():
    results.update(q.get())

print(results)

As mentioned multiprocessing pool is much slower than basic threading. Using queues as proposeded in some answers here is a very effective alternative. I have use it with dictionaries in order to be able run a lot of small threads and recuperate multiple answers by combining them with dictionaries:

#!/usr/bin/env python3

import threading
# use Queue for python2
import queue
import random

LETTERS = 'abcdefghijklmnopqrstuvwxyz'
LETTERS = [ x for x in LETTERS ]

NUMBERS = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

def randoms(k, q):
    result = dict()
    result['letter'] = random.choice(LETTERS)
    result['number'] = random.choice(NUMBERS)
    q.put({k: result})

threads = list()
q = queue.Queue()
results = dict()

for name in ('alpha', 'oscar', 'yankee',):
    threads.append( threading.Thread(target=randoms, args=(name, q)) )
    threads[-1].start()
_ = [ t.join() for t in threads ]
while not q.empty():
    results.update(q.get())

print(results)

回答 15

GuySoft的想法很棒,但是我认为对象不一定必须继承自Thread,并且可以从接口中删除start():

from threading import Thread
import queue
class ThreadWithReturnValue(object):
    def __init__(self, target=None, args=(), **kwargs):
        self._que = queue.Queue()
        self._t = Thread(target=lambda q,arg1,kwargs1: q.put(target(*arg1, **kwargs1)) ,
                args=(self._que, args, kwargs), )
        self._t.start()

    def join(self):
        self._t.join()
        return self._que.get()


def foo(bar):
    print('hello {0}'.format(bar))
    return "foo"

twrv = ThreadWithReturnValue(target=foo, args=('world!',))

print(twrv.join())   # prints foo

GuySoft’s idea is great, but I think the object does not necessarily have to inherit from Thread and start() could be removed from interface:

from threading import Thread
import queue
class ThreadWithReturnValue(object):
    def __init__(self, target=None, args=(), **kwargs):
        self._que = queue.Queue()
        self._t = Thread(target=lambda q,arg1,kwargs1: q.put(target(*arg1, **kwargs1)) ,
                args=(self._que, args, kwargs), )
        self._t.start()

    def join(self):
        self._t.join()
        return self._que.get()


def foo(bar):
    print('hello {0}'.format(bar))
    return "foo"

twrv = ThreadWithReturnValue(target=foo, args=('world!',))

print(twrv.join())   # prints foo

回答 16

一种常见的解决方案是foo使用类似这样的装饰器包装函数

result = queue.Queue()

def task_wrapper(*args):
    result.put(target(*args))

然后整个代码可能像这样

result = queue.Queue()

def task_wrapper(*args):
    result.put(target(*args))

threads = [threading.Thread(target=task_wrapper, args=args) for args in args_list]

for t in threads:
    t.start()
    while(True):
        if(len(threading.enumerate()) < max_num):
            break
for t in threads:
    t.join()
return result

注意

一个重要的问题是返回值可能是无序的。(实际上,return value不一定将保存到queue,因为您可以选择任意线程安全的数据结构)

One usual solution is to wrap your function foo with a decorator like

result = queue.Queue()

def task_wrapper(*args):
    result.put(target(*args))

Then the whole code may looks like that

result = queue.Queue()

def task_wrapper(*args):
    result.put(target(*args))

threads = [threading.Thread(target=task_wrapper, args=args) for args in args_list]

for t in threads:
    t.start()
    while(True):
        if(len(threading.enumerate()) < max_num):
            break
for t in threads:
    t.join()
return result

Note

One important issue is that the return values may be unorderred. (In fact, the return value is not necessarily saved to the queue, since you can choose arbitrary thread-safe data structure )


回答 17

为什么不只使用全局变量?

import threading


class myThread(threading.Thread):
    def __init__(self, ind, lock):
        threading.Thread.__init__(self)
        self.ind = ind
        self.lock = lock

    def run(self):
        global results
        with self.lock:
            results.append(self.ind)



results = []
lock = threading.Lock()
threads = [myThread(x, lock) for x in range(1, 4)]
for t in threads:
    t.start()
for t in threads:
    t.join()
print(results)

Why don’t just use global variable?

import threading


class myThread(threading.Thread):
    def __init__(self, ind, lock):
        threading.Thread.__init__(self)
        self.ind = ind
        self.lock = lock

    def run(self):
        global results
        with self.lock:
            results.append(self.ind)



results = []
lock = threading.Lock()
threads = [myThread(x, lock) for x in range(1, 4)]
for t in threads:
    t.start()
for t in threads:
    t.join()
print(results)

回答 18

Kindall在Python3中的答案

class ThreadWithReturnValue(Thread):
    def __init__(self, group=None, target=None, name=None,
                 args=(), kwargs={}, *, daemon=None):
        Thread.__init__(self, group, target, name, args, kwargs, daemon)
        self._return = None 

    def run(self):
        try:
            if self._target:
                self._return = self._target(*self._args, **self._kwargs)
        finally:
            del self._target, self._args, self._kwargs 

    def join(self,timeout=None):
        Thread.join(self,timeout)
        return self._return

Kindall’s answer in Python3

class ThreadWithReturnValue(Thread):
    def __init__(self, group=None, target=None, name=None,
                 args=(), kwargs={}, *, daemon=None):
        Thread.__init__(self, group, target, name, args, kwargs, daemon)
        self._return = None 

    def run(self):
        try:
            if self._target:
                self._return = self._target(*self._args, **self._kwargs)
        finally:
            del self._target, self._args, self._kwargs 

    def join(self,timeout=None):
        Thread.join(self,timeout)
        return self._return

回答 19

如果仅要从函数调用中验证True或False,我发现一个更简单的解决方案是更新全局列表。

import threading

lists = {"A":"True", "B":"True"}

def myfunc(name: str, mylist):
    for i in mylist:
        if i == 31:
            lists[name] = "False"
            return False
        else:
            print("name {} : {}".format(name, i))

t1 = threading.Thread(target=myfunc, args=("A", [1, 2, 3, 4, 5, 6], ))
t2 = threading.Thread(target=myfunc, args=("B", [11, 21, 31, 41, 51, 61], ))
t1.start()
t2.start()
t1.join()
t2.join()

for value in lists.values():
    if value == False:
        # Something is suspicious 
        # Take necessary action 

如果您想查找任何一个线程是否返回了错误的状态以采取必要的操作,这将对您有所帮助。

If only True or False is to be validated from a function’s call, a simpler solution I find is updating a global list.

import threading

lists = {"A":"True", "B":"True"}

def myfunc(name: str, mylist):
    for i in mylist:
        if i == 31:
            lists[name] = "False"
            return False
        else:
            print("name {} : {}".format(name, i))

t1 = threading.Thread(target=myfunc, args=("A", [1, 2, 3, 4, 5, 6], ))
t2 = threading.Thread(target=myfunc, args=("B", [11, 21, 31, 41, 51, 61], ))
t1.start()
t2.start()
t1.join()
t2.join()

for value in lists.values():
    if value == False:
        # Something is suspicious 
        # Take necessary action 

This is more helpful where you want to find if any one of the threads had returned a false status to take the necessary action.