__future__ frequently appears in Python modules. I do not understand what __future__ is for and how/when to use it even after reading the Python’s __future__ doc.
Can anyone explain with examples?
A few answers regarding the basic usage of __future__ I’ve received seemed correct.
However, I need to understand one more thing regarding how __future__ works:
The most confusing concept for me is how a current python release includes features for future releases, and how a program using a feature from a future release can be be compiled successfully in the current version of Python.
I am guessing that the current release is packaged with potential features for the future. However, the features are available only by using __future__ because they are not the current standard. Let me know if I am right.
With __future__ module’s inclusion, you can slowly be accustomed to incompatible changes or to such ones introducing new keywords.
E.g., for using context managers, you had to do from __future__ import with_statement in 2.5, as the with keyword was new and shouldn’t be used as variable names any longer. In order to use with as a Python keyword in Python 2.5 or older, you will need to use the import from above.
Another example is
from __future__ import division
print 8/7 # prints 1.1428571428571428
print 8//7 # prints 1
Without the __future__ stuff, both print statements would print 1.
The internal difference is that without that import, / is mapped to the __div__() method, while with it, __truediv__() is used. (In any case, // calls __floordiv__().)
Apropos print: print becomes a function in 3.x, losing its special property as a keyword. So it is the other way round.
>>> print
>>> from __future__ import print_function
>>> print
<built-in function print>
>>>
You’re not actually using an import statement, but a future statement. You’re reading the wrong docs, as you’re not actually importing that module.
Future statements are special — they change how your Python module is parsed, which is why they must be at the top of the file. They give new — or different — meaning to words or symbols in your file. From the docs:
A future statement is a directive to the compiler that a particular module should be compiled using syntax or semantics that will be available in a specified future release of Python. The future statement is intended to ease migration to future versions of Python that introduce incompatible changes to the language. It allows use of the new features on a per-module basis before the release in which the feature becomes standard.
If you actually want to import the __future__ module, just do
__future__is a pseudo-module which programmers can use to enable new language features which are not compatible with the current interpreter. For example, the expression 11/4 currently evaluates to 2. If the module in which it is executed had enabled true division by executing:
from __future__ import division
the expression 11/4 would evaluate to 2.75. By importing the __future__ module and evaluating its variables, you can see when a new feature was first added to the language and when it will become the default:
The from __future__ import nested_scopes directive will allow for this feature to be enabled.
generators
Introduced generator functions such as the one below to save state between successive function calls:
def fib():
a, b = 0, 1
while 1:
yield b
a, b = b, a+b
division
Classic division is used in Python 2.x versions. Meaning that some division statements return a reasonable approximation of division (“true division”) and others return the floor (“floor division”). Starting in Python 3.0, true division is specified by x/y, whereas floor division is specified by x//y.
The from __future__ import division directive forces the use of Python 3.0 style division.
absolute_import
Allows for parenthesis to enclose multiple import statements. For example:
from Tkinter import Tk, Frame, Button, Entry, Canvas, Text, \
LEFT, DISABLED, NORMAL, RIDGE, END
Or:
from Tkinter import Tk, Frame, Button, Entry, Canvas, Text
from Tkinter import LEFT, DISABLED, NORMAL, RIDGE, END
with_statement
Adds the statement with as a keyword in Python to eliminate the need for try/finally statements. Common uses of this are when doing file I/O such as:
with open('workfile', 'r') as f:
read_data = f.read()
print_function:
Forces the use of Python 3 parenthesis-style print() function call instead of the print MESSAGE style statement.
unicode_literals
Introduces the literal syntax for the bytes object. Meaning that statements such as bytes('Hello world', 'ascii') can be simply expressed as b'Hello world'.
generator_stop
Replaces the use of the StopIteration exception used inside generator functions with the RuntimeError exception.
One other use not mentioned above is that the __future__ statement also requires the use of Python 2.1+ interpreters since using an older version will throw a runtime exception.
Or is it like saying “Since this is python v2.7, use that different ‘print’ function that has also been added to python v2.7, after it was added in python 3. So my ‘print’ will no longer be statements (eg print “message” ) but functions (eg, print(“message”, options). That way when my code is run in python 3, ‘print’ will not break.”
In
from __future__ import print_function
print_function is the module containing the new implementation of ‘print’ as per how it is behaving in python v3.
from __future__ import print_function
classStar(object):def __init__(self,count):
self.count = count
def start(self):for i in range(1,self.count):for j in range (i):print('*', end='')# PEP 3105: print As a Function print()
a =Star(5)
a.start()Output:**********
After Python 3.0 onward, print is no longer just a statement, its a function instead. and is included in PEP 3105.
Also I think the Python 3.0 package has still these special functionality. Lets see its usability through a traditional “Pyramid program” in Python:
from __future__ import print_function
class Star(object):
def __init__(self,count):
self.count = count
def start(self):
for i in range(1,self.count):
for j in range (i):
print('*', end='') # PEP 3105: print As a Function
print()
a = Star(5)
a.start()
Output:
*
**
***
****
If we use normal print function, we won’t be able to achieve the same output, since print() comes with a extra newline. So every time the inner for loop execute, it will print * onto the next line.
The branch’s name is issue/34/oscar-0.6 so I did pip install https://github.com/tangentlabs/django-oscar-paypal.git@/issue/34/oscar-0.6 but its returning a 404.
Using pip with git+ to clone a repository can be extremely slow (test with https://github.com/django/django@stable/1.6.x for example, it will take a few minutes). The fastest thing I’ve found, which works with GitHub and BitBucket, is:
Here, the master branch is generally named default.
This will make your requirements.txt installing much faster.
Some other answers mention variations required when placing the package to be installed into your requirements.txt. Note that with this archive syntax, the leading -e and trailing #egg=blah-blah are not required, and you can just simply paste the URL, so your requirements.txt looks like:
You used the egg files install procedure.
This procedure supports installing over git, git+http, git+https, git+ssh, git+git and git+file. Some of these are mentioned.
It’s good you can use branches, tags, or hashes to install.
@Steve_K noted it can be slow to install with “git+” and proposed installing via zip file:
How do I create a GUID in Python that is platform independent? I hear there is a method using ActivePython on Windows but it’s Windows only because it uses COM. Is there a method using plain Python?
>>>import uuid>>># make a UUID based on the host ID and current time>>> uuid.uuid1()
UUID('a8098c1a-f86e-11da-bd1a-00112444be1e')>>># make a UUID using an MD5 hash of a namespace UUID and a name>>> uuid.uuid3(uuid.NAMESPACE_DNS,'python.org')
UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e')>>># make a random UUID>>> uuid.uuid4()
UUID('16fd2706-8baf-433b-82eb-8c7fada847da')>>># make a UUID using a SHA-1 hash of a namespace UUID and a name>>> uuid.uuid5(uuid.NAMESPACE_DNS,'python.org')
UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d')>>># make a UUID from a string of hex digits (braces and hyphens ignored)>>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}')>>># convert a UUID to a string of hex digits in standard form>>> str(x)'00010203-0405-0607-0809-0a0b0c0d0e0f'>>># get the raw 16 bytes of the UUID>>> x.bytes'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f'>>># make a UUID from a 16-byte string>>> uuid.UUID(bytes=x.bytes)
UUID('00010203-0405-0607-0809-0a0b0c0d0e0f')
>>> import uuid
>>> # make a UUID based on the host ID and current time
>>> uuid.uuid1()
UUID('a8098c1a-f86e-11da-bd1a-00112444be1e')
>>> # make a UUID using an MD5 hash of a namespace UUID and a name
>>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org')
UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e')
>>> # make a random UUID
>>> uuid.uuid4()
UUID('16fd2706-8baf-433b-82eb-8c7fada847da')
>>> # make a UUID using a SHA-1 hash of a namespace UUID and a name
>>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org')
UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d')
>>> # make a UUID from a string of hex digits (braces and hyphens ignored)
>>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}')
>>> # convert a UUID to a string of hex digits in standard form
>>> str(x)
'00010203-0405-0607-0809-0a0b0c0d0e0f'
>>> # get the raw 16 bytes of the UUID
>>> x.bytes
'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f'
>>> # make a UUID from a 16-byte string
>>> uuid.UUID(bytes=x.bytes)
UUID('00010203-0405-0607-0809-0a0b0c0d0e0f')
I use GUIDs as random keys for database type operations.
The hexadecimal form, with the dashes and extra characters seem unnecessarily long to me. But I also like that strings representing hexadecimal numbers are very safe in that they do not contain characters that can cause problems in some situations such as ‘+’,’=’, etc..
Instead of hexadecimal, I use a url-safe base64 string. The following does not conform to any UUID/GUID spec though (other than having the required amount of randomness).
import base64
import uuid
# get a UUID - URL safe, Base64
def get_a_uuid():
r_uuid = base64.urlsafe_b64encode(uuid.uuid4().bytes)
return r_uuid.replace('=', '')
If you need to pass UUID for a primary key for your model or unique field then below code returns the UUID object –
import uuid
uuid.uuid4()
If you need to pass UUID as a parameter for URL you can do like below code –
import uuid
str(uuid.uuid4())
If you want the hex value for a UUID you can do the below one –
import uuid
uuid.uuid4().hex
回答 5
此功能是完全可配置的,并根据指定的格式生成唯一的uid
例如:-[8,4,4,4,12],这是提到的格式,它将生成以下uuid
LxoYNyXe-7hbQ-caJt-DSdU-PDAht56cMEWi
import random as rdef generate_uuid():
random_string =''
random_str_seq ="0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
uuid_format =[8,4,4,4,12]for n in uuid_format:for i in range(0,n):
random_string += str(random_str_seq[r.randint(0, len(random_str_seq)-1)])if n !=12:
random_string +='-'return random_string
This function is fully configurable and generates unique uid based on the format specified
eg:- [8, 4, 4, 4, 12] , this is the format mentioned and it will generate the following uuid
LxoYNyXe-7hbQ-caJt-DSdU-PDAht56cMEWi
import random as r
def generate_uuid():
random_string = ''
random_str_seq = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
uuid_format = [8, 4, 4, 4, 12]
for n in uuid_format:
for i in range(0,n):
random_string += str(random_str_seq[r.randint(0, len(random_str_seq) - 1)])
if n != 12:
random_string += '-'
return random_string
from typing importOptionalimport reimport subprocessimport uuiddef get_windows_uuid()->Optional[uuid.UUID]:try:# Ask Windows for the device's permanent UUID. Throws if command missing/fails.
txt = subprocess.check_output("wmic csproduct get uuid").decode()# Attempt to extract the UUID from the command's result.
match = re.search(r"\bUUID\b[\s\r\n]+([^\s\r\n]+)", txt)if match isnotNone:
txt = match.group(1)if txt isnotNone:# Remove the surrounding whitespace (newlines, space, etc)# and useless dashes etc, by only keeping hex (0-9 A-F) chars.
txt = re.sub(r"[^0-9A-Fa-f]+","", txt)# Ensure we have exactly 32 characters (16 bytes).if len(txt)==32:return uuid.UUID(txt)except:pass# Silence subprocess exception.returnNoneprint(get_windows_uuid())
from typing import Optional
import re
import subprocess
import uuid
def get_windows_uuid() -> Optional[uuid.UUID]:
try:
# Ask Windows for the device's permanent UUID. Throws if command missing/fails.
txt = subprocess.check_output("wmic csproduct get uuid").decode()
# Attempt to extract the UUID from the command's result.
match = re.search(r"\bUUID\b[\s\r\n]+([^\s\r\n]+)", txt)
if match is not None:
txt = match.group(1)
if txt is not None:
# Remove the surrounding whitespace (newlines, space, etc)
# and useless dashes etc, by only keeping hex (0-9 A-F) chars.
txt = re.sub(r"[^0-9A-Fa-f]+", "", txt)
# Ensure we have exactly 32 characters (16 bytes).
if len(txt) == 32:
return uuid.UUID(txt)
except:
pass # Silence subprocess exception.
return None
print(get_windows_uuid())
Uses Windows API to get the computer’s permanent UUID, then processes the string to ensure it’s a valid UUID, and lastly returns a Python object (https://docs.python.org/3/library/uuid.html) which gives you convenient ways to use the data (such as 128-bit integer, hex string, etc).
Good luck!
PS: The subprocess call could probably be replaced with ctypes directly calling Windows kernel/DLLs. But for my purposes this function is all I need. It does strong validation and produces correct results.
import random import string # defining function for random # string id with parameter def ran_gen(size, chars=string.ascii_uppercase + string.digits):return''.join(random.choice(chars)for x in range(size))# function call for random string # generation with size 8 and string print(ran_gen(8,"AEIOSUMA23"))
Check this post, helped me a lot. In short, the best option for me was:
import random
import string
# defining function for random
# string id with parameter
def ran_gen(size, chars=string.ascii_uppercase + string.digits):
return ''.join(random.choice(chars) for x in range(size))
# function call for random string
# generation with size 8 and string
print (ran_gen(8, "AEIOSUMA23"))
Because I needed just 4-6 random characters instead of bulky GUID.
If this is for Python 2.x or for Python 3.2+, you can also use callable(). It used to be deprecated, but is now undeprecated, so you can use it again. You can read the discussion here: http://bugs.python.org/issue10518. You can do this with:
callable(obj)
If this is for Python 3.x but before 3.2, check if the object has a __call__ attribute. You can do this with:
hasattr(obj, '__call__')
The oft-suggested types.FunctionTypes approach is not correct because it fails to cover many cases that you would presumably want it to pass, like with builtins:
The proper way to check properties of duck-typed objects is to ask them if they quack, not to see if they fit in a duck-sized container. Don’t use types.FunctionType unless you have a very specific idea of what a function is.
Builtin types that don’t have constructors in the built-in namespace (e.g. functions, generators, methods) are in the types module. You can use types.FunctionType in an isinstance call:
In [1]: import types
In [2]: types.FunctionType
Out[2]: <type 'function'>
In [3]: def f(): pass
...:
In [4]: isinstance(f, types.FunctionType)
Out[4]: True
In [5]: isinstance(lambda x : None, types.FunctionType)
Out[5]: True
Note that this uses a very specific notion of “function” that is usually not what you need. For example, it rejects zip (technically a class):
If you’re doing something specific to types.FunctionType instances, like decompiling their bytecode or inspecting closure variables, use types.FunctionType, but if you just need an object to be callable like a function, use callable.
The accepted answer was at the time it was offered thought to be correct. As it
turns out, there is no substitute for callable(), which is back in Python
3.2: Specifically, callable() checks the tp_call field of the object being
tested. There is no plain Python equivalent. Most of the suggested tests are
correct most of the time:
We can throw a monkey-wrench into this by removing the __call__ from the
class. And just to keep things extra exciting, add a fake __call__ to the instance!
>>> del Spam.__call__
>>> can_o_spam.__call__ = lambda *args: 'OK?'
Notice this really isn’t callable:
>>> can_o_spam()
Traceback (most recent call last):
...
TypeError: 'Spam' object is not callable
callable() returns the correct result:
>>> callable(can_o_spam)
False
But hasattr is wrong:
>>> hasattr(can_o_spam, '__call__')
True
can_o_spam does have that attribute after all; it’s just not used when calling
the instance.
Even more subtle, isinstance() also gets this wrong:
Because we used this check earlier and later deleted the method, abc.ABCMeta
caches the result. Arguably this is a bug in abc.ABCMeta. That said,
there’s really no possible way it could produce a more accurate result than
the result than by using callable() itself, since the typeobject->tp_call
slot method is not accessible in any other way.
callable(x)will return true if the object passed can be called in Python, but the function does not exist in Python 3.0, and properly speaking will not distinguish between:
class A(object):
def __call__(self):
return 'Foo'
def B():
return 'Bar'
a = A()
b = B
print type(a), callable(a)
print type(b), callable(b)
You’ll get <class 'A'> True and <type function> True as output.
isinstance works perfectly well to determine if something is a function (try isinstance(b, types.FunctionType)); if you’re really interested in knowing if something can be called, you can either use hasattr(b, '__call__') or just try it.
This, of course, won’t tell you whether it’s callable but throws a TypeError when it executes, or isn’t callable in the first place. That may not matter to you.
If you want to detect everything that syntactically looks like a function: a function, method, built-in fun/meth, lambda … but exclude callable objects (objects with __call__ method defined), then try this one:
I compared this with the code of is*() checks in inspect module and the expression above is much more complete, especially if your goal is filtering out any functions or detecting regular properties of an object.
class A():def __call__(self, a,b):print(a,b)def func1(self, a, b):print("[classfunction]:", a, b)@classmethoddef func2(cls, a,b):print("[classmethod]:", a, b)@staticmethoddef func3(a,b):print("[staticmethod]:", a, b)def func(a,b):print("[function]", a,b)
定义函子:
#(1.1) built-in function
builtins_func = open
#(1.2) ordinary function
ordinary_func = func
#(1.3) lambda expression
lambda_func =lambda a : func(a,4)#(1.4) functools.partial
partial_func = functools.partial(func, b=4)#(2.1) callable class instance
class_callable_instance = A()#(2.2) ordinary class function
class_ordinary_func = A.func1
#(2.3) bound class method
class_bound_method = A.func2
#(2.4) static class method
class_static_func = A.func3
定义函子列表和类型列表:
## list of functors
xfuncs =[builtins_func, ordinary_func, lambda_func, partial_func, class_callable_instance, class_ordinary_func, class_bound_method, class_static_func]## list of type
xtypes =[types.BuiltinFunctionType, types.FunctionType, types.MethodType, types.LambdaType, functools.partial]
判断函子是否可调用。如您所见,它们都是可调用的。
res =[callable(xfunc)for xfunc in xfuncs]print("functors callable:")print(res)"""
functors callable:
[True, True, True, True, True, True, True, True]
"""
判断函子的类型(types.XXX)。那么函子的类型并不完全相同。
res =[[isinstance(xfunc, xtype)for xtype in xtypes]for xfunc in xfuncs]## output the resultprint("functors' types")for(row, xfunc)in zip(res, xfuncs):print(row, xfunc)"""
functors' types
[True, False, False, False, False] <built-in function open>
[False, True, False, True, False] <function func at 0x7f1b5203e048>
[False, True, False, True, False] <function <lambda> at 0x7f1b5081fd08>
[False, False, False, False, True] functools.partial(<function func at 0x7f1b5203e048>, b=4)
[False, False, False, False, False] <__main__.A object at 0x7f1b50870cc0>
[False, True, False, True, False] <function A.func1 at 0x7f1b5081fb70>
[False, False, True, False, False] <bound method A.func2 of <class '__main__.A'>>
[False, True, False, True, False] <function A.func3 at 0x7f1b5081fc80>
"""
If you have learned C++, you must be familiar with function object or functor, means any object that can be called as if it is a function.
In C++, an ordinary function is a function object, and so is a function pointer; more generally, so is an object of a class that defines operator(). In C++11 and greater, the lambda expression is the functor too.
Similarity, in Python, those functors are all callable. An ordinary function can be callable, a lambda expression can be callable, a functional.partial can be callable, the instances of class with a __call__() method can be callable.
Ok, go back to question : I have a variable, x, and I want to know whether it is pointing to a function or not.
If you want to judge weather the object acts like a function, then the callable method suggested by @John Feminella is ok.
If you want to judge whether a object is just an ordinary function or not( not a callable class instance, or a lambda expression), then the xtypes.XXX suggested by @Ryan is a better choice.
class A():
def __call__(self, a,b):
print(a,b)
def func1(self, a, b):
print("[classfunction]:", a, b)
@classmethod
def func2(cls, a,b):
print("[classmethod]:", a, b)
@staticmethod
def func3(a,b):
print("[staticmethod]:", a, b)
def func(a,b):
print("[function]", a,b)
Define the functors:
#(1.1) built-in function
builtins_func = open
#(1.2) ordinary function
ordinary_func = func
#(1.3) lambda expression
lambda_func = lambda a : func(a,4)
#(1.4) functools.partial
partial_func = functools.partial(func, b=4)
#(2.1) callable class instance
class_callable_instance = A()
#(2.2) ordinary class function
class_ordinary_func = A.func1
#(2.3) bound class method
class_bound_method = A.func2
#(2.4) static class method
class_static_func = A.func3
Define the functors’ list and the types’ list:
## list of functors
xfuncs = [builtins_func, ordinary_func, lambda_func, partial_func, class_callable_instance, class_ordinary_func, class_bound_method, class_static_func]
## list of type
xtypes = [types.BuiltinFunctionType, types.FunctionType, types.MethodType, types.LambdaType, functools.partial]
Judge wether the functor is callable. As you can see, they all are callable.
res = [callable(xfunc) for xfunc in xfuncs]
print("functors callable:")
print(res)
"""
functors callable:
[True, True, True, True, True, True, True, True]
"""
Judge the functor’s type( types.XXX). Then the types of functors are not all the same.
res = [[isinstance(xfunc, xtype) for xtype in xtypes] for xfunc in xfuncs]
## output the result
print("functors' types")
for (row, xfunc) in zip(res, xfuncs):
print(row, xfunc)
"""
functors' types
[True, False, False, False, False] <built-in function open>
[False, True, False, True, False] <function func at 0x7f1b5203e048>
[False, True, False, True, False] <function <lambda> at 0x7f1b5081fd08>
[False, False, False, False, True] functools.partial(<function func at 0x7f1b5203e048>, b=4)
[False, False, False, False, False] <__main__.A object at 0x7f1b50870cc0>
[False, True, False, True, False] <function A.func1 at 0x7f1b5081fb70>
[False, False, True, False, False] <bound method A.func2 of <class '__main__.A'>>
[False, True, False, True, False] <function A.func3 at 0x7f1b5081fc80>
"""
I draw a table of callable functor’s types using the data.
Then you can choose the functors’ types that suitable.
#check inspect.isfunction and type.FunctionTypeIn[46]: inspect.isfunction(detect_function)Out[46]:TrueIn[47]: inspect.isfunction(hasattr)Out[47]:FalseIn[48]: isinstance(detect_function, types.FunctionType)Out[48]:TrueIn[49]: isinstance(getattr, types.FunctionType)Out[49]:False#so they both just applied to judge the user-definded
As the accepted answer, John Feminella stated that:
The proper way to check properties of duck-typed objects is to ask them if they quack, not to see if they fit in a duck-sized container. The “compare it directly” approach will give the wrong answer for many functions, like builtins.
Even though, there’re two libs to distinguish functions strictly, I draw an exhaustive comparable table:
The “duck typing” is a preferred solution for general purpose:
def detect_function(obj):
return hasattr(obj,"__call__")
In [26]: detect_function(detect_function)
Out[26]: True
In [27]: callable(detect_function)
Out[27]: True
As for the builtins function
In [43]: callable(hasattr)
Out[43]: True
When go one more step to check if builtin function or user-defined funtion
#check inspect.isfunction and type.FunctionType
In [46]: inspect.isfunction(detect_function)
Out[46]: True
In [47]: inspect.isfunction(hasattr)
Out[47]: False
In [48]: isinstance(detect_function, types.FunctionType)
Out[48]: True
In [49]: isinstance(getattr, types.FunctionType)
Out[49]: False
#so they both just applied to judge the user-definded
Determine if builtin function
In [50]: isinstance(getattr, types.BuiltinFunctionType)
Out[50]: True
In [51]: isinstance(detect_function, types.BuiltinFunctionType)
Out[51]: False
Summary
Employ callable to duck type checking a function,
Use types.BuiltinFunctionType if you have further specified demand.
回答 11
函数只是带有__call__方法的类,因此您可以执行
hasattr(obj,'__call__')
例如:
>>> hasattr(x,'__call__')True>>> x =2>>> hasattr(x,'__call__')False
>>>def x():...raiseTypeError...>>> hasattr(x,'__call__')True# Correct>>>try:... x()...exceptTypeError:...print"x was not callable"...
x was not callable # Wrong!
That is the “best” way of doing it, but depending on why you need to know if it’s callable or note, you could just put it in a try/execpt block:
try:
x()
except TypeError:
print "was not callable"
It’s arguable if try/except is more Python’y than doing if hasattr(x, '__call__'): x().. I would say hasattr is more accurate, since you wont accidently catch the wrong TypeError, for example:
>>> def x():
... raise TypeError
...
>>> hasattr(x, '__call__')
True # Correct
>>> try:
... x()
... except TypeError:
... print "x was not callable"
...
x was not callable # Wrong!
Instead of checking for '__call__' (which is not exclusive to functions), you can check whether a user-defined function has attributes func_name, func_doc, etc. This does not work for methods.
Another way of checking is using the isfunction() method from the inspect module.
>>> import inspect
>>> inspect.isfunction(x)
True
To check if an object is a method, use inspect.ismethod()
回答 14
由于类也具有__call__方法,因此我建议另一个解决方案:
class A(object):def __init__(self):passdef __call__(self):print'I am a Class'MyClass= A()def foo():passprint hasattr(foo.__class__,'func_name')# Returns Trueprint hasattr(A.__class__,'func_name')# Returns False as expectedprint hasattr(foo,'__call__')# Returns Trueprint hasattr(A,'__call__')# (!) Returns True while it is not a function
The solutions using hasattr(obj, '__call__') and callable(.) mentioned in some of the answers have a main drawback: both also return True for classes and instances of classes with a __call__() method. Eg.
callable is a very good solution. However, I wanted to treat this the opposite way of John Feminella. Instead of treating it like this saying:
The proper way to check properties of duck-typed objects is to ask them if they quack, not to see if they fit in a duck-sized container. The “compare it directly” approach will give the wrong answer for many functions, like builtins.
We’ll treat it like this:
The proper way to check if something is a duck is not to see if it can quack, but rather to see if it truly is a duck through several filters, instead of just checking if it seems like a duck from the surface.
How Would We Implement It
The ‘types’ module has plenty of classes to detect functions, the most useful being types.FunctionType, but there are also plenty of others, like a method type, a built in type, and a lambda type. We also will consider a ‘functools.partial’ object as being a function.
The simple way we check if it is a function is by using an isinstance condition on all of these types. Previously, I wanted to make a base class which inherits from all of the above, but I am unable to do that, as Python does not allow us to inherit from some of the above classes.
Here’s a table of what classes can classify what functions:
Above function table by kinght-金
The Code Which Does It
Now, this is the code which does all of the work we described from above.
from types import BuiltinFunctionType, BuiltinMethodType, FunctionType, MethodType, LambdaType
from functools import partial
def is_function(obj):
return isinstance(obj, (BuiltinFunctionType, BuiltinMethodType, FunctionType, MethodType, LambdaType, partial))
#-------------------------------------------------
def my_func():
pass
def add_both(x, y):
return x + y
class a:
def b(self):
pass
check = [
is_function(lambda x: x + x),
is_function(my_func),
is_function(a.b),
is_function(partial),
is_function(partial(add_both, 2))
]
print(check)
>>> [True, True, True, False, True]
The one false was is_function(partial), because that’s a class, not a function, and this is exactly functions, not classes. Here is a preview for you to try out the code from.
Conclusion
callable(obj) is the preferred method to check if an object is a function if you want to go by duck-typing over absolutes.
Our custom is_function(obj), maybe with some edits is the preferred method to check if an object is a function if you don’t any count callable class instance as a function, but only functions defined built-in, or with lambda, def, or partial.
And I think that wraps it all up. Have a good day!
回答 19
在Python3中,我想出了type (f) == type (lambda x:x)产生Trueif f是否为函数的结果False。但是我想我更喜欢isinstance (f, types.FunctionType),感觉不太特别。我想做type (f) is function,但这不起作用。
In Python3 I came up with type (f) == type (lambda x:x) which yields True if f is a function and False if it is not. But I think I prefer isinstance (f, types.FunctionType), which feels less ad hoc. I wanted to do type (f) is function, but that doesn’t work.
回答 20
在先前的答复之后,我想到了这一点:
from pprint import pprint
def print_callables_of(obj):
li =[]for name in dir(obj):
attr = getattr(obj, name)if hasattr(attr,'__call__'):
li.append(name)
pprint(li)
from pprint import pprint
def print_callables_of(obj):
li = []
for name in dir(obj):
attr = getattr(obj, name)
if hasattr(attr, '__call__'):
li.append(name)
pprint(li)
回答 21
您可以尝试以下方法:
if obj.__class__.__name__ in['function','builtin_function_or_method']:print('probably a function')
甚至更怪异的东西:
if"function"in lower(obj.__class__.__name__):print('probably a function')
If PyPy can solve these great challenges, what are its weaknesses that are preventing wider adoption? That is to say, what’s preventing someone like me, a typical Python developer, from switching to PyPy right now?
NOTE: PyPy is more mature and better supported now than it was in 2013, when this question was asked. Avoid drawing conclusions from out-of-date information.
PyPy, as others have been quick to mention, has tenuous support for C extensions. It has support, but typically at slower-than-Python speeds and it’s iffy at best. Hence a lot of modules simply require CPython. PyPy doesn’t support numpyPyPy now supports numpy. Some extensions are still not supported (Pandas, SciPy, etc.), take a look at the list of supported packages before making the change.
Python 3 support is experimental at the moment.has just reached stable! As of 20th June 2014, PyPy3 2.3.1 – Fulcrum is out!
PyPy sometimes isn’t actually faster for “scripts”, which a lot of people use Python for. These are the short-running programs that do something simple and small. Because PyPy is a JIT compiler its main advantages come from long run times and simple types (such as numbers). Frankly, PyPy’s pre-JIT speeds are pretty bad compared to CPython.
Inertia. Moving to PyPy often requires retooling, which for some people and organizations is simply too much work.
Those are the main reasons that affect me, I’d say.
That site does not claim PyPy is 6.3 times faster than CPython. To quote:
The geometric average of all benchmarks is 0.16 or 6.3 times faster than CPython
This is a very different statement to the blanket statement you made, and when you understand the difference, you’ll understand at least one set of reasons why you can’t just say “use PyPy”. It might sound like I’m nit-picking, but understanding why these two statements are totally different is vital.
To break that down:
The statement they make only applies to the benchmarks they’ve used. It says absolutely nothing about your program (unless your program is exactly the same as one of their benchmarks).
The statement is about an average of a group of benchmarks. There is no claim that running PyPy will give a 6.3 times improvement even for the programs they have tested.
There is no claim that PyPy will even run all the programs that CPython runs at all, let alone faster.
Because pypy is not 100% compatible, takes 8 gigs of ram to compile, is a moving target, and highly experimental, where cpython is stable, the default target for module builders for 2 decades (including c extensions that don’t work on pypy), and already widely deployed.
Pypy will likely never be the reference implementation, but it is a good tool to have.
The second question is easier to answer: you basically can use PyPy as a drop-in replacement if all your code is pure Python. However, many widely used libraries (including some of the standard library) are written in C and compiled as Python extensions. Some of these can be made to work with PyPy, some can’t. PyPy provides the same “forward-facing” tool as Python — that is, it is Python — but its innards are different, so tools that interface with those innards won’t work.
As for the first question, I imagine it is sort of a Catch-22 with the first: PyPy has been evolving rapidly in an effort to improve speed and enhance interoperability with other code. This has made it more experimental than official.
I think it’s possible that if PyPy gets into a stable state, it may start getting more widely used. I also think it would be great for Python to move away from its C underpinnings. But it won’t happen for a while. PyPy hasn’t yet reached the critical mass where it is almost useful enough on its own to do everything you’d want, which would motivate people to fill in the gaps.
I did a small benchmark on this topic. While many of the other posters have made good points about compatibility, my experience has been that PyPy isn’t that much faster for just moving around bits. For many uses of Python, it really only exists to translate bits between two or more services. For example, not many web applications are performing CPU intensive analysis of datasets. Instead, they take some bytes from a client, store them in some sort of database, and later return them to other clients. Sometimes the format of the data is changed.
The BDFL and the CPython developers are a remarkably intelligent group of people and have a managed to help CPython perform excellent in such a scenario. Here’s a shameless blog plug: http://www.hydrogen18.com/blog/unpickling-buffers.html . I’m using Stackless, which is derived from CPython and retains the full C module interface. I didn’t find any advantage to using PyPy in that case.
Q: If PyPy can solve these great challenges (speed, memory consumption, parallelism) in comparison to CPython, what are its weaknesses that are preventing wider adoption?
A: First, there is little evidence that the PyPy team can solve the speed problem in general. Long-term evidence is showing that PyPy runs certain Python codes slower than CPython and this drawback seems to be rooted very deeply in PyPy.
Secondly, the current version of PyPy consumes much more memory than CPython in a rather large set of cases. So PyPy didn’t solve the memory consumption problem yet.
Whether PyPy solves the mentioned great challenges and will in general be faster, less memory hungry, and more friendly to parallelism than CPython is an open question that cannot be solved in the short term. Some people are betting that PyPy will never be able to offer a general solution enabling it to dominate CPython 2.7 and 3.3 in all cases.
If PyPy succeeds to be better than CPython in general, which is questionable, the main weakness affecting its wider adoption will be its compatibility with CPython. There also exist issues such as the fact that CPython runs on a wider range of CPUs and OSes, but these issues are much less important compared to PyPy’s performance and CPython-compatibility goals.
Q: Why can’t I do drop in replacement of CPython with PyPy now?
A: PyPy isn’t 100% compatible with CPython because it isn’t simulating CPython under the hood. Some programs may still depend on CPython’s unique features that are absent in PyPy such as C bindings, C implementations of Python object&methods, or the incremental nature of CPython’s garbage collector.
CPython has reference counting and garbage collection, PyPy has garbage collection only.
So objects tend to be deleted earlier and __del__ is called in a more predictable way in CPython. Some software relies on this behavior, thus they are not ready for migrating to PyPy.
Some other software works with both, but uses less memory with CPython, because unused objects are freed earlier. (I don’t have any measurements to indicate how significant this is and what other implementation details affect the memory use.)
For a lot of projects, there is actually 0% difference between the different pythons in terms of speed. That is those that are dominated by engineering time and where all pythons have the same amount of library support.
To make this simple: PyPy provides the speed that’s lacked by CPython but sacrifices its compatibility. Most people, however, choose Python for its flexibility and its “battery-included” feature (high compatibility), not for its speed (it’s still preferred though).
回答 9
我发现了一些例子,其中PyPy比Python慢。但是:仅在Windows上。
C:\Users\User>python -m timeit -n10 -s"from sympy import isprime""isprime(2**521-1);isprime(2**1279-1)"10 loops, best of 3:294 msec per loop
C:\Users\User>pypy -m timeit -n10 -s"from sympy import isprime""isprime(2**521-1);isprime(2**1279-1)"10 loops, best of 3:1.33 sec per loop
For Project Euler problems, I make frequent use of PyPy, and for simple numerical calculations often from __future__ import division is sufficient for my purposes, but Python 3 support is still being worked on as of 2018, with your best bet being on 64-bit Linux. Windows PyPy3.5 v6.0, the latest as of December 2018, is in beta.
For example, Python 3.7 introduced dataclasses and Python 3.8 introduced fstring =.
There might be other features in Python 3.7 and Python 3.8 which are more important to you. The point is that PyPy does not support Python 3.7 or Python 3.8 at the moment.
Reply from @Ib33X is awesome. If you want to remove every empty string, after stripped. you need to use the strip method too. Otherwise, it will return the empty string too if it has white spaces. Like, ” ” will be valid too for that answer. So, can be achieved by.
strings = ["first", "", "second ", " "]
[x.strip() for x in strings if x.strip()]
The answer for this will be ["first", "second"].
If you want to use filter method instead, you can do like
list(filter(lambda item: item.strip(), strings)). This is give the same result.
回答 5
代替if x,我将使用if X!=”来消除空字符串。像这样:
str_list =[x for x in str_list if x !='']
这将在列表中保留“无”数据类型。此外,如果您的列表中有整数,并且0是其中的一个,它也将被保留。
例如,
str_list =[None,'',0,"Hi",'',"Hello"][x for x in str_list if x !=''][None,0,"Hi","Hello"]
Depending on the size of your list, it may be most efficient if you use list.remove() rather than create a new list:
l = ["1", "", "3", ""]
while True:
try:
l.remove("")
except ValueError:
break
This has the advantage of not creating a new list, but the disadvantage of having to search from the beginning each time, although unlike using while '' in l as proposed above, it only requires searching once per occurrence of '' (there is certainly a way to keep the best of both methods, but it is more complicated).
Keep in mind that if you want to keep the white spaces within a string, you may remove them unintentionally using some approaches.
If you have this list
[‘hello world’, ‘ ‘, ”, ‘hello’]
what you may want [‘hello world’,’hello’]
first trim the list to convert any type of white space to empty string:
space_to_empty = [x.strip() for x in _text_list]
then remove empty string from them list
space_clean_list = [x for x in space_to_empty if x]
回答 8
用途filter:
newlist=filter(lambda x: len(x)>0, oldlist)
如所指出的,使用过滤器的缺点是它比替代方法慢。而且,lambda通常很昂贵。
或者,您可以选择最简单,最迭代的方法:
# I am assuming listtext is the original list containing (possibly) empty itemsfor item in listtext:if item:
newlist.append(str(item))# You can remove str() based on the content of your original list
The drawbacks of using filter as pointed out is that it is slower than alternatives; also, lambda is usually costly.
Or you can go for the simplest and the most iterative of all:
# I am assuming listtext is the original list containing (possibly) empty items
for item in listtext:
if item:
newlist.append(str(item))
# You can remove str() based on the content of your original list
this is the most intuitive of the methods and does it in decent time.
As reported by Aziz Altofilter(None, lstr) does not remove empty strings with a space ' ' but if you are sure lstr contains only string you can use filter(str.strip, lstr)
import datetime
mylist =[]
today = datetime.date.today()
mylist.append(today)print mylist[0]# print the date object, not the container ;-)2008-11-22# It's better to always use str() because :print"This is a new day : ", mylist[0]# will work>>>Thisis a new day :2008-11-22print"This is a new day : "+ mylist[0]# will crash>>> cannot concatenate 'str'and'datetime.date' objectsprint"This is a new day : "+ str(mylist[0])>>>Thisis a new day :2008-11-22
In Python, dates are objects. Therefore, when you manipulate them, you manipulate objects, not strings, not timestamps nor anything.
Any object in Python have TWO string representations:
The regular representation that is used by “print”, can be get using the str() function. It is most of the time the most common human readable format and is used to ease display. So str(datetime.datetime(2008, 11, 22, 19, 53, 42)) gives you '2008-11-22 19:53:42'.
The alternative representation that is used to represent the object nature (as a data). It can be get using the repr() function and is handy to know what kind of data your manipulating while you are developing or debugging. repr(datetime.datetime(2008, 11, 22, 19, 53, 42)) gives you 'datetime.datetime(2008, 11, 22, 19, 53, 42)'.
What happened is that when you have printed the date using “print”, it used str() so you could see a nice date string. But when you have printed mylist, you have printed a list of objects and Python tried to represent the set of data, using repr().
The How: what do you want to do with that?
Well, when you manipulate dates, keep using the date objects all long the way. They got thousand of useful methods and most of the Python API expect dates to be objects.
When you want to display them, just use str(). In Python, the good practice is to explicitly cast everything. So just when it’s time to print, get a string representation of your date using str(date).
One last thing. When you tried to print the dates, you printed mylist. If you want to print a date, you must print the date objects, not their container (the list).
E.G, you want to print all the date in a list :
for date in mylist :
print str(date)
Note that in that specific case, you can even omit str() because print will use it for you. But it should not become a habit :-)
Practical case, using your code
import datetime
mylist = []
today = datetime.date.today()
mylist.append(today)
print mylist[0] # print the date object, not the container ;-)
2008-11-22
# It's better to always use str() because :
print "This is a new day : ", mylist[0] # will work
>>> This is a new day : 2008-11-22
print "This is a new day : " + mylist[0] # will crash
>>> cannot concatenate 'str' and 'datetime.date' objects
print "This is a new day : " + str(mylist[0])
>>> This is a new day : 2008-11-22
Advanced date formatting
Dates have a default representation, but you may want to print them in a specific format. In that case, you can get a custom string representation using the strftime() method.
strftime() expects a string pattern explaining how you want to format your date.
E.G :
print today.strftime('We are the %d, %b %Y')
>>> 'We are the 22, Nov 2008'
All the letter after a "%" represent a format for something :
Since PEP3101, every object can have its own format used automatically by the method format of any string. In the case of the datetime, the format is the same used in
strftime. So you can do the same as above like this:
print "We are the {:%d, %b %Y}".format(today)
>>> 'We are the 22, Nov 2008'
The advantage of this form is that you can also convert other objects at the same time.
With the introduction of Formatted string literals (since Python 3.6, 2016-12-23) this can be written as
Dates can automatically adapt to the local language and culture if you use them the right way, but it’s a bit complicated. Maybe for another question on SO(Stack Overflow) ;-)
%a Locale’s abbreviated weekday name.%A Locale’s full weekday name.%b Locale’s abbreviated month name.%B Locale’s full month name.%c Locale’s appropriate date and time representation.%d Day of the month as a decimal number [01,31].%f Microsecondas a decimal number [0,999999], zero-padded on the left
%H Hour(24-hour clock)as a decimal number [00,23].%I Hour(12-hour clock)as a decimal number [01,12].%j Day of the year as a decimal number [001,366].%m Monthas a decimal number [01,12].%M Minuteas a decimal number [00,59].%p Locale’s equivalent of either AM or PM.%S Secondas a decimal number [00,61].%U Week number of the year (Sundayas the first day of the week)%w Weekdayas a decimal number [0(Sunday),6].%W Week number of the year (Mondayas the first day of the week)%x Locale’s appropriate date representation.%X Locale’s appropriate time representation.%y Year without century as a decimal number [00,99].%Y Yearwith century as a decimal number.%z UTC offset in the form +HHMM or-HHMM.%Z Time zone name (empty string if the object is naive).%% A literal '%' character.
这就是我们可以使用Python中的datetime和time模块来做的事情
import time
import datetime
print"Time in seconds since the epoch: %s"%time.time()print"Current date and time: ", datetime.datetime.now()print"Or like this: ", datetime.datetime.now().strftime("%y-%m-%d-%H-%M")print"Current year: ", datetime.date.today().strftime("%Y")print"Month of year: ", datetime.date.today().strftime("%B")print"Week number of the year: ", datetime.date.today().strftime("%W")print"Weekday of the week: ", datetime.date.today().strftime("%w")print"Day of year: ", datetime.date.today().strftime("%j")print"Day of the month : ", datetime.date.today().strftime("%d")print"Day of week: ", datetime.date.today().strftime("%A")
这将打印出如下内容:
Timein seconds since the epoch:1349271346.46Current date and time:2012-10-0315:35:46.461491Or like this:12-10-03-15-35Current year:2012Month of year:OctoberWeek number of the year:40Weekday of the week:3Day of year:277Day of the month :03Day of week:Wednesday
The date, datetime, and time objects all support a strftime(format) method,
to create a string representing the time under the control of an explicit format
string.
Here is a list of the format codes with their directive and meaning.
%a Locale’s abbreviated weekday name.
%A Locale’s full weekday name.
%b Locale’s abbreviated month name.
%B Locale’s full month name.
%c Locale’s appropriate date and time representation.
%d Day of the month as a decimal number [01,31].
%f Microsecond as a decimal number [0,999999], zero-padded on the left
%H Hour (24-hour clock) as a decimal number [00,23].
%I Hour (12-hour clock) as a decimal number [01,12].
%j Day of the year as a decimal number [001,366].
%m Month as a decimal number [01,12].
%M Minute as a decimal number [00,59].
%p Locale’s equivalent of either AM or PM.
%S Second as a decimal number [00,61].
%U Week number of the year (Sunday as the first day of the week)
%w Weekday as a decimal number [0(Sunday),6].
%W Week number of the year (Monday as the first day of the week)
%x Locale’s appropriate date representation.
%X Locale’s appropriate time representation.
%y Year without century as a decimal number [00,99].
%Y Year with century as a decimal number.
%z UTC offset in the form +HHMM or -HHMM.
%Z Time zone name (empty string if the object is naive).
%% A literal '%' character.
This is what we can do with the datetime and time modules in Python
import time
import datetime
print "Time in seconds since the epoch: %s" %time.time()
print "Current date and time: ", datetime.datetime.now()
print "Or like this: ", datetime.datetime.now().strftime("%y-%m-%d-%H-%M")
print "Current year: ", datetime.date.today().strftime("%Y")
print "Month of year: ", datetime.date.today().strftime("%B")
print "Week number of the year: ", datetime.date.today().strftime("%W")
print "Weekday of the week: ", datetime.date.today().strftime("%w")
print "Day of year: ", datetime.date.today().strftime("%j")
print "Day of the month : ", datetime.date.today().strftime("%d")
print "Day of week: ", datetime.date.today().strftime("%A")
That will print out something like this:
Time in seconds since the epoch: 1349271346.46
Current date and time: 2012-10-03 15:35:46.461491
Or like this: 12-10-03-15-35
Current year: 2012
Month of year: October
Week number of the year: 40
Weekday of the week: 3
Day of year: 277
Day of the month : 03
Day of week: Wednesday
>>> import time
>>> time.strftime("%Y-%m-%d %H:%M")
'2013-11-19 09:38'
回答 5
# convert date time to regular format.
d_date = datetime.datetime.now()
reg_format_date = d_date.strftime("%Y-%m-%d %I:%M:%S %p")print(reg_format_date)# some other date formats.
reg_format_date = d_date.strftime("%d %B %Y %I:%M:%S %p")print(reg_format_date)
reg_format_date = d_date.strftime("%Y-%m-%d %H:%M:%S")print(reg_format_date)
The date/time format directives are not documented as part of the Format String Syntax but rather in date, datetime, and time‘s strftime() documentation. The are based on the 1989 C Standard, but include some ISO 8601 directives since Python 3.6.
I hate the idea of importing too many modules for convenience. I would rather work with available module which in this case is datetime rather than calling a new module time.
A quick disclaimer for my answer – I’ve only been learning Python for about 2 weeks, so I am by no means an expert; therefore, my explanation may not be the best and I may use incorrect terminology. Anyway, here it goes.
I noticed in your code that when you declared your variable today = datetime.date.today() you chose to name your variable with the name of a built-in function.
When your next line of code mylist.append(today) appended your list, it appended the entire string datetime.date.today(), which you had previously set as the value of your today variable, rather than just appending today().
A simple solution, albeit maybe not one most coders would use when working with the datetime module, is to change the name of your variable.
import datetime
import time
months = ["Unknown","January","Febuary","Marchh","April","May","June","July","August","September","October","November","December"]
datetimeWrite = (time.strftime("%d-%m-%Y "))
date = time.strftime("%d")
month= time.strftime("%m")
choices = {'01': 'Jan', '02':'Feb','03':'Mar','04':'Apr','05':'May','06': 'Jun','07':'Jul','08':'Aug','09':'Sep','10':'Oct','11':'Nov','12':'Dec'}
result = choices.get(month, 'default')
year = time.strftime("%Y")
Date = date+"-"+result+"-"+year
print Date
In this way you can get Date formatted like this example: 22-Jun-2017
回答 21
我不太了解,但是可以pandas用来获取正确格式的时间:
>>>import pandas as pd
>>> pd.to_datetime('now')Timestamp('2018-10-07 06:03:30')>>>print(pd.to_datetime('now'))2018-10-0706:03:47>>> pd.to_datetime('now').date()
datetime.date(2018,10,7)>>>print(pd.to_datetime('now').date())2018-10-07>>>
和:
>>> l=[]>>> l.append(pd.to_datetime('now').date())>>> l
[datetime.date(2018,10,7)]>>> map(str,l)<map object at 0x0000005F67CCDF98>>>> list(map(str,l))['2018-10-07']
# Install setuptools to be able to download the following
sudo apt-get install python-setuptools# Install statlib for lightweight statistical tools
sudo easy_install statlib# Install construct for packing/unpacking binary data
sudo easy_install construct
I just installed the python modules: construct and statlib with setuptools like this:
# Install setuptools to be able to download the following
sudo apt-get install python-setuptools
# Install statlib for lightweight statistical tools
sudo easy_install statlib
# Install construct for packing/unpacking binary data
sudo easy_install construct
I want to be able to (programmatically) check their versions. Is there an equivalent to python --version I can run from the command line?
For an individual module, you can try the __version__ attribute, however there are modules without it:
$ python -c "import requests; print(requests.__version__)"
2.14.2
$ python -c "import lxml; print(lxml.__version__)"
Traceback (most recent call last):
File "<string>", line 1, in <module>
AttributeError: 'module' object has no attribute '__version__'
Lastly, as the commands in your question are prefixed with sudo, it appears you’re installing to the global python environment. Strongly advise to take look into python virtual environment managers, for example virtualenvwrapper
Use pkg_resources module distributed with setuptools library. Note that the string that you pass to get_distribution method should correspond to the PyPI entry.
Note that the string that you pass to the get_distribution method should be the package name as registered in PyPI, not the module name that you are trying to import.
Unfortunately these aren’t always the same (e.g. you do pip install memcached, but import memcache).
回答 3
我认为这可以帮助您,但请先安装show软件包才能运行,pip show然后使用show查找版本!
sudo pip install show# in order to get package version execute the below command
sudo pip show YOUR_PACKAGE_NAME | grep Version
module.__version__ is a good first thing to try, but it doesn’t always work.
If you don’t want to shell out, and you’re using pip 8 or 9, you can still use pip.get_installed_distributions() to get versions from within Python:
update:the solution here works in pip 8 and 9, but in pip 10 the function has been moved from pip.get_installed_distributions to pip._internal.utils.misc.get_installed_distributions to explicitly indicate that it’s not for external use. It’s not a good idea to rely on it if you’re using pip 10+.
import pip
pip.get_installed_distributions() # -> [distribute 0.6.16 (...), ...]
[
pkg.key + ': ' + pkg.version
for pkg in pip.get_installed_distributions()
if pkg.key in ['setuptools', 'statlib', 'construct']
] # -> nicely filtered list of ['setuptools: 3.3', ...]
回答 7
先前的答案不能解决我的问题,但是这段代码可以解决:
import sys
for name, module in sorted(sys.modules.items()):if hasattr(module,'__version__'):print name, module.__version__
Since python 3.8 it’s included in the standard library.
Then, to check a package’s version (in this example lxml) run:
>>> from importlib_metadata import version
>>> version('lxml')
'4.3.1'
Keep in mind that this works only for packages installed from PyPI. Also, you must pass a package name as an argument to the version method, rather than a module name that this package provides (although they’re usually the same).
first add python, pip to your environment variables. so that you can execute your commands from command prompt. then simply give python command.
then import the package
–>import scrapy
then print the version name
–>print(scrapy.__version__)
This will definitely work
回答 12
假设我们正在使用Jupyter Notebook(如果使用Terminal,请删除感叹号):
1)如果软件包(例如xgboost)是通过pip安装的:
!pip show xgboost
!pip freeze | grep xgboost
!pip list | grep xgboost
Note 1: We must regard the python version. If we have installed different versions of python, we have to open the terminal in the python version we are interested in. For example, opening the terminal with python3.8 can (surely will) give a different version of a library than opening with python3.5 or python2.7.
Note 2: We avoid using the print function, because its behavior depends on python2 or python3. We do not need it, the terminal will show the value of the expression.
This works in Jupyter Notebook on Windows, too! As long as Jupyter is launched from a bash-compliant command line such as Git Bash (MingW64), the solutions given in many of the answers can be used in Jupyter Notebook on Windows systems with one tiny tweak.
I’m running windows 10 Pro with Python installed via Anaconda, and the following code works when I launch Jupyter via Git Bash (but does not when I launch from the Anaconda prompt).
The tweak: Add an exclamation mark (!) in front of pip to make it !pip.
>>>!pip show lxml | grep Version
Version: 4.1.0
>>>!pip freeze | grep lxml
lxml==4.1.0
>>>!pip list | grep lxml
lxml 4.1.0
>>>!pip show lxml
Name: lxml
Version: 4.1.0
Summary: Powerful and Pythonic XML processing library combining libxml2/libxslt with the ElementTree API.
Home-page: http://lxml.de/
Author: lxml dev team
Author-email: lxml-dev@lxml.de
License: BSD
Location: c:\users\karls\anaconda2\lib\site-packages
Requires:
Required-by: jupyter-contrib-nbextensions
回答 17
快速的python程序列出所有包装(您可以将其复制到requirements.txt)
from pip._internal.utils.misc import get_installed_distributions
print_log =''for module in sorted(get_installed_distributions(), key=lambda x: x.key):
print_log += module.key +'~='+ module.version +'\n'print(print_log)
I found it quite unreliable to use the various tools available (including the best one pkg_resources mentioned by Jakub Kukul’ answer), as most of them do not cover all cases. For example
built-in modules
modules not installed but just added to the python path (by your IDE for example)
two versions of the same module available (one in python path superseding the one installed)
Since we needed a reliable way to get the version of any package, module or submodule, I ended up writing getversion. It is quite simple to use:
from getversion import get_module_version
import foo
version, details = get_module_version(foo)
Building on Jakub Kukul’s answer I found a more reliable way to solve this problem.
The main problem of that approach is that requires the packages to be installed “conventionally” (and that does not include using pip install --user), or be in the system PATH at Python initialisation.
To get around that you can use pkg_resources.find_distributions(path_to_search). This basically searches for distributions that would be importable if path_to_search was in the system PATH.
We can iterate through this generator like this:
avail_modules = {}
distros = pkg_resources.find_distributions(path_to_search)
for d in distros:
avail_modules[d.key] = d.version
This will return a dictionary having modules as keys and their version as value. This approach can be extended to a lot more than version number.
Thanks to Jakub Kukul for pointing to the right direction
I am using an XML file which contains size fields that specify the size of value. I must parse this XML and do my coding. When I want to change the value of a particular field, I will check the size field of that value. Here I want to compare whether the new value that I’m gong to enter is of the same size as in XML. I need to check the size of new value. In case of a string I can say its the length. But in case of int, float, etc. I am confused.
Just use the sys.getsizeof function defined in the sys module.
sys.getsizeof(object[, default]):
Return the size of an object in bytes.
The object can be any type of object.
All built-in objects will return
correct results, but this does not
have to hold true for third-party
extensions as it is implementation
specific.
The default argument allows to define
a value which will be returned if the
object type does not provide means to
retrieve the size and would cause a
TypeError.
getsizeof calls the object’s
__sizeof__ method and adds an additional garbage collector overhead
if the object is managed by the
garbage collector.
Empty
Bytes type scaling notes
28 int +4 bytes about every 30 powers of 2
37 bytes +1 byte per additional byte
49 str +1-4 per additional character (depending on max width)
48 tuple +8 per additional item
64 list +8 for each additional
224 set 5th increases to 736; 21nd, 2272; 85th, 8416; 341, 32992
240 dict 6th increases to 368; 22nd, 1184; 43rd, 2280; 86th, 4704; 171st, 9320
136 func def does not include default args and other attrs
1056 class def no slots
56 class inst has a __dict__ attr, same scaling as dict above
888 class def with slots
16 __slots__ seems to store in mutable tuple-like structure
first slot grows to 48, and so on.
/* This over-allocates proportional to the list size, making room
* for additional growth. The over-allocation is mild, but is
* enough to give linear-time amortized behavior over a long
* sequence of appends() in the presence of a poorly-performing
* system realloc().
* The growth pattern is: 0, 4, 8, 16, 25, 35, 46, 58, 72, 88, ...
* Note: new_allocated won't overflow because the largest possible value
* is PY_SSIZE_T_MAX * (9 / 8) + 6 which always fits in a size_t.
*/
new_allocated =(size_t)newsize +(newsize >>3)+(newsize <9?3:6);
历史数据
Python 2.7分析,通过guppy.hpy和确认sys.getsizeof:
Bytes type empty + scaling notes
24 int NA
28 long NA
37 str + 1 byte per additional character
52 unicode + 4 bytes per additional character
56 tuple + 8 bytes per additional item
72 list + 32 for first, 8 for each additional
232 set sixth item increases to 744; 22nd, 2280; 86th, 8424
280 dict sixth item increases to 1048; 22nd, 3352; 86th, 12568 *
120 func def does not include default args and other attrs
64 class inst has a __dict__ attr, same scaling as dict above
16 __slots__ class with slots has no dict, seems to store in
mutable tuple-like structure.
904 class def has a proxy __dict__ structure for class attrs
104 old class makes sense, less stuff, has real dict though.
How do I determine the size of an object in Python?
The answer, “Just use sys.getsizeof” is not a complete answer.
That answer does work for builtin objects directly, but it does not account for what those objects may contain, specifically, what types, such as custom objects, tuples, lists, dicts, and sets contain. They can contain instances each other, as well as numbers, strings and other objects.
A More Complete Answer
Using 64 bit Python 3.6 from the Anaconda distribution, with sys.getsizeof, I have determined the minimum size of the following objects, and note that sets and dicts preallocate space so empty ones don’t grow again until after a set amount (which may vary by implementation of the language):
Python 3:
Empty
Bytes type scaling notes
28 int +4 bytes about every 30 powers of 2
37 bytes +1 byte per additional byte
49 str +1-4 per additional character (depending on max width)
48 tuple +8 per additional item
64 list +8 for each additional
224 set 5th increases to 736; 21nd, 2272; 85th, 8416; 341, 32992
240 dict 6th increases to 368; 22nd, 1184; 43rd, 2280; 86th, 4704; 171st, 9320
136 func def does not include default args and other attrs
1056 class def no slots
56 class inst has a __dict__ attr, same scaling as dict above
888 class def with slots
16 __slots__ seems to store in mutable tuple-like structure
first slot grows to 48, and so on.
How do you interpret this? Well say you have a set with 10 items in it. If each item is 100 bytes each, how big is the whole data structure? The set is 736 itself because it has sized up one time to 736 bytes. Then you add the size of the items, so that’s 1736 bytes in total
Some caveats for function and class definitions:
Note each class definition has a proxy __dict__ (48 bytes) structure for class attrs. Each slot has a descriptor (like a property) in the class definition.
Slotted instances start out with 48 bytes on their first element, and increase by 8 each additional. Only empty slotted objects have 16 bytes, and an instance with no data makes very little sense.
Also, each function definition has code objects, maybe docstrings, and other possible attributes, even a __dict__.
Also note that we use sys.getsizeof() because we care about the marginal space usage, which includes the garbage collection overhead for the object, from the docs:
getsizeof() calls the object’s __sizeof__ method and adds an
additional garbage collector overhead if the object is managed by the
garbage collector.
Also note that resizing lists (e.g. repetitively appending to them) causes them to preallocate space, similarly to sets and dicts. From the listobj.c source code:
/* This over-allocates proportional to the list size, making room
* for additional growth. The over-allocation is mild, but is
* enough to give linear-time amortized behavior over a long
* sequence of appends() in the presence of a poorly-performing
* system realloc().
* The growth pattern is: 0, 4, 8, 16, 25, 35, 46, 58, 72, 88, ...
* Note: new_allocated won't overflow because the largest possible value
* is PY_SSIZE_T_MAX * (9 / 8) + 6 which always fits in a size_t.
*/
new_allocated = (size_t)newsize + (newsize >> 3) + (newsize < 9 ? 3 : 6);
Historical data
Python 2.7 analysis, confirmed with guppy.hpy and sys.getsizeof:
Bytes type empty + scaling notes
24 int NA
28 long NA
37 str + 1 byte per additional character
52 unicode + 4 bytes per additional character
56 tuple + 8 bytes per additional item
72 list + 32 for first, 8 for each additional
232 set sixth item increases to 744; 22nd, 2280; 86th, 8424
280 dict sixth item increases to 1048; 22nd, 3352; 86th, 12568 *
120 func def does not include default args and other attrs
64 class inst has a __dict__ attr, same scaling as dict above
16 __slots__ class with slots has no dict, seems to store in
mutable tuple-like structure.
904 class def has a proxy __dict__ structure for class attrs
104 old class makes sense, less stuff, has real dict though.
I think 8 bytes per additional item to reference makes a lot of sense on a 64 bit machine. Those 8 bytes point to the place in memory the contained item is at. The 4 bytes are fixed width for unicode in Python 2, if I recall correctly, but in Python 3, str becomes a unicode of width equal to the max width of the characters.
We want a function that searches the elements in lists, tuples, sets, dicts, obj.__dict__‘s, and obj.__slots__, as well as other things we may not have yet thought of.
We want to rely on gc.get_referents to do this search because it works at the C level (making it very fast). The downside is that get_referents can return redundant members, so we need to ensure we don’t double count.
Classes, modules, and functions are singletons – they exist one time in memory. We’re not so interested in their size, as there’s not much we can do about them – they’re a part of the program. So we’ll avoid counting them if they happen to be referenced.
We’re going to use a blacklist of types so we don’t include the entire program in our size count.
import sys
from types import ModuleType, FunctionType
from gc import get_referents
# Custom objects know their class.
# Function objects seem to know way too much, including modules.
# Exclude modules as well.
BLACKLIST = type, ModuleType, FunctionType
def getsize(obj):
"""sum size of object & members."""
if isinstance(obj, BLACKLIST):
raise TypeError('getsize() does not take argument of type: '+ str(type(obj)))
seen_ids = set()
size = 0
objects = [obj]
while objects:
need_referents = []
for obj in objects:
if not isinstance(obj, BLACKLIST) and id(obj) not in seen_ids:
seen_ids.add(id(obj))
size += sys.getsizeof(obj)
need_referents.append(obj)
objects = get_referents(*need_referents)
return size
To contrast this with the following whitelisted function, most objects know how to traverse themselves for the purposes of garbage collection (which is approximately what we’re looking for when we want to know how expensive in memory certain objects are. This functionality is used by gc.get_referents.) However, this measure is going to be much more expansive in scope than we intended if we are not careful.
For example, functions know quite a lot about the modules they are created in.
Another point of contrast is that strings that are keys in dictionaries are usually interned so they are not duplicated. Checking for id(key) will also allow us to avoid counting duplicates, which we do in the next section. The blacklist solution skips counting keys that are strings altogether.
To cover most of these types myself, instead of relying on the gc module, I wrote this recursive function to try to estimate the size of most Python objects, including most builtins, types in the collections module, and custom types (slotted and otherwise).
This sort of function gives much more fine-grained control over the types we’re going to count for memory usage, but has the danger of leaving types out:
import sys
from numbers import Number
from collections import Set, Mapping, deque
try: # Python 2
zero_depth_bases = (basestring, Number, xrange, bytearray)
iteritems = 'iteritems'
except NameError: # Python 3
zero_depth_bases = (str, bytes, Number, range, bytearray)
iteritems = 'items'
def getsize(obj_0):
"""Recursively iterate to sum size of object & members."""
_seen_ids = set()
def inner(obj):
obj_id = id(obj)
if obj_id in _seen_ids:
return 0
_seen_ids.add(obj_id)
size = sys.getsizeof(obj)
if isinstance(obj, zero_depth_bases):
pass # bypass remaining control flow and return
elif isinstance(obj, (tuple, list, Set, deque)):
size += sum(inner(i) for i in obj)
elif isinstance(obj, Mapping) or hasattr(obj, iteritems):
size += sum(inner(k) + inner(v) for k, v in getattr(obj, iteritems)())
# Check for custom object instances - may subclass above too
if hasattr(obj, '__dict__'):
size += inner(vars(obj))
if hasattr(obj, '__slots__'): # can have __slots__ with __dict__
size += sum(inner(getattr(obj, s)) for s in obj.__slots__ if hasattr(obj, s))
return size
return inner(obj_0)
And I tested it rather casually (I should unittest it):
This implementation breaks down on class definitions and function definitions because we don’t go after all of their attributes, but since they should only exist once in memory for the process, their size really doesn’t matter too much.
And if you need other view on live data, Pympler’s
module muppy is used for on-line monitoring of a Python application
and module Class Tracker provides off-line analysis of the lifetime of
selected Python objects.
回答 3
对于numpy数组,getsizeof它不起作用-对于我来说,由于某种原因它总是返回40:
from pylab import*from sys import getsizeof
A = rand(10)
B = rand(10000)
This can be more complicated than it looks depending on how you want to count things. For instance, if you have a list of ints, do you want the size of the list containing the references to the ints? (ie. list only, not what is contained in it), or do you want to include the actual data pointed to, in which case you need to deal with duplicate references, and how to prevent double-counting when two objects contain references to the same object.
You may want to take a look at one of the python memory profilers, such as pysizer to see if they meet your needs.
import sys
def get_size(obj, seen=None):"""Recursively finds size of objects"""
size = sys.getsizeof(obj)if seen isNone:
seen = set()
obj_id = id(obj)if obj_id in seen:return0# Important mark as seen *before* entering recursion to gracefully handle# self-referential objects
seen.add(obj_id)if isinstance(obj, dict):
size += sum([get_size(v, seen)for v in obj.values()])
size += sum([get_size(k, seen)for k in obj.keys()])elif hasattr(obj,'__dict__'):
size += get_size(obj.__dict__, seen)elif hasattr(obj,'__iter__')andnot isinstance(obj,(str, bytes, bytearray)):
size += sum([get_size(i, seen)for i in obj])return size
Having run into this problem many times myself, I wrote up a small function (inspired by @aaron-hall’s answer) & tests that does what I would have expected sys.getsizeof to do:
EDIT: Attaching the code below for easy reference. To see the most up-to-date code, please check the github link.
import sys
def get_size(obj, seen=None):
"""Recursively finds size of objects"""
size = sys.getsizeof(obj)
if seen is None:
seen = set()
obj_id = id(obj)
if obj_id in seen:
return 0
# Important mark as seen *before* entering recursion to gracefully handle
# self-referential objects
seen.add(obj_id)
if isinstance(obj, dict):
size += sum([get_size(v, seen) for v in obj.values()])
size += sum([get_size(k, seen) for k in obj.keys()])
elif hasattr(obj, '__dict__'):
size += get_size(obj.__dict__, seen)
elif hasattr(obj, '__iter__') and not isinstance(obj, (str, bytes, bytearray)):
size += sum([get_size(i, seen) for i in obj])
return size
Use sys.getsizeof() if you DON’T want to include sizes of linked (nested) objects.
However, if you want to count sub-objects nested in lists, dicts, sets, tuples – and usually THIS is what you’re looking for – use the recursive deep sizeof() function as shown below:
If you don’t need the exact size of the object but roughly to know how big it is, one quick (and dirty) way is to let the program run, sleep for an extended period of time, and check the memory usage (ex: Mac’s activity monitor) by this particular python process. This would be effective when you are trying to find the size of one single large object in a python process. For example, I recently wanted to check the memory usage of a new data structure and compare it with that of Python’s set data structure. First I wrote the elements (words from a large public domain book) to a set, then checked the size of the process, and then did the same thing with the other data structure. I found out the Python process with a set is taking twice as much memory as the new data structure. Again, you wouldn’t be able to exactly say the memory used by the process is equal to the size of the object. As the size of the object gets large, this becomes close as the memory consumed by the rest of the process becomes negligible compared to the size of the object you are trying to monitor.
回答 11
您可以使用如下所述的getSizeof()来确定对象的大小
import sys
str1 ="one"
int_element=5print("Memory size of '"+str1+"' = "+str(sys.getsizeof(str1))+" bytes")print("Memory size of '"+ str(int_element)+"' = "+str(sys.getsizeof(int_element))+" bytes")
import pygame as pg
import os
import psutil
import time
process = psutil.Process(os.getpid())
pg.init()
vocab =['hello','me','you','she','he','they','we','should','why?','necessarily','do','that']
font = pg.font.SysFont("monospace",100,True)
dct ={}
newMem = process.memory_info().rss # don't mind this lineStr= f'store '+ f'Nothing \tsurface use about '.expandtabs(15)+ \
f'0\t bytes'.expandtabs(9)# don't mind this assignment too
usedMem = process.memory_info().rss
for word in vocab:
dct[word]= font.render(word,True, pg.Color("#000000"))
time.sleep(0.1)# wait a moment# get total used memory of this script:
newMem = process.memory_info().rss
Str= f'store '+ f'{word}\tsurface use about '.expandtabs(15)+ \
f'{newMem - usedMem}\t bytes'.expandtabs(9)print(Str)
usedMem = newMem
在我的Windows 10(python 3.7.3)上,输出为:
store hello surface use about 225280 bytes
store me surface use about 61440 bytes
store you surface use about 94208 bytes
store she surface use about 81920 bytes
store he surface use about 53248 bytes
store they surface use about 114688 bytes
store we surface use about 57344 bytes
store should surface use about 172032 bytes
store why? surface use about 110592 bytes
store necessarily surface use about 311296 bytes
store do surface use about 57344 bytes
store that surface use about 110592 bytes
I use this trick… May won’t be accurate on small objects, but I think it’s much more accurate for a complex object (like pygame surface) rather than sys.getsizeof()
import pygame as pg
import os
import psutil
import time
process = psutil.Process(os.getpid())
pg.init()
vocab = ['hello', 'me', 'you', 'she', 'he', 'they', 'we',
'should', 'why?', 'necessarily', 'do', 'that']
font = pg.font.SysFont("monospace", 100, True)
dct = {}
newMem = process.memory_info().rss # don't mind this line
Str = f'store ' + f'Nothing \tsurface use about '.expandtabs(15) + \
f'0\t bytes'.expandtabs(9) # don't mind this assignment too
usedMem = process.memory_info().rss
for word in vocab:
dct[word] = font.render(word, True, pg.Color("#000000"))
time.sleep(0.1) # wait a moment
# get total used memory of this script:
newMem = process.memory_info().rss
Str = f'store ' + f'{word}\tsurface use about '.expandtabs(15) + \
f'{newMem - usedMem}\t bytes'.expandtabs(9)
print(Str)
usedMem = newMem
On my windows 10, python 3.7.3, the output is:
store hello surface use about 225280 bytes
store me surface use about 61440 bytes
store you surface use about 94208 bytes
store she surface use about 81920 bytes
store he surface use about 53248 bytes
store they surface use about 114688 bytes
store we surface use about 57344 bytes
store should surface use about 172032 bytes
store why? surface use about 110592 bytes
store necessarily surface use about 311296 bytes
store do surface use about 57344 bytes
store that surface use about 110592 bytes
Any solutions using the HOST or HOSTNAME environment variables are not portable. Even if it works on your system when you run it, it may not work when run in special environments such as cron.
On some systems, the hostname is set in the environment. If that is the case for you, the os module can pull it out of the environment via os.getenv. For example, if HOSTNAME is the environment variable containing what you want, the following will get it:
import os
system_name = os.getenv('HOSTNAME')
Update: As noted in the comments, this doesn’t always work, as not everyone’s environment is set up this way. I believe that at the time I initially answered this I was using this solution as it was the first thing I’d found in a web search and it worked for me at the time. Due to the lack of portability I probably wouldn’t use this now. However, I am leaving this answer for reference purposes. FWIW, it does eliminate the need for other imports if your environment has the system name and you are already importing the os module. Test it – if it doesn’t work in all the environments in which you expect your program to operate, use one of the other solutions provided.