Python 实用宝典

Question 1

I’m looking at how to do file input and output in Python. I’ve written the following code to read a list of names (one per line) from a file into another file while checking a name against the names in the file and appending text to the occurrences in the file. The code works. Could it be done better?

I’d wanted to use the with open(... statement for both input and output files but can’t see how they could be in the same block meaning I’d need to store the names in a temporary location.

def filter(txt, oldfile, newfile):
    '''\
    Read a list of names from a file line by line into an output file.
    If a line begins with a particular name, insert a string of text
    after the name before appending the line to the output file.
    '''

    outfile = open(newfile, 'w')
    with open(oldfile, 'r', encoding='utf-8') as infile:
        for line in infile:
            if line.startswith(txt):
                line = line[0:len(txt)] + ' - Truly a great person!\n'
            outfile.write(line)

    outfile.close()
    return # Do I gain anything by including this?

# input the name you want to check against
text = input('Please enter the name of a great person: ')    
letsgo = filter(text,'Spanish', 'Spanish2')

Question 2

Python allows putting multiple open() statements in a single with. You comma-separate them. Your code would then be:

def filter(txt, oldfile, newfile):
    '''\
    Read a list of names from a file line by line into an output file.
    If a line begins with a particular name, insert a string of text
    after the name before appending the line to the output file.
    '''

    with open(newfile, 'w') as outfile, open(oldfile, 'r', encoding='utf-8') as infile:
        for line in infile:
            if line.startswith(txt):
                line = line[0:len(txt)] + ' - Truly a great person!\n'
            outfile.write(line)

# input the name you want to check against
text = input('Please enter the name of a great person: ')    
letsgo = filter(text,'Spanish', 'Spanish2')

And no, you don’t gain anything by putting an explicit return at the end of your function. You can use return to exit early, but you had it at the end, and the function will exit without it. (Of course with functions that return a value, you use the return to specify the value to return.)

Using multiple open() items with with was not supported in Python 2.5 when the with statement was introduced, or in Python 2.6, but it is supported in Python 2.7 and Python 3.1 or newer.

http://docs.python.org/reference/compound_stmts.html#the-with-statement http://docs.python.org/release/3.1/reference/compound_stmts.html#the-with-statement

If you are writing code that must run in Python 2.5, 2.6 or 3.0, nest the with statements as the other answers suggested or use contextlib.nested.

Question 3

Use nested blocks like this,

with open(newfile, 'w') as outfile:
    with open(oldfile, 'r', encoding='utf-8') as infile:
        # your logic goes right here

Question 4

You can nest your with blocks. Like this:

with open(newfile, 'w') as outfile:
    with open(oldfile, 'r', encoding='utf-8') as infile:
        for line in infile:
            if line.startswith(txt):
                line = line[0:len(txt)] + ' - Truly a great person!\n'
            outfile.write(line)

This is better than your version because you guarantee that outfile will be closed even if your code encounters exceptions. Obviously you could do that with try/finally, but with is the right way to do this.

Or, as I have just learnt, you can have multiple context managers in a with statement as described by @steveha. That seems to me to be a better option than nesting.

And for your final minor question, the return serves no real purpose. I would remove it.

Question 5

Sometimes, you might want to open a variable amount of files and treat each one the same, you can do this with contextlib

from contextlib import ExitStack
filenames = [file1.txt, file2.txt, file3.txt]

with open('outfile.txt', 'a') as outfile:
    with ExitStack() as stack:
        file_pointers = [stack.enter_context(open(file, 'r')) for file in filenames]                
            for fp in file_pointers:
                outfile.write(fp.read())

Question 6

I installed Python 3.x (besides Python 2.x on Ubuntu) and slowly started to pair modules I use in Python 2.x.

So I wonder, what approach should I take to make my life easy by using pip for both Python 2.x and Python 3.x?

Question 7

The approach you should take is to install pip for Python 3.2.

You do this in the following way:

$ curl -O https://bootstrap.pypa.io/get-pip.py
$ sudo python3.2 get-pip.py

Then, you can install things for Python 3.2 with pip-3.2, and install things for Python 2-7 with pip-2.7. The pip command will end up pointing to one of these, but I’m not sure which, so you will have to check.

Question 8

What you can also do is to use apt-get:

apt-get install python3-pip

In my experience this works pretty fluent too, plus you get all the benefits from apt-get.

Question 9

First, install Python 3 pip using:

sudo apt-get install python3-pip

Then, to use Python 3 pip use:

pip3 install <module-name>

For Python 2 pip use:

pip install <module-name>

Question 10

If you don’t want to have to specify the version every time you use pip:

Install pip:

$ curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python3

and export the path:

$ export PATH=/Library/Frameworks/Python.framework/Versions/<version number>/bin:$PATH

Question 11

The shortest way:

python3 -m pip install package
python -m pip install package

Question 12

This worked for me on OS X: (I say this because sometimes is a pain that mac has “its own” version of every open source tool, and you cannot remove it because “its improvements” make it unique for other apple stuff to work, and if you remove it things start falling appart)

I followed the steps provided by @Lennart Regebro to get pip for python 3, nevertheless pip for python 2 was still first on the path, so… what I did is to create a symbolic link to python 3 inside /usr/bin (in deed I did the same to have my 2 pythons running in peace):

ln -s /Library/Frameworks/Python.framework/Versions/3.4/bin/pip /usr/bin/pip3

Notice that I added a 3 at the end, so basically what you have to do is to use pip3 instead of just pip.

The post is old but I hope this helps someone someday. this should theoretically work for any LINUX system.

Question 13

On Suse Linux 13.2, pip calls python3, but pip2 is available to use the older python version.

Question 14

In Windows, first installed Python 3.7 and then Python 2.7. Then, use command prompt:

pip install python2-module-name

pip3 install python3-module-name

That’s all

Question 15

Please note that on msys2 I’ve found these commands to be helpful:

$ pacman -S python3-pip
$ pip3 install --upgrade pip
$ pip3 install --user package_name

Question 16

Thought this is old question, I think I have better solution

To use pip for a python 2.x environment, use this command –

py -2 -m pip install -r requirements.txt
To use pip for python 3.x environment, use this command –

py -3 -m pip install -r requirements.txt

Question 17

What’s the difference between raise and raise from in Python?

try:
    raise ValueError
except Exception as e:
    raise IndexError

which yields

Traceback (most recent call last):
  File "tmp.py", line 2, in <module>
    raise ValueError
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tmp.py", line 4, in <module>
    raise IndexError
IndexError

and

try:
    raise ValueError
except Exception as e:
    raise IndexError from e

which yields

Traceback (most recent call last):
  File "tmp.py", line 2, in <module>
    raise ValueError
ValueError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "tmp.py", line 4, in <module>
    raise IndexError from e
IndexError

Question 18

The difference is that when you use from, the __cause__ attribute is set and the message states that the exception was directly caused by. If you omit the from then no __cause__ is set, but the __context__ attribute may be set as well, and the traceback then shows the context as during handling something else happened.

Setting the __context__ happens if you used raise in an exception handler; if you used raise anywhere else no __context__ is set either.

If a __cause__ is set, a __suppress_context__ = True flag is also set on the exception; when __suppress_context__ is set to True, the __context__ is ignored when printing a traceback.

When raising from a exception handler where you don’t want to show the context (don’t want a during handling another exception happened message), then use raise ... from None to set __suppress_context__ to True.

In other words, Python sets a context on exceptions so you can introspect where an exception was raised, letting you see if another exception was replaced by it. You can also add a cause to an exception, making the traceback explicit about the other exception (use different wording), and the context is ignored (but can still be introspected when debugging). Using raise ... from None lets you suppress the context being printed.

See the raise statement documenation:

The from clause is used for exception chaining: if given, the second expression must be another exception class or instance, which will then be attached to the raised exception as the __cause__ attribute (which is writable). If the raised exception is not handled, both exceptions will be printed:
>>> try:
...     print(1 / 0)
... except Exception as exc:
...     raise RuntimeError("Something bad happened") from exc
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ZeroDivisionError: int division or modulo by zero

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
RuntimeError: Something bad happened
A similar mechanism works implicitly if an exception is raised inside an exception handler or a finally clause: the previous exception is then attached as the new exception’s __context__ attribute:
>>> try:
...     print(1 / 0)
... except:
...     raise RuntimeError("Something bad happened")
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ZeroDivisionError: int division or modulo by zero

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
RuntimeError: Something bad happened

Also see the Built-in Exceptions documentation for details on the context and cause information attached to exceptions.

Question 19

I am using Python 3.5.1. I read the document and the package section here: https://docs.python.org/3/tutorial/modules.html#packages

Now, I have the following structure:

/home/wujek/Playground/a/b/module.py

module.py:

class Foo:
    def __init__(self):
        print('initializing Foo')

Now, while in /home/wujek/Playground:

~/Playground $ python3
>>> import a.b.module
>>> a.b.module.Foo()
initializing Foo
<a.b.module.Foo object at 0x100a8f0b8>

Similarly, now in home, superfolder of Playground:

~ $ PYTHONPATH=Playground python3
>>> import a.b.module
>>> a.b.module.Foo()
initializing Foo
<a.b.module.Foo object at 0x10a5fee10>

Actually, I can do all kinds of stuff:

~ $ PYTHONPATH=Playground python3
>>> import a
>>> import a.b
>>> import Playground.a.b

Why does this work? I though there needed to be __init__.py files (empty ones would work) in both a and b for module.py to be importable when the Python path points to the Playground folder?

This seems to have changed from Python 2.7:

~ $ PYTHONPATH=Playground python
>>> import a
ImportError: No module named a
>>> import a.b
ImportError: No module named a.b
>>> import a.b.module
ImportError: No module named a.b.module

With __init__.py in both ~/Playground/a and ~/Playground/a/b it works fine.

Question 20

Python 3.3+ has Implicit Namespace Packages that allow it to create a packages without an __init__.py file.

Allowing implicit namespace packages means that the requirement to provide an __init__.py file can be dropped completely, and affected … .

The old way with __init__.py files still works as in Python 2.

Question 21

IMPORTANT

@Mike’s answer is correct but too imprecise. It is true that Python 3.3+ supports Implicit Namespace Packages that allows it to create a package without an __init__.py file.

This however, ONLY applies to EMPTY __init__.py files. So EMPTY __init__.py files are no longer necessary and can be omitted. If you want to run a particular initialization script when the package or any of its modules or sub-packages are imported, you still require an __init__.py file. This is a great Stack Overflow answer for why you would want to use an __init__.py file to do some further initialization in case you wondering why this is in any way useful.

Directory Structure Example:

  parent_package/
     __init__.py            <- EMPTY, NOT NECESSARY in Python 3.3+
     child_package/
          __init__.py       <- STILL REQUIRED if you want to run an initialization script
          child1.py
          child2.py
          child3.py

parent_package/child_package/__init__.py:

print("from parent")

EXAMPLES

The below examples demonstrate how the initialization script is executed when the child_package or one of its modules is imported.

Example 1:

from parent_package import child_package  # prints "from parent"

Example 2:

from parent_package.child_package import child1  # prints "from parent"

Question 22

If you have setup.py in your project and you use find_packages() within it, it is necessary to have an __init__.py file in every directory for packages to be automatically found.

Packages are only recognized if they include an __init__.py file

UPD: If you want to use implicit namespace packages without __init__.py you just have to use find_namespace_packages() instead

Docs

Question 23

I would say that one should omit the __init__.py only if one wants to have the implicit namespace package. If you don’t know what it means, you probably don’t want it and therefore you should continue to use the __init__.py even in Python 3.

Question 24

The program is supposed to take in two names, and if they are the same length it should check if they are the same word. If it’s the same word it will print “The names are the same”. If they are the same length but with different letters it will print “The names are different but the same length”. The part I’m having a problem with is in the bottom 4 lines.

#!/usr/bin/env python
# Enter your code for "What's In (The Length Of) A Name?" here.
name1 = input("Enter name 1: ")
name2 = input("Enter name 2: ")
len(name1)
len(name2)
if len(name1) == len(name2):
    if name1 == name2:
        print ("The names are the same")
    else:
        print ("The names are different, but are the same length")
    if len(name1) > len(name2):
        print ("'{0}' is longer than '{1}'"% name1, name2)
    elif len(name1) < len(name2):
        print ("'{0}'is longer than '{1}'"% name2, name1)

When I run this code it displays:

Traceback (most recent call last):
  File "program.py", line 13, in <module>
    print ("'{0}' is longer than '{1}'"% name1, name2)
TypeError: not all arguments converted during string formatting

Any suggestions are highly appreciated.

Question 25

You’re mixing different format functions.

The old-style % formatting uses % codes for formatting:

'It will cost $%d dollars.' % 95

The new-style {} formatting uses {} codes and the .format method

'It will cost ${0} dollars.'.format(95)

Note that with old-style formatting, you have to specify multiple arguments using a tuple:

'%d days and %d nights' % (40, 40)

In your case, since you’re using {} format specifiers, use .format:

"'{0}' is longer than '{1}'".format(name1, name2)

Question 26

The error is in your string formatting.

The correct way to use traditional string formatting using the ‘%’ operator is to use a printf-style format string (Python documentation for this here: http://docs.python.org/2/library/string.html#format-string-syntax):

"'%s' is longer than '%s'" % (name1, name2)

However, the ‘%’ operator will probably be deprecated in the future. The new PEP 3101 way of doing things is like this:

"'{0}' is longer than '{1}'".format(name1, name2)

Question 27

For me, This error was caused when I was attempting to pass in a tuple into the string format method.

I found the solution from this question/answer

Copying and pasting the correct answer from the link (NOT MY WORK):

>>> thetuple = (1, 2, 3)
>>> print "this is a tuple: %s" % (thetuple,)
this is a tuple: (1, 2, 3)

Making a singleton tuple with the tuple of interest as the only item, i.e. the (thetuple,) part, is the key bit here.

Question 28

In my case, it’s because I need only a single %s, i missing values input.

Question 29

In addition to the other two answers, I think the indentations are also incorrect in the last two conditions. The conditions are that one name is longer than the other and they need to start with ‘elif’ and with no indentations. If you put it within the first condition (by giving it four indentations from the margin), it ends up being contradictory because the lengths of the names cannot be equal and different at the same time.

    else:
        print ("The names are different, but are the same length")
elif len(name1) > len(name2):
    print ("{0} is longer than {1}".format(name1, name2))

Question 30

There is a combination of issues as pointed out in a few of the other answers.

As pointed out by nneonneo you are mixing different String Formatting methods.
As pointed out by GuyP your indentation is off as well.

I’ve provided both the example of .format as well as passing tuples to the argument specifier of %s. In both cases the indentation has been fixed so greater/less than checks are outside of when length matches. Also changed subsequent if statements to elif’s so they only run if the prior same level statement was False.

String Formatting with .format

name1 = input("Enter name 1: ")
name2 = input("Enter name 2: ")
len(name1)
len(name2)
if len(name1) == len(name2):
    if name1 == name2:
        print ("The names are the same")
    else:
        print ("The names are different, but are the same length")
elif len(name1) > len(name2):
    print ("{0} is longer than {1}".format(name1, name2))
elif len(name1) < len(name2):
    print ("{0} is longer than {1}".format(name2, name1))

String Formatting with %s and a tuple

name1 = input("Enter name 1: ")
name2 = input("Enter name 2: ")
len(name1)
len(name2)
if len(name1) == len(name2):
    if name1 == name2:
        print ("The names are the same")
    else:
        print ("The names are different, but are the same length")
elif len(name1) > len(name2):
    print ("%s is longer than %s" % (name1, name2))
elif len(name1) < len(name2):
    print ("%s is longer than %s" % (name2, name1))

Question 31

In python 3.7 and above there is a new and easy way. here is the syntax:

name = "Eric"
age = 74
f"Hello, {name}. You are {age}."

OutPut:

Hello, Eric. You are 74.

Question 32

I encounter the error as well,

_mysql_exceptions.ProgrammingError: not all arguments converted during string formatting

But list args work well.

I use mysqlclient python lib. The lib looks like not to accept tuple args. To pass list args like ['arg1', 'arg2'] will work.

Question 33

django raw sql query in view

"SELECT * FROM VendorReport_vehicledamage WHERE requestdate BETWEEN '{0}' AND '{1}'".format(date_from, date_to)

models.py

class VehicleDamage(models.Model):
    requestdate = models.DateTimeField("requestdate")
    vendor_name = models.CharField("vendor_name", max_length=50)
    class Meta:
        managed=False

views.py

def location_damageReports(request):
    #static date for testing
    date_from = '2019-11-01'
    date_to = '2019-21-01'
    vehicle_damage_reports = VehicleDamage.objects.raw("SELECT * FROM VendorReport_vehicledamage WHERE requestdate BETWEEN '{0}' AND '{1}'".format(date_from, date_to))
    damage_report = DashboardDamageReportSerializer(vehicle_damage_reports, many=True)
    data={"data": damage_report.data}
    return HttpResponse(json.dumps(data), content_type="application/json")

Note: using python 3.5 and django 1.11

Question 34

For me, as I was storing many values within a single print call, the solution was to create a separate variable to store the data as a tuple and then call the print function.

x = (f"{id}", f"{name}", f"{age}")
print(x)

Question 35

Suppose I have a function:

def get_some_date(some_argument: int=None) -> %datetime_or_None%:
    if some_argument is not None and some_argument == 1:
        return datetime.utcnow()
    else:
        return None

How do I specify the return type for something that can be None?

Question 36

You’re looking for Optional.

Since your return type can either be datetime (as returned from datetime.utcnow()) or None you should use Optional[datetime]:

from typing import Optional

def get_some_date(some_argument: int=None) -> Optional[datetime]:
    # as defined

From the documentation on typing, Optional is shorthand for:

Optional[X] is equivalent to Union[X, None].

where Union[X, Y] means a value of type X or Y.

If you want to be explicit due to concerns that others might stumble on Optional and not realize it’s meaning, you could always use Union:

from typing import Union

def get_some_date(some_argument: int=None) -> Union[datetime, None]:

But I doubt this is a good idea, Optional is an indicative name and it does save a couple of keystrokes.

_{As pointed out in the comments by @Michael0x2a Union[T, None] is tranformed to Union[T, type(None)] so no need to use type here.}

Visually these might differ but programatically, in both cases, the result is exactly the same; Union[datetime.datetime, NoneType] will be the type stored in get_some_date.__annotations__^*:

>>> from typing import get_type_hints
>>> print(get_type_hints(get_some_date))
{'return': typing.Union[datetime.datetime, NoneType],
 'some_argument': typing.Union[int, NoneType]}

^*Use typing.get_type_hints to grab the objects’ __annotations__ attribute instead of accessing it directly.

Question 37

I recently came across a syntax I never seen before when I learned python nor in most tutorials, the .. notation, it looks something like this:

f = 1..__truediv__ # or 1..__div__ for python 2

print(f(8)) # prints 0.125

I figured it was exactly the same as (except it’s longer, of course):

f = lambda x: (1).__truediv__(x)
print(f(8)) # prints 0.125 or 1//8

But my questions are:

How can it do that?
What does it actually mean with the two dots?
How can you use it in a more complex statement (if possible)?

This will probably save me many lines of code in the future…:)

Question 38

What you have is a float literal without the trailing zero, which you then access the __truediv__ method of. It’s not an operator in itself; the first dot is part of the float value, and the second is the dot operator to access the objects properties and methods.

You can reach the same point by doing the following.

>>> f = 1.
>>> f
1.0
>>> f.__floordiv__
<method-wrapper '__floordiv__' of float object at 0x7f9fb4dc1a20>

Another example

>>> 1..__add__(2.)
3.0

Here we add 1.0 to 2.0, which obviously yields 3.0.

Question 39

The question is already sufficiently answered (i.e. @Paul Rooneys answer) but it’s also possible to verify the correctness of these answers.

Let me recap the existing answers: The .. is not a single syntax element!

You can check how the source code is “tokenized”. These tokens represent how the code is interpreted:

>>> from tokenize import tokenize
>>> from io import BytesIO

>>> s = "1..__truediv__"
>>> list(tokenize(BytesIO(s.encode('utf-8')).readline))
[...
 TokenInfo(type=2 (NUMBER), string='1.', start=(1, 0), end=(1, 2), line='1..__truediv__'),
 TokenInfo(type=53 (OP), string='.', start=(1, 2), end=(1, 3), line='1..__truediv__'),
 TokenInfo(type=1 (NAME), string='__truediv__', start=(1, 3), end=(1, 14), line='1..__truediv__'),
 ...]

So the string 1. is interpreted as number, the second . is an OP (an operator, in this case the “get attribute” operator) and the __truediv__ is the method name. So this is just accessing the __truediv__ method of the float 1.0.

Another way of viewing the generated bytecode is to disassemble it. This actually shows the instructions that are performed when some code is executed:

>>> import dis

>>> def f():
...     return 1..__truediv__

>>> dis.dis(f)
  4           0 LOAD_CONST               1 (1.0)
              3 LOAD_ATTR                0 (__truediv__)
              6 RETURN_VALUE

Which basically says the same. It loads the attribute __truediv__ of the constant 1.0.

Regarding your question

And how can you use it in a more complex statement (if possible)?

Even though it’s possible you should never write code like that, simply because it’s unclear what the code is doing. So please don’t use it in more complex statements. I would even go so far that you shouldn’t use it in so “simple” statements, at least you should use parenthesis to separate the instructions:

f = (1.).__truediv__

this would be definetly more readable – but something along the lines of:

from functools import partial
from operator import truediv
f = partial(truediv, 1.0)

would be even better!

The approach using partial also preserves python’s data model (the 1..__truediv__ approach does not!) which can be demonstrated by this little snippet:

>>> f1 = 1..__truediv__
>>> f2 = partial(truediv, 1.)

>>> f2(1+2j)  # reciprocal of complex number - works
(0.2-0.4j)
>>> f2('a')   # reciprocal of string should raise an exception
TypeError: unsupported operand type(s) for /: 'float' and 'str'

>>> f1(1+2j)  # reciprocal of complex number - works but gives an unexpected result
NotImplemented
>>> f1('a')   # reciprocal of string should raise an exception but it doesn't
NotImplemented

This is because 1. / (1+2j) is not evaluated by float.__truediv__ but with complex.__rtruediv__ – operator.truediv makes sure the reverse operation is called when the normal operation returns NotImplemented but you don’t have these fallbacks when you operate on __truediv__ directly. This loss of “expected behaviour” is the main reason why you (normally) shouldn’t use magic methods directly.

Question 40

Two dots together may be a little awkward at first:

f = 1..__truediv__ # or 1..__div__ for python 2

But it is the same as writing:

f = 1.0.__truediv__ # or 1.0.__div__ for python 2

Because float literals can be written in three forms:

normal_float = 1.0
short_float = 1.  # == 1.0
prefixed_float = .1  # == 0.1

Question 41

What is f = 1..__truediv__?

f is a bound special method on a float with a value of one. Specifically,

1.0 / x

in Python 3, invokes:

(1.0).__truediv__(x)

Evidence:

class Float(float):
    def __truediv__(self, other):
        print('__truediv__ called')
        return super(Float, self).__truediv__(other)

and:

>>> one = Float(1)
>>> one/2
__truediv__ called
0.5

If we do:

f = one.__truediv__

We retain a name bound to that bound method

>>> f(2)
__truediv__ called
0.5
>>> f(3)
__truediv__ called
0.3333333333333333

If we were doing that dotted lookup in a tight loop, this could save a little time.

Parsing the Abstract Syntax Tree (AST)

We can see that parsing the AST for the expression tells us that we are getting the __truediv__ attribute on the floating point number, 1.0:

>>> import ast
>>> ast.dump(ast.parse('1..__truediv__').body[0])
"Expr(value=Attribute(value=Num(n=1.0), attr='__truediv__', ctx=Load()))"

You could get the same resulting function from:

f = float(1).__truediv__

Or

f = (1.0).__truediv__

Deduction

We can also get there by deduction.

Let’s build it up.

1 by itself is an int:

>>> 1
1
>>> type(1)
<type 'int'>

1 with a period after it is a float:

>>> 1.
1.0
>>> type(1.)
<type 'float'>

The next dot by itself would be a SyntaxError, but it begins a dotted lookup on the instance of the float:

>>> 1..__truediv__
<method-wrapper '__truediv__' of float object at 0x0D1C7BF0>

No one else has mentioned this – This is now a “bound method” on the float, 1.0:

>>> f = 1..__truediv__
>>> f
<method-wrapper '__truediv__' of float object at 0x127F3CD8>
>>> f(2)
0.5
>>> f(3)
0.33333333333333331

We could accomplish the same function much more readably:

>>> def divide_one_by(x):
...     return 1.0/x
...     
>>> divide_one_by(2)
0.5
>>> divide_one_by(3)
0.33333333333333331

Performance

The downside of the divide_one_by function is that it requires another Python stack frame, making it somewhat slower than the bound method:

>>> def f_1():
...     for x in range(1, 11):
...         f(x)
...         
>>> def f_2():
...     for x in range(1, 11):
...         divide_one_by(x)
...         
>>> timeit.repeat(f_1)
[2.5495760687176485, 2.5585621018805469, 2.5411816588331888]
>>> timeit.repeat(f_2)
[3.479687248616699, 3.46196088706062, 3.473726342237768]

Of course, if you can just use plain literals, that’s even faster:

>>> def f_3():
...     for x in range(1, 11):
...         1.0/x
...         
>>> timeit.repeat(f_3)
[2.1224895628296281, 2.1219930218637728, 2.1280188256941983]

Question 42

From pip install --help:

 --user      Install to the Python user install directory for your platform. Typically ~/.local/, or %APPDATA%\Python on
             Windows. (See the Python documentation for site.USER_BASE for full details.)

The documentation for site.USER_BASE is a terrifying wormhole of interesting *NIX subject matter that I don’t understand.

What is the purpose of --user in plain english? Why would intalling the package to ~/.local/ matter? Why not just put an executable somewhere in my $PATH?

Question 43

pip defaults to installing Python packages to a system directory (such as /usr/local/lib/python3.4). This requires root access.

--user makes pip install packages in your home directory instead, which doesn’t require any special privileges.

Question 44

--user installs in site.USER_SITE.

For my case, it was /Users/.../Library/Python/2.7/bin. So I have added that to my PATH (in ~/.bash_profile file):

export PATH=$PATH:/Users/.../Library/Python/2.7/bin

Question 45

Other answers mention site.USER_SITE as where Python packages get placed. If you’re looking for binaries, these go in {site.USER_BASE}/bin.

If you want to add this directory to your shell’s search path, use:

export PATH="${PATH}:$(python3 -c 'import site; print(site.USER_BASE)')/bin"

Question 46

Just a warning:

According to this issue, --user is currently not valid inside a virtual env’s pip, since a user location doesn’t really make sense for a virtual environment.

So do not use pip install --user some_pkg inside a virtual environment, otherwise, virtual environment’s pip will be confused. See this answer for more details.

Question 47

Best way to is install virtualenv and not require the --user confusion. You will get more flexibility and not worry about clobbering the different python versions and projects everytime you pip install a package.

https://virtualenv.pypa.io/en/stable/

Question 48

On macOS, the reason for using the --user flag is to make sure we don’t corrupt the libraries the OS relies on. A conservative approach for many macOS users is to avoid installing or updating pip with a command that requires sudo. Thus, this includes installing to /usr/local/bin…

Ref: Installing python for Neovim (https://github.com/zchee/deoplete-jedi/wiki/Setting-up-Python-for-Neovim)

I’m not all clear why installing into /usr/local/bin is a risk on a Mac given the fact that the system only relies on python binaries in /Library/Frameworks/ and /usr/bin. I suspect it’s because as noted above, installing into /usr/local/bin requires sudo which opens the door to making a costly mistake with the system libraries. Thus, installing into ~/.local/bin is a sure fire way to avoid this risk.

Ref: Using python on a Mac (https://docs.python.org/2/using/mac.html)

Finally, to the degree there is a benefit of installing packages into the /usr/local/bin, I wonder if it makes sense to change the owner of the directory from root to user? This would avoid having to use sudo while still protecting against making system-dependent changes.* Is this a security default a relic of how Unix systems were more often used in the past (as servers)? Or at minimum, just a good way to go for Mac users not hosting a server?

*Note: Mac’s System Integrity Protection (SIP) feature also seems to protect the user from changing the system-dependent libraries.

– E

Question 49

Without Virtual Environments

pip <command> --user changes the scope of the current pip command to work on the current user account’s local python package install location, rather than the system-wide package install location, which is the default.

See User Installs in the PIP User Guide.

This only really matters on a multi-user machine. Anything installed to the system location will be visible to all users, so installing to the user location will keep that package installation separate from other users (they will not see it, and would have to install it themselves separately to use it). Because there can be version conflicts, installing a package with dependencies needed by other packages can cause problems, so it’s best not to push all packages a given user uses to the system install location.

If it is a single-user machine, there is little or no difference to installing to the --user location. It will be installed to a different folder, that may or may not need to be added to the path, depending on the package and how it’s used (many packages install command-line tools that must be on the path to run from a shell).
If it is a multi-user machine, --user is preferred to using root/sudo or requiring administrator installation and affecting the Python environment of every user, except in cases of general packages that the administrator wants to make available to all users by default.
- Note: Per comments, on most Unix/Linux installs it has been pointed out that system installs should use the general package manager, such as apt, rather than pip.

With Virtual Environments

See more about installing packages with virtual environments in the Python Packaging documentation.
Read about how to create and use virtual environments, and the venv command in the Python VENV docs.

The --user option in an active venv/virtualenv environment will install to the local user python location (same as without a virtual environment).

Packages are installed to the virtual environment by default, but if you use --user it will force it to install outside the virtual environments, in the users python script directory (in Windows, this currently is c:\users\<username>\appdata\roaming\python\python37\scripts for me with Python 3.7).

However, you won’t be able to access a system or user install from within virtual environment (even if you used --user while in a virtual environment).

If you install a virtual environment with the --system-site-packages argument, you will have access to the system script folder for python. I believe this included the user python script folder as well, but I’m unsure. However, there may be unintended consequences for this and it is not the intended way to use virtual environments.

Location of the Python System and Local User Install Folders

You can find the location of the user install folder for python with python -m site --user-base. I’m finding conflicting information in Q&A’s, the documentation and actually using this command on my PC as to what the defaults are, but they are underneath the user home directory (~ shortcut in *nix, and c:\users\<username> typically for Windows).

Other Details

The --user option is not a valid for every command. For example pip uninstall will find and uninstall packages wherever they were installed (in the user folder, virtual environment folder, etc.) and the --user option is not valid.

Things installed with pip install --user will be installed in a local location that will only be seen by the current user account, and will not require root access (on *nix) or administrator access (on Windows).

The --user option modifies all pip commands that accept it to see/operate on the user install folder, so if you use pip list --user it will only show you packages installed with pip install --user.

Question 50

Why not just put an executable somewhere in my $PATH

~/.local/bin directory is theoretically expected to be in your $PATH.

According to these people it’s a bug not adding it in the $PATH when using systemd.

This answer explains it more extensively.

But even if your distro includes the ~/.local/bin directory to the $PATH, it might be in the following form (inside ~/.profile):

if [ -d "$HOME/.local/bin" ] ; then
    PATH="$HOME/.local/bin:$PATH"
fi

which would require you to logout and login again, if the directory was not there before.

Question 51

Here is my code:

import imaplib
from email.parser import HeaderParser

conn = imaplib.IMAP4_SSL('imap.gmail.com')
conn.login('example@gmail.com', 'password')
conn.select()
conn.search(None, 'ALL')
data = conn.fetch('1', '(BODY[HEADER])')
header_data = data[1][0][1].decode('utf-8')

at this point I get the error message

AttributeError: 'str' object has no attribute 'decode'

Python 3 doesn’t have decode anymore, am I right? how can I fix this?

Also, in:

data = conn.fetch('1', '(BODY[HEADER])')

I am selecting only the 1st email. How do I select all?

Question 52

You are trying to decode an object that is already decoded. You have a str, there is no need to decode from UTF-8 anymore.

Simply drop the .decode('utf-8') part:

header_data = data[1][0][1]

As for your fetch() call, you are explicitly asking for just the first message. Use a range if you want to retrieve more messages. See the documentation:

The message_set options to commands below is a string specifying one or more messages to be acted upon. It may be a simple message number ('1'), a range of message numbers ('2:4'), or a group of non-contiguous ranges separated by commas ('1:3,6:9'). A range can contain an asterisk to indicate an infinite upper bound ('3:*').

Question 53

Begin with Python 3, all string is unicode object.

  a = 'Happy New Year' # Python 3
  b = unicode('Happy New Year') # Python 2

the code before are same. So I think you should remove the .decode('utf-8'). Because you have already get the unicode object.

Question 54

Use it by this Method:

str.encode().decode()

Question 55

For Python3

html = """\\u003Cdiv id=\\u0022contenedor\\u0022\\u003E \\u003Ch2 class=\\u0022text-left m-b-2\\u0022\\u003EInformaci\\u00f3n del veh\\u00edculo de patente AA345AA\\u003C\\/h2\\u003E\\n\\n\\n\\n \\u003Cdiv class=\\u0022panel panel-default panel-disabled m-b-2\\u0022\\u003E\\n \\u003Cdiv class=\\u0022panel-body\\u0022\\u003E\\n \\u003Ch2 class=\\u0022table_title m-b-2\\u0022\\u003EInformaci\\u00f3n del Registro Automotor\\u003C\\/h2\\u003E\\n \\u003Cdiv class=\\u0022col-md-6\\u0022\\u003E\\n \\u003Clabel class=\\u0022control-label\\u0022\\u003ERegistro Seccional\\u003C\\/label\\u003E\\n \\u003Cp\\u003ESAN MIGUEL N\\u00b0 1\\u003C\\/p\\u003E\\n \\u003Clabel class=\\u0022control-label\\u0022\\u003EDirecci\\u00f3n\\u003C\\/label\\u003E\\n \\u003Cp\\u003EMAESTRO ANGEL D\\u0027ELIA 766\\u003C\\/p\\u003E\\n \\u003Clabel class=\\u0022control-label\\u0022\\u003EPiso\\u003C\\/label\\u003E\\n \\u003Cp\\u003EPB\\u003C\\/p\\u003E\\n \\u003Clabel class=\\u0022control-label\\u0022\\u003EDepartamento\\u003C\\/label\\u003E\\n \\u003Cp\\u003E-\\u003C\\/p\\u003E\\n \\u003Clabel class=\\u0022control-label\\u0022\\u003EC\\u00f3digo postal\\u003C\\/label\\u003E\\n \\u003Cp\\u003E1663\\u003C\\/p\\u003E\\n \\u003C\\/div\\u003E\\n \\u003Cdiv class=\\u0022col-md-6\\u0022\\u003E\\n \\u003Clabel class=\\u0022control-label\\u0022\\u003ELocalidad\\u003C\\/label\\u003E\\n \\u003Cp\\u003ESAN MIGUEL\\u003C\\/p\\u003E\\n \\u003Clabel class=\\u0022control-label\\u0022\\u003EProvincia\\u003C\\/label\\u003E\\n \\u003Cp\\u003EBUENOS AIRES\\u003C\\/p\\u003E\\n \\u003Clabel class=\\u0022control-label\\u0022\\u003ETel\\u00e9fono\\u003C\\/label\\u003E\\n \\u003Cp\\u003E(11)46646647\\u003C\\/p\\u003E\\n \\u003Clabel class=\\u0022control-label\\u0022\\u003EHorario\\u003C\\/label\\u003E\\n \\u003Cp\\u003E08:30 a 12:30\\u003C\\/p\\u003E\\n \\u003C\\/div\\u003E\\n \\u003C\\/div\\u003E\\n\\u003C\\/div\\u003E \\n\\n\\u003Cp class=\\u0022text-center m-t-3 m-b-1 hidden-print\\u0022\\u003E\\n \\u003Ca href=\\u0022javascript:window.print();\\u0022 class=\\u0022btn btn-default\\u0022\\u003EImprim\\u00ed la consulta\\u003C\\/a\\u003E \\u0026nbsp; \\u0026nbsp;\\n \\u003Ca href=\\u0022\\u0022 class=\\u0022btn use-ajax btn-primary\\u0022\\u003EHacer otra consulta\\u003C\\/a\\u003E\\n\\u003C\\/p\\u003E\\n\\u003C\\/div\\u003E"""

print(html.replace("\\/", "/").encode().decode('unicode_escape'))

Question 56

I’m not familiar with the library, but if your problem is that you don’t want a byte array, one easy way is to specify an encoding type straight in a cast:

>>> my_byte_str
b'Hello World'

>>> str(my_byte_str, 'utf-8')
'Hello World'

Question 57

It s already decoded in Python3, Try directly it should work.

Question 58

Other answers sort of hint at it, but the problem may arise from expecting a bytes object. In Python 3, decode is valid when you have an object of class bytes. Running encode before decode may “fix” the problem, but it is a useless pair of operations that suggest the problem us upstream.

Question 59

Although I have never needed this, it just struck me that making an immutable object in Python could be slightly tricky. You can’t just override __setattr__, because then you can’t even set attributes in the __init__. Subclassing a tuple is a trick that works:

class Immutable(tuple):

    def __new__(cls, a, b):
        return tuple.__new__(cls, (a, b))

    @property
    def a(self):
        return self[0]

    @property
    def b(self):
        return self[1]

    def __str__(self):
        return "<Immutable {0}, {1}>".format(self.a, self.b)

    def __setattr__(self, *ignored):
        raise NotImplementedError

    def __delattr__(self, *ignored):
        raise NotImplementedError

But then you have access to the a and b variables through self[0] and self[1], which is annoying.

Is this possible in Pure Python? If not, how would I do it with a C extension?

(Answers that work only in Python 3 are acceptable).

Update:

So subclassing tuple is the way to do it in Pure Python, which works well except for the additional possibility of accessing the data by [0], [1] etc. So, to complete this question all that is missing is howto do it “properly” in C, which I suspect would be quite simple, by just not implementing any geititem or setattribute, etc. But instead of doing it myself, I offer a bounty for that, because I’m lazy. :)

Question 60

Yet another solution I just thought of: The simplest way to get the same behaviour as your original code is

Immutable = collections.namedtuple("Immutable", ["a", "b"])

It does not solve the problem that attributes can be accessed via [0] etc., but at least it’s considerably shorter and provides the additional advantage of being compatible with pickle and copy.

namedtuple creates a type similar to what I described in this answer, i.e. derived from tuple and using __slots__. It is available in Python 2.6 or above.

Question 61

The easiest way to do this is using __slots__:

class A(object):
    __slots__ = []

Instances of A are immutable now, since you can’t set any attributes on them.

If you want the class instances to contain data, you can combine this with deriving from tuple:

from operator import itemgetter
class Point(tuple):
    __slots__ = []
    def __new__(cls, x, y):
        return tuple.__new__(cls, (x, y))
    x = property(itemgetter(0))
    y = property(itemgetter(1))

p = Point(2, 3)
p.x
# 2
p.y
# 3

Edit: If you want to get rid of indexing either, you can override __getitem__():

class Point(tuple):
    __slots__ = []
    def __new__(cls, x, y):
        return tuple.__new__(cls, (x, y))
    @property
    def x(self):
        return tuple.__getitem__(self, 0)
    @property
    def y(self):
        return tuple.__getitem__(self, 1)
    def __getitem__(self, item):
        raise TypeError

Note that you can’t use operator.itemgetter for the properties in thise case, since this would rely on Point.__getitem__() instead of tuple.__getitem__(). Fuerthermore this won’t prevent the use of tuple.__getitem__(p, 0), but I can hardly imagine how this should constitute a problem.

I don’t think the “right” way of creating an immutable object is writing a C extension. Python usually relies on library implementers and library users being consenting adults, and instead of really enforcing an interface, the interface should be clearly stated in the documentation. This is why I don’t consider the possibility of circumventing an overridden __setattr__() by calling object.__setattr__() a problem. If someone does this, it’s on her own risk.

Question 62

..howto do it “properly” in C..

You could use Cython to create an extension type for Python:

cdef class Immutable:
    cdef readonly object a, b
    cdef object __weakref__ # enable weak referencing support

    def __init__(self, a, b):
        self.a, self.b = a, b

It works both Python 2.x and 3.

Tests

# compile on-the-fly
import pyximport; pyximport.install() # $ pip install cython
from immutable import Immutable

o = Immutable(1, 2)
assert o.a == 1, str(o.a)
assert o.b == 2

try: o.a = 3
except AttributeError:
    pass
else:
    assert 0, 'attribute must be readonly'

try: o[1]
except TypeError:
    pass
else:
    assert 0, 'indexing must not be supported'

try: o.c = 1
except AttributeError:
    pass
else:
    assert 0, 'no new attributes are allowed'

o = Immutable('a', [])
assert o.a == 'a'
assert o.b == []

o.b.append(3) # attribute may contain mutable object
assert o.b == [3]

try: o.c
except AttributeError:
    pass
else:
    assert 0, 'no c attribute'

o = Immutable(b=3,a=1)
assert o.a == 1 and o.b == 3

try: del o.b
except AttributeError:
    pass
else:
    assert 0, "can't delete attribute"

d = dict(b=3, a=1)
o = Immutable(**d)
assert o.a == d['a'] and o.b == d['b']

o = Immutable(1,b=3)
assert o.a == 1 and o.b == 3

try: object.__setattr__(o, 'a', 1)
except AttributeError:
    pass
else:
    assert 0, 'attributes are readonly'

try: object.__setattr__(o, 'c', 1)
except AttributeError:
    pass
else:
    assert 0, 'no new attributes'

try: Immutable(1,c=3)
except TypeError:
    pass
else:
    assert 0, 'accept only a,b keywords'

for kwd in [dict(a=1), dict(b=2)]:
    try: Immutable(**kwd)
    except TypeError:
        pass
    else:
        assert 0, 'Immutable requires exactly 2 arguments'

If you don’t mind indexing support then collections.namedtuple suggested by @Sven Marnach is preferrable:

Immutable = collections.namedtuple("Immutable", "a b")

Question 63

Another idea would be to completely disallow __setattr__ and use object.__setattr__ in the constructor:

class Point(object):
    def __init__(self, x, y):
        object.__setattr__(self, "x", x)
        object.__setattr__(self, "y", y)
    def __setattr__(self, *args):
        raise TypeError
    def __delattr__(self, *args):
        raise TypeError

Of course you could use object.__setattr__(p, "x", 3) to modify a Point instance p, but your original implementation suffers from the same problem (try tuple.__setattr__(i, "x", 42) on an Immutable instance).

You can apply the same trick in your original implementation: get rid of __getitem__(), and use tuple.__getitem__() in your property functions.

Question 64

You could create a @immutable decorator that either overrides the __setattr__ and change the __slots__ to an empty list, then decorate the __init__ method with it.

Edit: As the OP noted, changing the __slots__ attribute only prevents the creation of new attributes, not the modification.

Edit2: Here’s an implementation:

Edit3: Using __slots__ breaks this code, because if stops the creation of the object’s __dict__. I’m looking for an alternative.

Edit4: Well, that’s it. It’s a but hackish, but works as an exercise :-)

class immutable(object):
    def __init__(self, immutable_params):
        self.immutable_params = immutable_params

    def __call__(self, new):
        params = self.immutable_params

        def __set_if_unset__(self, name, value):
            if name in self.__dict__:
                raise Exception("Attribute %s has already been set" % name)

            if not name in params:
                raise Exception("Cannot create atribute %s" % name)

            self.__dict__[name] = value;

        def __new__(cls, *args, **kws):
            cls.__setattr__ = __set_if_unset__

            return super(cls.__class__, cls).__new__(cls, *args, **kws)

        return __new__

class Point(object):
    @immutable(['x', 'y'])
    def __new__(): pass

    def __init__(self, x, y):
        self.x = x
        self.y = y

p = Point(1, 2) 
p.x = 3 # Exception: Attribute x has already been set
p.z = 4 # Exception: Cannot create atribute z

问题：如何使用open with语句打开文件

回答 0

回答 1

回答 2

回答 3

问题：如何在Python 3.x和Python 2.x上使用pip

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

问题：Python“提高”用法

回答 0

问题：Python 3.3+中的软件包不需要__init__.py吗

回答 0

回答 1

回答 2

回答 3

问题：TypeError：并非在字符串格式化python期间转换所有参数

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

使用.format进行字符串格式化

使用％s和元组进行字符串格式化

String Formatting with .format

String Formatting with %s and a tuple

回答 6

回答 7

回答 8

回答 9

问题：如何使用类型提示指定“可空”返回类型

回答 0

问题：什么是“ 1 ..__ truediv__”？Python是否具有..（“点点”）表示法语法？

回答 0

回答 1

回答 2

回答 3

什么f = 1..__truediv__啊

解析抽象语法树（AST）

扣除

性能

What is f = 1..__truediv__?

Parsing the Abstract Syntax Tree (AST)

Deduction

Performance

问题：“ pip install –user…”的目的是什么？

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

没有虚拟环境

使用虚拟环境

Python系统和本地用户安装文件夹的位置

其他详情

Without Virtual Environments

With Virtual Environments

Location of the Python System and Local User Install Folders

Other Details

回答 7

问题：’str’对象没有属性’decode’。Python 3错误？

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

问题：如何在Python中制作一个不变的对象？

回答 0

问题：Python 3.3+中的软件包不需要init.py吗

问题：什么是“ 1 .. truediv”？Python是否具有..（“点点”）表示法语法？

什么`f = 1..truediv`啊

What is `f = 1..truediv`?

就像一个 `dict`

Just Like a `dict`