Python 实用宝典

Question 1

Using Matplotlib, I want to plot a 2D heat map. My data is an n-by-n Numpy array, each with a value between 0 and 1. So for the (i, j) element of this array, I want to plot a square at the (i, j) coordinate in my heat map, whose color is proportional to the element’s value in the array.

How can I do this?

Question 2

The imshow() function with parameters interpolation='nearest' and cmap='hot' should do what you want.

import matplotlib.pyplot as plt
import numpy as np

a = np.random.random((16, 16))
plt.imshow(a, cmap='hot', interpolation='nearest')
plt.show()

Question 3

Seaborn takes care of a lot of the manual work and automatically plots a gradient at the side of the chart etc.

import numpy as np
import seaborn as sns
import matplotlib.pylab as plt

uniform_data = np.random.rand(10, 12)
ax = sns.heatmap(uniform_data, linewidth=0.5)
plt.show()

Or, you can even plot upper / lower left / right triangles of square matrices, for example a correlation matrix which is square and is symmetric, so plotting all values would be redundant anyway.

corr = np.corrcoef(np.random.randn(10, 200))
mask = np.zeros_like(corr)
mask[np.triu_indices_from(mask)] = True
with sns.axes_style("white"):
    ax = sns.heatmap(corr, mask=mask, vmax=.3, square=True,  cmap="YlGnBu")
    plt.show()

Question 4

For a 2d numpy array, simply use imshow() may help you:

import matplotlib.pyplot as plt
import numpy as np


def heatmap2d(arr: np.ndarray):
    plt.imshow(arr, cmap='viridis')
    plt.colorbar()
    plt.show()


test_array = np.arange(100 * 100).reshape(100, 100)
heatmap2d(test_array)

This code produces a continuous heatmap.

You can choose another built-in colormap from here.

Question 5

I would use matplotlib’s pcolor/pcolormesh function since it allows nonuniform spacing of the data.

Example taken from matplotlib:

import matplotlib.pyplot as plt
import numpy as np

# generate 2 2d grids for the x & y bounds
y, x = np.meshgrid(np.linspace(-3, 3, 100), np.linspace(-3, 3, 100))

z = (1 - x / 2. + x ** 5 + y ** 3) * np.exp(-x ** 2 - y ** 2)
# x and y are bounds, so z should be the value *inside* those bounds.
# Therefore, remove the last value from the z array.
z = z[:-1, :-1]
z_min, z_max = -np.abs(z).max(), np.abs(z).max()

fig, ax = plt.subplots()

c = ax.pcolormesh(x, y, z, cmap='RdBu', vmin=z_min, vmax=z_max)
ax.set_title('pcolormesh')
# set the limits of the plot to the limits of the data
ax.axis([x.min(), x.max(), y.min(), y.max()])
fig.colorbar(c, ax=ax)

plt.show()

Question 6

Here’s how to do it from a csv:

import numpy as np
import matplotlib.pyplot as plt
from scipy.interpolate import griddata

# Load data from CSV
dat = np.genfromtxt('dat.xyz', delimiter=' ',skip_header=0)
X_dat = dat[:,0]
Y_dat = dat[:,1]
Z_dat = dat[:,2]

# Convert from pandas dataframes to numpy arrays
X, Y, Z, = np.array([]), np.array([]), np.array([])
for i in range(len(X_dat)):
        X = np.append(X, X_dat[i])
        Y = np.append(Y, Y_dat[i])
        Z = np.append(Z, Z_dat[i])

# create x-y points to be used in heatmap
xi = np.linspace(X.min(), X.max(), 1000)
yi = np.linspace(Y.min(), Y.max(), 1000)

# Z is a matrix of x-y values
zi = griddata((X, Y), Z, (xi[None,:], yi[:,None]), method='cubic')

# I control the range of my colorbar by removing data 
# outside of my range of interest
zmin = 3
zmax = 12
zi[(zi<zmin) | (zi>zmax)] = None

# Create the contour plot
CS = plt.contourf(xi, yi, zi, 15, cmap=plt.cm.rainbow,
                  vmax=zmax, vmin=zmin)
plt.colorbar()  
plt.show()

where dat.xyz is in the form

x1 y1 z1
x2 y2 z2
...

Question 7

How to draw a rectangle on an image, like this:

import matplotlib.pyplot as plt
from PIL import Image
import numpy as np
im = np.array(Image.open('dog.png'), dtype=np.uint8)
plt.imshow(im)

I don’t know how to proceed.

Question 8

You can add a Rectangle patch to the matplotlib Axes.

For example (using the image from the tutorial here):

import matplotlib.pyplot as plt
import matplotlib.patches as patches
from PIL import Image
import numpy as np

im = np.array(Image.open('stinkbug.png'), dtype=np.uint8)

# Create figure and axes
fig,ax = plt.subplots(1)

# Display the image
ax.imshow(im)

# Create a Rectangle patch
rect = patches.Rectangle((50,100),40,30,linewidth=1,edgecolor='r',facecolor='none')

# Add the patch to the Axes
ax.add_patch(rect)

plt.show()

Question 9

You need use patches.

import matplotlib.pyplot as plt
import matplotlib.patches as patches

fig2 = plt.figure()
ax2 = fig2.add_subplot(111, aspect='equal')

ax2.add_patch(
     patches.Rectangle(
        (0.1, 0.1),
        0.5,
        0.5,
        fill=False      # remove background
     ) ) 
fig2.savefig('rect2.png', dpi=90, bbox_inches='tight')

Question 10

There is no need for subplots, and pyplot can display PIL images, so this can be simplified further:

import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
from PIL import Image

im = Image.open('stinkbug.png')

# Display the image
plt.imshow(im)

# Get the current reference
ax = plt.gca()

# Create a Rectangle patch
rect = Rectangle((50,100),40,30,linewidth=1,edgecolor='r',facecolor='none')

# Add the patch to the Axes
ax.add_patch(rect)

Or, the short version:

import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
from PIL import Image

# Display the image
plt.imshow(Image.open('stinkbug.png'))

# Add the patch to the Axes
plt.gca().add_patch(Rectangle((50,100),40,30,linewidth=1,edgecolor='r',facecolor='none'))

Question 11

From my understanding matplotlib is a plotting library.

If you want to change the image data (e.g. draw a rectangle on an image), you could use PIL’s ImageDraw, OpenCV, or something similar.

Here is PIL’s ImageDraw method to draw a rectangle.

Here is one of OpenCV’s methods for drawing a rectangle.

Your question asked about Matplotlib, but probably should have just asked about drawing a rectangle on an image.

Here is another question which addresses what I think you wanted to know: Draw a rectangle and a text in it using PIL

Question 12

I’ve been trying to add the Python path to the command line on Windows 7, yet no matter the method I try, nothing seems to work. I’ve used the set command, I’ve tried adding it through the Edit Environment variables prompt etc.

Further more if I run the set command on the command line it lists this

python = c:\python27

Yet it still doesn’t recognize the Python command.

Reading the documentation, and various other sources hasn’t seemed to help.

Edit: Just to clarify further, I’ve appended the path of the Python executable to PATH in edit environment prompt. Doesn’t seem to work.

Question 13

Hold Win and press Pause.
Click Advanced System Settings.
Click Environment Variables.
Append ;C:\python27 to the Path variable.
Restart Command Prompt.

Question 14

When setting Environmental Variables in Windows, I have gone wrong on many, many occasions. I thought I should share a few of my past mistakes here hoping that it might help someone. (These apply to all Environmental Variables, not just when setting Python Path)

Watch out for these possible mistakes:

Kill and reopen your shell window: Once you make a change to the ENVIRONMENTAL Variables, you have to restart the window you are testing it on.
NO SPACES when setting the Variables. Make sure that you are adding the ;C:\Python27 WITHOUT any spaces. (It is common to try C:\SomeOther; C:\Python27 That space (␣) after the semicolon is not okay.)
USE A BACKWARD SLASH when spelling out your full path. You will see forward slashes when you try echo $PATH but only backward slashes have worked for me.
DO NOT ADD a final backslash. Only C:\Python27 NOT C:\Python27\

Hope this helps someone.

Question 15

Open cmd.exe with administrator privileges (right click on app). Then type:

setx path “%path%;C:\Python27;”

Remember to end with a semi-colon and don’t include a trailing slash.

Question 16

I’ve had a problem with this for a LONG time. I added it to my path in every way I could think of but here’s what finally worked for me:

Right click on “My computer”
Click “Properties”
Click “Advanced system settings” in the side panel
Click “Environment Variables”
Click the “New” below system variables
in name enter pythonexe (or anything you want)
in value enter the path to your python (example: C:\Python32\)
Now edit the Path variable (in the system part) and add %pythonexe%; to the end of what’s already there

IDK why this works but it did for me.

then try typing “python” into your command line and it should work!

Edit:

Lately I’ve been using this program which seems to work pretty well. There’s also this one which looks pretty good too, although I’ve never tried it.

Question 17

Try adding this python.bat file to System32 folder and the command line will now run python when you type in python

python.bat

@C:\Python27\python.exe %*

Source:

https://github.com/KartikTalwar/dotfiles/blob/master/bat/python.bat

Question 18

You can set the path from the current cmd window using the PATH = command. That will only add it for the current cmd instance. if you want to add it permanently, you should add it to system variables. (Computer > Advanced System Settings > Environment Variables)

You would goto your cmd instance, and put in PATH=C:/Python27/;%PATH%.

Question 19

Make sure you don’t add a space before the new directory.

Good: old;old;old;new

Bad: old;old;old; new

Question 20

Python comes with a small utility that does just this. From the command line run:

c:\python27\tools\scripts\win_add2path.py

Make sure you close the command window (with exit or the close button) and open it again.

Question 21

The following program will add the python executable path and the subdir Scripts (which is where e.g. pip and easy_install are installed) to your environment. It finds the path to the python executable from the registry key binding the .py extension. It will remove old python paths in your environment. Works with XP (and probably Vista) as well. It only uses modules that come with the basic windows installer.

# coding: utf-8

import sys
import os
import time
import _winreg
import ctypes

def find_python():
    """
    retrieves the commandline for .py extensions from the registry
    """
    hKey = _winreg.OpenKey(_winreg.HKEY_CLASSES_ROOT,
                           r'Python.File\shell\open\command')
    # get the default value
    value, typ = _winreg.QueryValueEx (hKey, None)
    program = value.split('"')[1]
    if not program.lower().endswith(r'\python.exe'):
        return None
    return os.path.dirname(program)

def extend_path(pypath, remove=False, verbose=0, remove_old=True,
                script=False):
    """
    extend(pypath) adds pypath to the PATH env. variable as defined in the
    registry, and then notifies applications (e.g. the desktop) of this change.
    !!! Already opened DOS-Command prompts are not updated. !!!
    Newly opened prompts will have the new path (inherited from the 
    updated windows explorer desktop)
    options:
    remove (default unset), remove from PATH instead of extend PATH
    remove_old (default set), removes any (old) python paths first
    script (default unset), try to add/remove the Scripts subdirectory 
        of pypath (pip, easy_install) as well
    """
    _sd = 'Scripts' # scripts subdir
    hKey = _winreg.OpenKey (_winreg.HKEY_LOCAL_MACHINE,
               r'SYSTEM\CurrentControlSet\Control\Session Manager\Environment',
               0, _winreg.KEY_READ | _winreg.KEY_SET_VALUE)

    value, typ = _winreg.QueryValueEx (hKey, "PATH")
    vals = value.split(';')
    assert isinstance(vals, list)
    if not remove and remove_old:
        new_vals = []
        for v in vals:
            pyexe = os.path.join(v, 'python.exe')
            if v != pypath and os.path.exists(pyexe):
                if verbose > 0:
                    print 'removing from PATH:', v
                continue
            if script and v != os.path.join(pypath, _sd) and \
               os.path.exists(v.replace(_sd, pyexe)):
                if verbose > 0:
                    print 'removing from PATH:', v
                continue
            new_vals.append(v)
        vals = new_vals
    if remove:
        try:
            vals.remove(pypath)
        except ValueError:
            if verbose > 0:
                print 'path element', pypath, 'not found'
            return
        if script:
            try:
                vals.remove(os.path.join(pypath, _sd))
            except ValueError:
                pass
            print 'removing from PATH:', pypath
    else:
        if pypath in vals:
            if verbose > 0:
                print 'path element', pypath, 'already in PATH'
            return
        vals.append(pypath)
        if verbose > 1:
            print 'adding to PATH:', pypath
        if script:
            if not pypath + '\\Scripts' in vals:
                vals.append(pypath + '\\Scripts')
            if verbose > 1:
                print 'adding to PATH:', pypath + '\\Scripts'
    _winreg.SetValueEx(hKey, "PATH", 0, typ, ';'.join(vals) )
    _winreg.SetValueEx(hKey, "OLDPATH", 0, typ, value )
    _winreg.FlushKey(hKey)
    # notify other programs
    SendMessage = ctypes.windll.user32.SendMessageW
    HWND_BROADCAST = 0xFFFF
    WM_SETTINGCHANGE = 0x1A
    SendMessage(HWND_BROADCAST, WM_SETTINGCHANGE, 0, u'Environment')
    if verbose > 1:
        print 'Do not forget to restart any command prompts'

if __name__ == '__main__':
    remove = '--remove' in sys.argv
    script = '--noscripts' not in sys.argv
    extend_path(find_python(), verbose=2, remove=remove, script=script)

Question 22

I know this post is old but I’d like to add that the solutions assume admin privs. If you don’t have those you can:

Go to control panel, type path (this is Windows 7 now so that’s in the Search box) and click “Edit Environment variables for your account”. You’ll now see the Environment Variable dialog with “User variables” on the top and “System variables” below.

You can, as a user, click the top “New” button and add:

Variable name: PATH
Variable value: C:\Python27

(no spaces anywhere) and click OK. Once your command prompt is restarted, any PATH in the User variables is appended to the end of the System Path. It doesn’t replace the PATH in any other way.

If you want a specific full path set up, you’re better off creating a batch file like this little one:

@echo off
PATH C:\User\Me\Programs\mingw\bin;C:\User\Me\Programs;C:\Windows\system32
title Compiler Environment - %Username%@%Computername%
cmd

Call it “compiler.bat” or whatever and double click to start it. Or link to it. Or pin it etc…

Question 23

You need to make changes in your system variable
— Right click on “My computer”
— Click “Properties”
— Click “Advanced system settings” in the side panel
— Click on Environment Variable — You will two sections of user variable and system variable
— Under system variable section search for the variable ‘Path’ click on edit and add
"C:\Python27;" (without quotes) save it
— Now open command line type ‘path’ hit enter you will see path variable has been modified
— Now type python --version you will see the python version

And it is done

Question 24

For anyone trying to achieve this with Python 3.3+, the Windows installer now includes an option to add python.exe to the system search path. Read more in the docs.

Question 25

Working with Windows environment variables is always a horrible experience. Recently, I found an amazing tool called Rapid Environment Editor, which gives an awesomely simple GUI for managing them.

If you use chocolatey, you can install it using choco install rapidee. Otherwise, take a look at http://www.rapidee.com/en/download

Re-reading this, it sounds like a paid shill, but I swear I’m not! It’s just been one of the most useful utilities in my toolkit for a while and I’m surprised no one seems to know about it.

Question 26

If Python was installed with another program, such as ArcGIS 10.1 in my case, then you also must include any extra folders that path to the python.exe in your Environment Variables.

So my Environment Variables looks like this:

System variables > Path > add ;C:\Python27\ArcGIS10.1

Question 27

This question is pretty old, but I just ran into a similar problem and my particular solution wasn’t listed here:

Make sure you don’t have a folder in your PATH that doesn’t exist.

In my case, I had a bunch of default folders (Windows, Powershell, Sql Server, etc) and then a custom C:\bin that I typically use, and then various other tweaks like c:\python17, etc. It turns out that the cmd processor was finding that c:\bin didn’t exist and then stopped processing the rest of the variable.

Also, I don’t know that I ever would have noticed this without PATH manager. It nicely highlighted the fact that that item was invalid.

Question 28

I just installed Python 3.3 on Windows 7 using the option “add python to PATH”.

In PATH variable, the installer automatically added a final backslash: C:\Python33\ and so it did not work on command prompt (i tried closing/opening the prompt several times)

I removed the final backslash and then it worked: C:\Python33

Thanks Ram Narasimhan for your tip #4 !

Question 29

I organized my python environment variable like this under Win7 64-bit using cmd.

I set the variable PYTHONPATH via environment variable menue of windows and added %PYTHONPATH% to the PATH variable:

...;%PYTHONPATH%

The cmd shell expands the variable correctly to this:

C:\>echo %PYTHONPATH%
C:\python27;c:\python27\lib;C:\python27\scripts

Do not forget to restart cmd shell after changing PATH.

Question 30

write that on your Command Prompt:

set Path=%path%

Replace %path% by the Path of your Python Folder Example:

set Path=C:/Python27

Question 31

If you have got frustrated by setting the path for the python just download the new version of python uninstall the older version of the python and while installing the new version it will ask whether to set path mark that and install

its the best way

Question 32

I’ve got a Python project using PyDev in Eclipse, and PyDev keeps generating false errors for my code. I have a module settings that defines a settings object. I import that in module b and assign an attribute with:

from settings import settings
settings.main = object()

In some of my code–but not all of it, statements like:

from settings import settings
print settings.main

… generate “Undefined variable from import: main” messages in the Eclipse code error pane, even though the code runs without a problem. How can I correct these?

Question 33

For code in your project, the only way is adding a declaration saying that you expected that — possibly protected by an if False so that it doesn’t execute (the static code-analysis only sees what you see, not runtime info — if you opened that module yourself, you’d have no indication that main was expected).

To overcome this there are some choices:

If it is some external module, it’s possible to add it to the forced builtins so that PyDev spawns a shell for it to obtain runtime information (see http://pydev.org/manual_101_interpreter.html for details) — i.e.: mostly, PyDev will import the module in a shell and do a dir(module) and dir on the classes found in the module to present completions and make code analysis.
You can use Ctrl+1 (Cmd+1 for Mac) in a line with an error and PyDev will present you an option to add a comment to ignore that error.
It’s possible to create a stub module and add it to the predefined completions (http://pydev.org/manual_101_interpreter.html also has details on that).

Question 34

I’m using opencv which relies on binaries etc so I have scripts where every other line has this silly error. Python is a dynamic language so such occasions shouldn’t be considered errors.

I removed these errors altogether by going to:

Window -> Preferences -> PyDev -> Editor -> Code Analysis -> Undefined -> Undefined Variable From Import -> Ignore

And that’s that.

It may also be, Window -> Preferences -> PyDev -> Editor -> Code Analysis -> Imports -> Import not found -> Ignore

Question 35

The post marked as answer gives a workaround, not a solution.

This solution works for me:

Go to Window - Preferences - PyDev - Interpreters - Python Interpreter
Go to the Forced builtins tab
Click on New...
Type the name of the module (multiprocessing in my case) and click OK

Not only will the error messages disappear, the module members will also be recognized.

Question 36

I was having a similar problem with an Eclipse/PyDev project. In this project the root directory of the python code was a sub-directory of the project.

--> MyProject
 + --> src         Root of python code
   + --> module1     A module 
   + --> module2     Another module
 + --> docs
 + --> test

When the project was debugged or run everything was fine as the working directory was set to the correct place. However the PyDev code analysis was failing to find any imports from module1 or module2.

Solution was to edit the project properties -> PyDev – PYTHONPATH section and remove /MyProject from the source folders tab and add /MyProject/src to it instead.

Question 37

This worked for me:

step 1) Removing the interpreter, auto configuring it again

step 2) Window – Preferences – PyDev – Interpreters – Python Interpreter Go to the Forced builtins tab Click on New… Type the name of the module (curses in my case) and click OK

step 3) Right click in the project explorer on whichever module is giving errors. Go to PyDev->Code analysis.

Question 38

I had the same problem. I am using Python and Eclipse on Windows. The code was running just fine, but eclipse show errors everywhere. After I changed the name of the folder ‘Lib’ to ‘lib’ (C:\Python27\lib), the problem was solved. It seems that if the capitalization of the letters doesn’t match the one in the configuration file, this will sometimes cause problems (but it seems like not always, because the error checking was fine for long time before the problems suddenly appeared for no obvious reason).

Question 39

An approximation of what I was doing:

import module.submodule

class MyClass:
    constant = submodule.constant

To which pylint said: E: 4,15: Undefined variable 'submodule' (undefined-variable)

I resolved this by changing my import like:

from module.submodule import CONSTANT

class MyClass:
    constant = CONSTANT

Note: I also renamed by imported variable to have an uppercase name to reflect its constant nature.

Question 40

It is possible you just need to re-configure your python path within Eclipse. See my answer to a similar question.

Question 41

in preferences –> PyDev –> PyLint under arguments to pass to PyLint add this line:

--generated-members=objects

you will need to do this for each generated . I found this by googling, but I lost the reference.

Question 42

Right click in the project explorer on whichever module is giving errors. Go to PyDev->Remove Error Markers.

Question 43

My answer doesn’t contribute anything new, just a concrete example I encountered.

import gtk.gdk

w = gtk.gdk.get_default_root_window()

PyDev showed the error message “Undefined variable from import: get_default_root_window()”

In the python shell you can see that this is a ‘built-in’ module as mentioned in a answer above:

>>> import gtk.gdk
>>> gtk.gdk
<module 'gtk.gdk' (built-in)>

Now under Window->Preferences->PyDev->Interpreters->Python Interpreter, I selected the tab ‘Forced Builtins’ and added ‘gtk.gdk’ to the list.

Now the error message didn’t show anymore.

Question 44

I find that these 2 steps work for me all the time:

Confirm (else add) the parent folder of the module to the PYTHONPATH.
Add FULL name of the module to forced builtins.

The things to note here:

Some popular modules install with some parent and child pair having the same name. In these cases you also have to add that parent to PYTHONPATH, in addition to its grandparent folder, which you already confirmed/added for everything else.
Use (for example) “google.appengine.api.memcache” when adding to forced builtins, NOT “memcache” only, where “google” in this example, is an immediate child of a folder defined in PYTHONPATH.

Question 45

If you’re sure that your script runs and that it is a false alarm, Go to Preferences > PyDev > Editor > Code Analysis. Demote the errors to warnings.

http://www.pydev.org/manual_adv_code_analysis.html

Question 46

What’s the difference between file and open in Python? When should I use which one? (Say I’m in 2.5)

Question 47

You should always use open().

As the documentation states:

When opening a file, it’s preferable to use open() instead of invoking this constructor directly. file is more suited to type testing (for example, writing “isinstance(f, file)”).

Also, file() has been removed since Python 3.0.

Question 48

Two reasons: The python philosophy of “There ought to be one way to do it” and file is going away.

file is the actual type (using e.g. file('myfile.txt') is calling its constructor). open is a factory function that will return a file object.

In python 3.0 file is going to move from being a built-in to being implemented by multiple classes in the io library (somewhat similar to Java with buffered readers, etc.)

Question 49

file() is a type, like an int or a list. open() is a function for opening files, and will return a file object.

This is an example of when you should use open:

f = open(filename, 'r')
for line in f:
    process(line)
f.close()

This is an example of when you should use file:

class LoggingFile(file):
    def write(self, data):
        sys.stderr.write("Wrote %d bytes\n" % len(data))
        super(LoggingFile, self).write(data)

As you can see, there’s a good reason for both to exist, and a clear use-case for both.

Question 50

Functionally, the two are the same; open will call file anyway, so currently the difference is a matter of style. The Python docs recommend using open.

When opening a file, it’s preferable to use open() instead of invoking the file constructor directly.

The reason is that in future versions they is not guaranteed to be the same (open will become a factory function, which returns objects of different types depending on the path it’s opening).

Question 51

Only ever use open() for opening files. file() is actually being removed in 3.0, and it’s deprecated at the moment. They’ve had a sort of strange relationship, but file() is going now, so there’s no need to worry anymore.

The following is from the Python 2.6 docs. [bracket stuff] added by me.

When opening a file, it’s preferable to use open() instead of invoking this [file()] constructor directly. file is more suited to type testing (for example, writing isinstance(f, file)

Question 52

According to Mr Van Rossum, although open() is currently an alias for file() you should use open() because this might change in the future.

Question 53

My model:

class Sample(models.Model):
    users = models.ManyToManyField(User)

I want to save both user1 and user2 in that model:

user1 = User.objects.get(pk=1)
user2 = User.objects.get(pk=2)
sample_object = Sample(users=user1, users=user2)
sample_object.save()

I know that’s wrong, but I’m sure you get what I want to do. How would you do it ?

Question 54

You cannot create m2m relations from unsaved objects. If you have the pks, try this:

sample_object = Sample()
sample_object.save()
sample_object.users.add(1,2)

Update: After reading the saverio’s answer, I decided to investigate the issue a bit more in depth. Here are my findings.

This was my original suggestion. It works, but isn’t optimal. (Note: I’m using Bars and a Foo instead of Users and a Sample, but you get the idea).

bar1 = Bar.objects.get(pk=1)
bar2 = Bar.objects.get(pk=2)
foo = Foo()
foo.save()
foo.bars.add(bar1)
foo.bars.add(bar2)

It generates a whopping total of 7 queries:

SELECT "app_bar"."id", "app_bar"."name" FROM "app_bar" WHERE "app_bar"."id" = 1
SELECT "app_bar"."id", "app_bar"."name" FROM "app_bar" WHERE "app_bar"."id" = 2
INSERT INTO "app_foo" ("name") VALUES ()
SELECT "app_foo_bars"."bar_id" FROM "app_foo_bars" WHERE ("app_foo_bars"."foo_id" = 1  AND "app_foo_bars"."bar_id" IN (1))
INSERT INTO "app_foo_bars" ("foo_id", "bar_id") VALUES (1, 1)
SELECT "app_foo_bars"."bar_id" FROM "app_foo_bars" WHERE ("app_foo_bars"."foo_id" = 1  AND "app_foo_bars"."bar_id" IN (2))
INSERT INTO "app_foo_bars" ("foo_id", "bar_id") VALUES (1, 2)

I’m sure we can do better. You can pass multiple objects to the add() method:

bar1 = Bar.objects.get(pk=1)
bar2 = Bar.objects.get(pk=2)
foo = Foo()
foo.save()
foo.bars.add(bar1, bar2)

As we can see, passing multiple objects saves one SELECT:

SELECT "app_bar"."id", "app_bar"."name" FROM "app_bar" WHERE "app_bar"."id" = 1
SELECT "app_bar"."id", "app_bar"."name" FROM "app_bar" WHERE "app_bar"."id" = 2
INSERT INTO "app_foo" ("name") VALUES ()
SELECT "app_foo_bars"."bar_id" FROM "app_foo_bars" WHERE ("app_foo_bars"."foo_id" = 1  AND "app_foo_bars"."bar_id" IN (1, 2))
INSERT INTO "app_foo_bars" ("foo_id", "bar_id") VALUES (1, 1)
INSERT INTO "app_foo_bars" ("foo_id", "bar_id") VALUES (1, 2)

I wasn’t aware that you can also assign a list of objects:

bar1 = Bar.objects.get(pk=1)
bar2 = Bar.objects.get(pk=2)
foo = Foo()
foo.save()
foo.bars = [bar1, bar2]

Unfortunately, that creates one additional SELECT:

SELECT "app_bar"."id", "app_bar"."name" FROM "app_bar" WHERE "app_bar"."id" = 1
SELECT "app_bar"."id", "app_bar"."name" FROM "app_bar" WHERE "app_bar"."id" = 2
INSERT INTO "app_foo" ("name") VALUES ()
SELECT "app_foo_bars"."id", "app_foo_bars"."foo_id", "app_foo_bars"."bar_id" FROM "app_foo_bars" WHERE "app_foo_bars"."foo_id" = 1
SELECT "app_foo_bars"."bar_id" FROM "app_foo_bars" WHERE ("app_foo_bars"."foo_id" = 1  AND "app_foo_bars"."bar_id" IN (1, 2))
INSERT INTO "app_foo_bars" ("foo_id", "bar_id") VALUES (1, 1)
INSERT INTO "app_foo_bars" ("foo_id", "bar_id") VALUES (1, 2)

Let’s try to assign a list of pks, as saverio suggested:

foo = Foo()
foo.save()
foo.bars = [1,2]

As we don’t fetch the two Bars, we save two SELECT statements, resulting in a total of 5:

INSERT INTO "app_foo" ("name") VALUES ()
SELECT "app_foo_bars"."id", "app_foo_bars"."foo_id", "app_foo_bars"."bar_id" FROM "app_foo_bars" WHERE "app_foo_bars"."foo_id" = 1
SELECT "app_foo_bars"."bar_id" FROM "app_foo_bars" WHERE ("app_foo_bars"."foo_id" = 1  AND "app_foo_bars"."bar_id" IN (1, 2))
INSERT INTO "app_foo_bars" ("foo_id", "bar_id") VALUES (1, 1)
INSERT INTO "app_foo_bars" ("foo_id", "bar_id") VALUES (1, 2)

And the winner is:

foo = Foo()
foo.save()
foo.bars.add(1,2)

Passing pks to add() gives us a total of 4 queries:

INSERT INTO "app_foo" ("name") VALUES ()
SELECT "app_foo_bars"."bar_id" FROM "app_foo_bars" WHERE ("app_foo_bars"."foo_id" = 1  AND "app_foo_bars"."bar_id" IN (1, 2))
INSERT INTO "app_foo_bars" ("foo_id", "bar_id") VALUES (1, 1)
INSERT INTO "app_foo_bars" ("foo_id", "bar_id") VALUES (1, 2)

Question 55

For future visitors, you can create an object and all of its m2m objects in 2 queries using the new bulk_create in django 1.4. Note that this is only usable if you don’t require any pre or post-processing on the data with save() methods or signals. What you insert is exactly what will be in the DB

You can do this without specifying a “through” model on the field. For completeness, the example below creates a blank Users model to mimic what the original poster was asking.

from django.db import models

class Users(models.Model):
    pass

class Sample(models.Model):
    users = models.ManyToManyField(Users)

Now, in a shell or other code, create 2 users, create a sample object, and bulk add the users to that sample object.

Users().save()
Users().save()

# Access the through model directly
ThroughModel = Sample.users.through

users = Users.objects.filter(pk__in=[1,2])

sample_object = Sample()
sample_object.save()

ThroughModel.objects.bulk_create([
    ThroughModel(users_id=users[0].pk, sample_id=sample_object.pk),
    ThroughModel(users_id=users[1].pk, sample_id=sample_object.pk)
])

Question 56

Django 1.9
A quick example:

sample_object = Sample()
sample_object.save()

list_of_users = DestinationRate.objects.all()
sample_object.users.set(list_of_users)

Question 57

RelatedObjectManagers are different “attributes” than fields in a Model. The simplest way to achieve what you are looking for is

sample_object = Sample.objects.create()
sample_object.users = [1, 2]

That’s the same as assigning a User list, without the additional queries and the model building.

If the number of queries is what bothers you (instead of simplicity), then the optimal solution requires three queries:

sample_object = Sample.objects.create()
sample_id = sample_object.id
sample_object.users.through.objects.create(user_id=1, sample_id=sample_id)
sample_object.users.through.objects.create(user_id=2, sample_id=sample_id)

This will work because we already know that the ‘users’ list is empty, so we can create mindlessly.

Question 58

You could replace the set of related objects in this way (new in Django 1.9):

new_list = [user1, user2, user3]
sample_object.related_set.set(new_list)

Question 59

If someone is looking to do David Marbles answer on a self referring ManyToMany field. The ids of the through model are called: “to_’model_name_id” and “from_’model_name’_id”.

If that doesn’t work you can check the django connection.

Question 60

Despite a ton of Googling, I can’t find any docs for pip’s command line options/arguments. What does pip install -U mean? Does anyone have a link to a list of pip’s options and arguments?

Question 61

Type pip install -h to list help:

-U, –upgrade Upgrade all packages to the newest available version

So, if you already have a package installed, it will upgrade the package for you. Without the -U switch it’ll tell you the package is already installed and exit.

Each pip subcommand has its own help listing. pip -h shows you overall help, and pip [subcommand] -h gives you help for that sub command, such as install.

You can also find the full reference documentation online; the General Options section covers switches available for every pip subcommand, while each subcommand has a separate Options section to cover subcommand-specific switches; see the pip install options section, for example.

Question 62

I’m working in Python and using Flask. When I run my main Python file on my computer, it works perfectly, but when I activate venv and run the Flask Python file in the terminal, it says that my main Python file has “No Module Named bs4.” Any comments or advice is greatly appreciated.

Question 63

Activate the virtualenv, and then install BeautifulSoup4:

$ pip install BeautifulSoup4

When you installed bs4 with easy_install, you installed it system-wide. So your system python can import it, but not your virtualenv python. If you do not need bs4 to be installed in your system python path, uninstall it and keep it in your virtualenv.

For more information about virtualenvs, read this

Question 64

For python2.x:

sudo pip install BeautifulSoup4

For python3:

sudo apt-get install python3-bs4

问题：使用Matplotlib绘制2D热图

回答 0

回答 1

回答 2

回答 3

回答 4

问题：matplotlib：如何在图像上绘制矩形

回答 0

回答 1

回答 2

回答 3

问题：在Windows 7上添加Python路径

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

回答 10

回答 11

回答 12

回答 13

回答 14

回答 15

回答 16

回答 17

回答 18

问题：如何修复PyDev“导入时未定义变量”错误？

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

我通过更改导入来解决此问题，例如：

I resolved this by changing my import like:

回答 7

回答 8

回答 9

回答 10

回答 11

回答 12

问题：Python-何时使用文件vs打开

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

问题：如何为具有多对多字段的Django模型创建对象？

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

问题：pip install -U中的“ -U”选项代表什么

回答 0

问题：ImportError：没有名为bs4的模块（BeautifulSoup）

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

回答 10

回答 11

回答 12

回答 13

回答 14

回答 15