标签归档:Python

如何将单独的Pan​​das DataFrame绘制为子图?

问题:如何将单独的Pan​​das DataFrame绘制为子图?

我有一些Pandas DataFrame共享相同的价值规模,但是具有不同的列和索引。调用时df.plot(),会得到单独的绘图图像。我真正想要的是将它们与子图放置在同一块图上,但是不幸的是,我未能提出解决方案,并且希望获得一些帮助。

I have a few Pandas DataFrames sharing the same value scale, but having different columns and indices. When invoking df.plot(), I get separate plot images. what I really want is to have them all in the same plot as subplots, but I’m unfortunately failing to come up with a solution to how and would highly appreciate some help.


回答 0

您可以使用matplotlib手动创建子图,然后使用ax关键字在特定的子图上绘制数据框。例如,对于4个子图(2×2):

import matplotlib.pyplot as plt

fig, axes = plt.subplots(nrows=2, ncols=2)

df1.plot(ax=axes[0,0])
df2.plot(ax=axes[0,1])
...

axes是一个包含不同子图轴的数组,您只需通过index即可访问一个axes
如果要共享x轴,则可以提供sharex=Trueplt.subplots

You can manually create the subplots with matplotlib, and then plot the dataframes on a specific subplot using the ax keyword. For example for 4 subplots (2×2):

import matplotlib.pyplot as plt

fig, axes = plt.subplots(nrows=2, ncols=2)

df1.plot(ax=axes[0,0])
df2.plot(ax=axes[0,1])
...

Here axes is an array which holds the different subplot axes, and you can access one just by indexing axes.
If you want a shared x-axis, then you can provide sharex=True to plt.subplots.


回答 1

您可以看到例如 在演示joris答案的文档中。另外,从文档,您也可以设置subplots=Truelayout=(,)大熊猫内plot功能:

df.plot(subplots=True, layout=(1,2))

你也可以使用fig.add_subplot()这需要插曲电网参数,如221,222,223,224,等,在后描述这里。可以在此ipython笔记本中看到有关熊猫数据框(包括子图)的漂亮绘图示例。

You can see e.gs. in the documentation demonstrating joris answer. Also from the documentation, you could also set subplots=True and layout=(,) within the pandas plot function:

df.plot(subplots=True, layout=(1,2))

You could also use fig.add_subplot() which takes subplot grid parameters such as 221, 222, 223, 224, etc. as described in the post here. Nice examples of plot on pandas data frame, including subplots, can be seen in this ipython notebook.


回答 2

您可以使用熟悉的Matplotlib样式调用a figuresubplot,但是只需使用即可指定当前轴plt.gca()。一个例子:

plt.figure(1)
plt.subplot(2,2,1)
df.A.plot() #no need to specify for first axis
plt.subplot(2,2,2)
df.B.plot(ax=plt.gca())
plt.subplot(2,2,3)
df.C.plot(ax=plt.gca())

等等…

You can use the familiar Matplotlib style calling a figure and subplot, but you simply need to specify the current axis using plt.gca(). An example:

plt.figure(1)
plt.subplot(2,2,1)
df.A.plot() #no need to specify for first axis
plt.subplot(2,2,2)
df.B.plot(ax=plt.gca())
plt.subplot(2,2,3)
df.C.plot(ax=plt.gca())

etc…


回答 3

您可以使用matplotlib通过绘制所有数据框列表的简单技巧来绘制多个熊猫数据框的多个子图。然后使用for循环绘制子图。

工作代码:

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
# dataframe sample data
df1 = pd.DataFrame(np.random.rand(10,2)*100, columns=['A', 'B'])
df2 = pd.DataFrame(np.random.rand(10,2)*100, columns=['A', 'B'])
df3 = pd.DataFrame(np.random.rand(10,2)*100, columns=['A', 'B'])
df4 = pd.DataFrame(np.random.rand(10,2)*100, columns=['A', 'B'])
df5 = pd.DataFrame(np.random.rand(10,2)*100, columns=['A', 'B'])
df6 = pd.DataFrame(np.random.rand(10,2)*100, columns=['A', 'B'])
#define number of rows and columns for subplots
nrow=3
ncol=2
# make a list of all dataframes 
df_list = [df1 ,df2, df3, df4, df5, df6]
fig, axes = plt.subplots(nrow, ncol)
# plot counter
count=0
for r in range(nrow):
    for c in range(ncol):
        df_list[count].plot(ax=axes[r,c])
        count=+1

使用此代码,您可以在任何配置中绘制子图。您只需要定义行nrow数和列数即可ncol。另外,您需要列出df_list要绘制的数据框。

You can plot multiple subplots of multiple pandas data frames using matplotlib with a simple trick of making a list of all data frame. Then using the for loop for plotting subplots.

Working code:

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
# dataframe sample data
df1 = pd.DataFrame(np.random.rand(10,2)*100, columns=['A', 'B'])
df2 = pd.DataFrame(np.random.rand(10,2)*100, columns=['A', 'B'])
df3 = pd.DataFrame(np.random.rand(10,2)*100, columns=['A', 'B'])
df4 = pd.DataFrame(np.random.rand(10,2)*100, columns=['A', 'B'])
df5 = pd.DataFrame(np.random.rand(10,2)*100, columns=['A', 'B'])
df6 = pd.DataFrame(np.random.rand(10,2)*100, columns=['A', 'B'])
#define number of rows and columns for subplots
nrow=3
ncol=2
# make a list of all dataframes 
df_list = [df1 ,df2, df3, df4, df5, df6]
fig, axes = plt.subplots(nrow, ncol)
# plot counter
count=0
for r in range(nrow):
    for c in range(ncol):
        df_list[count].plot(ax=axes[r,c])
        count=+1

Using this code you can plot subplots in any configuration. You need to just define number of rows nrow and number of columns ncol. Also, you need to make list of data frames df_list which you wanted to plot.


回答 4

您可以使用此:

fig = plt.figure()
ax = fig.add_subplot(221)
plt.plot(x,y)

ax = fig.add_subplot(222)
plt.plot(x,z)
...

plt.show()

You can use this:

fig = plt.figure()
ax = fig.add_subplot(221)
plt.plot(x,y)

ax = fig.add_subplot(222)
plt.plot(x,z)
...

plt.show()

回答 5

您可能根本不需要使用熊猫。这是猫的频率的matplotlib图:

x = np.linspace(0, 2*np.pi, 400)
y = np.sin(x**2)

f, axes = plt.subplots(2, 1)
for c, i in enumerate(axes):
  axes[c].plot(x, y)
  axes[c].set_title('cats')
plt.tight_layout()

You may not need to use Pandas at all. Here’s a matplotlib plot of cat frequencies:

x = np.linspace(0, 2*np.pi, 400)
y = np.sin(x**2)

f, axes = plt.subplots(2, 1)
for c, i in enumerate(axes):
  axes[c].plot(x, y)
  axes[c].set_title('cats')
plt.tight_layout()

回答 6

以上面的@joris响应为基础,如果您已经建立了对该子图的引用,则也可以使用该引用。例如,

ax1 = plt.subplot2grid((50,100), (0, 0), colspan=20, rowspan=10)
...

df.plot.barh(ax=ax1, stacked=True)

Building on @joris response above, if you have already established a reference to the subplot, you can use the reference as well. For example,

ax1 = plt.subplot2grid((50,100), (0, 0), colspan=20, rowspan=10)
...

df.plot.barh(ax=ax1, stacked=True)

回答 7

如何从具有长(整洁)数据的数据帧字典中创建多个图

  • 假设条件

    • 有一个包含多个整齐数据框架的字典
      • 通过读取文件创建
      • 通过将单个数据帧分为多个数据帧来创建
    • 类别cat可能重叠,但是所有数据框可能不包含的所有值cat
    • hue='cat'
  • 因为要遍历数据帧,所以不能保证每个图的颜色都相同

    • 需要根据'cat'所有数据框的唯一值创建自定义颜色图
    • 由于颜色相同,因此在图的侧面放置一个图例,而不要在每个图上都放置图例

导入和综合数据

import pandas as pd
import numpy as np  # used for random data
import random  # used for random data
import matplotlib.pyplot as plt
from matplotlib.patches import Patch  # for custom legend
import seaborn as sns
import math import ceil  # determine correct number of subplot


# synthetic data
df_dict = dict()
for i in range(1, 7):
    np.random.seed(i)
    random.seed(i)
    data_length = 100
    data = {'cat': [random.choice(['A', 'B', 'C']) for _ in range(data_length)],
            'x': np.random.rand(data_length),
            'y': np.random.rand(data_length)}
    df_dict[i] = pd.DataFrame(data)


# display(df_dict[1].head())

  cat         x         y
0   A  0.417022  0.326645
1   C  0.720324  0.527058
2   A  0.000114  0.885942
3   B  0.302333  0.357270
4   A  0.146756  0.908535

创建颜色映射并绘制

# create color mapping based on all unique values of cat
unique_cat = {cat for v in df_dict.values() for cat in v.cat.unique()}  # get unique cats
colors = sns.color_palette('husl', n_colors=len(unique_cat))  # get a number of colors
cmap = dict(zip(unique_cat, colors))  # zip values to colors

# iterate through dictionary and plot
col_nums = 3  # how many plots per row
row_nums = math.ceil(len(df_dict) / col_nums)  # how many rows of plots
plt.figure(figsize=(10, 5))  # change the figure size as needed
for i, (k, v) in enumerate(df_dict.items(), 1):
    plt.subplot(row_nums, col_nums, i)  # create subplots
    p = sns.scatterplot(data=v, x='x', y='y', hue='cat', palette=cmap)
    p.legend_.remove()  # remove the individual plot legends
    plt.title(f'DataFrame: {k}')

plt.tight_layout()
# create legend from cmap
patches = [Patch(color=v, label=k) for k, v in cmap.items()]
# place legend outside of plot; change the right bbox value to move the legend up or down
plt.legend(handles=patches, bbox_to_anchor=(1.06, 1.2), loc='center left', borderaxespad=0)
plt.show()

How to create multiple plots from a dictionary of dataframes with long (tidy) data

  • Assumptions

    • There is a dictionary of multiple dataframes of tidy data
      • Created by reading in from files
      • Created by separating a single dataframe into multiple dataframes
    • The categories, cat, may be overlapping, but all dataframes may not contain all values of cat
    • hue='cat'
  • Because dataframes are being iterated through, there’s not guarantee that colors will be mapped the same for each plot

    • A custom color map needs to be created from the unique 'cat' values for all the dataframes
    • Since the colors will be the same, place one legend to the side of the plots, instead of a legend in every plot

Imports and synthetic data

import pandas as pd
import numpy as np  # used for random data
import random  # used for random data
import matplotlib.pyplot as plt
from matplotlib.patches import Patch  # for custom legend
import seaborn as sns
import math import ceil  # determine correct number of subplot


# synthetic data
df_dict = dict()
for i in range(1, 7):
    np.random.seed(i)
    random.seed(i)
    data_length = 100
    data = {'cat': [random.choice(['A', 'B', 'C']) for _ in range(data_length)],
            'x': np.random.rand(data_length),
            'y': np.random.rand(data_length)}
    df_dict[i] = pd.DataFrame(data)


# display(df_dict[1].head())

  cat         x         y
0   A  0.417022  0.326645
1   C  0.720324  0.527058
2   A  0.000114  0.885942
3   B  0.302333  0.357270
4   A  0.146756  0.908535

Create color mappings and plot

# create color mapping based on all unique values of cat
unique_cat = {cat for v in df_dict.values() for cat in v.cat.unique()}  # get unique cats
colors = sns.color_palette('husl', n_colors=len(unique_cat))  # get a number of colors
cmap = dict(zip(unique_cat, colors))  # zip values to colors

# iterate through dictionary and plot
col_nums = 3  # how many plots per row
row_nums = math.ceil(len(df_dict) / col_nums)  # how many rows of plots
plt.figure(figsize=(10, 5))  # change the figure size as needed
for i, (k, v) in enumerate(df_dict.items(), 1):
    plt.subplot(row_nums, col_nums, i)  # create subplots
    p = sns.scatterplot(data=v, x='x', y='y', hue='cat', palette=cmap)
    p.legend_.remove()  # remove the individual plot legends
    plt.title(f'DataFrame: {k}')

plt.tight_layout()
# create legend from cmap
patches = [Patch(color=v, label=k) for k, v in cmap.items()]
# place legend outside of plot; change the right bbox value to move the legend up or down
plt.legend(handles=patches, bbox_to_anchor=(1.06, 1.2), loc='center left', borderaxespad=0)
plt.show()


如何在CPU上运行Tensorflow

问题:如何在CPU上运行Tensorflow

我已经在Ubuntu 14.04上安装了GPU版本的tensorflow。

我在GPU服务器上,张量流可以访问可用的GPU。

我想在CPU上运行tensorflow。

通常我可以env CUDA_VISIBLE_DEVICES=0在GPU号上运行。0。

我该如何在CPU之间进行选择?

我不喜欢重新编写我的代码 with tf.device("/cpu:0"):

I have installed the GPU version of tensorflow on an Ubuntu 14.04.

I am on a GPU server where tensorflow can access the available GPUs.

I want to run tensorflow on the CPUs.

Normally I can use env CUDA_VISIBLE_DEVICES=0 to run on GPU no. 0.

How can I pick between the CPUs instead?

I am not intersted in rewritting my code with with tf.device("/cpu:0"):


回答 0

您可以device_counttf.Session以下方式应用参数:

config = tf.ConfigProto(
        device_count = {'GPU': 0}
    )
sess = tf.Session(config=config)

另请参阅protobuf配置文件:

tensorflow/core/framework/config.proto

You can apply device_count parameter per tf.Session:

config = tf.ConfigProto(
        device_count = {'GPU': 0}
    )
sess = tf.Session(config=config)

See also protobuf config file:

tensorflow/core/framework/config.proto


回答 1

您还可以将环境变量设置为

CUDA_VISIBLE_DEVICES=""

无需修改源代码。

You can also set the environment variable to

CUDA_VISIBLE_DEVICES=""

without having to modify the source code.


回答 2

如果以上答案不起作用,请尝试:

os.environ['CUDA_VISIBLE_DEVICES'] = '-1'

If the above answers don’t work, try:

os.environ['CUDA_VISIBLE_DEVICES'] = '-1'

回答 3

对我来说,只有CUDA_VISIBLE_DEVICES精确设置才行-1

作品:

import os
import tensorflow as tf

os.environ['CUDA_VISIBLE_DEVICES'] = '-1'

if tf.test.gpu_device_name():
    print('GPU found')
else:
    print("No GPU found")

# No GPU found

难道工作:

import os
import tensorflow as tf

os.environ['CUDA_VISIBLE_DEVICES'] = ''    

if tf.test.gpu_device_name():
    print('GPU found')
else:
    print("No GPU found")

# GPU found

For me, only setting CUDA_VISIBLE_DEVICES to precisely -1 works:

Works:

import os
import tensorflow as tf

os.environ['CUDA_VISIBLE_DEVICES'] = '-1'

if tf.test.gpu_device_name():
    print('GPU found')
else:
    print("No GPU found")

# No GPU found

Does not work:

import os
import tensorflow as tf

os.environ['CUDA_VISIBLE_DEVICES'] = ''    

if tf.test.gpu_device_name():
    print('GPU found')
else:
    print("No GPU found")

# GPU found

回答 4

只需使用下面的代码。

import os
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'

Just using the code below.

import os
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'

回答 5

在某些系统中,必须指定:

import os
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"]=""  # or even "-1"

在导入张量流之前。

In some systems one have to specify:

import os
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"]=""  # or even "-1"

BEFORE importing tensorflow.


回答 6

您可以使用tf.config.set_visible_devices。一种可能的功能,它允许您设置是否使用以及使用哪些GPU:

import tensorflow as tf

def set_gpu(gpu_ids_list):
    gpus = tf.config.list_physical_devices('GPU')
    if gpus:
        try:
            gpus_used = [gpus[i] for i in gpu_ids_list]
            tf.config.set_visible_devices(gpus_used, 'GPU')
            logical_gpus = tf.config.experimental.list_logical_devices('GPU')
            print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPU")
        except RuntimeError as e:
            # Visible devices must be set before GPUs have been initialized
            print(e)

假设你是用4个GPU的系统上,你希望只使用两个GPU,在一个具有id = 0和一个有id = 2,那么你的代码的第一个命令,导入库后,将是:

set_gpu([0, 2])

对于您的情况,仅使用CPU,可以使用一个空列表调用该函数

set_gpu([])

为了完整起见,如果要避免运行时初始化将分配设备上的所有内存,可以使用tf.config.experimental.set_memory_growth。最后,用于管理要使用的设备的功能(动态占用GPU的内存)变为:

import tensorflow as tf

def set_gpu(gpu_ids_list):
    gpus = tf.config.list_physical_devices('GPU')
    if gpus:
        try:
            gpus_used = [gpus[i] for i in gpu_ids_list]
            tf.config.set_visible_devices(gpus_used, 'GPU')
            for gpu in gpus_used:
                tf.config.experimental.set_memory_growth(gpu, True)
            logical_gpus = tf.config.experimental.list_logical_devices('GPU')
            print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPU")
        except RuntimeError as e:
            # Visible devices must be set before GPUs have been initialized
            print(e)

You could use tf.config.set_visible_devices. One possible function that allows you to set if and which GPUs to use is:

import tensorflow as tf

def set_gpu(gpu_ids_list):
    gpus = tf.config.list_physical_devices('GPU')
    if gpus:
        try:
            gpus_used = [gpus[i] for i in gpu_ids_list]
            tf.config.set_visible_devices(gpus_used, 'GPU')
            logical_gpus = tf.config.experimental.list_logical_devices('GPU')
            print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPU")
        except RuntimeError as e:
            # Visible devices must be set before GPUs have been initialized
            print(e)

Suppose you are on a system with 4 GPUs and you want to use only two GPUs, the one with id = 0 and the one with id = 2, then the first command of your code, immediately after importing the libraries, would be:

set_gpu([0, 2])

In your case, to use only the CPU, you can invoke the function with an empty list:

set_gpu([])

For completeness, if you want to avoid that the runtime initialization will allocate all memory on the device, you can use tf.config.experimental.set_memory_growth. Finally, the function to manage which devices to use, occupying the GPUs memory dynamically, becomes:

import tensorflow as tf

def set_gpu(gpu_ids_list):
    gpus = tf.config.list_physical_devices('GPU')
    if gpus:
        try:
            gpus_used = [gpus[i] for i in gpu_ids_list]
            tf.config.set_visible_devices(gpus_used, 'GPU')
            for gpu in gpus_used:
                tf.config.experimental.set_memory_growth(gpu, True)
            logical_gpus = tf.config.experimental.list_logical_devices('GPU')
            print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPU")
        except RuntimeError as e:
            # Visible devices must be set before GPUs have been initialized
            print(e)

回答 7

在安装级别上的另一种可能的解决方案是寻找仅CPU的变体:https : //www.tensorflow.org/install/pip#package-location

就我而言,这给出了:

pip3 install https://storage.googleapis.com/tensorflow/windows/cpu/tensorflow_cpu-2.2.0-cp38-cp38-win_amd64.whl

只需选择正确的版本。如在此答案中说明的使用venv的奖励积分。

Another possible solution on installation level would be to look for the CPU only variant: https://www.tensorflow.org/install/pip#package-location

In my case, this gives right now:

pip3 install https://storage.googleapis.com/tensorflow/windows/cpu/tensorflow_cpu-2.2.0-cp38-cp38-win_amd64.whl

Just select the correct version. Bonus points for using a venv like explained eg in this answer.


Django:如何管理开发和生产设置?

问题:Django:如何管理开发和生产设置?

我一直在开发一个基本的应用程序。现在在部署阶段,很明显,我需要本地设置和生产设置。

很高兴知道以下内容:

  • 如何最好地处理开发和生产设置。
  • 如何仅在开发环境中保留django-debug-toolbar之类的应用程序。
  • 开发和部署设置的其他任何技巧和最佳做法。

I have been developing a basic app. Now at the deployment stage it has become clear I have need for both a local settings and production settings.

It would be great to know the following:

  • How best to deal with development and production settings.
  • How to keep apps such as django-debug-toolbar only in a development environment.
  • Any other tips and best practices for development and deployment settings.

回答 0

DJANGO_SETTINGS_MODULE环境变量,其设置文件Django的控件将加载。

因此,您可以为各自的环境创建单独的配置文件(请注意,它们当然可以import *来自单独的“共享设置”文件),并用于DJANGO_SETTINGS_MODULE控制使用哪个文件。

这是如何做:

如Django文档中所述:

DJANGO_SETTINGS_MODULE的值应采用Python路径语法,例如mysite.settings。请注意,设置模块应位于Python导入搜索路径上。

所以,让我们假设你创建myapp/production_settings.pymyapp/test_settings.py在源存储库。

在这种情况下,您将分别设置DJANGO_SETTINGS_MODULE=myapp.production_settings使用前者和DJANGO_SETTINGS_MODULE=myapp.test_settings后者。


从现在开始,问题归结为设置DJANGO_SETTINGS_MODULE环境变量。

设置DJANGO_SETTINGS_MODULE使用脚本或shell

然后,您可以使用引导脚本或流程管理器来加载正确的设置(通过设置环境),或者仅在启动Django:之前从您的Shell中运行它export DJANGO_SETTINGS_MODULE=myapp.production_settings

请注意,您可以随时从Shell运行此导出-它不需要存在于您的.bashrc容器中。

DJANGO_SETTINGS_MODULE使用流程管理器进行设置

如果您不喜欢编写用于设置环境的引导脚本(并且有很好的理由!),我建议您使用流程管理器:


最后,请注意,您可以利用PYTHONPATH变量将设置存储在完全不同的位置(例如,在生产服务器上,将其存储在中/etc/)。这允许将配置与应用程序文件分开。您可能想要或不想要它,这取决于您的应用程序的结构。

The DJANGO_SETTINGS_MODULE environment variable controls which settings file Django will load.

You therefore create separate configuration files for your respective environments (note that they can of course both import * from a separate, “shared settings” file), and use DJANGO_SETTINGS_MODULE to control which one to use.

Here’s how:

As noted in the Django documentation:

The value of DJANGO_SETTINGS_MODULE should be in Python path syntax, e.g. mysite.settings. Note that the settings module should be on the Python import search path.

So, let’s assume you created myapp/production_settings.py and myapp/test_settings.py in your source repository.

In that case, you’d respectively set DJANGO_SETTINGS_MODULE=myapp.production_settings to use the former and DJANGO_SETTINGS_MODULE=myapp.test_settings to use the latter.


From here on out, the problem boils down to setting the DJANGO_SETTINGS_MODULE environment variable.

Setting DJANGO_SETTINGS_MODULE using a script or a shell

You can then use a bootstrap script or a process manager to load the correct settings (by setting the environment), or just run it from your shell before starting Django: export DJANGO_SETTINGS_MODULE=myapp.production_settings.

Note that you can run this export at any time from a shell — it does not need to live in your .bashrc or anything.

Setting DJANGO_SETTINGS_MODULE using a Process Manager

If you’re not fond of writing a bootstrap script that sets the environment (and there are very good reasons to feel that way!), I would recommend using a process manager:


Finally, note that you can take advantage of the PYTHONPATH variable to store the settings in a completely different location (e.g. on a production server, storing them in /etc/). This allows for separating configuration from application files. You may or may not want that, it depends on how your app is structured.


回答 1

默认情况下,使用生产设置,但settings_dev.py在与文件相同的文件夹中创建一个名为的settings.py文件。在此处添加替代,例如DEBUG=True

在将用于开发的计算机上,将此添加到您的~/.bashrc文件中:

export DJANGO_DEVELOPMENT=true

settings.py文件底部,添加以下内容。

# Override production variables if DJANGO_DEVELOPMENT env variable is set
if os.environ.get('DJANGO_DEVELOPMENT'):
    from settings_dev import *  # or specific overrides

(请注意,*通常应避免在Python 中导入)

默认情况下,生产服务器将不覆盖任何内容。做完了!

与其他答案相比,该答案更简单,因为它不需要更新PYTHONPATH或设置DJANGO_SETTINGS_MODULE,而每次只允许您处理一个django项目。

By default use production settings, but create a file called settings_dev.py in the same folder as your settings.py file. Add overrides there, such as DEBUG=True.

On the computer that will be used for development, add this to your ~/.bashrc file:

export DJANGO_DEVELOPMENT=true

At the bottom of your settings.py file, add the following.

# Override production variables if DJANGO_DEVELOPMENT env variable is set
if os.environ.get('DJANGO_DEVELOPMENT'):
    from settings_dev import *  # or specific overrides

(Note that importing * should generally be avoided in Python)

By default the production servers will not override anything. Done!

Compared to the other answers, this one is simpler because it doesn’t require updating PYTHONPATH, or setting DJANGO_SETTINGS_MODULE which only allows you to work on one django project at a time.


回答 2

通常每个环境有一个设置文件,还有一个共享设置文件:

/myproject/
  settings.production.py
  settings.development.py
  shared_settings.py

我的每个环境文件都有:

try:
    from shared_settings import *
except ImportError:
    pass

这使我可以在必要时覆盖共享设置(通过在该节下面添加修改)。

然后,通过将其链接到settings.py来选择要使用的设置文件:

ln -s settings.development.py settings.py

I usually have one settings file per environment, and a shared settings file:

/myproject/
  settings.production.py
  settings.development.py
  shared_settings.py

Each of my environment files has:

try:
    from shared_settings import *
except ImportError:
    pass

This allows me to override shared settings if necessary (by adding the modifications below that stanza).

I then select which settings files to use by linking it in to settings.py:

ln -s settings.development.py settings.py

回答 3

这是我通过6个简单步骤完成的操作:

  1. 在项目目录中创建一个文件夹,并将其命名settings

    项目结构:

    myproject/
           myapp1/
           myapp2/              
           myproject/
                  settings/
  2. 创建内部4个Python文件settings目录,即__init__.pybase.pydev.pyprod.py

    设置文件:

    settings/
         __init__.py
         base.py
         prod.py
         dev.py 
  3. 打开__init__.py并填写以下内容:

    初始化 .py:

    from .base import *
    # you need to set "myproject = 'prod'" as an environment variable
    # in your OS (on which your website is hosted)
    if os.environ['myproject'] == 'prod':
       from .prod import *
    else:
       from .dev import *
  4. 打开base.py并填写所有常用设置(将在生产和开发中使用)。例如:

    base.py:

    import os
    ...
    INSTALLED_APPS = [...]
    MIDDLEWARE = [...]
    TEMPLATES = [{...}]
    ...
    STATIC_URL = '/static/'
    STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
    MEDIA_ROOT = os.path.join(BASE_DIR, '/path/')
    MEDIA_URL = '/path/'
  5. 打开dev.py并包含特定于开发的内容,例如:

    dev.py:

    DEBUG = True
    ALLOWED_HOSTS = ['localhost']
    ...
  6. 打开prod.py并包含特定于生产的内容,例如:

    prod.py:

    DEBUG = False
    ALLOWED_HOSTS = ['www.example.com']
    LOGGING = [...]
    ...

This is how I do it in 6 easy steps:

  1. Create a folder inside your project directory and name it settings.

    Project structure:

    myproject/
           myapp1/
           myapp2/              
           myproject/
                  settings/
    
  2. Create four python files inside of the settings directory namely __init__.py, base.py, dev.py and prod.py

    Settings files:

    settings/
         __init__.py
         base.py
         prod.py
         dev.py 
    
  3. Open __init__.py and fill it with the following content:

    init.py:

    from .base import *
    # you need to set "myproject = 'prod'" as an environment variable
    # in your OS (on which your website is hosted)
    if os.environ['myproject'] == 'prod':
       from .prod import *
    else:
       from .dev import *
    
  4. Open base.py and fill it with all the common settings (that will be used in both production as well as development.) for example:

    base.py:

    import os
    ...
    INSTALLED_APPS = [...]
    MIDDLEWARE = [...]
    TEMPLATES = [{...}]
    ...
    STATIC_URL = '/static/'
    STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
    MEDIA_ROOT = os.path.join(BASE_DIR, '/path/')
    MEDIA_URL = '/path/'
    
  5. Open dev.py and include that stuff which is development specific for example:

    dev.py:

    DEBUG = True
    ALLOWED_HOSTS = ['localhost']
    ...
    
  6. Open prod.py and include that stuff which is production specific for example:

    prod.py:

    DEBUG = False
    ALLOWED_HOSTS = ['www.example.com']
    LOGGING = [...]
    ...
    

回答 4

创建多个settings*.py文件,推断出每个环境需要更改的变量。然后在主settings.py文件的末尾:

try:
  from settings_dev import *
except ImportError:
  pass

settings_*为每个阶段保留单独的文件。

settings_dev.py文件顶部,添加以下内容:

import sys
globals().update(vars(sys.modules['settings']))

导入需要修改的变量。

Wiki条目提供有关如何拆分设置的更多想法。

Create multiple settings*.py files, extrapolating the variables that need to change per environment. Then at the end of your master settings.py file:

try:
  from settings_dev import *
except ImportError:
  pass

You keep the separate settings_* files for each stage.

At the top of your settings_dev.py file, add this:

import sys
globals().update(vars(sys.modules['settings']))

To import variables that you need to modify.

This wiki entry has more ideas on how to split your settings.


回答 5

我使用了很棒的django-configuration,所有设置都存储在我的计算机中settings.py

from configurations import Configuration

class Base(Configuration):
    # all the base settings here...
    BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
    ...

class Develop(Base):
    # development settings here...
    DEBUG = True 
    ...

class Production(Base):
    # production settings here...
    DEBUG = False

要配置Django项目,我只需遵循docs即可

I use the awesome django-configurations, and all the settings are stored in my settings.py:

from configurations import Configuration

class Base(Configuration):
    # all the base settings here...
    BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
    ...

class Develop(Base):
    # development settings here...
    DEBUG = True 
    ...

class Production(Base):
    # production settings here...
    DEBUG = False

To configure the Django project I just followed the docs.


回答 6

这是我们使用的方法:

  • 一个settings将设置分成多个文件以增强可读性的模块;
  • 一个.env.json文件,用于存储我们要从git存储库中排除或特定于环境的凭据和参数;
  • 一个env.py文件来读取.env.json文件

考虑以下结构:

...
.env.json           # the file containing all specific credentials and parameters
.gitignore          # the .gitignore file to exclude `.env.json`
project_name/       # project dir (the one which django-admin.py creates)
  accounts/         # project's apps
    __init__.py
    ...
  ...
  env.py            # the file to load credentials
  settings/
    __init__.py     # main settings file
    database.py     # database conf
    storage.py      # storage conf
    ...
venv                # virtualenv
...

.env.json喜欢:

{
    "debug": false,
    "allowed_hosts": ["mydomain.com"],
    "django_secret_key": "my_very_long_secret_key",
    "db_password": "my_db_password",
    "db_name": "my_db_name",
    "db_user": "my_db_user",
    "db_host": "my_db_host",
}

project_name/env.py

<!-- language: lang-python -->
import json
import os


def get_credentials():
    env_file_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
    with open(os.path.join(env_file_dir, '.env.json'), 'r') as f:
        creds = json.loads(f.read())
    return creds


credentials = get_credentials()

我们可以进行以下设置:

<!-- language: lang-py -->
# project_name/settings/__init__.py
from project_name.env import credentials
from project_name.settings.database import *
from project_name.settings.storage import *
...

SECRET_KEY = credentials.get('django_secret_key')

DEBUG = credentials.get('debug')

ALLOWED_HOSTS = credentials.get('allowed_hosts', [])

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',

    ...
]

if DEBUG:
    INSTALLED_APPS += ['debug_toolbar']

...

# project_name/settings/database.py
from project_name.env import credentials

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': credentials.get('db_name', ''),
        'USER': credentials.get('db_user', ''),
        'HOST': credentials.get('db_host', ''),
        'PASSWORD': credentials.get('db_password', ''),
        'PORT': '5432',
    }
}

该解决方案的好处是:

  • 用于本地开发的用户专用凭证和配置,而无需修改git存储库;
  • 针对特定环境的配置,例如,您可以具有三个不同的环境,其中具有三个不同的环境,.env.json例如dev,stagging和production;
  • 凭证不在存储库中

希望对您有所帮助,请告诉我您是否对此解决方案有所了解。

Here is the approach we use :

  • a settings module to split settings into multiple files for readability ;
  • a .env.json file to store credentials and parameters that we want excluded from our git repository, or that are environment specific ;
  • an env.py file to read the .env.json file

Considering the following structure :

...
.env.json           # the file containing all specific credentials and parameters
.gitignore          # the .gitignore file to exclude `.env.json`
project_name/       # project dir (the one which django-admin.py creates)
  accounts/         # project's apps
    __init__.py
    ...
  ...
  env.py            # the file to load credentials
  settings/
    __init__.py     # main settings file
    database.py     # database conf
    storage.py      # storage conf
    ...
venv                # virtualenv
...

With .env.json like :

{
    "debug": false,
    "allowed_hosts": ["mydomain.com"],
    "django_secret_key": "my_very_long_secret_key",
    "db_password": "my_db_password",
    "db_name": "my_db_name",
    "db_user": "my_db_user",
    "db_host": "my_db_host",
}

And project_name/env.py :

<!-- language: lang-python -->
import json
import os


def get_credentials():
    env_file_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
    with open(os.path.join(env_file_dir, '.env.json'), 'r') as f:
        creds = json.loads(f.read())
    return creds


credentials = get_credentials()

We can have the following settings:

<!-- language: lang-py -->
# project_name/settings/__init__.py
from project_name.env import credentials
from project_name.settings.database import *
from project_name.settings.storage import *
...

SECRET_KEY = credentials.get('django_secret_key')

DEBUG = credentials.get('debug')

ALLOWED_HOSTS = credentials.get('allowed_hosts', [])

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',

    ...
]

if DEBUG:
    INSTALLED_APPS += ['debug_toolbar']

...

# project_name/settings/database.py
from project_name.env import credentials

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': credentials.get('db_name', ''),
        'USER': credentials.get('db_user', ''),
        'HOST': credentials.get('db_host', ''),
        'PASSWORD': credentials.get('db_password', ''),
        'PORT': '5432',
    }
}

the benefits of this solution are :

  • user specific credentials and configurations for local development without modifying the git repository ;
  • environment specific configuration, you can have for example three different environments with three different .env.json like dev, stagging and production ;
  • credentials are not in the repository

I hope this helps, just let me know if you see any caveats with this solution.


回答 7

我使用以下文件结构:

project/
   ...
   settings/
   settings/common.py
   settings/local.py
   settings/prod.py
   settings/__init__.py -> local.py

所以,__init__.py是(在Windows LN在UNIX或mklink)的链接,local.py或者可以是prod.py这样的配置仍然是在project.settings模块清洁和有组织的,如果你想使用一个特定的配置,你可以使用环境变量DJANGO_SETTINGS_MODULEproject.settings.prod,如果你需要在生产环境中运行命令。

在文件prod.pylocal.py

from .shared import *

DATABASE = {
    ...
}

并且该shared.py文件在没有特定配置的情况下保持全局状态。

I use the folloring file structure:

project/
   ...
   settings/
   settings/common.py
   settings/local.py
   settings/prod.py
   settings/__init__.py -> local.py

So __init__.py is a link (ln in unix or mklink in windows) to local.py or can be to prod.py so the configuration is still in the project.settings module is clean and organized, and if you want to use a particular config you can use the environment variable DJANGO_SETTINGS_MODULE to project.settings.prod if you need to run a command for production environment.

In the files prod.py and local.py:

from .shared import *

DATABASE = {
    ...
}

and the shared.py file keeps as global without specific configs.


回答 8

建立CS01的答案:

如果您在使用环境变量时遇到问题,请将其值设置为字符串(例如,我做了DJANGO_DEVELOPMENT="true")。

我还更改了cs01的文件工作流程,如下所示:

#settings.py
import os
if os.environ.get('DJANGO_DEVELOPMENT') is not None:
    from settings_dev import * 
else:
    from settings_production import *
#settings_dev.py
development settings go here
#settings_production.py
production settings go here

这样,Django无需在运行适当的设置文件之前通读整个设置文件。如果您的生产文件仅需要生产服务器上的内容,则此解决方案非常有用。

注意:在Python 3中,导入的文件需要.附加一个附件(例如from .settings_dev import *

building off cs01’s answer:

if you’re having problems with the environment variable, set its value to a string (e.g. I did DJANGO_DEVELOPMENT="true").

I also changed cs01’s file workflow as follows:

#settings.py
import os
if os.environ.get('DJANGO_DEVELOPMENT') is not None:
    from settings_dev import * 
else:
    from settings_production import *
#settings_dev.py
development settings go here
#settings_production.py
production settings go here

This way, Django doesn’t have to read through the entirety of a settings file before running the appropriate settings file. This solution comes in handy if your production file needs stuff that’s only on your production server.

Note: in Python 3, imported files need to have a . appended (e.g. from .settings_dev import *)


回答 9

如果要保留1个设置文件,并且您的开发操作系统与生产操作系统不同,则可以将其放在settings.py的底部:

from sys import platform
if platform == "linux" or platform == "linux2":
    # linux
    # some special setting here for when I'm on my prod server
elif platform == "darwin":
    # OS X
    # some special setting here for when I'm developing on my mac
elif platform == "win32":
    # Windows...
    # some special setting here for when I'm developing on my pc

阅读更多:如何检查Python中的操作系统?

If you want to keep 1 settings file, and your development operating system is different than your production operating system, you can put this at the bottom of your settings.py:

from sys import platform
if platform == "linux" or platform == "linux2":
    # linux
    # some special setting here for when I'm on my prod server
elif platform == "darwin":
    # OS X
    # some special setting here for when I'm developing on my mac
elif platform == "win32":
    # Windows...
    # some special setting here for when I'm developing on my pc

Read more: How do I check the operating system in Python?


回答 10

这似乎已经得到了回答,但是我将其与版本控制结合使用的方法如下:

在与我也添加到.gitignore的本地开发环境上的设置相同的目录中设置一个env.py文件:

env.py:

#!usr/bin/python

DJANGO_ENV = True
ALLOWED_HOSTS = ['127.0.0.1', 'dev.mywebsite.com']

.gitignore:

mywebsite/env.py

settings.py:

if os.path.exists(os.getcwd() + '/env.py'):
    #env.py is excluded using the .gitignore file - when moving to production we can automatically set debug mode to off:
    from env import *
else:
    DJANGO_ENV = False

DEBUG = DJANGO_ENV

我只是发现它有效并且更加优雅-使用env.py可以很容易地看到我们的本地环境变量,并且我们可以在没有多个settings.py文件或类似文件的情况下处理所有这些问题。这种方法允许使用各种我们不想在生产服务器上设置的局部环境变量。通过版本控制使用.gitignore,我们还保持所有内容无缝集成。

This seems to have been answered, however a method which I use as combined with version control is the following:

Setup a env.py file in the same directory as settings on my local development environment that I also add to .gitignore:

env.py:

#!usr/bin/python

DJANGO_ENV = True
ALLOWED_HOSTS = ['127.0.0.1', 'dev.mywebsite.com']

.gitignore:

mywebsite/env.py

settings.py:

if os.path.exists(os.getcwd() + '/env.py'):
    #env.py is excluded using the .gitignore file - when moving to production we can automatically set debug mode to off:
    from env import *
else:
    DJANGO_ENV = False

DEBUG = DJANGO_ENV

I just find this works and is far more elegant – with env.py it is easy to see our local environment variables and we can handle all of this without multiple settings.py files or the likes. This methods allows for all sorts of local environment variables to be used that we wouldn’t want set on our production server. Utilising the .gitignore via version control we are also keeping everything seamlessly integrated.


回答 11

使用settings.py进行生产。在同一目录中创建settings_dev.py替代。

# settings_dev.py

from .settings import * 

DEBUG = False

在开发机器上,使用以下命令运行Django应用:

DJANGO_SETTINGS_MODULE=<your_app_name>.settings_dev python3 manage.py runserver

在生产机器上运行,就好像您只是拥有settings.py而已。

优点

  1. settings.py (用于生产)完全不知道存在任何其他环境。
  2. 要查看prod和dev之间的区别,您只需查看一个位置- settings_dev.py。无需配置收集散落settings_prod.pysettings_dev.pysettings_shared.py
  3. 如果在解决生产问题后有人将设置添加到您的产品配置中,您可以放心,该设置也会出现在您的开发配置中(除非被显式覆盖)。因此,不同配置文件之间的差异将被最小化。

Use settings.py for production. In the same directory create settings_dev.py for overrides.

# settings_dev.py

from .settings import * 

DEBUG = False

On a dev machine run your Django app with:

DJANGO_SETTINGS_MODULE=<your_app_name>.settings_dev python3 manage.py runserver

On a prod machine run as if you just had settings.py and nothing else.

ADVANTAGES

  1. settings.py (used for production) is completely agnostic to the fact that any other environments even exist.
  2. To see the difference between prod and dev you just look into a single location – settings_dev.py. No need to gather configurations scattered across settings_prod.py, settings_dev.py and settings_shared.py.
  3. If someone adds a setting to your prod config after troubleshooting a production issue you can rest assured that it will appear in your dev config as well (unless explicitly overridden). Thus the divergence between different config files will be minimized.

回答 12

对于设置文件的问题,我选择复制

Project
   |---__init__.py   [ write code to copy setting file from subdir to current dir]
   |---settings.py  (do not commit this file to git)
   |---setting1_dir
   |         |--  settings.py
   |---setting2_dir
   |         |--  settings.py

当您运行django时,将运行__init__py。这时,settings.py in setting1_dir将更换settings.py in Project

如何选择不同的环境?

  • __init__.py直接修改。
  • 制作一个bash文件进行修改__init__.py
  • 在linux中修改env,然后__init__.py读取此变量。

为什么要用这种方式?

因为我不喜欢同一目录中的太多文件,所以太多的文件会使其他合作伙伴感到困惑,并且对于IDE来说不是很好(IDE无法找到我们使用的文件)

如果您不想看到所有这些详细信息,可以将项目分为两部分。

  1. 制作像Spring Initializr这样的小工具,仅用于设置您的项目。(像复制文件一样)
  2. 您的项目代码

For the problem of setting files, I choose to copy

Project
   |---__init__.py   [ write code to copy setting file from subdir to current dir]
   |---settings.py  (do not commit this file to git)
   |---setting1_dir
   |         |--  settings.py
   |---setting2_dir
   |         |--  settings.py

When you run django, __init__py will be ran. At this time , settings.py in setting1_dir will replace settings.py in Project.

How to choose different env?

  • modify __init__.py directly.
  • make a bash file to modify __init__.py.
  • modify env in linux, and then let __init__.py read this variable.

Why use to this way?

Because I don’t like so many files in the same directory, too many files will confuse other partners and not very well for IDE.(IDE cannot find what file we use)

If you do not want to see all these details, you can divide project into two part.

  1. make your small tool like Spring Initializr, just for setup your project.(do sth like copy file)
  2. your project code

回答 13

我正在使用不同的app.yaml文件来更改Google Cloud App Engine中环境之间的配置。

您可以使用此命令在终端命令中创建代理连接:

./cloud_sql_proxy -instances=<INSTANCE_CONNECTION_NAME>=tcp:1433

https://cloud.google.com/sql/docs/sqlserver/connect-admin-proxy#macos-64-bit

档案:app.yaml

# [START django_app]
service: development
runtime: python37

env_variables:
  DJANGO_DB_HOST: '/cloudsql/myproject:myregion:myinstance'
  DJANGO_DEBUG: True

handlers:
# This configures Google App Engine to serve the files in the app's static
# directory.
- url: /static
  static_dir: static/

# This handler routes all requests not caught above to your main app. It is
# required when static routes are defined, but can be omitted (along with
# the entire handlers section) when there are no static files defined.
- url: /.*
  script: auto
# [END django_app]

I’m using different app.yaml file to change configuration between environments in google cloud app engine.

You can use this to create a proxy connection in your terminal command:

./cloud_sql_proxy -instances=<INSTANCE_CONNECTION_NAME>=tcp:1433

https://cloud.google.com/sql/docs/sqlserver/connect-admin-proxy#macos-64-bit

File: app.yaml

# [START django_app]
service: development
runtime: python37

env_variables:
  DJANGO_DB_HOST: '/cloudsql/myproject:myregion:myinstance'
  DJANGO_DEBUG: True

handlers:
# This configures Google App Engine to serve the files in the app's static
# directory.
- url: /static
  static_dir: static/

# This handler routes all requests not caught above to your main app. It is
# required when static routes are defined, but can be omitted (along with
# the entire handlers section) when there are no static files defined.
- url: /.*
  script: auto
# [END django_app]

回答 14

这是我的解决方案,具有不同的开发,测试和生产环境

import socket

[...]

DEV_PC = 'PC059'
host_name = socket.gethostname()

if host_name == DEV_PC:
   #do something
   pass
elif [...]

This is my solution, with different environements for dev, test and prod

import socket

[...]

DEV_PC = 'PC059'
host_name = socket.gethostname()

if host_name == DEV_PC:
   #do something
   pass
elif [...]

清单理解条件中的`elif`

问题:清单理解条件中的`elif`

我们可以elif在列表理解中使用吗?

范例:

l = [1, 2, 3, 4, 5]

for values in l:
    if values==1:
        print 'yes'
    elif values==2:
        print 'no'
    else:
        print 'idle'

我们可以elif采用与上面的代码类似的方式将列表理解包括在内吗?

例如,答案如下:

['yes', 'no', 'idle', 'idle', 'idle']

到现在为止,我仅使用ifelse理解列表。

Can we use elif in list comprehension?

Example :

l = [1, 2, 3, 4, 5]

for values in l:
    if values==1:
        print 'yes'
    elif values==2:
        print 'no'
    else:
        print 'idle'

Can we include the elif in our list comprehension, in a similar fashion to the code above?

For example, an answer like:

['yes', 'no', 'idle', 'idle', 'idle']

Up until now, I have only used if and else in list comprehension.


回答 0

Python的条件表达式正是针对这种用例而设计的:

>>> l = [1, 2, 3, 4, 5]
>>> ['yes' if v == 1 else 'no' if v == 2 else 'idle' for v in l]
['yes', 'no', 'idle', 'idle', 'idle']

希望这可以帮助 :-)

Python’s conditional expressions were designed exactly for this sort of use-case:

>>> l = [1, 2, 3, 4, 5]
>>> ['yes' if v == 1 else 'no' if v == 2 else 'idle' for v in l]
['yes', 'no', 'idle', 'idle', 'idle']

回答 1

>>> d = {1: 'yes', 2: 'no'}
>>> [d.get(x, 'idle') for x in l]
['yes', 'no', 'idle', 'idle', 'idle']
>>> d = {1: 'yes', 2: 'no'}
>>> [d.get(x, 'idle') for x in l]
['yes', 'no', 'idle', 'idle', 'idle']

回答 2

可以的。

请注意,当您使用以下语法时:

['yes' if v == 1 else 'no' for v in l]

您正在使用if / else运算符的三进制形式(如果您熟悉C之类的语言,则类似于?:Construct:)(v == 1 ? 'yes' : 'no')

if / else运算符的三进制形式没有内置的“ elif”,但是您可以在“ else”条件下对其进行仿真:

['yes' if v == 1 else 'no' if v == 2 else 'idle' for v in l]

这就像在说:

for v in l:
    if v == 1 :
        print 'yes'
    else:
        if v == 2:
            print 'no'
        else:
            print 'idle'

因此,没有像您问的那样直接的“ elif”构造,但是可以使用嵌套的if / else语句来模拟它。

You can, sort of.

Note that when you use sytax like:

['yes' if v == 1 else 'no' for v in l]

You are using the ternary form of the if/else operator (if you’re familiar with languages like C, this is like the ?: construct: (v == 1 ? 'yes' : 'no')).

The ternary form of the if/else operator doesn’t have an ‘elif’ built in, but you can simulate it in the ‘else’ condition:

['yes' if v == 1 else 'no' if v == 2 else 'idle' for v in l]

This is like saying:

for v in l:
    if v == 1 :
        print 'yes'
    else:
        if v == 2:
            print 'no'
        else:
            print 'idle'

So there’s no direct ‘elif’ construct like you asked about, but it can be simulated with nested if/else statements.


回答 3

也许您想要这样:

l = [1, 2, 3, 4, 5] 

print ([['idle','no','yes'][2*(n==1)+(n==2)] for n in l])

Maybe you want this:

l = [1, 2, 3, 4, 5] 

print ([['idle','no','yes'][2*(n==1)+(n==2)] for n in l])

回答 4

您可以使用列表理解功能,以便从原始列表创建另一个列表。

>>> l = [1, 2, 3, 4, 5]
>>> result_map = {1: 'yes', 2: 'no'}
>>> [result_map[x] if x in result_map else 'idle' for x in l]
['yes', 'no', 'idle', 'idle', 'idle']

You can use list comprehension is you are going to create another list from original.

>>> l = [1, 2, 3, 4, 5]
>>> result_map = {1: 'yes', 2: 'no'}
>>> [result_map[x] if x in result_map else 'idle' for x in l]
['yes', 'no', 'idle', 'idle', 'idle']

回答 5

另一种简单的方法是使用条件列表理解,如下所示:

l=[1,2,3,4,5]
print [[["no","yes"][v==1],"idle"][v!=1 and v!=2] for v in l]

给您正确的答案:

[‘是’,’否’,’空闲’,’空闲’,’空闲’]

Another easy way is to use conditional list comprehension like this:

l=[1,2,3,4,5]
print [[["no","yes"][v==1],"idle"][v!=1 and v!=2] for v in l]

gives you the correct anwer:

[‘yes’, ‘no’, ‘idle’, ‘idle’, ‘idle’]


如何在Python matplotlib中使x轴和y轴的比例相等?

问题:如何在Python matplotlib中使x轴和y轴的比例相等?

我希望在方形图上画线。

天秤x-axisy-axis应该是相同的。

例如x的范围是0到10,在屏幕上是10厘米。y也必须在0到10的范围内,并且也必须是10 cm。

即使我迷失了窗口大小,也必须保持正方形。

目前,我的图形与窗口大小一起缩放。

我该如何实现?

更新:

我尝试了以下操作,但没有成功。

plt.xlim(-3, 3)
plt.ylim(-3, 3)
plt.axis('equal')

I wish to draw lines on a square graph.

The scales of x-axis and y-axis should be the same.

e.g. x ranges from 0 to 10 and it is 10cm on the screen. y has to also range from 0 to 10 and has to be also 10 cm.

The square shape has to be maintained, even if I mess around with the window size.

Currently, my graph scales together with the window size.

How may I achieve this?

UPDATE:

I tried the following, but it did not work.

plt.xlim(-3, 3)
plt.ylim(-3, 3)
plt.axis('equal')

回答 0

您需要对api进行更深入的研究:

from matplotlib import pyplot as plt
plt.plot(range(5))
plt.xlim(-3, 3)
plt.ylim(-3, 3)
plt.gca().set_aspect('equal', adjustable='box')
plt.draw()

set_aspect的文档

You need to dig a bit deeper into the api to do this:

from matplotlib import pyplot as plt
plt.plot(range(5))
plt.xlim(-3, 3)
plt.ylim(-3, 3)
plt.gca().set_aspect('equal', adjustable='box')
plt.draw()

doc for set_aspect


回答 1

plt.axis('scaled')

对我来说效果很好。

plt.axis('scaled')

works well for me.


回答 2

尝试类似的方法:

import pylab as p
p.plot(x,y)
p.axis('equal')
p.show()

Try something like:

import pylab as p
p.plot(x,y)
p.axis('equal')
p.show()

回答 3

文档plt.axis()。这个:

plt.axis('equal')

不起作用,因为它更改了轴的界限以使圆看起来是圆形。您想要的是:

plt.axis('square')

这将创建具有相等轴的正方形图。

See the documentation on plt.axis(). This:

plt.axis('equal')

doesn’t work because it changes the limits of the axis to make circles appear circular. What you want is:

plt.axis('square')

This creates a square plot with equal axes.


为Python项目构建Docker映像时如何避免重新安装软件包?

问题:为Python项目构建Docker映像时如何避免重新安装软件包?

我的Dockerfile就像

FROM my/base

ADD . /srv
RUN pip install -r requirements.txt
RUN python setup.py install

ENTRYPOINT ["run_server"]

每次构建新映像时,都必须重新安装依赖项,这在我所在的地区可能非常慢。

我想到的cache已安装软件包的一种方法是my/base用新的映像覆盖该映像,如下所示:

docker build -t new_image_1 .
docker tag new_image_1 my/base

因此,下一次我使用此Dockerfile进行构建时,我/基础已经安装了一些软件包。

但是此解决方案有两个问题:

  1. 并非总是可以覆盖基本图像
  2. 随着新图像的分层,基础图像变得越来越大

那么,我可以使用哪种更好的解决方案来解决此问题?

编辑##:

有关我机器上的docker的一些信息:

  test  docker version
Client version: 1.1.2
Client API version: 1.13
Go version (client): go1.2.1
Git commit (client): d84a070
Server version: 1.1.2
Server API version: 1.13
Go version (server): go1.2.1
Git commit (server): d84a070
  test  docker info
Containers: 0
Images: 56
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Dirs: 56
Execution Driver: native-0.2
Kernel Version: 3.13.0-29-generic
WARNING: No swap limit support

My Dockerfile is something like

FROM my/base

ADD . /srv
RUN pip install -r requirements.txt
RUN python setup.py install

ENTRYPOINT ["run_server"]

Every time I build a new image, dependencies have to be reinstalled, which could be very slow in my region.

One way I think of to cache packages that have been installed is to override the my/base image with newer images like this:

docker build -t new_image_1 .
docker tag new_image_1 my/base

So next time I build with this Dockerfile, my/base already has some packages installed.

But this solution has two problems:

  1. It is not always possible to override a base image
  2. The base image grow bigger and bigger as newer images are layered on it

So what better solution could I use to solve this problem?

EDIT##:

Some information about the docker on my machine:

☁  test  docker version
Client version: 1.1.2
Client API version: 1.13
Go version (client): go1.2.1
Git commit (client): d84a070
Server version: 1.1.2
Server API version: 1.13
Go version (server): go1.2.1
Git commit (server): d84a070
☁  test  docker info
Containers: 0
Images: 56
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Dirs: 56
Execution Driver: native-0.2
Kernel Version: 3.13.0-29-generic
WARNING: No swap limit support

回答 0

尝试构建一个如下所示的Dockerfile:

FROM my/base

WORKDIR /srv
ADD ./requirements.txt /srv/requirements.txt
RUN pip install -r requirements.txt
ADD . /srv
RUN python setup.py install

ENTRYPOINT ["run_server"]

Docker将在pip安装期间使用缓存,只要您不对进行任何更改requirements.txt,无论位于的其他代码文件是否.已更改。这是一个例子。


这是一个简单的Hello, World!程序:

$ tree
.
├── Dockerfile
├── requirements.txt
└── run.py   

0 directories, 3 file

# Dockerfile

FROM dockerfile/python
WORKDIR /srv
ADD ./requirements.txt /srv/requirements.txt
RUN pip install -r requirements.txt
ADD . /srv
CMD python /srv/run.py

# requirements.txt
pytest==2.3.4

# run.py
print("Hello, World")

docker build的输出:

Step 1 : WORKDIR /srv
---> Running in 22d725d22e10
---> 55768a00fd94
Removing intermediate container 22d725d22e10
Step 2 : ADD ./requirements.txt /srv/requirements.txt
---> 968a7c3a4483
Removing intermediate container 5f4e01f290fd
Step 3 : RUN pip install -r requirements.txt
---> Running in 08188205e92b
Downloading/unpacking pytest==2.3.4 (from -r requirements.txt (line 1))
  Running setup.py (path:/tmp/pip_build_root/pytest/setup.py) egg_info for package pytest
....
Cleaning up...
---> bf5c154b87c9
Removing intermediate container 08188205e92b
Step 4 : ADD . /srv
---> 3002a3a67e72
Removing intermediate container 83defd1851d0
Step 5 : CMD python /srv/run.py
---> Running in 11e69b887341
---> 5c0e7e3726d6
Removing intermediate container 11e69b887341
Successfully built 5c0e7e3726d6

让我们修改一下run.py

# run.py
print("Hello, Python")

尝试再次构建,下面是输出:

Sending build context to Docker daemon  5.12 kB
Sending build context to Docker daemon 
Step 0 : FROM dockerfile/python
---> f86d6993fc7b
Step 1 : WORKDIR /srv
---> Using cache
---> 55768a00fd94
Step 2 : ADD ./requirements.txt /srv/requirements.txt
---> Using cache
---> 968a7c3a4483
Step 3 : RUN pip install -r requirements.txt
---> Using cache
---> bf5c154b87c9
Step 4 : ADD . /srv
---> 9cc7508034d6
Removing intermediate container 0d7cf71eb05e
Step 5 : CMD python /srv/run.py
---> Running in f25c21135010
---> 4ffab7bc66c7
Removing intermediate container f25c21135010
Successfully built 4ffab7bc66c7

如上所示,这次docker在构建期间使用了缓存。现在,让我们更新requirements.txt

# requirements.txt

pytest==2.3.4
ipython

以下是docker build的输出:

Sending build context to Docker daemon  5.12 kB
Sending build context to Docker daemon 
Step 0 : FROM dockerfile/python
---> f86d6993fc7b
Step 1 : WORKDIR /srv
---> Using cache
---> 55768a00fd94
Step 2 : ADD ./requirements.txt /srv/requirements.txt
---> b6c19f0643b5
Removing intermediate container a4d9cb37dff0
Step 3 : RUN pip install -r requirements.txt
---> Running in 4b7a85a64c33
Downloading/unpacking pytest==2.3.4 (from -r requirements.txt (line 1))
  Running setup.py (path:/tmp/pip_build_root/pytest/setup.py) egg_info for package pytest

Downloading/unpacking ipython (from -r requirements.txt (line 2))
Downloading/unpacking py>=1.4.12 (from pytest==2.3.4->-r requirements.txt (line 1))
  Running setup.py (path:/tmp/pip_build_root/py/setup.py) egg_info for package py

Installing collected packages: pytest, ipython, py
  Running setup.py install for pytest

Installing py.test script to /usr/local/bin
Installing py.test-2.7 script to /usr/local/bin
  Running setup.py install for py

Successfully installed pytest ipython py
Cleaning up...
---> 23a1af3df8ed
Removing intermediate container 4b7a85a64c33
Step 4 : ADD . /srv
---> d8ae270eca35
Removing intermediate container 7f003ebc3179
Step 5 : CMD python /srv/run.py
---> Running in 510359cf9e12
---> e42fc9121a77
Removing intermediate container 510359cf9e12
Successfully built e42fc9121a77

请注意,在pip安装期间docker如何不使用缓存。如果它不起作用,请检查您的码头工人版本。

Client version: 1.1.2
Client API version: 1.13
Go version (client): go1.2.1
Git commit (client): d84a070
Server version: 1.1.2
Server API version: 1.13
Go version (server): go1.2.1
Git commit (server): d84a070

Try to build a Dockerfile which looks something like this:

FROM my/base

WORKDIR /srv
ADD ./requirements.txt /srv/requirements.txt
RUN pip install -r requirements.txt
ADD . /srv
RUN python setup.py install

ENTRYPOINT ["run_server"]

Docker will use cache during pip install as long as you do not make any changes to the requirements.txt, irrespective of the fact whether other code files at . were changed or not. Here’s an example.


Here’s a simple Hello, World! program:

$ tree
.
├── Dockerfile
├── requirements.txt
└── run.py   

0 directories, 3 file

# Dockerfile

FROM dockerfile/python
WORKDIR /srv
ADD ./requirements.txt /srv/requirements.txt
RUN pip install -r requirements.txt
ADD . /srv
CMD python /srv/run.py

# requirements.txt
pytest==2.3.4

# run.py
print("Hello, World")

The output of docker build:

Step 1 : WORKDIR /srv
---> Running in 22d725d22e10
---> 55768a00fd94
Removing intermediate container 22d725d22e10
Step 2 : ADD ./requirements.txt /srv/requirements.txt
---> 968a7c3a4483
Removing intermediate container 5f4e01f290fd
Step 3 : RUN pip install -r requirements.txt
---> Running in 08188205e92b
Downloading/unpacking pytest==2.3.4 (from -r requirements.txt (line 1))
  Running setup.py (path:/tmp/pip_build_root/pytest/setup.py) egg_info for package pytest
....
Cleaning up...
---> bf5c154b87c9
Removing intermediate container 08188205e92b
Step 4 : ADD . /srv
---> 3002a3a67e72
Removing intermediate container 83defd1851d0
Step 5 : CMD python /srv/run.py
---> Running in 11e69b887341
---> 5c0e7e3726d6
Removing intermediate container 11e69b887341
Successfully built 5c0e7e3726d6

Let’s modify run.py:

# run.py
print("Hello, Python")

Try to build again, below is the output:

Sending build context to Docker daemon  5.12 kB
Sending build context to Docker daemon 
Step 0 : FROM dockerfile/python
---> f86d6993fc7b
Step 1 : WORKDIR /srv
---> Using cache
---> 55768a00fd94
Step 2 : ADD ./requirements.txt /srv/requirements.txt
---> Using cache
---> 968a7c3a4483
Step 3 : RUN pip install -r requirements.txt
---> Using cache
---> bf5c154b87c9
Step 4 : ADD . /srv
---> 9cc7508034d6
Removing intermediate container 0d7cf71eb05e
Step 5 : CMD python /srv/run.py
---> Running in f25c21135010
---> 4ffab7bc66c7
Removing intermediate container f25c21135010
Successfully built 4ffab7bc66c7

As you can see above, this time docker uses cache during the build. Now, let’s update requirements.txt:

# requirements.txt

pytest==2.3.4
ipython

Below is the output of docker build:

Sending build context to Docker daemon  5.12 kB
Sending build context to Docker daemon 
Step 0 : FROM dockerfile/python
---> f86d6993fc7b
Step 1 : WORKDIR /srv
---> Using cache
---> 55768a00fd94
Step 2 : ADD ./requirements.txt /srv/requirements.txt
---> b6c19f0643b5
Removing intermediate container a4d9cb37dff0
Step 3 : RUN pip install -r requirements.txt
---> Running in 4b7a85a64c33
Downloading/unpacking pytest==2.3.4 (from -r requirements.txt (line 1))
  Running setup.py (path:/tmp/pip_build_root/pytest/setup.py) egg_info for package pytest

Downloading/unpacking ipython (from -r requirements.txt (line 2))
Downloading/unpacking py>=1.4.12 (from pytest==2.3.4->-r requirements.txt (line 1))
  Running setup.py (path:/tmp/pip_build_root/py/setup.py) egg_info for package py

Installing collected packages: pytest, ipython, py
  Running setup.py install for pytest

Installing py.test script to /usr/local/bin
Installing py.test-2.7 script to /usr/local/bin
  Running setup.py install for py

Successfully installed pytest ipython py
Cleaning up...
---> 23a1af3df8ed
Removing intermediate container 4b7a85a64c33
Step 4 : ADD . /srv
---> d8ae270eca35
Removing intermediate container 7f003ebc3179
Step 5 : CMD python /srv/run.py
---> Running in 510359cf9e12
---> e42fc9121a77
Removing intermediate container 510359cf9e12
Successfully built e42fc9121a77

Notice how docker didn’t use cache during pip install. If it doesn’t work, check your docker version.

Client version: 1.1.2
Client API version: 1.13
Go version (client): go1.2.1
Git commit (client): d84a070
Server version: 1.1.2
Server API version: 1.13
Go version (server): go1.2.1
Git commit (server): d84a070

回答 1

为了最大程度地减少网络活动,您可以指向pip主机上的缓存目录。

在主机的pip缓存目录绑定已安装到容器的pip缓存目录中的情况下运行docker容器。docker run命令应如下所示:

docker run -v $HOME/.cache/pip-docker/:/root/.cache/pip image_1

然后在您的Dockerfile中,将您的需求作为ENTRYPOINT语句(或CMD语句)的一部分而不是RUN命令来安装。这很重要,因为(如注释中所指出),在映像构建期间(RUN执行语句时)该安装不可用。Docker文件应如下所示:

FROM my/base

ADD . /srv

ENTRYPOINT ["sh", "-c", "pip install -r requirements.txt && python setup.py install && run_server"]

To minimise the network activity, you could point pip to a cache directory on your host machine.

Run your docker container with your host’s pip cache directory bind mounted into your container’s pip cache directory. docker run command should look like this:

docker run -v $HOME/.cache/pip-docker/:/root/.cache/pip image_1

Then in your Dockerfile install your requirements as a part of ENTRYPOINT statement (or CMD statement) instead of as a RUN command. This is important, because (as pointed out in comments) the mount is not available during image building (when RUN statements are executed). Docker file should look like this:

FROM my/base

ADD . /srv

ENTRYPOINT ["sh", "-c", "pip install -r requirements.txt && python setup.py install && run_server"]

回答 2

我知道这个问题已经有一些流行的答案。但是,有一种更新的方式可以为程序包管理器缓存文件。我认为,当BuildKit变得更加标准时,这可能是一个很好的答案。

从Docker 18.09开始,对BuildKit进行了实验性支持。BuildKit在Dockerfile中增加了对一些新功能的支持,包括对将外部卷安装RUN步骤中的实验性支持。这使我们可以为诸如之类的东西创建缓存$HOME/.cache/pip/

我们将使用以下requirements.txt文件作为示例:

Click==7.0
Django==2.2.3
django-appconf==1.0.3
django-compressor==2.3
django-debug-toolbar==2.0
django-filter==2.2.0
django-reversion==3.0.4
django-rq==2.1.0
pytz==2019.1
rcssmin==1.0.6
redis==3.3.4
rjsmin==1.1.0
rq==1.1.0
six==1.12.0
sqlparse==0.3.0

Python的典型示例Dockerfile如下所示:

FROM python:3.7
WORKDIR /usr/src/app
COPY requirements.txt /usr/src/app/
RUN pip install -r requirements.txt
COPY . /usr/src/app

使用DOCKER_BUILDKIT环境变量启用BuildKit后,我们可以pip在大约65秒内构建未缓存的步骤:

$ export DOCKER_BUILDKIT=1
$ docker build -t test .
[+] Building 65.6s (10/10) FINISHED                                                                                                                                             
 => [internal] load .dockerignore                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                            0.0s
 => [internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 120B                                                                                                                                       0.0s
 => [internal] load metadata for docker.io/library/python:3.7                                                                                                              0.5s
 => CACHED [1/4] FROM docker.io/library/python:3.7@sha256:6eaf19442c358afc24834a6b17a3728a45c129de7703d8583392a138ecbdb092                                                 0.0s
 => [internal] load build context                                                                                                                                          0.6s
 => => transferring context: 899.99kB                                                                                                                                      0.6s
 => CACHED [internal] helper image for file operations                                                                                                                     0.0s
 => [2/4] COPY requirements.txt /usr/src/app/                                                                                                                              0.5s
 => [3/4] RUN pip install -r requirements.txt                                                                                                                             61.3s
 => [4/4] COPY . /usr/src/app                                                                                                                                              1.3s
 => exporting to image                                                                                                                                                     1.2s
 => => exporting layers                                                                                                                                                    1.2s
 => => writing image sha256:d66a2720e81530029bf1c2cb98fb3aee0cffc2f4ea2aa2a0760a30fb718d7f83                                                                               0.0s
 => => naming to docker.io/library/test                                                                                                                                    0.0s

现在,让我们添加实验性标头并修改RUN缓存Python包的步骤:

# syntax=docker/dockerfile:experimental

FROM python:3.7
WORKDIR /usr/src/app
COPY requirements.txt /usr/src/app/
RUN --mount=type=cache,target=/root/.cache/pip pip install -r requirements.txt
COPY . /usr/src/app

继续并立即进行另一个构建。它应该花费相同的时间。但这一次是在新的缓存安装中缓存Python软件包:

$ docker build -t pythontest .
[+] Building 60.3s (14/14) FINISHED                                                                                                                                             
 => [internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 120B                                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                            0.0s
 => resolve image config for docker.io/docker/dockerfile:experimental                                                                                                      0.5s
 => CACHED docker-image://docker.io/docker/dockerfile:experimental@sha256:9022e911101f01b2854c7a4b2c77f524b998891941da55208e71c0335e6e82c3                                 0.0s
 => [internal] load .dockerignore                                                                                                                                          0.0s
 => [internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 120B                                                                                                                                       0.0s
 => [internal] load metadata for docker.io/library/python:3.7                                                                                                              0.5s
 => CACHED [1/4] FROM docker.io/library/python:3.7@sha256:6eaf19442c358afc24834a6b17a3728a45c129de7703d8583392a138ecbdb092                                                 0.0s
 => [internal] load build context                                                                                                                                          0.7s
 => => transferring context: 899.99kB                                                                                                                                      0.6s
 => CACHED [internal] helper image for file operations                                                                                                                     0.0s
 => [2/4] COPY requirements.txt /usr/src/app/                                                                                                                              0.6s
 => [3/4] RUN --mount=type=cache,target=/root/.cache/pip pip install -r requirements.txt                                                                                  53.3s
 => [4/4] COPY . /usr/src/app                                                                                                                                              2.6s
 => exporting to image                                                                                                                                                     1.2s
 => => exporting layers                                                                                                                                                    1.2s
 => => writing image sha256:0b035548712c1c9e1c80d4a86169c5c1f9e94437e124ea09e90aea82f45c2afc                                                                               0.0s
 => => naming to docker.io/library/test                                                                                                                                    0.0s

大约60秒。类似于我们的第一个版本。

对进行一些小的更改requirements.txt(例如在两个软件包之间添加新行)以强制高速缓存无效并再次运行:

$ docker build -t pythontest .
[+] Building 15.9s (14/14) FINISHED                                                                                                                                             
 => [internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 120B                                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                            0.0s
 => resolve image config for docker.io/docker/dockerfile:experimental                                                                                                      1.1s
 => CACHED docker-image://docker.io/docker/dockerfile:experimental@sha256:9022e911101f01b2854c7a4b2c77f524b998891941da55208e71c0335e6e82c3                                 0.0s
 => [internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 120B                                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                                          0.0s
 => [internal] load metadata for docker.io/library/python:3.7                                                                                                              0.5s
 => CACHED [1/4] FROM docker.io/library/python:3.7@sha256:6eaf19442c358afc24834a6b17a3728a45c129de7703d8583392a138ecbdb092                                                 0.0s
 => CACHED [internal] helper image for file operations                                                                                                                     0.0s
 => [internal] load build context                                                                                                                                          0.7s
 => => transferring context: 899.99kB                                                                                                                                      0.7s
 => [2/4] COPY requirements.txt /usr/src/app/                                                                                                                              0.6s
 => [3/4] RUN --mount=type=cache,target=/root/.cache/pip pip install -r requirements.txt                                                                                   8.8s
 => [4/4] COPY . /usr/src/app                                                                                                                                              2.1s
 => exporting to image                                                                                                                                                     1.1s
 => => exporting layers                                                                                                                                                    1.1s
 => => writing image sha256:fc84cd45482a70e8de48bfd6489e5421532c2dd02aaa3e1e49a290a3dfb9df7c                                                                               0.0s
 => => naming to docker.io/library/test                                                                                                                                    0.0s

仅约16秒!

由于不再下载所有Python软件包,因此我们获得了这种加速。它们由程序包管理器缓存(pip在这种情况下),并存储在缓存卷装载中。卷安装已提供给运行步骤,以便pip可以重复使用我们已经下载的软件包。这发生在任何Docker层缓存之外

大的时候收益应该更好requirements.txt

笔记:

  • 这是实验性的Dockerfile语法,应这样对待。您现在可能不想在生产中使用此版本。
  • 目前,BuildKit的东西在Docker Compose或其他直接使用Docker API的工具下不起作用。从1.25.0开始,Docker Compose中已对此提供支持。请参阅如何使用docker-compose启用BuildKit?
  • 目前没有任何直接接口可用于管理缓存。当您执行一次操作时,它将被清除docker system prune -a

希望这些功能可以使其融入Docker进行构建,并且BuildKit将成为默认设置。如果/当发生这种情况时,我将尝试更新此答案。

I understand this question has some popular answers already. But there is a newer way to cache files for package managers. I think it could be a good answer in the future when BuildKit becomes more standard.

As of Docker 18.09 there is experimental support for BuildKit. BuildKit adds support for some new features in the Dockerfile including experimental support for mounting external volumes into RUN steps. This allows us to create caches for things like $HOME/.cache/pip/.

We’ll use the following requirements.txt file as an example:

Click==7.0
Django==2.2.3
django-appconf==1.0.3
django-compressor==2.3
django-debug-toolbar==2.0
django-filter==2.2.0
django-reversion==3.0.4
django-rq==2.1.0
pytz==2019.1
rcssmin==1.0.6
redis==3.3.4
rjsmin==1.1.0
rq==1.1.0
six==1.12.0
sqlparse==0.3.0

A typical example Python Dockerfile might look like:

FROM python:3.7
WORKDIR /usr/src/app
COPY requirements.txt /usr/src/app/
RUN pip install -r requirements.txt
COPY . /usr/src/app

With BuildKit enabled using the DOCKER_BUILDKIT environment variable we can build the uncached pip step in about 65 seconds:

$ export DOCKER_BUILDKIT=1
$ docker build -t test .
[+] Building 65.6s (10/10) FINISHED                                                                                                                                             
 => [internal] load .dockerignore                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                            0.0s
 => [internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 120B                                                                                                                                       0.0s
 => [internal] load metadata for docker.io/library/python:3.7                                                                                                              0.5s
 => CACHED [1/4] FROM docker.io/library/python:3.7@sha256:6eaf19442c358afc24834a6b17a3728a45c129de7703d8583392a138ecbdb092                                                 0.0s
 => [internal] load build context                                                                                                                                          0.6s
 => => transferring context: 899.99kB                                                                                                                                      0.6s
 => CACHED [internal] helper image for file operations                                                                                                                     0.0s
 => [2/4] COPY requirements.txt /usr/src/app/                                                                                                                              0.5s
 => [3/4] RUN pip install -r requirements.txt                                                                                                                             61.3s
 => [4/4] COPY . /usr/src/app                                                                                                                                              1.3s
 => exporting to image                                                                                                                                                     1.2s
 => => exporting layers                                                                                                                                                    1.2s
 => => writing image sha256:d66a2720e81530029bf1c2cb98fb3aee0cffc2f4ea2aa2a0760a30fb718d7f83                                                                               0.0s
 => => naming to docker.io/library/test                                                                                                                                    0.0s

Now, let us add the experimental header and modify the RUN step to cache the Python packages:

# syntax=docker/dockerfile:experimental

FROM python:3.7
WORKDIR /usr/src/app
COPY requirements.txt /usr/src/app/
RUN --mount=type=cache,target=/root/.cache/pip pip install -r requirements.txt
COPY . /usr/src/app

Go ahead and do another build now. It should take the same amount of time. But this time it is caching the Python packages in our new cache mount:

$ docker build -t pythontest .
[+] Building 60.3s (14/14) FINISHED                                                                                                                                             
 => [internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 120B                                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                            0.0s
 => resolve image config for docker.io/docker/dockerfile:experimental                                                                                                      0.5s
 => CACHED docker-image://docker.io/docker/dockerfile:experimental@sha256:9022e911101f01b2854c7a4b2c77f524b998891941da55208e71c0335e6e82c3                                 0.0s
 => [internal] load .dockerignore                                                                                                                                          0.0s
 => [internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 120B                                                                                                                                       0.0s
 => [internal] load metadata for docker.io/library/python:3.7                                                                                                              0.5s
 => CACHED [1/4] FROM docker.io/library/python:3.7@sha256:6eaf19442c358afc24834a6b17a3728a45c129de7703d8583392a138ecbdb092                                                 0.0s
 => [internal] load build context                                                                                                                                          0.7s
 => => transferring context: 899.99kB                                                                                                                                      0.6s
 => CACHED [internal] helper image for file operations                                                                                                                     0.0s
 => [2/4] COPY requirements.txt /usr/src/app/                                                                                                                              0.6s
 => [3/4] RUN --mount=type=cache,target=/root/.cache/pip pip install -r requirements.txt                                                                                  53.3s
 => [4/4] COPY . /usr/src/app                                                                                                                                              2.6s
 => exporting to image                                                                                                                                                     1.2s
 => => exporting layers                                                                                                                                                    1.2s
 => => writing image sha256:0b035548712c1c9e1c80d4a86169c5c1f9e94437e124ea09e90aea82f45c2afc                                                                               0.0s
 => => naming to docker.io/library/test                                                                                                                                    0.0s

About 60 seconds. Similar to our first build.

Make a small change to the requirements.txt (such as adding a new line between two packages) to force a cache invalidation and run again:

$ docker build -t pythontest .
[+] Building 15.9s (14/14) FINISHED                                                                                                                                             
 => [internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 120B                                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                            0.0s
 => resolve image config for docker.io/docker/dockerfile:experimental                                                                                                      1.1s
 => CACHED docker-image://docker.io/docker/dockerfile:experimental@sha256:9022e911101f01b2854c7a4b2c77f524b998891941da55208e71c0335e6e82c3                                 0.0s
 => [internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 120B                                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                                          0.0s
 => [internal] load metadata for docker.io/library/python:3.7                                                                                                              0.5s
 => CACHED [1/4] FROM docker.io/library/python:3.7@sha256:6eaf19442c358afc24834a6b17a3728a45c129de7703d8583392a138ecbdb092                                                 0.0s
 => CACHED [internal] helper image for file operations                                                                                                                     0.0s
 => [internal] load build context                                                                                                                                          0.7s
 => => transferring context: 899.99kB                                                                                                                                      0.7s
 => [2/4] COPY requirements.txt /usr/src/app/                                                                                                                              0.6s
 => [3/4] RUN --mount=type=cache,target=/root/.cache/pip pip install -r requirements.txt                                                                                   8.8s
 => [4/4] COPY . /usr/src/app                                                                                                                                              2.1s
 => exporting to image                                                                                                                                                     1.1s
 => => exporting layers                                                                                                                                                    1.1s
 => => writing image sha256:fc84cd45482a70e8de48bfd6489e5421532c2dd02aaa3e1e49a290a3dfb9df7c                                                                               0.0s
 => => naming to docker.io/library/test                                                                                                                                    0.0s

Only about 16 seconds!

We are getting this speedup because we are no longer downloading all the Python packages. They were cached by the package manager (pip in this case) and stored in a cache volume mount. The volume mount is provided to the run step so that pip can reuse our already downloaded packages. This happens outside any Docker layer caching.

The gains should be much better on larger requirements.txt.

Notes:

  • This is experimental Dockerfile syntax and should be treated as such. You may not want to build with this in production at the moment.
  • The BuildKit stuff doesn’t work under Docker Compose or other tools that directly use the Docker API at the moment. There is now support for this in Docker Compose as of 1.25.0. See How do you enable BuildKit with docker-compose?
  • There isn’t any direct interface for managed the cache at the moment. It is purged when you do a docker system prune -a.

Hopefully, these features will make it into Docker for building and BuildKit will become the default. If / when that happens I will try to update this answer.


回答 3

我发现更好的方法是将Python site-packages目录添加为卷。

services:
    web:
        build: .
        command: python manage.py runserver 0.0.0.0:8000
        volumes:
            - .:/code
            -  /usr/local/lib/python2.7/site-packages/

这样,我可以只安装新的库而不必进行完全重建。

编辑:忽略此答案,以上jkukul的答案为我工作。我的意图是缓存site-packages文件夹。那看起来应该更像:

volumes:
   - .:/code
   - ./cached-packages:/usr/local/lib/python2.7/site-packages/

缓存下载文件夹虽然更干净。这也可以缓存轮子,因此可以正确地完成任务。

I found that a better way is to just add the Python site-packages directory as a volume.

services:
    web:
        build: .
        command: python manage.py runserver 0.0.0.0:8000
        volumes:
            - .:/code
            -  /usr/local/lib/python2.7/site-packages/

This way I can just pip install new libraries without having to do a full rebuild.

EDIT: Disregard this answer, jkukul’s answer above worked for me. My intent was to cache the site-packages folder. That would have looked something more like:

volumes:
   - .:/code
   - ./cached-packages:/usr/local/lib/python2.7/site-packages/

Caching the download folder is alot cleaner though. That also caches the wheels, so it properly achieves the task.


类型提示指定类型的列表

问题:类型提示指定类型的列表

使用Python 3的功能注释,可以指定包含在同类列表(或其他集合)中的项的类型,以便在PyCharm和其他IDE中进行类型提示。

一个int列表的伪python代码示例:

def my_func(l:list<int>):
    pass

我知道有可能使用Docstring …

def my_func(l):
    """
    :type l: list[int]
    """
    pass

…但是我更喜欢注释样式。

Using Python 3’s function annotations, it is possible to specify the type of items contained within a homogeneous list (or other collection) for the purpose of type hinting in PyCharm and other IDEs?

A pseudo-python code example for a list of int:

def my_func(l:list<int>):
    pass

I know it’s possible using Docstring…

def my_func(l):
    """
    :type l: list[int]
    """
    pass

… but I prefer the annotation style if it’s possible.


回答 0

回答我自己的问题;TLDR的答案是“

更新2

在2015年9月,Python 3.5发行了对Type Hints的支持,并包含一个新的打字模块。这允许指定集合中包含的类型。截至2015年11月,JetBrains PyCharm 5.0完全支持Python 3.5包含类型提示,如下所示。

更新1

截至2015年5月,PEP0484(类型提示)已被正式接受。草案的实现也可以在github的ambv / typehinting下找到

原始答案

自2014年8月起,我已经确认无法使用Python 3类型注释在集合中指定类型(例如:字符串列表)。

诸如reStructuredText或Sphinx之类的格式化文档字符串的使用是可行的替代方法,并且受各种IDE支持。

看来,Guido还本着mypy的精神考虑了扩展类型注释的想法:http ://mail.python.org/pipermail/python-ideas/2014-August/028618.html

Answering my own question; the TLDR answer is No Yes.

Update 2

In September 2015, Python 3.5 was released with support for Type Hints and includes a new typing module. This allows for the specification of types contained within collections. As of November 2015, JetBrains PyCharm 5.0 fully supports Python 3.5 to include Type Hints as illustrated below.

Update 1

As of May 2015, PEP0484 (Type Hints) has been formally accepted. The draft implementation is also available at github under ambv/typehinting.

Original Answer

As of Aug 2014, I have confirmed that it is not possible to use Python 3 type annotations to specify types within collections (ex: a list of strings).

The use of formatted docstrings such as reStructuredText or Sphinx are viable alternatives and supported by various IDEs.

It also appears that Guido is mulling over the idea of extending type annotations in the spirit of mypy: http://mail.python.org/pipermail/python-ideas/2014-August/028618.html


回答 1

现在Python 3.5正式发布了,这里有Type Hints支持模块- typing以及相关的List通用容器 “类型”。

换句话说,现在您可以执行以下操作:

from typing import List

def my_func(l: List[int]):
    pass

Now that Python 3.5 is officially out, there is the Type Hints supporting module – typing and the relevant List “type” for the generic containers.

In other words, now you can do:

from typing import List

def my_func(l: List[int]):
    pass

回答 2

PEP 484起已添加类型注释

from . import Monitor
from typing import List, Set, Tuple, Dict


active_monitors = [] # type: List[Monitor]
# or
active_monitors: List[Monitor] = []

# bonus
active_monitors: Set[Monitor] = set()
monitor_pair: Tuple[Monitor, Monitor] = (Monitor(), Monitor())
monitor_dict: Dict[str, Monitor] = {'codename': Monitor()}

# nested
monitor_pair_list: List[Dict[str, Monitor]] = [{'codename': Monitor()}]

目前,这对于使用Python 3.6.4的PyCharm来说是有用的

Pycharm中的示例图片

Type comments have been added since PEP 484

from . import Monitor
from typing import List, Set, Tuple, Dict


active_monitors = [] # type: List[Monitor]
# or
active_monitors: List[Monitor] = []

# bonus
active_monitors: Set[Monitor] = set()
monitor_pair: Tuple[Monitor, Monitor] = (Monitor(), Monitor())
monitor_dict: Dict[str, Monitor] = {'codename': Monitor()}

# nested
monitor_pair_list: List[Dict[str, Monitor]] = [{'codename': Monitor()}]

This is currently working for me on PyCharm with Python 3.6.4

Example Picture in Pycharm


回答 3

在BDFL的支持下,现在几乎可以确定python(可能是3.5)将通过函数注释为类型提示提供标准化的语法。

https://www.python.org/dev/peps/pep-0484/

正如PEP中所引用的那样,有一个名为mypy的实验型类型检查器(有点像pylint,但对于类型而言)已经使用此标准,并且不需要任何新语法。

http://mypy-lang.org/

With support from the BDFL, it’s almost certain now that python (probably 3.5) will provide a standardized syntax for type hints via function annotations.

https://www.python.org/dev/peps/pep-0484/

As referenced in the PEP, there is an experimental type-checker (kind of like pylint, but for types) called mypy that already uses this standard, and doesn’t require any new syntax.

http://mypy-lang.org/


回答 4

从Python 3.9开始,内置类型在类型注释方面是通用的(请参阅PEP 585)。这允许直接指定元素的类型:

def my_func(l: list[int]):
    pass

各种工具可能早于Python 3.9支持此语法。如果在运行时未检查注释,则使用引号或语法有效__future__.annotations

# quoted
def my_func(l: 'list[int]'):
    pass
# postponed evaluation of annotation
from __future__ import annotations

def my_func(l: list[int]):
    pass

As of Python 3.9, builtin types are generic with respect to type annotations (see PEP 585). This allows to directly specify the type of elements:

def my_func(l: list[int]):
    pass

Various tools may support this syntax earlier than Python 3.9. When annotations are not inspected at runtime, the syntax is valid using quoting or __future__.annotations.

# quoted
def my_func(l: 'list[int]'):
    pass
# postponed evaluation of annotation
from __future__ import annotations

def my_func(l: list[int]):
    pass

如何使用Python / Django执行HTML解码/编码?

问题:如何使用Python / Django执行HTML解码/编码?

我有一个HTML编码的字符串:

'''&lt;img class=&quot;size-medium wp-image-113&quot;\
 style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot;\
 src=&quot;http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg&quot;\
 alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;'''

我想将其更改为:

<img class="size-medium wp-image-113" style="margin-left: 15px;" 
  title="su1" src="http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg" 
  alt="" width="300" height="194" /> 

我希望将其注册为HTML,以便浏览器将其呈现为图像,而不是显示为文本。

字符串的存储方式是这样的,因为我正在使用一种名为的网络抓取工具BeautifulSoup,它将“扫描”网页并从中获取某些内容,然后以该格式返回字符串。

我已经找到了如何在C#中而不是在Python中执行此操作。有人可以帮我吗?

有关

I have a string that is HTML encoded:

'''&lt;img class=&quot;size-medium wp-image-113&quot;\
 style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot;\
 src=&quot;http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg&quot;\
 alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;'''

I want to change that to:

<img class="size-medium wp-image-113" style="margin-left: 15px;" 
  title="su1" src="http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg" 
  alt="" width="300" height="194" /> 

I want this to register as HTML so that it is rendered as an image by the browser instead of being displayed as text.

The string is stored like that because I am using a web-scraping tool called BeautifulSoup, it “scans” a web-page and gets certain content from it, then returns the string in that format.

I’ve found how to do this in C# but not in Python. Can someone help me out?

Related


回答 0

给定Django用例,对此有两个答案。这是它的django.utils.html.escape功能,以供参考:

def escape(html):
    """Returns the given HTML with ampersands, quotes and carets encoded."""
    return mark_safe(force_unicode(html).replace('&', '&amp;').replace('<', '&l
t;').replace('>', '&gt;').replace('"', '&quot;').replace("'", '&#39;'))

为了解决这个问题,Jake的答案中描述的Cheetah函数应该起作用,但是缺少单引号。此版本包含一个更新的元组,并且替换顺序相反,以避免出现对称问题:

def html_decode(s):
    """
    Returns the ASCII decoded version of the given HTML string. This does
    NOT remove normal HTML tags like <p>.
    """
    htmlCodes = (
            ("'", '&#39;'),
            ('"', '&quot;'),
            ('>', '&gt;'),
            ('<', '&lt;'),
            ('&', '&amp;')
        )
    for code in htmlCodes:
        s = s.replace(code[1], code[0])
    return s

unescaped = html_decode(my_string)

但是,这不是一般的解决方案。仅适用于以编码的字符串django.utils.html.escape。更笼统地说,坚持使用标准库是一个好主意:

# Python 2.x:
import HTMLParser
html_parser = HTMLParser.HTMLParser()
unescaped = html_parser.unescape(my_string)

# Python 3.x:
import html.parser
html_parser = html.parser.HTMLParser()
unescaped = html_parser.unescape(my_string)

# >= Python 3.5:
from html import unescape
unescaped = unescape(my_string)

建议:将未转义的HTML存储在数据库中可能更有意义。如果可能的话,值得一探的是从BeautifulSoup获得未转义的结果,并完全避免此过程。

对于Django,转义仅在模板渲染期间发生;因此,为了防止转义,您只需告诉模板引擎不要转义您的字符串即可。为此,请在模板中使用以下选项之一:

{{ context_var|safe }}
{% autoescape off %}
    {{ context_var }}
{% endautoescape %}

Given the Django use case, there are two answers to this. Here is its django.utils.html.escape function, for reference:

def escape(html):
    """Returns the given HTML with ampersands, quotes and carets encoded."""
    return mark_safe(force_unicode(html).replace('&', '&amp;').replace('<', '&l
t;').replace('>', '&gt;').replace('"', '&quot;').replace("'", '&#39;'))

To reverse this, the Cheetah function described in Jake’s answer should work, but is missing the single-quote. This version includes an updated tuple, with the order of replacement reversed to avoid symmetric problems:

def html_decode(s):
    """
    Returns the ASCII decoded version of the given HTML string. This does
    NOT remove normal HTML tags like <p>.
    """
    htmlCodes = (
            ("'", '&#39;'),
            ('"', '&quot;'),
            ('>', '&gt;'),
            ('<', '&lt;'),
            ('&', '&amp;')
        )
    for code in htmlCodes:
        s = s.replace(code[1], code[0])
    return s

unescaped = html_decode(my_string)

This, however, is not a general solution; it is only appropriate for strings encoded with django.utils.html.escape. More generally, it is a good idea to stick with the standard library:

# Python 2.x:
import HTMLParser
html_parser = HTMLParser.HTMLParser()
unescaped = html_parser.unescape(my_string)

# Python 3.x:
import html.parser
html_parser = html.parser.HTMLParser()
unescaped = html_parser.unescape(my_string)

# >= Python 3.5:
from html import unescape
unescaped = unescape(my_string)

As a suggestion: it may make more sense to store the HTML unescaped in your database. It’d be worth looking into getting unescaped results back from BeautifulSoup if possible, and avoiding this process altogether.

With Django, escaping only occurs during template rendering; so to prevent escaping you just tell the templating engine not to escape your string. To do that, use one of these options in your template:

{{ context_var|safe }}
{% autoescape off %}
    {{ context_var }}
{% endautoescape %}

回答 1

使用标准库:

  • HTML转义

    try:
        from html import escape  # python 3.x
    except ImportError:
        from cgi import escape  # python 2.x
    
    print(escape("<"))
    
  • HTML转义

    try:
        from html import unescape  # python 3.4+
    except ImportError:
        try:
            from html.parser import HTMLParser  # python 3.x (<3.4)
        except ImportError:
            from HTMLParser import HTMLParser  # python 2.x
        unescape = HTMLParser().unescape
    
    print(unescape("&gt;"))
    

With the standard library:

  • HTML Escape

    try:
        from html import escape  # python 3.x
    except ImportError:
        from cgi import escape  # python 2.x
    
    print(escape("<"))
    
  • HTML Unescape

    try:
        from html import unescape  # python 3.4+
    except ImportError:
        try:
            from html.parser import HTMLParser  # python 3.x (<3.4)
        except ImportError:
            from HTMLParser import HTMLParser  # python 2.x
        unescape = HTMLParser().unescape
    
    print(unescape("&gt;"))
    

回答 2

对于html编码,标准库中有cgi.escape

>> help(cgi.escape)
cgi.escape = escape(s, quote=None)
    Replace special characters "&", "<" and ">" to HTML-safe sequences.
    If the optional flag quote is true, the quotation mark character (")
    is also translated.

对于html解码,我使用以下代码:

import re
from htmlentitydefs import name2codepoint
# for some reason, python 2.5.2 doesn't have this one (apostrophe)
name2codepoint['#39'] = 39

def unescape(s):
    "unescape HTML code refs; c.f. http://wiki.python.org/moin/EscapingHtml"
    return re.sub('&(%s);' % '|'.join(name2codepoint),
              lambda m: unichr(name2codepoint[m.group(1)]), s)

对于更复杂的事情,我使用BeautifulSoup。

For html encoding, there’s cgi.escape from the standard library:

>> help(cgi.escape)
cgi.escape = escape(s, quote=None)
    Replace special characters "&", "<" and ">" to HTML-safe sequences.
    If the optional flag quote is true, the quotation mark character (")
    is also translated.

For html decoding, I use the following:

import re
from htmlentitydefs import name2codepoint
# for some reason, python 2.5.2 doesn't have this one (apostrophe)
name2codepoint['#39'] = 39

def unescape(s):
    "unescape HTML code refs; c.f. http://wiki.python.org/moin/EscapingHtml"
    return re.sub('&(%s);' % '|'.join(name2codepoint),
              lambda m: unichr(name2codepoint[m.group(1)]), s)

For anything more complicated, I use BeautifulSoup.


回答 3

如果编码字符集受到相对限制,请使用daniel的解决方案。否则,请使用众多HTML解析库之一。

我喜欢BeautifulSoup,因为它可以处理格式错误的XML / HTML:

http://www.crummy.com/software/BeautifulSoup/

对于您的问题,他们的文档中有一个示例

from BeautifulSoup import BeautifulStoneSoup
BeautifulStoneSoup("Sacr&eacute; bl&#101;u!", 
                   convertEntities=BeautifulStoneSoup.HTML_ENTITIES).contents[0]
# u'Sacr\xe9 bleu!'

Use daniel’s solution if the set of encoded characters is relatively restricted. Otherwise, use one of the numerous HTML-parsing libraries.

I like BeautifulSoup because it can handle malformed XML/HTML :

http://www.crummy.com/software/BeautifulSoup/

for your question, there’s an example in their documentation

from BeautifulSoup import BeautifulStoneSoup
BeautifulStoneSoup("Sacr&eacute; bl&#101;u!", 
                   convertEntities=BeautifulStoneSoup.HTML_ENTITIES).contents[0]
# u'Sacr\xe9 bleu!'

回答 4

在Python 3.4+中:

import html

html.unescape(your_string)

In Python 3.4+:

import html

html.unescape(your_string)

回答 5

请参阅此页面底部的Python Wiki,至少有2个选项可以“取消转义” html。

See at the bottom of this page at Python wiki, there are at least 2 options to “unescape” html.


回答 6

丹尼尔的评论作为答案:

“转义仅发生在Django模板渲染期间。因此,不需要进行转义-您只需告诉模板引擎不要转义。{{context_var | safe}}或{%autoescape off%} {{context_var}} { %endautoescape%}”

Daniel’s comment as an answer:

“escaping only occurs in Django during template rendering. Therefore, there’s no need for an unescape – you just tell the templating engine not to escape. either {{ context_var|safe }} or {% autoescape off %}{{ context_var }}{% endautoescape %}”


回答 7

我在以下位置找到了很好的功能:http : //snippets.dzone.com/posts/show/4569

def decodeHtmlentities(string):
    import re
    entity_re = re.compile("&(#?)(\d{1,5}|\w{1,8});")

    def substitute_entity(match):
        from htmlentitydefs import name2codepoint as n2cp
        ent = match.group(2)
        if match.group(1) == "#":
            return unichr(int(ent))
        else:
            cp = n2cp.get(ent)

            if cp:
                return unichr(cp)
            else:
                return match.group()

    return entity_re.subn(substitute_entity, string)[0]

I found a fine function at: http://snippets.dzone.com/posts/show/4569

def decodeHtmlentities(string):
    import re
    entity_re = re.compile("&(#?)(\d{1,5}|\w{1,8});")

    def substitute_entity(match):
        from htmlentitydefs import name2codepoint as n2cp
        ent = match.group(2)
        if match.group(1) == "#":
            return unichr(int(ent))
        else:
            cp = n2cp.get(ent)

            if cp:
                return unichr(cp)
            else:
                return match.group()

    return entity_re.subn(substitute_entity, string)[0]

回答 8

如果有人在寻找通过django模板执行此操作的简单方法,则可以始终使用以下过滤器:

<html>
{{ node.description|safe }}
</html>

我有一些来自供应商的数据,我发布的所有内容实际上都是在呈现的页面上写的html标签,就像您在查看源代码一样。上面的代码极大地帮助了我。希望这对其他人有帮助。

干杯!!

If anyone is looking for a simple way to do this via the django templates, you can always use filters like this:

<html>
{{ node.description|safe }}
</html>

I had some data coming from a vendor and everything I posted had html tags actually written on the rendered page as if you were looking at the source. The above code helped me greatly. Hope this helps others.

Cheers!!


回答 9

即使这是一个非常老的问题,也可能有效。

的Django 1.5.5

In [1]: from django.utils.text import unescape_entities
In [2]: unescape_entities('&lt;img class=&quot;size-medium wp-image-113&quot; style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; src=&quot;http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg&quot; alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;')
Out[2]: u'<img class="size-medium wp-image-113" style="margin-left: 15px;" title="su1" src="http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg" alt="" width="300" height="194" />'

Even though this is a really old question, this may work.

Django 1.5.5

In [1]: from django.utils.text import unescape_entities
In [2]: unescape_entities('&lt;img class=&quot;size-medium wp-image-113&quot; style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; src=&quot;http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg&quot; alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;')
Out[2]: u'<img class="size-medium wp-image-113" style="margin-left: 15px;" title="su1" src="http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg" alt="" width="300" height="194" />'

回答 10

我在猎豹的源代码中找到了这个(这里

htmlCodes = [
    ['&', '&amp;'],
    ['<', '&lt;'],
    ['>', '&gt;'],
    ['"', '&quot;'],
]
htmlCodesReversed = htmlCodes[:]
htmlCodesReversed.reverse()
def htmlDecode(s, codes=htmlCodesReversed):
    """ Returns the ASCII decoded version of the given HTML string. This does
        NOT remove normal HTML tags like <p>. It is the inverse of htmlEncode()."""
    for code in codes:
        s = s.replace(code[1], code[0])
    return s

不确定为什么要反转列表,我认为它与编码方式有关,因此对于您而言,可能不需要反转。另外,如果我是我,我会将htmlCodes更改为元组列表,而不是列表列表…尽管这将在我的库中进行:)

我也注意到您的标题也要求编码,所以这是猎豹的编码功能。

def htmlEncode(s, codes=htmlCodes):
    """ Returns the HTML encoded version of the given string. This is useful to
        display a plain ASCII text string on a web page."""
    for code in codes:
        s = s.replace(code[0], code[1])
    return s

I found this in the Cheetah source code (here)

htmlCodes = [
    ['&', '&amp;'],
    ['<', '&lt;'],
    ['>', '&gt;'],
    ['"', '&quot;'],
]
htmlCodesReversed = htmlCodes[:]
htmlCodesReversed.reverse()
def htmlDecode(s, codes=htmlCodesReversed):
    """ Returns the ASCII decoded version of the given HTML string. This does
        NOT remove normal HTML tags like <p>. It is the inverse of htmlEncode()."""
    for code in codes:
        s = s.replace(code[1], code[0])
    return s

not sure why they reverse the list, I think it has to do with the way they encode, so with you it may not need to be reversed. Also if I were you I would change htmlCodes to be a list of tuples rather than a list of lists… this is going in my library though :)

i noticed your title asked for encode too, so here is Cheetah’s encode function.

def htmlEncode(s, codes=htmlCodes):
    """ Returns the HTML encoded version of the given string. This is useful to
        display a plain ASCII text string on a web page."""
    for code in codes:
        s = s.replace(code[0], code[1])
    return s

回答 11

您也可以使用django.utils.html.escape

from django.utils.html import escape

something_nice = escape(request.POST['something_naughty'])

You can also use django.utils.html.escape

from django.utils.html import escape

something_nice = escape(request.POST['something_naughty'])

回答 12

以下是使用module的python函数htmlentitydefs。这不是完美的。htmlentitydefs我所拥有的版本不完整,它假设所有实体都解码到一个代码点,这对于像这样的实体是错误的&NotEqualTilde;

http://www.w3.org/TR/html5/named-character-references.html

NotEqualTilde;     U+02242 U+00338    ≂̸

尽管有这些警告,但这里是代码。

def decodeHtmlText(html):
    """
    Given a string of HTML that would parse to a single text node,
    return the text value of that node.
    """
    # Fast path for common case.
    if html.find("&") < 0: return html
    return re.sub(
        '&(?:#(?:x([0-9A-Fa-f]+)|([0-9]+))|([a-zA-Z0-9]+));',
        _decode_html_entity,
        html)

def _decode_html_entity(match):
    """
    Regex replacer that expects hex digits in group 1, or
    decimal digits in group 2, or a named entity in group 3.
    """
    hex_digits = match.group(1)  # '&#10;' -> unichr(10)
    if hex_digits: return unichr(int(hex_digits, 16))
    decimal_digits = match.group(2)  # '&#x10;' -> unichr(0x10)
    if decimal_digits: return unichr(int(decimal_digits, 10))
    name = match.group(3)  # name is 'lt' when '&lt;' was matched.
    if name:
        decoding = (htmlentitydefs.name2codepoint.get(name)
            # Treat &GT; like &gt;.
            # This is wrong for &Gt; and &Lt; which HTML5 adopted from MathML.
            # If htmlentitydefs included mappings for those entities,
            # then this code will magically work.
            or htmlentitydefs.name2codepoint.get(name.lower()))
        if decoding is not None: return unichr(decoding)
    return match.group(0)  # Treat "&noSuchEntity;" as "&noSuchEntity;"

Below is a python function that uses module htmlentitydefs. It is not perfect. The version of htmlentitydefs that I have is incomplete and it assumes that all entities decode to one codepoint which is wrong for entities like &NotEqualTilde;:

http://www.w3.org/TR/html5/named-character-references.html

NotEqualTilde;     U+02242 U+00338    ≂̸

With those caveats though, here’s the code.

def decodeHtmlText(html):
    """
    Given a string of HTML that would parse to a single text node,
    return the text value of that node.
    """
    # Fast path for common case.
    if html.find("&") < 0: return html
    return re.sub(
        '&(?:#(?:x([0-9A-Fa-f]+)|([0-9]+))|([a-zA-Z0-9]+));',
        _decode_html_entity,
        html)

def _decode_html_entity(match):
    """
    Regex replacer that expects hex digits in group 1, or
    decimal digits in group 2, or a named entity in group 3.
    """
    hex_digits = match.group(1)  # '&#10;' -> unichr(10)
    if hex_digits: return unichr(int(hex_digits, 16))
    decimal_digits = match.group(2)  # '&#x10;' -> unichr(0x10)
    if decimal_digits: return unichr(int(decimal_digits, 10))
    name = match.group(3)  # name is 'lt' when '&lt;' was matched.
    if name:
        decoding = (htmlentitydefs.name2codepoint.get(name)
            # Treat &GT; like &gt;.
            # This is wrong for &Gt; and &Lt; which HTML5 adopted from MathML.
            # If htmlentitydefs included mappings for those entities,
            # then this code will magically work.
            or htmlentitydefs.name2codepoint.get(name.lower()))
        if decoding is not None: return unichr(decoding)
    return match.group(0)  # Treat "&noSuchEntity;" as "&noSuchEntity;"

回答 13

这是解决此问题的最简单方法-

{% autoescape on %}
   {{ body }}
{% endautoescape %}

从此页面

This is the easiest solution for this problem –

{% autoescape on %}
   {{ body }}
{% endautoescape %}

From this page.


回答 14

在Django和Python中搜索此问题的最简单解决方案,我发现您可以使用内置函数来转义/转义html代码。

我将您的html代码保存在scraped_html和中clean_html

scraped_html = (
    '&lt;img class=&quot;size-medium wp-image-113&quot; '
    'style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; '
    'src=&quot;http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg&quot; '
    'alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;'
)
clean_html = (
    '<img class="size-medium wp-image-113" style="margin-left: 15px;" '
    'title="su1" src="http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg" '
    'alt="" width="300" height="194" />'
)

Django的

您需要Django> = 1.0

逃生

取消抓取的 HTML代码的转义,可以使用django.utils.text.unescape_entities,其中:

将所有命名和数字字符引用转换为相应的unicode字符。

>>> from django.utils.text import unescape_entities
>>> clean_html == unescape_entities(scraped_html)
True

逃逸

要转义干净的html代码,可以使用django.utils.html.escape,其中:

返回给定文本,该文本带有与符号,引号和尖括号,并编码为在HTML中使用。

>>> from django.utils.html import escape
>>> scraped_html == escape(clean_html)
True

Python

您需要Python> = 3.4

逃生

取消抓取的 html代码,可以使用html.unescape,其中:

转换所有命名和数字字符引用(例如&gt;&#62;&x3e;到对应的Unicode字符字符串s)。

>>> from html import unescape
>>> clean_html == unescape(scraped_html)
True

逃逸

要转义干净的html代码,可以使用html.escape,其中:

转换角色&<>在字符串s到HTML安全序列。

>>> from html import escape
>>> scraped_html == escape(clean_html)
True

Searching the simplest solution of this question in Django and Python I found you can use builtin theirs functions to escape/unescape html code.

Example

I saved your html code in scraped_html and clean_html:

scraped_html = (
    '&lt;img class=&quot;size-medium wp-image-113&quot; '
    'style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; '
    'src=&quot;http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg&quot; '
    'alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;'
)
clean_html = (
    '<img class="size-medium wp-image-113" style="margin-left: 15px;" '
    'title="su1" src="http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg" '
    'alt="" width="300" height="194" />'
)

Django

You need Django >= 1.0

unescape

To unescape your scraped html code you can use django.utils.text.unescape_entities which:

Convert all named and numeric character references to the corresponding unicode characters.

>>> from django.utils.text import unescape_entities
>>> clean_html == unescape_entities(scraped_html)
True

escape

To escape your clean html code you can use django.utils.html.escape which:

Returns the given text with ampersands, quotes and angle brackets encoded for use in HTML.

>>> from django.utils.html import escape
>>> scraped_html == escape(clean_html)
True

Python

You need Python >= 3.4

unescape

To unescape your scraped html code you can use html.unescape which:

Convert all named and numeric character references (e.g. &gt;, &#62;, &x3e;) in the string s to the corresponding unicode characters.

>>> from html import unescape
>>> clean_html == unescape(scraped_html)
True

escape

To escape your clean html code you can use html.escape which:

Convert the characters &, < and > in string s to HTML-safe sequences.

>>> from html import escape
>>> scraped_html == escape(clean_html)
True

为什么在Python中“ if not someobj:”优于“ if someobj == None:”?

问题:为什么在Python中“ if not someobj:”优于“ if someobj == None:”?

我看过几个这样的代码示例:

if not someobj:
    #do something

但我想知道为什么不这样做:

if someobj == None:
    #do something

有什么区别吗?一个人比另一个人有优势吗?

I’ve seen several examples of code like this:

if not someobj:
    #do something

But I’m wondering why not doing:

if someobj == None:
    #do something

Is there any difference? Does one have an advantage over the other?


回答 0

在第一个测试中,Python尝试将对象转换为bool值(如果尚未转换为值)。粗略地,我们在问对象:您有意义吗?这可以通过以下算法完成:

  1. 如果对象具有__nonzero__特殊方法(数字内置intfloat),则调用此方法。它必须要么返回bool其随后直接使用值,或者int被认为是值False如果等于零。

  2. 否则,如果对象有一个__len__特殊的方法(如做容器内置插件,listdictsettuple,…),它会调用这个方法,考虑一个容器False,如果它是空的(长度为零)。

  3. 否则,该对象被认为是True,除非它是None在这种情况下,它被认为是False

在第二个测试中,将对象与进行相等性比较None。在这里,我们要问对象“您等于这个其他值吗?” 这可以通过以下算法完成:

  1. 如果对象具有__eq__方法,则将其调用,然后将返回值转换为一个bool值,并用于确定的结果if

  2. 否则,如果对象具有__cmp__方法,则将其调用。此函数必须返回一个int指示两个对象的顺序的指示符(-1if self < other0if self == other+1if self > other)。

  3. 否则,比较对象的身份(即,它们引用了同一对象,可以由is操作员进行测试)。

使用is运算符可以进行另一项测试。我们会问对象“您是这个特定对象吗?”

通常,我建议对非数字值使用第一个测试,如果要比较具有相同性质(两个字符串,两个数字,…)的对象,并且仅在以下情况下检查身份,请使用该测试是否相等使用哨兵值(例如,None未针对成员字段初始化,或使用getattr__getitem__方法时未初始化)。

总而言之,我们有:

>>> class A(object):
...    def __repr__(self):
...        return 'A()'
...    def __nonzero__(self):
...        return False

>>> class B(object):
...    def __repr__(self):
...        return 'B()'
...    def __len__(self):
...        return 0

>>> class C(object):
...    def __repr__(self):
...        return 'C()'
...    def __cmp__(self, other):
...        return 0

>>> class D(object):
...    def __repr__(self):
...        return 'D()'
...    def __eq__(self, other):
...        return True

>>> for obj in ['', (), [], {}, 0, 0., A(), B(), C(), D(), None]:
...     print '%4s: bool(obj) -> %5s, obj == None -> %5s, obj is None -> %5s' % \
...         (repr(obj), bool(obj), obj == None, obj is None)
  '': bool(obj) -> False, obj == None -> False, obj is None -> False
  (): bool(obj) -> False, obj == None -> False, obj is None -> False
  []: bool(obj) -> False, obj == None -> False, obj is None -> False
  {}: bool(obj) -> False, obj == None -> False, obj is None -> False
   0: bool(obj) -> False, obj == None -> False, obj is None -> False
 0.0: bool(obj) -> False, obj == None -> False, obj is None -> False
 A(): bool(obj) -> False, obj == None -> False, obj is None -> False
 B(): bool(obj) -> False, obj == None -> False, obj is None -> False
 C(): bool(obj) ->  True, obj == None ->  True, obj is None -> False
 D(): bool(obj) ->  True, obj == None ->  True, obj is None -> False
None: bool(obj) -> False, obj == None ->  True, obj is None ->  True

In the first test, Python try to convert the object to a bool value if it is not already one. Roughly, we are asking the object : are you meaningful or not ? This is done using the following algorithm :

  1. If the object has a __nonzero__ special method (as do numeric built-ins, int and float), it calls this method. It must either return a bool value which is then directly used, or an int value that is considered False if equal to zero.

  2. Otherwise, if the object has a __len__ special method (as do container built-ins, list, dict, set, tuple, …), it calls this method, considering a container False if it is empty (length is zero).

  3. Otherwise, the object is considered True unless it is None in which case, it is considered False.

In the second test, the object is compared for equality to None. Here, we are asking the object, “Are you equal to this other value?” This is done using the following algorithm :

  1. If the object has a __eq__ method, it is called, and the return value is then converted to a boolvalue and used to determine the outcome of the if.

  2. Otherwise, if the object has a __cmp__ method, it is called. This function must return an int indicating the order of the two object (-1 if self < other, 0 if self == other, +1 if self > other).

  3. Otherwise, the object are compared for identity (ie. they are reference to the same object, as can be tested by the is operator).

There is another test possible using the is operator. We would be asking the object, “Are you this particular object?”

Generally, I would recommend to use the first test with non-numerical values, to use the test for equality when you want to compare objects of the same nature (two strings, two numbers, …) and to check for identity only when using sentinel values (None meaning not initialized for a member field for exemple, or when using the getattr or the __getitem__ methods).

To summarize, we have :

>>> class A(object):
...    def __repr__(self):
...        return 'A()'
...    def __nonzero__(self):
...        return False

>>> class B(object):
...    def __repr__(self):
...        return 'B()'
...    def __len__(self):
...        return 0

>>> class C(object):
...    def __repr__(self):
...        return 'C()'
...    def __cmp__(self, other):
...        return 0

>>> class D(object):
...    def __repr__(self):
...        return 'D()'
...    def __eq__(self, other):
...        return True

>>> for obj in ['', (), [], {}, 0, 0., A(), B(), C(), D(), None]:
...     print '%4s: bool(obj) -> %5s, obj == None -> %5s, obj is None -> %5s' % \
...         (repr(obj), bool(obj), obj == None, obj is None)
  '': bool(obj) -> False, obj == None -> False, obj is None -> False
  (): bool(obj) -> False, obj == None -> False, obj is None -> False
  []: bool(obj) -> False, obj == None -> False, obj is None -> False
  {}: bool(obj) -> False, obj == None -> False, obj is None -> False
   0: bool(obj) -> False, obj == None -> False, obj is None -> False
 0.0: bool(obj) -> False, obj == None -> False, obj is None -> False
 A(): bool(obj) -> False, obj == None -> False, obj is None -> False
 B(): bool(obj) -> False, obj == None -> False, obj is None -> False
 C(): bool(obj) ->  True, obj == None ->  True, obj is None -> False
 D(): bool(obj) ->  True, obj == None ->  True, obj is None -> False
None: bool(obj) -> False, obj == None ->  True, obj is None ->  True

回答 1

这些实际上都是不好的做法。曾几何时,随意将None和False视为相似是可以的。但是,从Python 2.2开始,这不是最好的策略。

首先,当您进行测试if xif not x某种测试时,Python必须隐式转换x为布尔值。该bool函数的规则描述了许多错误的事物。其他一切都是真的。如果x的值开头不是正确的布尔值,那么这种隐式转换实际上并不是最清晰的表达方式。

在Python 2.2之前,没有bool函数,因此不清楚。

其次,您不应该真正使用进行测试== None。您应该使用is Noneis not None

请参阅PEP 8,Python代码样式指南

- Comparisons to singletons like None should always be done with
  'is' or 'is not', never the equality operators.

  Also, beware of writing "if x" when you really mean "if x is not None"
  -- e.g. when testing whether a variable or argument that defaults to
  None was set to some other value.  The other value might have a type
  (such as a container) that could be false in a boolean context!

有多少个单身人士?五:NoneTrueFalseNotImplementedEllipsis。由于您实际上不太可能使用NotImplementedEllipsis,因此您永远不会说if x is True(因为if x它很清楚),因此您只会进行测试None

These are actually both poor practices. Once upon a time, it was considered OK to casually treat None and False as similar. However, since Python 2.2 this is not the best policy.

First, when you do an if x or if not x kind of test, Python has to implicitly convert x to boolean. The rules for the bool function describe a raft of things which are False; everything else is True. If the value of x wasn’t properly boolean to begin with, this implicit conversion isn’t really the clearest way to say things.

Before Python 2.2, there was no bool function, so it was even less clear.

Second, you shouldn’t really test with == None. You should use is None and is not None.

See PEP 8, Style Guide for Python Code.

- Comparisons to singletons like None should always be done with
  'is' or 'is not', never the equality operators.

  Also, beware of writing "if x" when you really mean "if x is not None"
  -- e.g. when testing whether a variable or argument that defaults to
  None was set to some other value.  The other value might have a type
  (such as a container) that could be false in a boolean context!

How many singletons are there? Five: None, True, False, NotImplemented and Ellipsis. Since you’re really unlikely to use NotImplemented or Ellipsis, and you would never say if x is True (because simply if x is a lot clearer), you’ll only ever test None.


回答 2

因为None不是唯一被认为是错误的事情。

if not False:
    print "False is false."
if not 0:
    print "0 is false."
if not []:
    print "An empty list is false."
if not ():
    print "An empty tuple is false."
if not {}:
    print "An empty dict is false."
if not "":
    print "An empty string is false."

False0()[]{}""是从各个不同的None,所以你的两段代码是等价的。

此外,请考虑以下事项:

>>> False == 0
True
>>> False == ()
False

if object:不是相等性检查。0()[]None{},等都是彼此各不相同,但他们都评价为False。

这是短路表达式背后的“魔术”:

foo = bar and spam or eggs

简写为:

if bar:
    foo = spam
else:
    foo = eggs

尽管您确实应该写:

foo = spam if bar else egg

Because None is not the only thing that is considered false.

if not False:
    print "False is false."
if not 0:
    print "0 is false."
if not []:
    print "An empty list is false."
if not ():
    print "An empty tuple is false."
if not {}:
    print "An empty dict is false."
if not "":
    print "An empty string is false."

False, 0, (), [], {} and "" are all different from None, so your two code snippets are not equivalent.

Moreover, consider the following:

>>> False == 0
True
>>> False == ()
False

if object: is not an equality check. 0, (), [], None, {}, etc. are all different from each other, but they all evaluate to False.

This is the “magic” behind short circuiting expressions like:

foo = bar and spam or eggs

which is shorthand for:

if bar:
    foo = spam
else:
    foo = eggs

although you really should write:

foo = spam if bar else egg

回答 3

PEP 8 -风格指南Python代码建议使用不是,如果你正在测试无岬

- Comparisons to singletons like None should always be done with
  'is' or 'is not', never the equality operators.

另一方面,如果要测试的不只是无度,则应使用布尔运算符。

PEP 8 — Style Guide for Python Code recommends to use is or is not if you are testing for None-ness

- Comparisons to singletons like None should always be done with
  'is' or 'is not', never the equality operators.

On the other hand if you are testing for more than None-ness, you should use the boolean operator.


回答 4

如果你问

if not spam:
    print "Sorry. No SPAM."

垃圾邮件__nonzero__方法被调用。从Python手册中:

__nonzero__self)调用以实现真值测试,并内置操作bool(); 应该返回False或True,或者它们的等效整数0或1。如果未定义此方法,则调用__len __()(如果已定义)(请参见下文)。如果一个类未定义__len __()或__nonzero __(),则其所有实例均被视为true。

如果你问

if spam == None:
    print "Sorry. No SPAM here either."

使用参数None调用垃圾邮件__eq__方法。

有关自定义可能性的更多信息,请参见https://docs.python.org/reference/datamodel.html#basic-customization上的Python文档。

If you ask

if not spam:
    print "Sorry. No SPAM."

the __nonzero__ method of spam gets called. From the Python manual:

__nonzero__(self) Called to implement truth value testing, and the built-in operation bool(); should return False or True, or their integer equivalents 0 or 1. When this method is not defined, __len__() is called, if it is defined (see below). If a class defines neither __len__() nor __nonzero__(), all its instances are considered true.

If you ask

if spam == None:
    print "Sorry. No SPAM here either."

the __eq__ method of spam gets called with the argument None.

For more information of the customization possibilities have a look at the Python documenation at https://docs.python.org/reference/datamodel.html#basic-customization


回答 5

这两个比较有不同的目的。前者检查某物的布尔值,第二者检查无值的身份。

These two comparisons serve different purposes. The former checks for boolean value of something, the second checks for identity with None value.


回答 6

对于一个例子,第一个例子更短,看起来更好。与其他帖子一样,您选择的内容还取决于您真正想要进行比较的内容。

For one the first example is shorter and looks nicer. As per the other posts what you choose also depends on what you really want to do with the comparison.


回答 7

答案是“取决于”。

在这种情况下,如果我认为0,“”,[]和False(列表不详尽)等效于None,则使用第一个示例。

The answer is “it depends”.

I use the first example if I consider 0, “”, [] and False (list not exhaustive) to be equivalent to None in this context.


回答 8

就个人而言,我选择了一种跨语言的一致方法:if (var)仅当var声明为布尔值时(或如此定义,在C中,我们没有特定的类型),我才做(或等效)。我什至给这些变量加上前缀bbVar实际上是这样),以确保我不会在这里偶然使用其他类型。
我真的不喜欢将隐式强制转换为布尔值,甚至在有许多复杂规则时也是如此。

当然,人们会不同意。有些人走得更远,我if (bVar == true)在工作中的Java代码中看到(对我来说太多余了!),另一些人则喜欢太多紧凑的语法,while (line = getNextLine())对我来说太模棱两可了。

Personally, I chose a consistent approach across languages: I do if (var) (or equivalent) only if var is declared as boolean (or defined as such, in C we don’t have a specific type). I even prefix these variables with a b (so it would be bVar actually) to be sure I won’t accidentally use another type here.
I don’t really like implicit casting to boolean, even less when there are numerous, complex rules.

Of course, people will disagree. Some go farther, I see if (bVar == true) in the Java code at my work (too redundant for my taste!), others love too much compact syntax, going while (line = getNextLine()) (too ambiguous for me).


使用python map和其他功能工具

问题:使用python map和其他功能工具

这相当不错,但是我正在尝试学习/理解python中的函数式编程。如下代码:

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest(foo, bar):
    print foo, bar

map(maptest, foos, bars)

生成:

1.0 1
2.0 2
3.0 3
4.0 None
5.0 None

问:有没有一种方法可以使用map或python中的任何其他功能工具来产生以下内容而没有循环等。

1.0 [1,2,3]
2.0 [1,2,3]
3.0 [1,2,3]
4.0 [1,2,3]
5.0 [1,2,3]

顺便提一下,如果foo和bar之间存在依赖关系,则实现将如何更改。例如

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3,4,5]

并打印:

1.0 [2,3,4,5]
2.0 [1,3,4,5]
3.0 [1,2,4,5]
...

PS:我知道如何使用if,循环和/或生成器天真地做到这一点,但是我想学习如何使用功能性工具来实现这一点。是否只是在maptest中添加if语句或在maptest内部将其他过滤器映射应用于bar的情况?

This is quite n00bish, but I’m trying to learn/understand functional programming in python. The following code:

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest(foo, bar):
    print foo, bar

map(maptest, foos, bars)

produces:

1.0 1
2.0 2
3.0 3
4.0 None
5.0 None

Q. Is there a way to use map or any other functional tools in python to produce the following without loops etc.

1.0 [1,2,3]
2.0 [1,2,3]
3.0 [1,2,3]
4.0 [1,2,3]
5.0 [1,2,3]

Just as a side note how would the implementation change if there is a dependency between foo and bar. e.g.

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3,4,5]

and print:

1.0 [2,3,4,5]
2.0 [1,3,4,5]
3.0 [1,2,4,5]
...

P.S: I know how to do it naively using if, loops and/or generators, but I’d like to learn how to achieve the same using functional tools. Is it just a case of adding an if statement to maptest or apply another filter map to bars internally within maptest?


回答 0

最简单的方法是不bars通过不同的功能,而直接从maptest以下位置访问它:

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest(foo):
    print foo, bars

map(maptest, foos)

使用原始maptest函数,您还可以在map以下代码中使用lambda函数:

map((lambda foo: maptest(foo, bars)), foos)

The easiest way would be not to pass bars through the different functions, but to access it directly from maptest:

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest(foo):
    print foo, bars

map(maptest, foos)

With your original maptest function you could also use a lambda function in map:

map((lambda foo: maptest(foo, bars)), foos)

回答 1

您是否熟悉其他功能语言?即,您是要学习python如何进行函数编程,还是要学习有关函数编程并使用python作为工具?

另外,您了解列表理解吗?

map(f, sequence)

与(*)直接等效:

[f(x) for x in sequence]

实际上,我认为map()曾经打算从python 3.0中删除它是多余的(那没有发生)。

map(f, sequence1, sequence2)

大致等于:

[f(x1, x2) for x1, x2 in zip(sequence1, sequence2)]

(在处理序列长度不同的情况时,它的处理方式有所不同。如您所见,map()当其中一个序列用完时,填入None,而zip()当最短序列停止时,则填满)

因此,为了解决您的特定问题,您尝试产生结果:

foos[0], bars
foos[1], bars
foos[2], bars
# etc.

您可以通过编写一个带有单个参数并打印它的函数,然后加上杠来做到这一点:

def maptest(x):
     print x, bars
map(maptest, foos)

或者,您可以创建一个如下所示的列表:

[bars, bars, bars, ] # etc.

并使用原始的maptest:

def maptest(x, y):
    print x, y

一种方法是事先显式构建列表:

barses = [bars] * len(foos)
map(maptest, foos, barses)

或者,您可以拉入itertools模块。 itertools包含许多巧妙的功能,可帮助您在python中进行功能风格的延迟评估编程。在这种情况下,我们需要itertools.repeat,当您对其进行迭代时,它将无限期地输出其参数。最后一个事实意味着,如果您这样做:

map(maptest, foos, itertools.repeat(bars))

map()只要参数之一仍在产生输出,您就会得到无穷的输出,因为它一直持续下去。但是,itertools.imap就像map(),但最短的可迭代停止就停止。

itertools.imap(maptest, foos, itertools.repeat(bars))

希望这可以帮助 :-)

(*)在python 3.0中有些不同。在那里,map()本质上返回一个生成器表达式。

Are you familiar with other functional languages? i.e. are you trying to learn how python does functional programming, or are you trying to learn about functional programming and using python as the vehicle?

Also, do you understand list comprehensions?

map(f, sequence)

is directly equivalent (*) to:

[f(x) for x in sequence]

In fact, I think map() was once slated for removal from python 3.0 as being redundant (that didn’t happen).

map(f, sequence1, sequence2)

is mostly equivalent to:

[f(x1, x2) for x1, x2 in zip(sequence1, sequence2)]

(there is a difference in how it handles the case where the sequences are of different length. As you saw, map() fills in None when one of the sequences runs out, whereas zip() stops when the shortest sequence stops)

So, to address your specific question, you’re trying to produce the result:

foos[0], bars
foos[1], bars
foos[2], bars
# etc.

You could do this by writing a function that takes a single argument and prints it, followed by bars:

def maptest(x):
     print x, bars
map(maptest, foos)

Alternatively, you could create a list that looks like this:

[bars, bars, bars, ] # etc.

and use your original maptest:

def maptest(x, y):
    print x, y

One way to do this would be to explicitely build the list beforehand:

barses = [bars] * len(foos)
map(maptest, foos, barses)

Alternatively, you could pull in the itertools module. itertools contains many clever functions that help you do functional-style lazy-evaluation programming in python. In this case, we want itertools.repeat, which will output its argument indefinitely as you iterate over it. This last fact means that if you do:

map(maptest, foos, itertools.repeat(bars))

you will get endless output, since map() keeps going as long as one of the arguments is still producing output. However, itertools.imap is just like map(), but stops as soon as the shortest iterable stops.

itertools.imap(maptest, foos, itertools.repeat(bars))

Hope this helps :-)

(*) It’s a little different in python 3.0. There, map() essentially returns a generator expression.


回答 2

这是您要寻找的解决方案:

>>> foos = [1.0, 2.0, 3.0, 4.0, 5.0]
>>> bars = [1, 2, 3]
>>> [(x, bars) for x in foos]
[(1.0, [1, 2, 3]), (2.0, [1, 2, 3]), (3.0, [1, 2, 3]), (4.0, [1, 2, 3]), (5.0, [
1, 2, 3])]

我建议使用列表理解([(x, bars) for x in foos]部分)而不是使用地图,因为它避免了每次迭代时函数调用的开销(这可能非常重要)。如果只打算在for循环中使用它,则可以通过使用生成器理解来获得更好的速度:

>>> y = ((x, bars) for x in foos)
>>> for z in y:
...     print z
...
(1.0, [1, 2, 3])
(2.0, [1, 2, 3])
(3.0, [1, 2, 3])
(4.0, [1, 2, 3])
(5.0, [1, 2, 3])

区别在于生成器理解迟缓地加载

更新 针对此评论:

当然,您知道您不复制栏,所有条目都是相同的栏列表。因此,如果您修改其中的任何一个(包括原始条),那么您将修改所有的它们。

我想这是一个正确的观点。我可以想到两种解决方案。最有效的可能是这样的:

tbars = tuple(bars)
[(x, tbars) for x in foos]

由于元组是不可变的,因此这将防止通过此列表理解的结果(或通过该路线生成器理解)的结果来修改钢筋。如果确实需要修改每个结果,则可以执行以下操作:

from copy import copy
[(x, copy(bars)) for x in foos]

但是,这在内存使用和速度方面都可能会有些昂贵,因此我建议您不要这样做,除非您确实需要添加每个内存。

Here’s the solution you’re looking for:

>>> foos = [1.0, 2.0, 3.0, 4.0, 5.0]
>>> bars = [1, 2, 3]
>>> [(x, bars) for x in foos]
[(1.0, [1, 2, 3]), (2.0, [1, 2, 3]), (3.0, [1, 2, 3]), (4.0, [1, 2, 3]), (5.0, [
1, 2, 3])]

I’d recommend using a list comprehension (the [(x, bars) for x in foos] part) over using map as it avoids the overhead of a function call on every iteration (which can be very significant). If you’re just going to use it in a for loop, you’ll get better speeds by using a generator comprehension:

>>> y = ((x, bars) for x in foos)
>>> for z in y:
...     print z
...
(1.0, [1, 2, 3])
(2.0, [1, 2, 3])
(3.0, [1, 2, 3])
(4.0, [1, 2, 3])
(5.0, [1, 2, 3])

The difference is that the generator comprehension is lazily loaded.

UPDATE In response to this comment:

Of course you know, that you don’t copy bars, all entries are the same bars list. So if you modify any one of them (including original bars), you modify all of them.

I suppose this is a valid point. There are two solutions to this that I can think of. The most efficient is probably something like this:

tbars = tuple(bars)
[(x, tbars) for x in foos]

Since tuples are immutable, this will prevent bars from being modified through the results of this list comprehension (or generator comprehension if you go that route). If you really need to modify each and every one of the results, you can do this:

from copy import copy
[(x, copy(bars)) for x in foos]

However, this can be a bit expensive both in terms of memory usage and in speed, so I’d recommend against it unless you really need to add to each one of them.


回答 3

函数式编程是关于创建无副作用的代码。

map是功能列表转换的抽象。您可以使用它来获取一系列序列并将其转换为其他序列。

您正在尝试将其用作迭代器。不要那样做 :)

这是一个示例,说明如何使用地图构建所需的列表。有较短的解决方案(我只是使用理解力),但这将帮助您了解哪种地图效果更好:

def my_transform_function(input):
    return [input, [1, 2, 3]]

new_list = map(my_transform, input_list)

请注意,此时您仅完成了数据操作。现在您可以打印它:

for n,l in new_list:
    print n, ll

-我不确定“没有循环”是什么意思。fp并不是要避免循环(您无法访问列表中的每个项目都无法对其进行检查)。这是关于避免副作用,从而减少错误。

Functional programming is about creating side-effect-free code.

map is a functional list transformation abstraction. You use it to take a sequence of something and turn it into a sequence of something else.

You are trying to use it as an iterator. Don’t do that. :)

Here is an example of how you might use map to build the list you want. There are shorter solutions (I’d just use comprehensions), but this will help you understand what map does a bit better:

def my_transform_function(input):
    return [input, [1, 2, 3]]

new_list = map(my_transform, input_list)

Notice at this point, you’ve only done a data manipulation. Now you can print it:

for n,l in new_list:
    print n, ll

— I’m not sure what you mean by ‘without loops.’ fp isn’t about avoiding loops (you can’t examine every item in a list without visiting each one). It’s about avoiding side-effects, thus writing fewer bugs.


回答 4

>>> from itertools import repeat
>>> for foo, bars in zip(foos, repeat(bars)):
...     print foo, bars
... 
1.0 [1, 2, 3]
2.0 [1, 2, 3]
3.0 [1, 2, 3]
4.0 [1, 2, 3]
5.0 [1, 2, 3]
>>> from itertools import repeat
>>> for foo, bars in zip(foos, repeat(bars)):
...     print foo, bars
... 
1.0 [1, 2, 3]
2.0 [1, 2, 3]
3.0 [1, 2, 3]
4.0 [1, 2, 3]
5.0 [1, 2, 3]

回答 5

import itertools

foos=[1.0, 2.0, 3.0, 4.0, 5.0]
bars=[1, 2, 3]

print zip(foos, itertools.cycle([bars]))
import itertools

foos=[1.0, 2.0, 3.0, 4.0, 5.0]
bars=[1, 2, 3]

print zip(foos, itertools.cycle([bars]))

回答 6

以下是该map(function, *sequences)函数的参数概述:

  • function 是函数的名称。
  • sequences是任意数量的序列,通常是列表或元组。 map同时迭代它们并将当前值提供给function。这就是为什么序列数应等于函数的参数数的原因。

听起来您正在尝试迭代某些function参数,但保持其他参数不变,但是不幸的map是不支持该参数。我发现一个向Python添加此类功能的旧建议,但是map构造是如此干净且完善,以至于我怀疑这样的东西是否会实现。

像其他人建议的那样,使用诸如全局变量或列表理解之类的解决方法。

Here’s an overview of the parameters to the map(function, *sequences) function:

  • function is the name of your function.
  • sequences is any number of sequences, which are usually lists or tuples. map will iterate over them simultaneously and give the current values to function. That’s why the number of sequences should equal the number of parameters to your function.

It sounds like you’re trying to iterate for some of function‘s parameters but keep others constant, and unfortunately map doesn’t support that. I found an old proposal to add such a feature to Python, but the map construct is so clean and well-established that I doubt something like that will ever be implemented.

Use a workaround like global variables or list comprehensions, as others have suggested.


回答 7

这样可以吗?

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest2(bar):
  print bar

def maptest(foo):
  print foo
  map(maptest2, bars)

map(maptest, foos)

Would this do it?

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest2(bar):
  print bar

def maptest(foo):
  print foo
  map(maptest2, bars)

map(maptest, foos)

回答 8

这个怎么样:

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest(foo, bar):
    print foo, bar

map(maptest, foos, [bars]*len(foos))

How about this:

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest(foo, bar):
    print foo, bar

map(maptest, foos, [bars]*len(foos))