标签归档：chained-assignment

熊猫使用什么规则生成视图与副本？

2021年8月17日 Python实用宝典

问题：熊猫使用什么规则生成视图与副本？

我对Pandas决定从数据框中进行选择是原始数据框的副本或原始数据视图时使用的规则感到困惑。

例如，如果我有

df = pd.DataFrame(np.random.randn(8,8), columns=list('ABCDEFGH'), index=range(1,9))

我了解a会query传回副本，因此类似

foo = df.query('2 < index <= 5')
foo.loc[:,'E'] = 40

将对原始数据帧无效df。我也了解标量或命名切片返回一个视图，因此对它们的赋值（例如

df.iloc[3] = 70

要么

df.ix[1,'B':'E'] = 222

会改变df。但是当涉及到更复杂的案件时，我迷失了。例如，

df[df.C <= df.B] = 7654321

变化df，但是

df[df.C <= df.B].ix[:,'B':'E']

才不是。

是否有一个熊猫正在使用的简单规则，我只是想念它？在这些特定情况下发生了什么；尤其是，如何更改满足特定查询的数据框中的所有值（或值的子集）（就像我在上面的最后一个示例中尝试的那样）？

注意：这和这个问题不一样；并且我已经阅读了文档，但并未对此有所启发。我还阅读了有关此主题的“相关”问题，但我仍然缺少Pandas使用的简单规则，以及如何将其应用于（例如）修改值（或值的子集）在满足特定查询的数据框中。

I’m confused about the rules Pandas uses when deciding that a selection from a dataframe is a copy of the original dataframe, or a view on the original.

If I have, for example,

df = pd.DataFrame(np.random.randn(8,8), columns=list('ABCDEFGH'), index=range(1,9))

I understand that a query returns a copy so that something like

foo = df.query('2 < index <= 5')
foo.loc[:,'E'] = 40

will have no effect on the original dataframe, df. I also understand that scalar or named slices return a view, so that assignments to these, such as

df.iloc[3] = 70

df.ix[1,'B':'E'] = 222

will change df. But I’m lost when it comes to more complicated cases. For example,

df[df.C <= df.B] = 7654321

changes df, but

df[df.C <= df.B].ix[:,'B':'E']

does not.

Is there a simple rule that Pandas is using that I’m just missing? What’s going on in these specific cases; and in particular, how do I change all values (or a subset of values) in a dataframe that satisfy a particular query (as I’m attempting to do in the last example above)?

Note: This is not the same as this question; and I have read the documentation, but am not enlightened by it. I’ve also read through the “Related” questions on this topic, but I’m still missing the simple rule Pandas is using, and how I’d apply it to — for example — modify the values (or a subset of values) in a dataframe that satisfy a particular query.

回答 0

这是规则，其后是覆盖：

所有操作都会生成一个副本
如果inplace=True提供，它将原位修改；只有一些操作支持这一点
设置的索引器，例如.loc/.iloc/.iat/.at将原地设置。
到达单一类型对象的索引器几乎总是一个视图（取决于内存布局，这可能不是原因，这不可靠）。这主要是为了提高效率。（上面的示例用于.query；它将始终返回的副本，其值为numexpr）
到达多类型对象的索引器始终是副本。

您的例子 chained indexing

df[df.C <= df.B].loc[:,'B':'E']

不能保证能正常工作（因此您不应该这样做）。

而是：

df.loc[df.C <= df.B, 'B':'E']

因为这更快，并且将始终有效

链式索引是2个单独的python操作，因此无法可靠地被熊猫拦截（您通常会得到SettingWithCopyWarning，但也不是100％可检测到的）。您所指出的dev文档提供了更全面的说明。

Here’s the rules, subsequent override:

All operations generate a copy
If inplace=True is provided, it will modify in-place; only some operations support this
An indexer that sets, e.g. .loc/.iloc/.iat/.at will set inplace.
An indexer that gets on a single-dtyped object is almost always a view (depending on the memory layout it may not be that’s why this is not reliable). This is mainly for efficiency. (the example from above is for .query; this will always return a copy as its evaluated by numexpr)
An indexer that gets on a multiple-dtyped object is always a copy.

Your example of chained indexing

df[df.C <= df.B].loc[:,'B':'E']

is not guaranteed to work (and thus you shoulld never do this).

Instead do:

df.loc[df.C <= df.B, 'B':'E']

as this is faster and will always work

The chained indexing is 2 separate python operations and thus cannot be reliably intercepted by pandas (you will oftentimes get a SettingWithCopyWarning, but that is not 100% detectable either). The dev docs, which you pointed, offer a much more full explanation.

知识问答

我为什么要在熊猫中复制数据框

2021年8月2日 Python实用宝典

问题：我为什么要在熊猫中复制数据框

当从父数据帧中选择子数据帧时，我注意到一些程序员使用该.copy()方法复制数据帧。

他们为什么要复制数据框？如果我不复制怎么办？

When selecting a sub dataframe from a parent dataframe, I noticed that some programmers make a copy of the data frame using the .copy() method. For example,

X = my_dataframe[features_list].copy()

…instead of just

X = my_dataframe[features_list]

Why are they making a copy of the data frame? What will happen if I don’t make a copy?

回答 0

这扩展了保罗的答案。在Pandas中，对DataFrame进行索引将返回对初始DataFrame的引用。因此，更改子集将更改初始DataFrame。因此，如果要确保不更改初始DataFrame，则需要使用该副本。考虑以下代码：

df = DataFrame({'x': [1,2]})
df_sub = df[0:1]
df_sub.x = -1
print(df)

你会得到：

x
0 -1
1  2

相反，以下内容使df保持不变：

df_sub_copy = df[0:1].copy()
df_sub_copy.x = -1

This expands on Paul’s answer. In Pandas, indexing a DataFrame returns a reference to the initial DataFrame. Thus, changing the subset will change the initial DataFrame. Thus, you’d want to use the copy if you want to make sure the initial DataFrame shouldn’t change. Consider the following code:

df = DataFrame({'x': [1,2]})
df_sub = df[0:1]
df_sub.x = -1
print(df)

You’ll get:

x
0 -1
1  2

In contrast, the following leaves df unchanged:

df_sub_copy = df[0:1].copy()
df_sub_copy.x = -1

回答 1

因为如果您不进行复制，那么即使您将dataFrame分配给其他名称，索引仍然可以在其他地方进行操作。

例如：

df2 = df
func1(df2)
func2(df)

func1可以通过修改df2来修改df，因此要避免这种情况：

df2 = df.copy()
func1(df2)
func2(df)

Because if you don’t make a copy then the indices can still be manipulated elsewhere even if you assign the dataFrame to a different name.

For example:

df2 = df
func1(df2)
func2(df)

func1 can modify df by modifying df2, so to avoid that:

df2 = df.copy()
func1(df2)
func2(df)

回答 2

必须提到返回的副本或视图取决于索引的类型。

大熊猫文档说：

返回视图与副本

关于何时返回数据视图的规则完全取决于NumPy。每当索引操作涉及标签数组或布尔向量时，结果将是副本。使用单个标签/标量索引和切片，例如df.ix [3：6]或df.ix [:,’A’]，将返回视图。

It’s necessary to mention that returning copy or view depends on kind of indexing.

The pandas documentation says:

Returning a view versus a copy

The rules about when a view on the data is returned are entirely dependent on NumPy. Whenever an array of labels or a boolean vector are involved in the indexing operation, the result will be a copy. With single label / scalar indexing and slicing, e.g. df.ix[3:6] or df.ix[:, ‘A’], a view will be returned.

回答 3

主要目的是避免链接索引并消除SettingWithCopyWarning。

在这里，链式索引就像 dfc['A'][0] = 111

该文件说，在返回视图或副本时，应避免链接索引。这是该文档中经过稍微修改的示例：

In [1]: import pandas as pd

In [2]: dfc = pd.DataFrame({'A':['aaa','bbb','ccc'],'B':[1,2,3]})

In [3]: dfc
Out[3]:
    A   B
0   aaa 1
1   bbb 2
2   ccc 3

In [4]: aColumn = dfc['A']

In [5]: aColumn[0] = 111
SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

In [6]: dfc
Out[6]:
    A   B
0   111 1
1   bbb 2
2   ccc 3

这aColumn是一个视图，而不是原始DataFrame的副本，因此修改aColumn也将导致原始数据dfc被修改。接下来，如果我们首先索引该行：

In [7]: zero_row = dfc.loc[0]

In [8]: zero_row['A'] = 222
SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

In [9]: dfc
Out[9]:
    A   B
0   111 1
1   bbb 2
2   ccc 3

这次zero_row是副本，因此原始文件dfc没有被修改。

从上面的两个示例中，我们可以看出是否要更改原始DataFrame是不明确的。如果您编写以下内容，则尤其危险：

In [10]: dfc.loc[0]['A'] = 333
SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

In [11]: dfc
Out[11]:
    A   B
0   111 1
1   bbb 2
2   ccc 3

这次根本没有用。在这里我们想要更改dfc，但实际上我们修改了一个中间值dfc.loc[0]，该中间值是一个副本，被立即丢弃。很难预测中间值是dfc.loc[0]还是dfc['A']视图或副本，因此无法保证是否会更新原始DataFrame。这就是为什么应该避免链接索引的原因，而pandas会SettingWithCopyWarning为这种链接索引更新生成。

现在是的使用.copy()。要消除该警告，请复制一份以明确表达您的意图：

In [12]: zero_row_copy = dfc.loc[0].copy()

In [13]: zero_row_copy['A'] = 444 # This time no warning

由于您正在修改副本，因此您知道原件dfc永远不会更改，并且您不希望更改。您的期望与行为匹配，然后SettingWithCopyWarning消失。

注意，如果确实要修改原始DataFrame，则文档建议您使用loc：

In [14]: dfc.loc[0,'A'] = 555

In [15]: dfc
Out[15]:
    A   B
0   555 1
1   bbb 2
2   ccc 3

The primary purpose is to avoid chained indexing and eliminate the SettingWithCopyWarning.

Here chained indexing is something like dfc['A'][0] = 111

The document said chained indexing should be avoided in Returning a view versus a copy. Here is a slightly modified example from that document:

In [1]: import pandas as pd

In [2]: dfc = pd.DataFrame({'A':['aaa','bbb','ccc'],'B':[1,2,3]})

In [3]: dfc
Out[3]:
    A   B
0   aaa 1
1   bbb 2
2   ccc 3

In [4]: aColumn = dfc['A']

In [5]: aColumn[0] = 111
SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

In [6]: dfc
Out[6]:
    A   B
0   111 1
1   bbb 2
2   ccc 3

Here the aColumn is a view and not a copy from the original DataFrame, so modifying aColumn will cause the original dfc be modified too. Next, if we index the row first:

In [7]: zero_row = dfc.loc[0]

In [8]: zero_row['A'] = 222
SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

In [9]: dfc
Out[9]:
    A   B
0   111 1
1   bbb 2
2   ccc 3

This time zero_row is a copy, so the original dfc is not modified.

From these two examples above, we see it’s ambiguous whether or not you want to change the original DataFrame. This is especially dangerous if you write something like the following:

In [10]: dfc.loc[0]['A'] = 333
SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

In [11]: dfc
Out[11]:
    A   B
0   111 1
1   bbb 2
2   ccc 3

This time it didn’t work at all. Here we wanted to change dfc, but we actually modified an intermediate value dfc.loc[0] that is a copy and is discarded immediately. It’s very hard to predict whether the intermediate value like dfc.loc[0] or dfc['A'] is a view or a copy, so it’s not guaranteed whether or not original DataFrame will be updated. That’s why chained indexing should be avoided, and pandas generates the SettingWithCopyWarning for this kind of chained indexing update.

Now is the use of .copy(). To eliminate the warning, make a copy to express your intention explicitly:

In [12]: zero_row_copy = dfc.loc[0].copy()

In [13]: zero_row_copy['A'] = 444 # This time no warning

Since you are modifying a copy, you know the original dfc will never change and you are not expecting it to change. Your expectation matches the behavior, then the SettingWithCopyWarning disappears.

Note, If you do want to modify the original DataFrame, the document suggests you use loc:

In [14]: dfc.loc[0,'A'] = 555

In [15]: dfc
Out[15]:
    A   B
0   555 1
1   bbb 2
2   ccc 3

回答 4

通常，对副本进行处理比对原始数据帧进行处理更为安全，除非您知道不再需要原始文件并希望继续使用可操纵的版本。通常，原始数据帧仍然可以与操纵版本进行比较，等等。因此，大多数人都在处理副本并最终合并。

In general it is safer to work on copies than on original data frames, except when you know that you won’t be needing the original anymore and want to proceed with the manipulated version. Normally, you would still have some use for the original data frame to compare with the manipulated version, etc. Therefore, most people work on copies and merge at the end.

回答 5

假设您具有以下数据框

df1
     A    B    C    D
4 -1.0 -1.0 -1.0 -1.0
5 -1.0 -1.0 -1.0 -1.0
6 -1.0 -1.0 -1.0 -1.0
6 -1.0 -1.0 -1.0 -1.0

当您想要创建df2与相同的其他对象时df1，无需copy

df2=df1
df2
     A    B    C    D
4 -1.0 -1.0 -1.0 -1.0
5 -1.0 -1.0 -1.0 -1.0
6 -1.0 -1.0 -1.0 -1.0
6 -1.0 -1.0 -1.0 -1.0

并只想如下修改df2值

df2.iloc[0,0]='changed'

df2
         A    B    C    D
4  changed -1.0 -1.0 -1.0
5       -1 -1.0 -1.0 -1.0
6       -1 -1.0 -1.0 -1.0
6       -1 -1.0 -1.0 -1.0

同时df1也要更改

df1
         A    B    C    D
4  changed -1.0 -1.0 -1.0
5       -1 -1.0 -1.0 -1.0
6       -1 -1.0 -1.0 -1.0
6       -1 -1.0 -1.0 -1.0

由于两个df相同object，因此我们可以使用id

id(df1)
140367679979600
id(df2)
140367679979600

因此，它们作为同一对象，一个更改另一个对象也将传递相同的值。

如果我们添加copy，和现在df1并且df2被认为是不同的object，如果我们对其中一个进行相同的更改，则另一个不会更改。

df2=df1.copy()
id(df1)
140367679979600
id(df2)
140367674641232

df1.iloc[0,0]='changedback'
df2
         A    B    C    D
4  changed -1.0 -1.0 -1.0
5       -1 -1.0 -1.0 -1.0
6       -1 -1.0 -1.0 -1.0
6       -1 -1.0 -1.0 -1.0

值得一提的是，当您对原始数据帧进行子集设置时，也可以安全地添加副本，以避免 SettingWithCopyWarning

Assumed you have data frame as below

df1
     A    B    C    D
4 -1.0 -1.0 -1.0 -1.0
5 -1.0 -1.0 -1.0 -1.0
6 -1.0 -1.0 -1.0 -1.0
6 -1.0 -1.0 -1.0 -1.0

When you would like create another df2 which is identical to df1, without copy

df2=df1
df2
     A    B    C    D
4 -1.0 -1.0 -1.0 -1.0
5 -1.0 -1.0 -1.0 -1.0
6 -1.0 -1.0 -1.0 -1.0
6 -1.0 -1.0 -1.0 -1.0

And would like modify the df2 value only as below

df2.iloc[0,0]='changed'

df2
         A    B    C    D
4  changed -1.0 -1.0 -1.0
5       -1 -1.0 -1.0 -1.0
6       -1 -1.0 -1.0 -1.0
6       -1 -1.0 -1.0 -1.0

At the same time the df1 is changed as well

df1
         A    B    C    D
4  changed -1.0 -1.0 -1.0
5       -1 -1.0 -1.0 -1.0
6       -1 -1.0 -1.0 -1.0
6       -1 -1.0 -1.0 -1.0

Since two df as same object, we can check it by using the id

id(df1)
140367679979600
id(df2)
140367679979600

So they as same object and one change another one will pass the same value as well.

If we add the copy, and now df1 and df2 are considered as different object, if we do the same change to one of them the other will not change.

df2=df1.copy()
id(df1)
140367679979600
id(df2)
140367674641232

df1.iloc[0,0]='changedback'
df2
         A    B    C    D
4  changed -1.0 -1.0 -1.0
5       -1 -1.0 -1.0 -1.0
6       -1 -1.0 -1.0 -1.0
6       -1 -1.0 -1.0 -1.0

Good to mention, when you subset the original dataframe, it is safe to add the copy as well in order to avoid the SettingWithCopyWarning

知识问答

将特定的选定列提取到新DataFrame中作为副本

2021年8月2日 Python实用宝典

问题：将特定的选定列提取到新DataFrame中作为副本

我有一个带有4列的pandas DataFrame，我想创建一个只有三个列的新 DataFrame 。这个问题类似于：从数据框中提取特定的列，但对于不是R的熊猫来说。以下代码不起作用，会引发错误，并且肯定不是熊猫的方式。

import pandas as pd
old = pd.DataFrame({'A' : [4,5], 'B' : [10,20], 'C' : [100,50], 'D' : [-30,-50]})
new = pd.DataFrame(zip(old.A, old.C, old.D)) # raises TypeError: data argument can't be an iterator

熊猫式的做法是什么？

I have a pandas DataFrame with 4 columns and I want to create a new DataFrame that only has three of the columns. This question is similar to: Extracting specific columns from a data frame but for pandas not R. The following code does not work, raises an error, and is certainly not the pandasnic way to do it.

import pandas as pd
old = pd.DataFrame({'A' : [4,5], 'B' : [10,20], 'C' : [100,50], 'D' : [-30,-50]})
new = pd.DataFrame(zip(old.A, old.C, old.D)) # raises TypeError: data argument can't be an iterator

What is the pandasnic way to do it?

回答 0

有一种方法可以做到，它实际上看起来类似于R

new = old[['A', 'C', 'D']].copy()

在这里，您只是从原始数据框中选择所需的列，并为这些列创建变量。如果您想完全修改新的数据框，则可能要.copy()避免使用SettingWithCopyWarning。

另一种方法是使用filter默认创建副本的方法：

new = old.filter(['A','B','D'], axis=1)

最后，根据原始数据框中的列数，使用a表示它可能更简洁drop（默认情况下也会创建一个副本）：

new = old.drop('B', axis=1)

There is a way of doing this and it actually looks similar to R

new = old[['A', 'C', 'D']].copy()

Here you are just selecting the columns you want from the original data frame and creating a variable for those. If you want to modify the new dataframe at all you’ll probably want to use .copy() to avoid a SettingWithCopyWarning.

An alternative method is to use filter which will create a copy by default:

new = old.filter(['A','B','D'], axis=1)

Finally, depending on the number of columns in your original dataframe, it might be more succinct to express this using a drop (this will also create a copy by default):

new = old.drop('B', axis=1)

回答 1

最简单的方法是

new = old[['A','C','D']]

。

The easiest way is

new = old[['A','C','D']]

回答 2

另一个更简单的方法似乎是：

new = pd.DataFrame([old.A, old.B, old.C]).transpose()

哪里old.column_name会给你一系列。列出所有要保留的列系列，并将其传递给DataFrame构造函数。我们需要进行转置来调整形状。

In [14]:pd.DataFrame([old.A, old.B, old.C]).transpose()
Out[14]: 
   A   B    C
0  4  10  100
1  5  20   50

Another simpler way seems to be:

new = pd.DataFrame([old.A, old.B, old.C]).transpose()

where old.column_name will give you a series. Make a list of all the column-series you want to retain and pass it to the DataFrame constructor. We need to do a transpose to adjust the shape.

In [14]:pd.DataFrame([old.A, old.B, old.C]).transpose()
Out[14]: 
   A   B    C
0  4  10  100
1  5  20   50

回答 3

通用功能形式

def select_columns(data_frame, column_names):
    new_frame = data_frame.loc[:, column_names]
    return new_frame

专门针对您上面的问题

selected_columns = ['A', 'C', 'D']
new = select_columns(old, selected_columns)

Generic functional form

def select_columns(data_frame, column_names):
    new_frame = data_frame.loc[:, column_names]
    return new_frame

Specific for your problem above

selected_columns = ['A', 'C', 'D']
new = select_columns(old, selected_columns)

回答 4

如果您想要一个新的数据框，则：

import pandas as pd
old = pd.DataFrame({'A' : [4,5], 'B' : [10,20], 'C' : [100,50], 'D' : [-30,-50]})
new=  old[['A', 'C', 'D']]

If you want to have a new data frame then:

import pandas as pd
old = pd.DataFrame({'A' : [4,5], 'B' : [10,20], 'C' : [100,50], 'D' : [-30,-50]})
new=  old[['A', 'C', 'D']]

回答 5

据我所知，使用过滤器功能时不一定需要指定轴。

new = old.filter(['A','B','D'])

返回与相同的数据框

new = old.filter(['A','B','D'], axis=1)

As far as I can tell, you don’t necessarily need to specify the axis when using the filter function.

new = old.filter(['A','B','D'])

returns the same dataframe as

new = old.filter(['A','B','D'], axis=1)

回答 6

按索引列：

# selected column index: 1, 6, 7
new = old.iloc[: , [1, 6, 7]].copy()

columns by index:

# selected column index: 1, 6, 7
new = old.iloc[: , [1, 6, 7]].copy()

知识问答

如何处理熊猫中的SettingWithCopyWarning？

2021年7月25日 Python实用宝典

问题：如何处理熊猫中的SettingWithCopyWarning？

背景

我刚刚将熊猫从0.11升级到0.13.0rc1。现在，该应用程序弹出了许多新警告。其中之一是这样的：

E:\FinReporter\FM_EXT.py:449: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_index,col_indexer] = value instead
  quote_df['TVol']   = quote_df['TVol']/TVOL_SCALE

我想知道到底是什么意思？我需要改变什么吗？

如果我坚持使用该如何警告quote_df['TVol'] = quote_df['TVol']/TVOL_SCALE？

产生错误的功能

def _decode_stock_quote(list_of_150_stk_str):
    """decode the webpage and return dataframe"""

    from cStringIO import StringIO

    str_of_all = "".join(list_of_150_stk_str)

    quote_df = pd.read_csv(StringIO(str_of_all), sep=',', names=list('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefg')) #dtype={'A': object, 'B': object, 'C': np.float64}
    quote_df.rename(columns={'A':'STK', 'B':'TOpen', 'C':'TPCLOSE', 'D':'TPrice', 'E':'THigh', 'F':'TLow', 'I':'TVol', 'J':'TAmt', 'e':'TDate', 'f':'TTime'}, inplace=True)
    quote_df = quote_df.ix[:,[0,3,2,1,4,5,8,9,30,31]]
    quote_df['TClose'] = quote_df['TPrice']
    quote_df['RT']     = 100 * (quote_df['TPrice']/quote_df['TPCLOSE'] - 1)
    quote_df['TVol']   = quote_df['TVol']/TVOL_SCALE
    quote_df['TAmt']   = quote_df['TAmt']/TAMT_SCALE
    quote_df['STK_ID'] = quote_df['STK'].str.slice(13,19)
    quote_df['STK_Name'] = quote_df['STK'].str.slice(21,30)#.decode('gb2312')
    quote_df['TDate']  = quote_df.TDate.map(lambda x: x[0:4]+x[5:7]+x[8:10])

    return quote_df

Background

I just upgraded my Pandas from 0.11 to 0.13.0rc1. Now, the application is popping out many new warnings. One of them like this:

E:\FinReporter\FM_EXT.py:449: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_index,col_indexer] = value instead
  quote_df['TVol']   = quote_df['TVol']/TVOL_SCALE

I want to know what exactly it means? Do I need to change something?

How should I suspend the warning if I insist to use quote_df['TVol'] = quote_df['TVol']/TVOL_SCALE?

The function that gives errors

def _decode_stock_quote(list_of_150_stk_str):
    """decode the webpage and return dataframe"""

    from cStringIO import StringIO

    str_of_all = "".join(list_of_150_stk_str)

    quote_df = pd.read_csv(StringIO(str_of_all), sep=',', names=list('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefg')) #dtype={'A': object, 'B': object, 'C': np.float64}
    quote_df.rename(columns={'A':'STK', 'B':'TOpen', 'C':'TPCLOSE', 'D':'TPrice', 'E':'THigh', 'F':'TLow', 'I':'TVol', 'J':'TAmt', 'e':'TDate', 'f':'TTime'}, inplace=True)
    quote_df = quote_df.ix[:,[0,3,2,1,4,5,8,9,30,31]]
    quote_df['TClose'] = quote_df['TPrice']
    quote_df['RT']     = 100 * (quote_df['TPrice']/quote_df['TPCLOSE'] - 1)
    quote_df['TVol']   = quote_df['TVol']/TVOL_SCALE
    quote_df['TAmt']   = quote_df['TAmt']/TAMT_SCALE
    quote_df['STK_ID'] = quote_df['STK'].str.slice(13,19)
    quote_df['STK_Name'] = quote_df['STK'].str.slice(21,30)#.decode('gb2312')
    quote_df['TDate']  = quote_df.TDate.map(lambda x: x[0:4]+x[5:7]+x[8:10])

    return quote_df

More error messages

E:\FinReporter\FM_EXT.py:449: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_index,col_indexer] = value instead
  quote_df['TVol']   = quote_df['TVol']/TVOL_SCALE
E:\FinReporter\FM_EXT.py:450: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_index,col_indexer] = value instead
  quote_df['TAmt']   = quote_df['TAmt']/TAMT_SCALE
E:\FinReporter\FM_EXT.py:453: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_index,col_indexer] = value instead
  quote_df['TDate']  = quote_df.TDate.map(lambda x: x[0:4]+x[5:7]+x[8:10])

回答 0

在SettingWithCopyWarning被创造的标志可能造成混淆的“链接”的任务，比如下面这并不总是按预期方式工作，特别是当第一选择返回一个副本。[ 有关背景讨论，请参见GH5390和GH5597。]

df[df['A'] > 2]['B'] = new_val  # new_val not set in df

该警告提出了如下重写建议：

df.loc[df['A'] > 2, 'B'] = new_val

但是，这不适合您的用法，相当于：

df = df[df['A'] > 2]
df['B'] = new_val

显然，您不关心将其写回到原始帧的写操作（因为您正在覆盖对它的引用），但是不幸的是，这种模式无法与第一个链式分配示例区分开。因此，（误报）警告。如果您想进一步阅读，可能会在建立索引的文档中解决误报的可能性。您可以通过以下分配安全地禁用此新警告。

import pandas as pd
pd.options.mode.chained_assignment = None  # default='warn'

The SettingWithCopyWarning was created to flag potentially confusing “chained” assignments, such as the following, which does not always work as expected, particularly when the first selection returns a copy. [see GH5390 and GH5597 for background discussion.]

df[df['A'] > 2]['B'] = new_val  # new_val not set in df

The warning offers a suggestion to rewrite as follows:

df.loc[df['A'] > 2, 'B'] = new_val

However, this doesn’t fit your usage, which is equivalent to:

df = df[df['A'] > 2]
df['B'] = new_val

While it’s clear that you don’t care about writes making it back to the original frame (since you are overwriting the reference to it), unfortunately this pattern cannot be differentiated from the first chained assignment example. Hence the (false positive) warning. The potential for false positives is addressed in the docs on indexing, if you’d like to read further. You can safely disable this new warning with the following assignment.

import pandas as pd
pd.options.mode.chained_assignment = None  # default='warn'

回答 1

SettingWithCopyWarning熊猫如何应对？

这篇文章的读者对象是：

想了解此警告的含义
想了解抑制此警告的不同方法
想了解如何改进其代码并遵循良好做法，以避免将来出现此警告。

设定

np.random.seed(0)
df = pd.DataFrame(np.random.choice(10, (3, 5)), columns=list('ABCDE'))
df
   A  B  C  D  E
0  5  0  3  3  7
1  9  3  5  2  4
2  7  6  8  8  1

什么是`SettingWithCopyWarning`？

要知道如何处理此警告，重要的是要理解它的含义以及为什么首先提出它。

过滤DataFrame时，可以对帧进行切片/索引以返回一个视图或copy，具体取决于内部布局和各种实现细节。顾名思义，“视图”是原始数据的视图，因此修改视图可能会修改原始对象。另一方面，“副本”是原始数据的复制，修改副本不会影响原始数据。

如其他答案所述SettingWithCopyWarning，创建时会标记“链接分配”操作。df在上面的设置中考虑。假设您要选择“ B”列中的所有值，其中“ A”列中的值>5。Pandas允许您以不同的方式执行此操作，其中某些方法比其他方法更正确。例如，

df[df.A > 5]['B']

1    3
2    6
Name: B, dtype: int64

和，

df.loc[df.A > 5, 'B']

1    3
2    6
Name: B, dtype: int64

这些返回相同的结果，因此，如果您仅读取这些值，则没有区别。那么，问题是什么呢？链式分配的问题在于，通常很难预测是否返回视图或副本，因此在尝试分配回值时，这在很大程度上成为一个问题。为了建立在前面的示例上，请考虑解释器如何执行此代码：

df.loc[df.A > 5, 'B'] = 4
# becomes
df.__setitem__((df.A > 5, 'B'), 4)

只需__setitem__调用一次即可df。OTOH，请考虑以下代码：

df[df.A > 5]['B'] = 4
# becomes
df.__getitem__(df.A > 5).__setitem__('B", 4)

现在，根据__getitem__返回的视图还是副本，__setitem__操作可能不起作用。

通常，您应将其loc用于基于标签的分配以及iloc基于整数/位置的分配，因为该规范保证它们始终在原始文件上运行。此外，要设置单个单元格，应使用at和iat。

可以在文档中找到更多信息。

注意使用进行的
所有布尔索引操作loc也可以使用进行iloc。唯一的区别是iloc期望索引的整数/位置或布尔值的numpy数组，以及列的整数/位置索引。

例如，
df.loc[df.A > 5, 'B'] = 4
可以写成nas
df.iloc[(df.A > 5).values, 1] = 4
和，
df.loc[1, 'A'] = 100
可以写成
df.iloc[1, 0] = 100
等等。

告诉我如何抑制警告！

考虑对的“ A”列进行的简单操作df。选择“ A”并除以2将发出警告，但该操作将起作用。

df2 = df[['A']]
df2['A'] /= 2
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/IPython/__main__.py:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

df2
     A
0  2.5
1  4.5
2  3.5

有两种方法可以直接静默此警告：

做一个 deepcopy

df2 = df[['A']].copy(deep=True)
df2['A'] /= 2

更改pd.options.mode.chained_assignment
可以设置为None，"warn"或"raise"。"warn"是默认值。None将完全抑制警告，并"raise"抛出SettingWithCopyError，阻止操作进行。
```
pd.options.mode.chained_assignment = None
df2['A'] /= 2
```

@Peter Cotton在评论中提出了一种不错的方法，即使用上下文管理器以非侵入方式更改模式（从此要点修改），仅在需要时才设置模式，然后将其重置为完成后的原始状态。

class ChainedAssignent:
    def __init__(self, chained=None):
        acceptable = [None, 'warn', 'raise']
        assert chained in acceptable, "chained must be in " + str(acceptable)
        self.swcw = chained

    def __enter__(self):
        self.saved_swcw = pd.options.mode.chained_assignment
        pd.options.mode.chained_assignment = self.swcw
        return self

    def __exit__(self, *args):
        pd.options.mode.chained_assignment = self.saved_swcw

用法如下：

# some code here
with ChainedAssignent():
    df2['A'] /= 2
# more code follows

或者，引发异常

with ChainedAssignent(chained='raise'):
    df2['A'] /= 2

SettingWithCopyError: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

“ XY问题”：我在做什么错？

很多时候，用户试图寻找抑制此异常的方法而没有完全理解为什么首先出现该异常。这是XY问题的一个很好的示例，用户尝试解决问题“ Y”，这实际上是根源问题“ X”的症状。将根据遇到此警告的常见问题提出问题，然后提出解决方案。

问题1
我有一个DataFrame

df
       A  B  C  D  E
    0  5  0  3  3  7
    1  9  3  5  2  4
    2  7  6  8  8  1

我想为“ A”> 5到1000分配值。我的预期输出是

      A  B  C  D  E
0     5  0  3  3  7
1  1000  3  5  2  4
2  1000  6  8  8  1

错误的方法：

df.A[df.A > 5] = 1000         # works, because df.A returns a view
df[df.A > 5]['A'] = 1000      # does not work
df.loc[df.A  5]['A'] = 1000   # does not work

正确使用方法loc：

df.loc[df.A > 5, 'A'] = 1000

问题2 ¹
我正在尝试将单元格（1，’D’）中的值设置为12345。我的预期输出是
   A  B  C      D  E
0  5  0  3      3  7
1  9  3  5  12345  4
2  7  6  8      8  1
我尝试了多种访问此单元格的方法，例如 df['D'][1]。做这个的最好方式是什么？

_{1.这个问题与警告并不特别相关，但是最好了解如何正确执行此特定操作，以避免将来可能出现警告的情况。}

您可以使用以下任何一种方法来执行此操作。

df.loc[1, 'D'] = 12345
df.iloc[1, 3] = 12345
df.at[1, 'D'] = 12345
df.iat[1, 3] = 12345

问题3
我试图根据某些条件对值进行子集化。我有一个DataFrame
   A  B  C  D  E
1  9  3  5  2  4
2  7  6  8  8  1
我想将“ D”中的值分配给123，以使“ C” ==5。我尝试过
df2.loc[df2.C == 5, 'D'] = 123
看起来不错，但我仍然可以 SettingWithCopyWarning！我该如何解决？

实际上，这可能是因为您的管道中的代码更高。您是否df2从更大的事物（例如

df2 = df[df.A > 5]

？在这种情况下，布尔索引将返回一个视图，因此df2将引用原始视图。您需要做的就是分配df2一个副本：

df2 = df[df.A > 5].copy()
# Or,
# df2 = df.loc[df.A > 5, :]

问题4
我试图将列“ C”从
   A  B  C  D  E
1  9  3  5  2  4
2  7  6  8  8  1
但是使用
df2.drop('C', axis=1, inplace=True)
抛出SettingWithCopyWarning。为什么会这样呢？

这是因为df2必须已通过其他切片操作将其创建为视图，例如

df2 = df[df.A > 5]

这里的解决方案是要么做copy()的df，或使用loc，如前。

How to deal with SettingWithCopyWarning in Pandas?

This post is meant for readers who,

Would like to understand what this warning means
Would like to understand different ways of suppressing this warning
Would like to understand how to improve their code and follow good practices to avoid this warning in the future.

Setup

np.random.seed(0)
df = pd.DataFrame(np.random.choice(10, (3, 5)), columns=list('ABCDE'))
df
   A  B  C  D  E
0  5  0  3  3  7
1  9  3  5  2  4
2  7  6  8  8  1

What is the `SettingWithCopyWarning`?

To know how to deal with this warning, it is important to understand what it means and why it is raised in the first place.

When filtering DataFrames, it is possible slice/index a frame to return either a view, or a copy, depending on the internal layout and various implementation details. A “view” is, as the term suggests, a view into the original data, so modifying the view may modify the original object. On the other hand, a “copy” is a replication of data from the original, and modifying the copy has no effect on the original.

As mentioned by other answers, the SettingWithCopyWarning was created to flag “chained assignment” operations. Consider df in the setup above. Suppose you would like to select all values in column “B” where values in column “A” is > 5. Pandas allows you to do this in different ways, some more correct than others. For example,

df[df.A > 5]['B']

1    3
2    6
Name: B, dtype: int64

And,

df.loc[df.A > 5, 'B']

1    3
2    6
Name: B, dtype: int64

These return the same result, so if you are only reading these values, it makes no difference. So, what is the issue? The problem with chained assignment, is that it is generally difficult to predict whether a view or a copy is returned, so this largely becomes an issue when you are attempting to assign values back. To build on the earlier example, consider how this code is executed by the interpreter:

df.loc[df.A > 5, 'B'] = 4
# becomes
df.__setitem__((df.A > 5, 'B'), 4)

With a single __setitem__ call to df. OTOH, consider this code:

df[df.A > 5]['B'] = 4
# becomes
df.__getitem__(df.A > 5).__setitem__('B", 4)

Now, depending on whether __getitem__ returned a view or a copy, the __setitem__ operation may not work.

In general, you should use loc for label-based assignment, and iloc for integer/positional based assignment, as the spec guarantees that they always operate on the original. Additionally, for setting a single cell, you should use at and iat.

Just tell me how to suppress the warning!

Consider a simple operation on the “A” column of df. Selecting “A” and dividing by 2 will raise the warning, but the operation will work.

df2 = df[['A']]
df2['A'] /= 2
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/IPython/__main__.py:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

df2
     A
0  2.5
1  4.5
2  3.5

There are a couple ways of directly silencing this warning:

Make a deepcopy

df2 = df[['A']].copy(deep=True)
df2['A'] /= 2

Change pd.options.mode.chained_assignment
Can be set to None, "warn", or "raise". "warn" is the default. None will suppress the warning entirely, and "raise" will throw a SettingWithCopyError, preventing the operation from going through.
```
pd.options.mode.chained_assignment = None
df2['A'] /= 2
```

@Peter Cotton in the comments, came up with a nice way of non-intrusively changing the mode (modified from this gist) using a context manager, to set the mode only as long as it is required, and the reset it back to the original state when finished.

class ChainedAssignent:
    def __init__(self, chained=None):
        acceptable = [None, 'warn', 'raise']
        assert chained in acceptable, "chained must be in " + str(acceptable)
        self.swcw = chained

    def __enter__(self):
        self.saved_swcw = pd.options.mode.chained_assignment
        pd.options.mode.chained_assignment = self.swcw
        return self

    def __exit__(self, *args):
        pd.options.mode.chained_assignment = self.saved_swcw

The usage is as follows:

# some code here
with ChainedAssignent():
    df2['A'] /= 2
# more code follows

Or, to raise the exception

with ChainedAssignent(chained='raise'):
    df2['A'] /= 2

SettingWithCopyError: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

The “XY Problem”: What am I doing wrong?

A lot of the time, users attempt to look for ways of suppressing this exception without fully understanding why it was raised in the first place. This is a good example of an XY problem, where users attempt to solve a problem “Y” that is actually a symptom of a deeper rooted problem “X”. Questions will be raised based on common problems that encounter this warning, and solutions will then be presented.

Question 1
I have a DataFrame
df
       A  B  C  D  E
    0  5  0  3  3  7
    1  9  3  5  2  4
    2  7  6  8  8  1
I want to assign values in col “A” > 5 to 1000. My expected output is
      A  B  C  D  E
0     5  0  3  3  7
1  1000  3  5  2  4
2  1000  6  8  8  1

Wrong way to do this:

df.A[df.A > 5] = 1000         # works, because df.A returns a view
df[df.A > 5]['A'] = 1000      # does not work
df.loc[df.A  5]['A'] = 1000   # does not work

Right way using loc:

df.loc[df.A > 5, 'A'] = 1000

Question 2¹
I am trying to set the value in cell (1, ‘D’) to 12345. My expected output is
   A  B  C      D  E
0  5  0  3      3  7
1  9  3  5  12345  4
2  7  6  8      8  1
I have tried different ways of accessing this cell, such as df['D'][1]. What is the best way to do this?

_{1. This question isn’t specifically related to the warning, but
it is good to understand how to do this particular operation correctly
so as to avoid situations where the warning could potentially arise in
future.}

You can use any of the following methods to do this.

df.loc[1, 'D'] = 12345
df.iloc[1, 3] = 12345
df.at[1, 'D'] = 12345
df.iat[1, 3] = 12345

Question 3
I am trying to subset values based on some condition. I have a DataFrame
   A  B  C  D  E
1  9  3  5  2  4
2  7  6  8  8  1
I would like to assign values in “D” to 123 such that “C” == 5. I tried
df2.loc[df2.C == 5, 'D'] = 123
Which seems fine but I am still getting the SettingWithCopyWarning! How do I fix this?

This is actually probably because of code higher up in your pipeline. Did you create df2 from something larger, like

df2 = df[df.A > 5]

? In this case, boolean indexing will return a view, so df2 will reference the original. What you’d need to do is assign df2 to a copy:

df2 = df[df.A > 5].copy()
# Or,
# df2 = df.loc[df.A > 5, :]

Question 4
I’m trying to drop column “C” in-place from
   A  B  C  D  E
1  9  3  5  2  4
2  7  6  8  8  1
But using
df2.drop('C', axis=1, inplace=True)
Throws SettingWithCopyWarning. Why is this happening?

This is because df2 must have been created as a view from some other slicing operation, such as

df2 = df[df.A > 5]

The solution here is to either make a copy() of df, or use loc, as before.

回答 2

通常，的目的SettingWithCopyWarning是向用户（尤其是新用户）显示他们可能正在使用副本，而不是他们认为的原始内容。这里是误报（IOW如果你知道你在做什么，它可能是好的）。一种可能就是简单地关闭（默认警告按照@Garrett的建议）警告。

这是另一个选择：

In [1]: df = DataFrame(np.random.randn(5, 2), columns=list('AB'))

In [2]: dfa = df.ix[:, [1, 0]]

In [3]: dfa.is_copy
Out[3]: True

In [4]: dfa['A'] /= 2
/usr/local/bin/ipython:1: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_index,col_indexer] = value instead
  #!/usr/local/bin/python

您可以将is_copy标志设置为False，以有效关闭该对象的检查：

In [5]: dfa.is_copy = False

In [6]: dfa['A'] /= 2

如果您明确复制，则不会发生进一步的警告：

In [7]: dfa = df.ix[:, [1, 0]].copy()

In [8]: dfa['A'] /= 2

OP在上面显示的代码是合法的，并且可能是我也可以做的，但从技术上讲，此警告是一种情况，不是误报。没有警告的另一种方法是通过进行选择操作reindex，例如

quote_df = quote_df.reindex(columns=['STK', ...])

要么，

quote_df = quote_df.reindex(['STK', ...], axis=1)  # v.0.21

In general the point of the SettingWithCopyWarning is to show users (and especially new users) that they may be operating on a copy and not the original as they think. There are false positives (IOW if you know what you are doing it could be ok). One possibility is simply to turn off the (by default warn) warning as @Garrett suggest.

Here is another option:

In [1]: df = DataFrame(np.random.randn(5, 2), columns=list('AB'))

In [2]: dfa = df.ix[:, [1, 0]]

In [3]: dfa.is_copy
Out[3]: True

In [4]: dfa['A'] /= 2
/usr/local/bin/ipython:1: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_index,col_indexer] = value instead
  #!/usr/local/bin/python

You can set the is_copy flag to False, which will effectively turn off the check, for that object:

In [5]: dfa.is_copy = False

In [6]: dfa['A'] /= 2

If you explicitly copy then no further warning will happen:

In [7]: dfa = df.ix[:, [1, 0]].copy()

In [8]: dfa['A'] /= 2

The code the OP is showing above, while legitimate, and probably something I do as well, is technically a case for this warning, and not a false positive. Another way to not have the warning would be to do the selection operation via reindex, e.g.

quote_df = quote_df.reindex(columns=['STK', ...])

Or,

quote_df = quote_df.reindex(['STK', ...], axis=1)  # v.0.21

回答 3

熊猫数据框复制警告

当您去做这样的事情时：

quote_df = quote_df.ix[:,[0,3,2,1,4,5,8,9,30,31]]

pandas.ix 在这种情况下将返回一个新的独立数据帧。

您决定在此数据框中更改的任何值都不会更改原始数据框。

这就是熊猫试图警告您的内容。

为什么 `.ix`是个坏主意

该.ix对象试图做的事情不只一件事，而且对于任何阅读过干净代码的人来说，这是一种强烈的气味。

给定此数据框：

df = pd.DataFrame({"a": [1,2,3,4], "b": [1,1,2,2]})

两种行为：

dfcopy = df.ix[:,["a"]]
dfcopy.a.ix[0] = 2

行为一：dfcopy现在是一个独立的数据框。改变它不会改变df

df.ix[0, "a"] = 3

行为二：更改原始数据框。

使用`.loc`替代

熊猫开发者意识到该.ix对象很臭（推测地），因此创建了两个新对象，这些对象有助于数据的获取和分配。（另一个是.iloc）

.loc 速度更快，因为它不会尝试创建数据副本。

.loc 旨在就地修改现有数据框，从而提高内存效率。

.loc 是可预测的，它具有一种行为。

解决方案

在代码示例中，您正在执行的操作是加载一个包含许多列的大文件，然后将其修改为较小的文件。

该pd.read_csv功能可以帮助您解决很多问题，还可以加快文件的加载速度。

所以不要这样做

quote_df = pd.read_csv(StringIO(str_of_all), sep=',', names=list('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefg')) #dtype={'A': object, 'B': object, 'C': np.float64}
quote_df.rename(columns={'A':'STK', 'B':'TOpen', 'C':'TPCLOSE', 'D':'TPrice', 'E':'THigh', 'F':'TLow', 'I':'TVol', 'J':'TAmt', 'e':'TDate', 'f':'TTime'}, inplace=True)
quote_df = quote_df.ix[:,[0,3,2,1,4,5,8,9,30,31]]

做这个

columns = ['STK', 'TPrice', 'TPCLOSE', 'TOpen', 'THigh', 'TLow', 'TVol', 'TAmt', 'TDate', 'TTime']
df = pd.read_csv(StringIO(str_of_all), sep=',', usecols=[0,3,2,1,4,5,8,9,30,31])
df.columns = columns

这只会读取您感兴趣的列，并正确命名它们。无需使用邪恶的.ix物体做神奇的事情。

Pandas dataframe copy warning

When you go and do something like this:

quote_df = quote_df.ix[:,[0,3,2,1,4,5,8,9,30,31]]

pandas.ix in this case returns a new, stand alone dataframe.

Any values you decide to change in this dataframe, will not change the original dataframe.

This is what pandas tries to warn you about.

Why `.ix` is a bad idea

The .ix object tries to do more than one thing, and for anyone who has read anything about clean code, this is a strong smell.

Given this dataframe:

df = pd.DataFrame({"a": [1,2,3,4], "b": [1,1,2,2]})

Two behaviors:

dfcopy = df.ix[:,["a"]]
dfcopy.a.ix[0] = 2

Behavior one: dfcopy is now a stand alone dataframe. Changing it will not change df

df.ix[0, "a"] = 3

Behavior two: This changes the original dataframe.

Use `.loc` instead

The pandas developers recognized that the .ix object was quite smelly[speculatively] and thus created two new objects which helps in the accession and assignment of data. (The other being .iloc)

.loc is faster, because it does not try to create a copy of the data.

.loc is meant to modify your existing dataframe inplace, which is more memory efficient.

.loc is predictable, it has one behavior.

The solution

What you are doing in your code example is loading a big file with lots of columns, then modifying it to be smaller.

The pd.read_csv function can help you out with a lot of this and also make the loading of the file a lot faster.

So instead of doing this

quote_df = pd.read_csv(StringIO(str_of_all), sep=',', names=list('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefg')) #dtype={'A': object, 'B': object, 'C': np.float64}
quote_df.rename(columns={'A':'STK', 'B':'TOpen', 'C':'TPCLOSE', 'D':'TPrice', 'E':'THigh', 'F':'TLow', 'I':'TVol', 'J':'TAmt', 'e':'TDate', 'f':'TTime'}, inplace=True)
quote_df = quote_df.ix[:,[0,3,2,1,4,5,8,9,30,31]]

Do this

columns = ['STK', 'TPrice', 'TPCLOSE', 'TOpen', 'THigh', 'TLow', 'TVol', 'TAmt', 'TDate', 'TTime']
df = pd.read_csv(StringIO(str_of_all), sep=',', usecols=[0,3,2,1,4,5,8,9,30,31])
df.columns = columns

This will only read the columns you are interested in, and name them properly. No need for using the evil .ix object to do magical stuff.

回答 4

在这里，我直接回答这个问题。怎么处理呢？

.copy(deep=False)切片后做一个。参见pandas.DataFrame.copy。

等等，切片不返回副本吗？毕竟，这是警告消息要说的内容？阅读详细答案：

import pandas as pd
df = pd.DataFrame({'x':[1,2,3]})

这给出了警告：

df0 = df[df.x>2]
df0['foo'] = 'bar'

这不是：

df1 = df[df.x>2].copy(deep=False)
df1['foo'] = 'bar'

两者df0和df1都是DataFrame对象，但它们之间的某些不同之处使熊猫能够打印警告。让我们找出它是什么。

import inspect
slice= df[df.x>2]
slice_copy = df[df.x>2].copy(deep=False)
inspect.getmembers(slice)
inspect.getmembers(slice_copy)

使用选择的差异工具，您将看到，除了几个地址之外，唯一的实质区别是：

|          | slice   | slice_copy |
| _is_copy | weakref | None       |

决定是否发出警告的方法是DataFrame._check_setitem_copy检查_is_copy。所以，你去。制作一个copy使您的DataFrame不_is_copy。

建议使用警告.loc，但如果在上使用.loc该框架_is_copy，您仍会收到相同的警告。误导？是。烦人吗你打赌有帮助吗？可能在使用链式分配时。但是它不能正确检测链条分配，并且会随意打印警告。

Here I answer the question directly. How to deal with it?

Make a .copy(deep=False) after you slice. See pandas.DataFrame.copy.

Wait, doesn’t a slice return a copy? After all, this is what the warning message is attempting to say? Read the long answer:

import pandas as pd
df = pd.DataFrame({'x':[1,2,3]})

This gives a warning:

df0 = df[df.x>2]
df0['foo'] = 'bar'

This does not:

df1 = df[df.x>2].copy(deep=False)
df1['foo'] = 'bar'

Both df0 and df1 are DataFrame objects, but something about them is different that enables pandas to print the warning. Let’s find out what it is.

import inspect
slice= df[df.x>2]
slice_copy = df[df.x>2].copy(deep=False)
inspect.getmembers(slice)
inspect.getmembers(slice_copy)

Using your diff tool of choice, you will see that beyond a couple of addresses, the only material difference is this:

|          | slice   | slice_copy |
| _is_copy | weakref | None       |

The method that decides whether to warn is DataFrame._check_setitem_copy which checks _is_copy. So here you go. Make a copy so that your DataFrame is not _is_copy.

The warning is suggesting to use .loc, but if you use .loc on a frame that _is_copy, you will still get the same warning. Misleading? Yes. Annoying? You bet. Helpful? Potentially, when chained assignment is used. But it cannot correctly detect chain assignment and prints the warning indiscriminately.

回答 5

这个话题确实让Pandas感到困惑。幸运的是，它有一个相对简单的解决方案。

问题在于，并不总是清楚数据过滤操作（例如loc）是否返回DataFrame的副本或视图。因此，这种过滤后的DataFrame的进一步使用可能会造成混淆。

简单的解决方案是（除非您需要处理非常大的数据集）：

每当需要更新任何值时，请始终确保在分配之前隐式复制DataFrame。

df  # Some DataFrame
df = df.loc[:, 0:2]  # Some filtering (unsure whether a view or copy is returned)
df = df.copy()  # Ensuring a copy is made
df[df["Name"] == "John"] = "Johny"  # Assignment can be done now (no warning)

This topic is really confusing with Pandas. Luckily, it has a relatively simple solution.

The problem is that it is not always clear whether data filtering operations (e.g. loc) return a copy or a view of the DataFrame. Further use of such filtered DataFrame could therefore be confusing.

The simple solution is (unless you need to work with very large sets of data):

Whenever you need to update any values, always make sure that you implicitely copy the DataFrame before the assignment.

df  # Some DataFrame
df = df.loc[:, 0:2]  # Some filtering (unsure whether a view or copy is returned)
df = df.copy()  # Ensuring a copy is made
df[df["Name"] == "John"] = "Johny"  # Assignment can be done now (no warning)

回答 6

为了消除任何疑问，我的解决方案是制作切片的深层副本，而不是常规副本。根据您的上下文，这可能不适用（内存限制/切片的大小，潜在的性能下降-特别是如果复制像对我一样在一个循环中发生，等等。）

需要明确的是，这是我收到的警告：

/opt/anaconda3/lib/python3.6/site-packages/ipykernel/__main__.py:54:
SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation:
http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy

插图

我怀疑是否由于我将一列放在切片的副本上而引发警告。虽然从技术上讲，它不是在切片副本中尝试设置值，但是这仍然是切片副本的修改。以下是我为确认怀疑而采取的（简化）步骤，希望它能对那些试图了解警告的人有所帮助。

示例1：在原件上放置一列会影响复印

我们已经知道了，但这是健康的提醒。这是不是警告是关于什么的。

>> data1 = {'A': [111, 112, 113], 'B':[121, 122, 123]}
>> df1 = pd.DataFrame(data1)
>> df1

    A   B
0   111 121
1   112 122
2   113 123


>> df2 = df1
>> df2

A   B
0   111 121
1   112 122
2   113 123

# Dropping a column on df1 affects df2
>> df1.drop('A', axis=1, inplace=True)
>> df2
    B
0   121
1   122
2   123

可以避免对df1进行更改以影响df2

>> data1 = {'A': [111, 112, 113], 'B':[121, 122, 123]}
>> df1 = pd.DataFrame(data1)
>> df1

A   B
0   111 121
1   112 122
2   113 123

>> import copy
>> df2 = copy.deepcopy(df1)
>> df2
A   B
0   111 121
1   112 122
2   113 123

# Dropping a column on df1 does not affect df2
>> df1.drop('A', axis=1, inplace=True)
>> df2
    A   B
0   111 121
1   112 122
2   113 123

示例2：在副本上放置一列可能会影响原始

这实际上说明了警告。

>> data1 = {'A': [111, 112, 113], 'B':[121, 122, 123]}
>> df1 = pd.DataFrame(data1)
>> df1

    A   B
0   111 121
1   112 122
2   113 123

>> df2 = df1
>> df2

    A   B
0   111 121
1   112 122
2   113 123

# Dropping a column on df2 can affect df1
# No slice involved here, but I believe the principle remains the same?
# Let me know if not
>> df2.drop('A', axis=1, inplace=True)
>> df1

B
0   121
1   122
2   123

可以避免对df2进行更改以影响df1

>> data1 = {'A': [111, 112, 113], 'B':[121, 122, 123]}
>> df1 = pd.DataFrame(data1)
>> df1

    A   B
0   111 121
1   112 122
2   113 123

>> import copy
>> df2 = copy.deepcopy(df1)
>> df2

A   B
0   111 121
1   112 122
2   113 123

>> df2.drop('A', axis=1, inplace=True)
>> df1

A   B
0   111 121
1   112 122
2   113 123

干杯!

To remove any doubt, my solution was to make a deep copy of the slice instead of a regular copy. This may not be applicable depending on your context (Memory constraints / size of the slice, potential for performance degradation – especially if the copy occurs in a loop like it did for me, etc…)

To be clear, here is the warning I received:

/opt/anaconda3/lib/python3.6/site-packages/ipykernel/__main__.py:54:
SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation:
http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy

Illustration

I had doubts that the warning was thrown because of a column I was dropping on a copy of the slice. While not technically trying to set a value in the copy of the slice, that was still a modification of the copy of the slice. Below are the (simplified) steps I have taken to confirm the suspicion, I hope it will help those of us who are trying to understand the warning.

Example 1: dropping a column on the original affects the copy

We knew that already but this is a healthy reminder. This is NOT what the warning is about.

>> data1 = {'A': [111, 112, 113], 'B':[121, 122, 123]}
>> df1 = pd.DataFrame(data1)
>> df1

    A   B
0   111 121
1   112 122
2   113 123


>> df2 = df1
>> df2

A   B
0   111 121
1   112 122
2   113 123

# Dropping a column on df1 affects df2
>> df1.drop('A', axis=1, inplace=True)
>> df2
    B
0   121
1   122
2   123

It is possible to avoid changes made on df1 to affect df2

>> data1 = {'A': [111, 112, 113], 'B':[121, 122, 123]}
>> df1 = pd.DataFrame(data1)
>> df1

A   B
0   111 121
1   112 122
2   113 123

>> import copy
>> df2 = copy.deepcopy(df1)
>> df2
A   B
0   111 121
1   112 122
2   113 123

# Dropping a column on df1 does not affect df2
>> df1.drop('A', axis=1, inplace=True)
>> df2
    A   B
0   111 121
1   112 122
2   113 123

Example 2: dropping a column on the copy may affect the original

This actually illustrates the warning.

>> data1 = {'A': [111, 112, 113], 'B':[121, 122, 123]}
>> df1 = pd.DataFrame(data1)
>> df1

    A   B
0   111 121
1   112 122
2   113 123

>> df2 = df1
>> df2

    A   B
0   111 121
1   112 122
2   113 123

# Dropping a column on df2 can affect df1
# No slice involved here, but I believe the principle remains the same?
# Let me know if not
>> df2.drop('A', axis=1, inplace=True)
>> df1

B
0   121
1   122
2   123

It is possible to avoid changes made on df2 to affect df1

>> data1 = {'A': [111, 112, 113], 'B':[121, 122, 123]}
>> df1 = pd.DataFrame(data1)
>> df1

    A   B
0   111 121
1   112 122
2   113 123

>> import copy
>> df2 = copy.deepcopy(df1)
>> df2

A   B
0   111 121
1   112 122
2   113 123

>> df2.drop('A', axis=1, inplace=True)
>> df1

A   B
0   111 121
1   112 122
2   113 123

Cheers!

回答 7

这应该工作：

quote_df.loc[:,'TVol'] = quote_df['TVol']/TVOL_SCALE

This should work:

quote_df.loc[:,'TVol'] = quote_df['TVol']/TVOL_SCALE

回答 8

有些人可能想简单地消除警告：

class SupressSettingWithCopyWarning:
    def __enter__(self):
        pd.options.mode.chained_assignment = None

    def __exit__(self, *args):
        pd.options.mode.chained_assignment = 'warn'

with SupressSettingWithCopyWarning():
    #code that produces warning

Some may want to simply suppress the warning:

class SupressSettingWithCopyWarning:
    def __enter__(self):
        pd.options.mode.chained_assignment = None

    def __exit__(self, *args):
        pd.options.mode.chained_assignment = 'warn'

with SupressSettingWithCopyWarning():
    #code that produces warning

回答 9

如果您已将切片分配给变量，并希望使用变量进行设置，如下所示：

df2 = df[df['A'] > 2]
df2['B'] = value

而且由于您的条件计算df2时间太长或出于某些其他原因，您不想使用Jeffs解决方案，那么您可以使用以下方法：

df.loc[df2.index.tolist(), 'B'] = value

df2.index.tolist() 返回df2中所有条目的索引，然后将这些索引用于设置原始数据帧中的B列。

If you have assigned the slice to a variable and want to set using the variable as in the following:

df2 = df[df['A'] > 2]
df2['B'] = value

And you do not want to use Jeffs solution because your condition computing df2 is to long or for some other reason, then you can use the following:

df.loc[df2.index.tolist(), 'B'] = value

df2.index.tolist() returns the indices from all entries in df2, which will then be used to set column B in the original dataframe.

回答 10

对我来说，此问题发生在下面的> simplified <示例中。我也能够解决它（希望有一个正确的解决方案）：

带有警告的旧代码：

def update_old_dataframe(old_dataframe, new_dataframe):
    for new_index, new_row in new_dataframe.iterrorws():
        old_dataframe.loc[new_index] = update_row(old_dataframe.loc[new_index], new_row)

def update_row(old_row, new_row):
    for field in [list_of_columns]:
        # line with warning because of chain indexing old_dataframe[new_index][field]
        old_row[field] = new_row[field]  
    return old_row

这打印了该行的警告 old_row[field] = new_row[field]

由于update_row方法中的行实际上是type Series，因此我将其替换为：

old_row.at[field] = new_row.at[field]

即用于访问/查找的方法Series。尽管两者都可以正常工作并且结果是相同的，但是通过这种方式，我不必禁用警告（=将其保留在其他地方的其他链索引问题中）。

我希望这可以帮助某人。

For me this issue occured in a following >simplified< example. And I was also able to solve it (hopefully with a correct solution):

old code with warning:

def update_old_dataframe(old_dataframe, new_dataframe):
    for new_index, new_row in new_dataframe.iterrorws():
        old_dataframe.loc[new_index] = update_row(old_dataframe.loc[new_index], new_row)

def update_row(old_row, new_row):
    for field in [list_of_columns]:
        # line with warning because of chain indexing old_dataframe[new_index][field]
        old_row[field] = new_row[field]  
    return old_row

This printed the warning for the line old_row[field] = new_row[field]

Since the rows in update_row method are actually type Series, I replaced the line with:

old_row.at[field] = new_row.at[field]

i.e. method for accessing/lookups for a Series. Eventhough both works just fine and the result is same, this way I don’t have to disable the warnings (=keep them for other chain indexing issues somewhere else).

I hope this may help someone.

回答 11

我相信您可以避免像这样的整个问题：

return (
    pd.read_csv(StringIO(str_of_all), sep=',', names=list('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefg')) #dtype={'A': object, 'B': object, 'C': np.float64}
    .rename(columns={'A':'STK', 'B':'TOpen', 'C':'TPCLOSE', 'D':'TPrice', 'E':'THigh', 'F':'TLow', 'I':'TVol', 'J':'TAmt', 'e':'TDate', 'f':'TTime'}, inplace=True)
    .ix[:,[0,3,2,1,4,5,8,9,30,31]]
    .assign(
        TClose=lambda df: df['TPrice'],
        RT=lambda df: 100 * (df['TPrice']/quote_df['TPCLOSE'] - 1),
        TVol=lambda df: df['TVol']/TVOL_SCALE,
        TAmt=lambda df: df['TAmt']/TAMT_SCALE,
        STK_ID=lambda df: df['STK'].str.slice(13,19),
        STK_Name=lambda df: df['STK'].str.slice(21,30)#.decode('gb2312'),
        TDate=lambda df: df.TDate.map(lambda x: x[0:4]+x[5:7]+x[8:10]),
    )
)

使用分配。从文档中：将新列分配给DataFrame，返回一个新对象（一个副本），其中除新列外还包含所有原始列。

参见汤姆·奥格斯珀格（Tom Augspurger）关于熊猫方法链接的文章：https ://tomaugspurger.github.io/method-chaining

You could avoid the whole problem like this, I believe:

return (
    pd.read_csv(StringIO(str_of_all), sep=',', names=list('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefg')) #dtype={'A': object, 'B': object, 'C': np.float64}
    .rename(columns={'A':'STK', 'B':'TOpen', 'C':'TPCLOSE', 'D':'TPrice', 'E':'THigh', 'F':'TLow', 'I':'TVol', 'J':'TAmt', 'e':'TDate', 'f':'TTime'}, inplace=True)
    .ix[:,[0,3,2,1,4,5,8,9,30,31]]
    .assign(
        TClose=lambda df: df['TPrice'],
        RT=lambda df: 100 * (df['TPrice']/quote_df['TPCLOSE'] - 1),
        TVol=lambda df: df['TVol']/TVOL_SCALE,
        TAmt=lambda df: df['TAmt']/TAMT_SCALE,
        STK_ID=lambda df: df['STK'].str.slice(13,19),
        STK_Name=lambda df: df['STK'].str.slice(21,30)#.decode('gb2312'),
        TDate=lambda df: df.TDate.map(lambda x: x[0:4]+x[5:7]+x[8:10]),
    )
)

Using Assign. From the documentation: Assign new columns to a DataFrame, returning a new object (a copy) with all the original columns in addition to the new ones.

See Tom Augspurger’s article on method chaining in pandas: https://tomaugspurger.github.io/method-chaining

回答 12

后续初学者问题/备注

也许是对其他像我这样的初学者的澄清（我来自R，似乎在幕后工作有所不同）。以下看起来无害且功能正常的代码不断产生SettingWithCopy警告，但我不知道为什么。我已经阅读并理解了带有“链式索引”的内容，但是我的代码不包含任何内容：

def plot(pdb, df, title, **kw):
    df['target'] = (df['ogg'] + df['ugg']) / 2
    # ...

但是后来，太晚了，我查看了plot（）函数的调用位置：

    df = data[data['anz_emw'] > 0]
    pixbuf = plot(pdb, df, title)

因此，“ df”不是数据帧，而是某种对象，它会以某种方式记住它是通过索引数据帧而创建的（因此是视图？），这将使plot（）中的行成为可能。

 df['target'] = ...

相当于

 data[data['anz_emw'] > 0]['target'] = ...

这是一个链接索引。我说对了吗？

无论如何，

def plot(pdb, df, title, **kw):
    df.loc[:,'target'] = (df['ogg'] + df['ugg']) / 2

固定它。

Followup beginner question / remark

Maybe a clarification for other beginners like me (I come from R which seems to work a bit differently under the hood). The following harmless-looking and functional code kept producing the SettingWithCopy warning, and I couldn’t figure out why. I had both read and understood the issued with “chained indexing”, but my code doesn’t contain any:

def plot(pdb, df, title, **kw):
    df['target'] = (df['ogg'] + df['ugg']) / 2
    # ...

But then, later, much too late, I looked at where the plot() function is called:

    df = data[data['anz_emw'] > 0]
    pixbuf = plot(pdb, df, title)

So “df” isn’t a data frame but an object that somehow remembers that it was created by indexing a data frame (so is that a view?) which would make the line in plot()

 df['target'] = ...

equivalent to

 data[data['anz_emw'] > 0]['target'] = ...

which is a chained indexing. Did I get that right?

Anyway,

def plot(pdb, df, title, **kw):
    df.loc[:,'target'] = (df['ogg'] + df['ugg']) / 2

fixed it.

回答 13

由于这个问题已经在现有答案中得到了充分的解释和讨论，因此我将为pandas上下文管理器提供一种简洁的方法，使用pandas.option_context（指向文档和示例的链接）-绝对不需要使用所有dunder方法和其他方法创建自定义类和口哨声。

首先，上下文管理器代码本身：

from contextlib import contextmanager

@contextmanager
def SuppressPandasWarning():
    with pd.option_context("mode.chained_assignment", None):
        yield

再举一个例子：

import pandas as pd
from string import ascii_letters

a = pd.DataFrame({"A": list(ascii_letters[0:4]), "B": range(0,4)})

mask = a["A"].isin(["c", "d"])
# Even shallow copy below is enough to not raise the warning, but why is a mystery to me.
b = a.loc[mask]  # .copy(deep=False)

# Raises the `SettingWithCopyWarning`
b["B"] = b["B"] * 2

# Does not!
with SuppressPandasWarning():
    b["B"] = b["B"] * 2

值得一提的是，这两个方法均未修改a，这对我来说有点令人惊讶，即使是带有df的浅表副本.copy(deep=False)也将阻止发出此警告（据我所知，浅表副本也应至少a也应进行修改，但它不会’t。pandas魔术。）。

As this question is already fully explained and discussed in existing answers I will just provide a neat pandas approach to the context manager using pandas.option_context (links to docs and example) – there is absolutely no need to create a custom class with all the dunder methods and other bells and whistles.

First the context manager code itself:

from contextlib import contextmanager

@contextmanager
def SuppressPandasWarning():
    with pd.option_context("mode.chained_assignment", None):
        yield

Then an example:

import pandas as pd
from string import ascii_letters

a = pd.DataFrame({"A": list(ascii_letters[0:4]), "B": range(0,4)})

mask = a["A"].isin(["c", "d"])
# Even shallow copy below is enough to not raise the warning, but why is a mystery to me.
b = a.loc[mask]  # .copy(deep=False)

# Raises the `SettingWithCopyWarning`
b["B"] = b["B"] * 2

# Does not!
with SuppressPandasWarning():
    b["B"] = b["B"] * 2

Worth noticing is that both approches do not modify a, which is a bit surprising to me, and even a shallow df copy with .copy(deep=False) would prevent this warning to be raised (as far as I understand shallow copy should at least modify a as well, but it doesn’t. pandas magic.).

回答 14

.apply()从使用该.query()方法的现有数据帧分配新数据帧时，我一直遇到这个问题。例如：

prop_df = df.query('column == "value"')
prop_df['new_column'] = prop_df.apply(function, axis=1)

将返回此错误。在这种情况下，似乎可以解决该错误的修补程序是将其更改为：

prop_df = df.copy(deep=True)
prop_df = prop_df.query('column == "value"')
prop_df['new_column'] = prop_df.apply(function, axis=1)

但是，由于必须进行新的复制，因此这在使用大型数据帧时效率不高。

如果.apply()在生成新列及其值时使用该方法，则可以通过添加以下方法来解决错误并提高效率.reset_index(drop=True)：

prop_df = df.query('column == "value"').reset_index(drop=True)
prop_df['new_column'] = prop_df.apply(function, axis=1)

I had been getting this issue with .apply() when assigning a new dataframe from a pre-existing dataframe on which i’ve used the .query() method. For instance:

prop_df = df.query('column == "value"')
prop_df['new_column'] = prop_df.apply(function, axis=1)

Would return this error. The fix that seems to resolve the error in this case is by changing this to:

prop_df = df.copy(deep=True)
prop_df = prop_df.query('column == "value"')
prop_df['new_column'] = prop_df.apply(function, axis=1)

However, this is NOT efficient especially when using large dataframes, due to having to make a new copy.

If you’re using the .apply() method in generating a new column and its values, a fix that resolves the error and is more efficient is by adding .reset_index(drop=True):

prop_df = df.query('column == "value"').reset_index(drop=True)
prop_df['new_column'] = prop_df.apply(function, axis=1)

知识问答

在Python Pandas中向现有DataFrame添加新列

2021年7月25日 Python实用宝典

问题：在Python Pandas中向现有DataFrame添加新列

我有以下索引的DataFrame，其中的命名列和行不是连续数字：

          a         b         c         d
2  0.671399  0.101208 -0.181532  0.241273
3  0.446172 -0.243316  0.051767  1.577318
5  0.614758  0.075793 -0.451460 -0.012493

我想'e'在现有数据框架中添加一个新列，并且不想更改数据框架中的任何内容（即，新列始终与DataFrame具有相同的长度）。

0   -0.335485
1   -1.166658
2   -0.385571
dtype: float64

如何e在上述示例中添加列？

I have the following indexed DataFrame with named columns and rows not- continuous numbers:

          a         b         c         d
2  0.671399  0.101208 -0.181532  0.241273
3  0.446172 -0.243316  0.051767  1.577318
5  0.614758  0.075793 -0.451460 -0.012493

I would like to add a new column, 'e', to the existing data frame and do not want to change anything in the data frame (i.e., the new column always has the same length as the DataFrame).

0   -0.335485
1   -1.166658
2   -0.385571
dtype: float64

How can I add column e to the above example?

回答 0

使用原始的df1索引创建系列：

df1['e'] = pd.Series(np.random.randn(sLength), index=df1.index)

编辑2015年
有人报告SettingWithCopyWarning使用此代码。
但是，该代码仍可以在当前的熊猫0.10.1版本中完美运行。

>>> sLength = len(df1['a'])
>>> df1
          a         b         c         d
6 -0.269221 -0.026476  0.997517  1.294385
8  0.917438  0.847941  0.034235 -0.448948

>>> df1['e'] = pd.Series(np.random.randn(sLength), index=df1.index)
>>> df1
          a         b         c         d         e
6 -0.269221 -0.026476  0.997517  1.294385  1.757167
8  0.917438  0.847941  0.034235 -0.448948  2.228131

>>> p.version.short_version
'0.16.1'

该SettingWithCopyWarning目标对数据帧的副本通知可能无效转让的。它不一定表示您做错了（它可能会触发误报），但从0.13.0起，它会让您知道有更多适合同一目的的方法。然后，如果收到警告，请遵循其建议：尝试使用.loc [row_index，col_indexer] = value代替

>>> df1.loc[:,'f'] = pd.Series(np.random.randn(sLength), index=df1.index)
>>> df1
          a         b         c         d         e         f
6 -0.269221 -0.026476  0.997517  1.294385  1.757167 -0.050927
8  0.917438  0.847941  0.034235 -0.448948  2.228131  0.006109
>>>

实际上，这是目前熊猫文档中描述的更有效的方法

编辑2017

如评论和@Alexander所示，当前最好将Series的值添加为DataFrame的新列的最佳方法是使用assign：

df1 = df1.assign(e=pd.Series(np.random.randn(sLength)).values)

Use the original df1 indexes to create the series:

df1['e'] = pd.Series(np.random.randn(sLength), index=df1.index)

Edit 2015
Some reported getting the SettingWithCopyWarning with this code.
However, the code still runs perfectly with the current pandas version 0.16.1.

>>> sLength = len(df1['a'])
>>> df1
          a         b         c         d
6 -0.269221 -0.026476  0.997517  1.294385
8  0.917438  0.847941  0.034235 -0.448948

>>> df1['e'] = pd.Series(np.random.randn(sLength), index=df1.index)
>>> df1
          a         b         c         d         e
6 -0.269221 -0.026476  0.997517  1.294385  1.757167
8  0.917438  0.847941  0.034235 -0.448948  2.228131

>>> p.version.short_version
'0.16.1'

The SettingWithCopyWarning aims to inform of a possibly invalid assignment on a copy of the Dataframe. It doesn’t necessarily say you did it wrong (it can trigger false positives) but from 0.13.0 it let you know there are more adequate methods for the same purpose. Then, if you get the warning, just follow its advise: Try using .loc[row_index,col_indexer] = value instead

>>> df1.loc[:,'f'] = pd.Series(np.random.randn(sLength), index=df1.index)
>>> df1
          a         b         c         d         e         f
6 -0.269221 -0.026476  0.997517  1.294385  1.757167 -0.050927
8  0.917438  0.847941  0.034235 -0.448948  2.228131  0.006109
>>>

In fact, this is currently the more efficient method as described in pandas docs

Edit 2017

As indicated in the comments and by @Alexander, currently the best method to add the values of a Series as a new column of a DataFrame could be using assign:

df1 = df1.assign(e=pd.Series(np.random.randn(sLength)).values)

回答 1

这是添加新列的简单方法： df['e'] = e

This is the simple way of adding a new column: df['e'] = e

回答 2

我想在现有数据框中添加新列’e’，并且不更改数据框中的任何内容。（该系列的长度总是与数据帧相同。）

我假设中的索引值e与中的索引值匹配df1。

初始化名为的新列e并为其分配系列中的值的最简单方法e：

df['e'] = e.values

分配（熊猫0.16.0+）

从Pandas 0.16.0开始，您还可以使用assign，它为DataFrame分配新列，并返回一个新对象（副本），该对象除包含新列外还包含所有原始列。

df1 = df1.assign(e=e.values)

按照此示例（还包括assign函数的源代码），您还可以包括多个列：

df = pd.DataFrame({'a': [1, 2], 'b': [3, 4]})
>>> df.assign(mean_a=df.a.mean(), mean_b=df.b.mean())
   a  b  mean_a  mean_b
0  1  3     1.5     3.5
1  2  4     1.5     3.5

在您的示例中：

np.random.seed(0)
df1 = pd.DataFrame(np.random.randn(10, 4), columns=['a', 'b', 'c', 'd'])
mask = df1.applymap(lambda x: x <-0.7)
df1 = df1[-mask.any(axis=1)]
sLength = len(df1['a'])
e = pd.Series(np.random.randn(sLength))

>>> df1
          a         b         c         d
0  1.764052  0.400157  0.978738  2.240893
2 -0.103219  0.410599  0.144044  1.454274
3  0.761038  0.121675  0.443863  0.333674
7  1.532779  1.469359  0.154947  0.378163
9  1.230291  1.202380 -0.387327 -0.302303

>>> e
0   -1.048553
1   -1.420018
2   -1.706270
3    1.950775
4   -0.509652
dtype: float64

df1 = df1.assign(e=e.values)

>>> df1
          a         b         c         d         e
0  1.764052  0.400157  0.978738  2.240893 -1.048553
2 -0.103219  0.410599  0.144044  1.454274 -1.420018
3  0.761038  0.121675  0.443863  0.333674 -1.706270
7  1.532779  1.469359  0.154947  0.378163  1.950775
9  1.230291  1.202380 -0.387327 -0.302303 -0.509652

首次引入此新功能时，可以在此处找到说明。

I would like to add a new column, ‘e’, to the existing data frame and do not change anything in the data frame. (The series always got the same length as a dataframe.)

I assume that the index values in e match those in df1.

The easiest way to initiate a new column named e, and assign it the values from your series e:

df['e'] = e.values

assign (Pandas 0.16.0+)

As of Pandas 0.16.0, you can also use assign, which assigns new columns to a DataFrame and returns a new object (a copy) with all the original columns in addition to the new ones.

df1 = df1.assign(e=e.values)

As per this example (which also includes the source code of the assign function), you can also include more than one column:

df = pd.DataFrame({'a': [1, 2], 'b': [3, 4]})
>>> df.assign(mean_a=df.a.mean(), mean_b=df.b.mean())
   a  b  mean_a  mean_b
0  1  3     1.5     3.5
1  2  4     1.5     3.5

In context with your example:

np.random.seed(0)
df1 = pd.DataFrame(np.random.randn(10, 4), columns=['a', 'b', 'c', 'd'])
mask = df1.applymap(lambda x: x <-0.7)
df1 = df1[-mask.any(axis=1)]
sLength = len(df1['a'])
e = pd.Series(np.random.randn(sLength))

>>> df1
          a         b         c         d
0  1.764052  0.400157  0.978738  2.240893
2 -0.103219  0.410599  0.144044  1.454274
3  0.761038  0.121675  0.443863  0.333674
7  1.532779  1.469359  0.154947  0.378163
9  1.230291  1.202380 -0.387327 -0.302303

>>> e
0   -1.048553
1   -1.420018
2   -1.706270
3    1.950775
4   -0.509652
dtype: float64

df1 = df1.assign(e=e.values)

>>> df1
          a         b         c         d         e
0  1.764052  0.400157  0.978738  2.240893 -1.048553
2 -0.103219  0.410599  0.144044  1.454274 -1.420018
3  0.761038  0.121675  0.443863  0.333674 -1.706270
7  1.532779  1.469359  0.154947  0.378163  1.950775
9  1.230291  1.202380 -0.387327 -0.302303 -0.509652

The description of this new feature when it was first introduced can be found here.

回答 3

似乎在最新的Pandas版本中，可行的方法是使用df.assign：

df1 = df1.assign(e=np.random.randn(sLength))

它不会产生SettingWithCopyWarning。

It seems that in recent Pandas versions the way to go is to use df.assign:

df1 = df1.assign(e=np.random.randn(sLength))

It doesn’t produce SettingWithCopyWarning.

回答 4

通过NumPy直接执行此操作将是最有效的：

df1['e'] = np.random.randn(sLength)

请注意，我最初的建议（很旧）是使用map（慢得多）：

df1['e'] = df1['a'].map(lambda x: np.random.random())

Doing this directly via NumPy will be the most efficient:

df1['e'] = np.random.randn(sLength)

Note my original (very old) suggestion was to use map (which is much slower):

df1['e'] = df1['a'].map(lambda x: np.random.random())

回答 5

超简单的列分配

将熊猫数据框实现为列的有序字典。

这意味着__getitem__ []不仅可以用于获取特定列，__setitem__ [] =还可以用于分配新列。

例如，只需使用[]访问器，就可以向该数据框添加一列

    size      name color
0    big      rose   red
1  small    violet  blue
2  small     tulip   red
3  small  harebell  blue

df['protected'] = ['no', 'no', 'no', 'yes']

    size      name color protected
0    big      rose   red        no
1  small    violet  blue        no
2  small     tulip   red        no
3  small  harebell  blue       yes

请注意，即使数据框的索引已关闭，此操作也有效。

df.index = [3,2,1,0]
df['protected'] = ['no', 'no', 'no', 'yes']
    size      name color protected
3    big      rose   red        no
2  small    violet  blue        no
1  small     tulip   red        no
0  small  harebell  blue       yes

[] =是要走的路，但要当心！

但是，如果您有一个pd.Series并尝试将其分配给索引关闭的数据帧，则会遇到麻烦。参见示例：

df['protected'] = pd.Series(['no', 'no', 'no', 'yes'])
    size      name color protected
3    big      rose   red       yes
2  small    violet  blue        no
1  small     tulip   red        no
0  small  harebell  blue        no

这是因为pd.Series默认情况下，a的索引从0枚举到n。而熊猫[] =方法试图 变得“聪明”

实际发生了什么。

使用[] =方法时，pandas使用左手数据帧的索引和右手序列的索引安静地执行外部联接或外部合并。df['column'] = series

边注

这很快就会引起认知失调，因为该[]=方法试图根据输入来做很多不同的事情，除非您只知道熊猫是如何工作的，否则无法预测结果。因此，我建议不要使用[]=in代码库，但是在笔记本中浏览数据时可以使用。

解决问题

如果您有一个pd.Series并且希望从上到下分配它，或者您正在编码生产性代码并且不确定索引顺序，那么为此类问题提供保护是值得的。

您可以将转换pd.Series为a np.ndarray或a list，这可以解决问题。

df['protected'] = pd.Series(['no', 'no', 'no', 'yes']).values

要么

df['protected'] = list(pd.Series(['no', 'no', 'no', 'yes']))

但这不是很明确。

某些编码器可能会说：“嘿，这看起来很多余，我将对其进行优化”。

显式方式

设置的索引pd.Series是的索引df是明确的。

df['protected'] = pd.Series(['no', 'no', 'no', 'yes'], index=df.index)

或更现实的说，您可能pd.Series已经有空了。

protected_series = pd.Series(['no', 'no', 'no', 'yes'])
protected_series.index = df.index

3     no
2     no
1     no
0    yes

现在可以分配

df['protected'] = protected_series

    size      name color protected
3    big      rose   red        no
2  small    violet  blue        no
1  small     tulip   red        no
0  small  harebell  blue       yes

另一种方式 `df.reset_index()`

由于索引不一致是问题所在，因此，如果您认为数据框的索引不应该指示事物，则可以简单地删除索引，这应该更快，但是它不是很干净，因为您的函数现在可能做两件事。

df.reset_index(drop=True)
protected_series.reset_index(drop=True)
df['protected'] = protected_series

    size      name color protected
0    big      rose   red        no
1  small    violet  blue        no
2  small     tulip   red        no
3  small  harebell  blue       yes

注意 `df.assign`

尽管df.assign让您更清楚地知道自己在做什么，但实际上却存在与上述相同的所有问题[]=

df.assign(protected=pd.Series(['no', 'no', 'no', 'yes']))
    size      name color protected
3    big      rose   red       yes
2  small    violet  blue        no
1  small     tulip   red        no
0  small  harebell  blue        no

请注意df.assign，您的专栏没有被调用self。会导致错误。这很df.assign 臭，因为函数中存在这些伪像。

df.assign(self=pd.Series(['no', 'no', 'no', 'yes'])
TypeError: assign() got multiple values for keyword argument 'self'

您可能会说，“好吧，那我就不使用了self”。但是谁知道这个函数将来会如何变化以支持新的论点。也许您的列名将成为熊猫新更新中的一个参数，从而导致升级问题。

Super simple column assignment

A pandas dataframe is implemented as an ordered dict of columns.

This means that the __getitem__ [] can not only be used to get a certain column, but __setitem__ [] = can be used to assign a new column.

For example, this dataframe can have a column added to it by simply using the [] accessor

    size      name color
0    big      rose   red
1  small    violet  blue
2  small     tulip   red
3  small  harebell  blue

df['protected'] = ['no', 'no', 'no', 'yes']

    size      name color protected
0    big      rose   red        no
1  small    violet  blue        no
2  small     tulip   red        no
3  small  harebell  blue       yes

Note that this works even if the index of the dataframe is off.

df.index = [3,2,1,0]
df['protected'] = ['no', 'no', 'no', 'yes']
    size      name color protected
3    big      rose   red        no
2  small    violet  blue        no
1  small     tulip   red        no
0  small  harebell  blue       yes

[]= is the way to go, but watch out!

However, if you have a pd.Series and try to assign it to a dataframe where the indexes are off, you will run in to trouble. See example:

df['protected'] = pd.Series(['no', 'no', 'no', 'yes'])
    size      name color protected
3    big      rose   red       yes
2  small    violet  blue        no
1  small     tulip   red        no
0  small  harebell  blue        no

This is because a pd.Series by default has an index enumerated from 0 to n. And the pandas [] = method tries to be “smart”

What actually is going on.

When you use the [] = method pandas is quietly performing an outer join or outer merge using the index of the left hand dataframe and the index of the right hand series. df['column'] = series

Side note

This quickly causes cognitive dissonance, since the []= method is trying to do a lot of different things depending on the input, and the outcome cannot be predicted unless you just know how pandas works. I would therefore advice against the []= in code bases, but when exploring data in a notebook, it is fine.

Going around the problem

If you have a pd.Series and want it assigned from top to bottom, or if you are coding productive code and you are not sure of the index order, it is worth it to safeguard for this kind of issue.

You could downcast the pd.Series to a np.ndarray or a list, this will do the trick.

df['protected'] = pd.Series(['no', 'no', 'no', 'yes']).values

df['protected'] = list(pd.Series(['no', 'no', 'no', 'yes']))

But this is not very explicit.

Some coder may come along and say “Hey, this looks redundant, I’ll just optimize this away”.

Explicit way

Setting the index of the pd.Series to be the index of the df is explicit.

df['protected'] = pd.Series(['no', 'no', 'no', 'yes'], index=df.index)

Or more realistically, you probably have a pd.Series already available.

protected_series = pd.Series(['no', 'no', 'no', 'yes'])
protected_series.index = df.index

3     no
2     no
1     no
0    yes

Can now be assigned

df['protected'] = protected_series

    size      name color protected
3    big      rose   red        no
2  small    violet  blue        no
1  small     tulip   red        no
0  small  harebell  blue       yes

Alternative way with `df.reset_index()`

Since the index dissonance is the problem, if you feel that the index of the dataframe should not dictate things, you can simply drop the index, this should be faster, but it is not very clean, since your function now probably does two things.

df.reset_index(drop=True)
protected_series.reset_index(drop=True)
df['protected'] = protected_series

    size      name color protected
0    big      rose   red        no
1  small    violet  blue        no
2  small     tulip   red        no
3  small  harebell  blue       yes

Note on `df.assign`

While df.assign make it more explicit what you are doing, it actually has all the same problems as the above []=

df.assign(protected=pd.Series(['no', 'no', 'no', 'yes']))
    size      name color protected
3    big      rose   red       yes
2  small    violet  blue        no
1  small     tulip   red        no
0  small  harebell  blue        no

Just watch out with df.assign that your column is not called self. It will cause errors. This makes df.assign smelly, since there are these kind of artifacts in the function.

df.assign(self=pd.Series(['no', 'no', 'no', 'yes'])
TypeError: assign() got multiple values for keyword argument 'self'

You may say, “Well, I’ll just not use self then”. But who knows how this function changes in the future to support new arguments. Maybe your column name will be an argument in a new update of pandas, causing problems with upgrading.

回答 6

最简单的方法：

data['new_col'] = list_of_values

data.loc[ : , 'new_col'] = list_of_values

这样，您可以在熊猫对象中设置新值时避免所谓的链接索引。单击此处以进一步阅读。

Easiest ways:-

data['new_col'] = list_of_values

data.loc[ : , 'new_col'] = list_of_values

This way you avoid what is called chained indexing when setting new values in a pandas object. Click here to read further.

回答 7

如果您要将整个新列设置为初始基值（例如None），则可以执行以下操作：df1['e'] = None

实际上，这将为单元分配“对象”类型。因此，稍后您可以将复杂的数据类型（如列表）放到单个单元格中。

If you want to set the whole new column to an initial base value (e.g. None), you can do this: df1['e'] = None

This actually would assign “object” type to the cell. So later you’re free to put complex data types, like list, into individual cells.

回答 8

我感到恐惧SettingWithCopyWarning，并且无法通过使用iloc语法进行修复。我的DataFrame是由ODBC源中的read_sql创建的。使用上面lowtech的建议，以下内容对我有用：

df.insert(len(df.columns), 'e', pd.Series(np.random.randn(sLength),  index=df.index))

这样可以很好地在最后插入列。我不知道这是否是最有效的，但我不喜欢警告消息。我认为有一个更好的解决方案，但我找不到它，并且我认为它取决于索引的某些方面。
注意。这只能工作一次，并且如果尝试覆盖现有列会给出错误消息。
注意如上所述，从0.16.0开始分配是最佳解决方案。请参阅文档http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.assign.html#pandas.DataFrame.assign 对于不覆盖中间值的数据流类型而言效果很好。

I got the dreaded SettingWithCopyWarning, and it wasn’t fixed by using the iloc syntax. My DataFrame was created by read_sql from an ODBC source. Using a suggestion by lowtech above, the following worked for me:

df.insert(len(df.columns), 'e', pd.Series(np.random.randn(sLength),  index=df.index))

This worked fine to insert the column at the end. I don’t know if it is the most efficient, but I don’t like warning messages. I think there is a better solution, but I can’t find it, and I think it depends on some aspect of the index.
Note. That this only works once and will give an error message if trying to overwrite and existing column.
Note As above and from 0.16.0 assign is the best solution. See documentation http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.assign.html#pandas.DataFrame.assign Works well for data flow type where you don’t overwrite your intermediate values.

回答 9

首先创建一个list_of_e具有相关数据的python 。
用这个： df['e'] = list_of_e

First create a python’s list_of_e that has relevant data.
Use this: df['e'] = list_of_e

回答 10

如果您要添加的列是一个系列变量，则只需：

df["new_columns_name"]=series_variable_name #this will do it for you

即使您要替换现有的列，此方法也能很好地工作，只需键入与要替换的列相同的new_columns_name，它将用新的系列数据覆盖现有的列数据。

If the column you are trying to add is a series variable then just :

df["new_columns_name"]=series_variable_name #this will do it for you

This works well even if you are replacing an existing column.just type the new_columns_name same as the column you want to replace.It will just overwrite the existing column data with the new series data.

回答 11

如果数据框和Series对象具有相同的index，则pandas.concat也可以在这里工作：

import pandas as pd
df
#          a            b           c           d
#0  0.671399     0.101208   -0.181532    0.241273
#1  0.446172    -0.243316    0.051767    1.577318
#2  0.614758     0.075793   -0.451460   -0.012493

e = pd.Series([-0.335485, -1.166658, -0.385571])    
e
#0   -0.335485
#1   -1.166658
#2   -0.385571
#dtype: float64

# here we need to give the series object a name which converts to the new  column name 
# in the result
df = pd.concat([df, e.rename("e")], axis=1)
df

#          a            b           c           d           e
#0  0.671399     0.101208   -0.181532    0.241273   -0.335485
#1  0.446172    -0.243316    0.051767    1.577318   -1.166658
#2  0.614758     0.075793   -0.451460   -0.012493   -0.385571

如果它们没有相同的索引：

e.index = df.index
df = pd.concat([df, e.rename("e")], axis=1)

If the data frame and Series object have the same index, pandas.concat also works here:

import pandas as pd
df
#          a            b           c           d
#0  0.671399     0.101208   -0.181532    0.241273
#1  0.446172    -0.243316    0.051767    1.577318
#2  0.614758     0.075793   -0.451460   -0.012493

e = pd.Series([-0.335485, -1.166658, -0.385571])    
e
#0   -0.335485
#1   -1.166658
#2   -0.385571
#dtype: float64

# here we need to give the series object a name which converts to the new  column name 
# in the result
df = pd.concat([df, e.rename("e")], axis=1)
df

#          a            b           c           d           e
#0  0.671399     0.101208   -0.181532    0.241273   -0.335485
#1  0.446172    -0.243316    0.051767    1.577318   -1.166658
#2  0.614758     0.075793   -0.451460   -0.012493   -0.385571

In case they don’t have the same index:

e.index = df.index
df = pd.concat([df, e.rename("e")], axis=1)

回答 12

万无一失：

df.loc[:, 'NewCol'] = 'New_Val'

例：

df = pd.DataFrame(data=np.random.randn(20, 4), columns=['A', 'B', 'C', 'D'])

df

           A         B         C         D
0  -0.761269  0.477348  1.170614  0.752714
1   1.217250 -0.930860 -0.769324 -0.408642
2  -0.619679 -1.227659 -0.259135  1.700294
3  -0.147354  0.778707  0.479145  2.284143
4  -0.529529  0.000571  0.913779  1.395894
5   2.592400  0.637253  1.441096 -0.631468
6   0.757178  0.240012 -0.553820  1.177202
7  -0.986128 -1.313843  0.788589 -0.707836
8   0.606985 -2.232903 -1.358107 -2.855494
9  -0.692013  0.671866  1.179466 -1.180351
10 -1.093707 -0.530600  0.182926 -1.296494
11 -0.143273 -0.503199 -1.328728  0.610552
12 -0.923110 -1.365890 -1.366202 -1.185999
13 -2.026832  0.273593 -0.440426 -0.627423
14 -0.054503 -0.788866 -0.228088 -0.404783
15  0.955298 -1.430019  1.434071 -0.088215
16 -0.227946  0.047462  0.373573 -0.111675
17  1.627912  0.043611  1.743403 -0.012714
18  0.693458  0.144327  0.329500 -0.655045
19  0.104425  0.037412  0.450598 -0.923387


df.drop([3, 5, 8, 10, 18], inplace=True)

df

           A         B         C         D
0  -0.761269  0.477348  1.170614  0.752714
1   1.217250 -0.930860 -0.769324 -0.408642
2  -0.619679 -1.227659 -0.259135  1.700294
4  -0.529529  0.000571  0.913779  1.395894
6   0.757178  0.240012 -0.553820  1.177202
7  -0.986128 -1.313843  0.788589 -0.707836
9  -0.692013  0.671866  1.179466 -1.180351
11 -0.143273 -0.503199 -1.328728  0.610552
12 -0.923110 -1.365890 -1.366202 -1.185999
13 -2.026832  0.273593 -0.440426 -0.627423
14 -0.054503 -0.788866 -0.228088 -0.404783
15  0.955298 -1.430019  1.434071 -0.088215
16 -0.227946  0.047462  0.373573 -0.111675
17  1.627912  0.043611  1.743403 -0.012714
19  0.104425  0.037412  0.450598 -0.923387

df.loc[:, 'NewCol'] = 0

df
           A         B         C         D  NewCol
0  -0.761269  0.477348  1.170614  0.752714       0
1   1.217250 -0.930860 -0.769324 -0.408642       0
2  -0.619679 -1.227659 -0.259135  1.700294       0
4  -0.529529  0.000571  0.913779  1.395894       0
6   0.757178  0.240012 -0.553820  1.177202       0
7  -0.986128 -1.313843  0.788589 -0.707836       0
9  -0.692013  0.671866  1.179466 -1.180351       0
11 -0.143273 -0.503199 -1.328728  0.610552       0
12 -0.923110 -1.365890 -1.366202 -1.185999       0
13 -2.026832  0.273593 -0.440426 -0.627423       0
14 -0.054503 -0.788866 -0.228088 -0.404783       0
15  0.955298 -1.430019  1.434071 -0.088215       0
16 -0.227946  0.047462  0.373573 -0.111675       0
17  1.627912  0.043611  1.743403 -0.012714       0
19  0.104425  0.037412  0.450598 -0.923387       0

Foolproof:

df.loc[:, 'NewCol'] = 'New_Val'

Example:

df = pd.DataFrame(data=np.random.randn(20, 4), columns=['A', 'B', 'C', 'D'])

df

           A         B         C         D
0  -0.761269  0.477348  1.170614  0.752714
1   1.217250 -0.930860 -0.769324 -0.408642
2  -0.619679 -1.227659 -0.259135  1.700294
3  -0.147354  0.778707  0.479145  2.284143
4  -0.529529  0.000571  0.913779  1.395894
5   2.592400  0.637253  1.441096 -0.631468
6   0.757178  0.240012 -0.553820  1.177202
7  -0.986128 -1.313843  0.788589 -0.707836
8   0.606985 -2.232903 -1.358107 -2.855494
9  -0.692013  0.671866  1.179466 -1.180351
10 -1.093707 -0.530600  0.182926 -1.296494
11 -0.143273 -0.503199 -1.328728  0.610552
12 -0.923110 -1.365890 -1.366202 -1.185999
13 -2.026832  0.273593 -0.440426 -0.627423
14 -0.054503 -0.788866 -0.228088 -0.404783
15  0.955298 -1.430019  1.434071 -0.088215
16 -0.227946  0.047462  0.373573 -0.111675
17  1.627912  0.043611  1.743403 -0.012714
18  0.693458  0.144327  0.329500 -0.655045
19  0.104425  0.037412  0.450598 -0.923387


df.drop([3, 5, 8, 10, 18], inplace=True)

df

           A         B         C         D
0  -0.761269  0.477348  1.170614  0.752714
1   1.217250 -0.930860 -0.769324 -0.408642
2  -0.619679 -1.227659 -0.259135  1.700294
4  -0.529529  0.000571  0.913779  1.395894
6   0.757178  0.240012 -0.553820  1.177202
7  -0.986128 -1.313843  0.788589 -0.707836
9  -0.692013  0.671866  1.179466 -1.180351
11 -0.143273 -0.503199 -1.328728  0.610552
12 -0.923110 -1.365890 -1.366202 -1.185999
13 -2.026832  0.273593 -0.440426 -0.627423
14 -0.054503 -0.788866 -0.228088 -0.404783
15  0.955298 -1.430019  1.434071 -0.088215
16 -0.227946  0.047462  0.373573 -0.111675
17  1.627912  0.043611  1.743403 -0.012714
19  0.104425  0.037412  0.450598 -0.923387

df.loc[:, 'NewCol'] = 0

df
           A         B         C         D  NewCol
0  -0.761269  0.477348  1.170614  0.752714       0
1   1.217250 -0.930860 -0.769324 -0.408642       0
2  -0.619679 -1.227659 -0.259135  1.700294       0
4  -0.529529  0.000571  0.913779  1.395894       0
6   0.757178  0.240012 -0.553820  1.177202       0
7  -0.986128 -1.313843  0.788589 -0.707836       0
9  -0.692013  0.671866  1.179466 -1.180351       0
11 -0.143273 -0.503199 -1.328728  0.610552       0
12 -0.923110 -1.365890 -1.366202 -1.185999       0
13 -2.026832  0.273593 -0.440426 -0.627423       0
14 -0.054503 -0.788866 -0.228088 -0.404783       0
15  0.955298 -1.430019  1.434071 -0.088215       0
16 -0.227946  0.047462  0.373573 -0.111675       0
17  1.627912  0.043611  1.743403 -0.012714       0
19  0.104425  0.037412  0.450598 -0.923387       0

回答 13

让我补充一点，就像hum3一样，.loc没有解决SettingWithCopyWarning，我不得不求助于df.insert()。在我的情况下，“假”链索引产生了误报 dict['a']['e']，其中'e'是新列，并且dict['a']是来自字典的DataFrame。

另请注意，如果您知道自己在做什么，则可以使用pd.options.mode.chained_assignment = None ，而可以使用此处提供的其他解决方案之一来切换警告。

Let me just add that, just like for hum3, .loc didn’t solve the SettingWithCopyWarning and I had to resort to df.insert(). In my case false positive was generated by “fake” chain indexing dict['a']['e'], where 'e' is the new column, and dict['a'] is a DataFrame coming from dictionary.

Also note that if you know what you are doing, you can switch of the warning using pd.options.mode.chained_assignment = None and than use one of the other solutions given here.

回答 14

要在数据框中的给定位置（0 <= loc <=列数）插入新列，只需使用Dataframe.insert：

DataFrame.insert(loc, column, value)

因此，如果要将列e添加到名为df的数据帧的末尾，则可以使用：

e = [-0.335485, -1.166658, -0.385571]    
DataFrame.insert(loc=len(df.columns), column='e', value=e)

value可以是Series，整数（在这种情况下，所有单元格都填充有该值）或类似数组的结构

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.insert.html

to insert a new column at a given location (0 <= loc <= amount of columns) in a data frame, just use Dataframe.insert:

DataFrame.insert(loc, column, value)

Therefore, if you want to add the column e at the end of a data frame called df, you can use:

e = [-0.335485, -1.166658, -0.385571]    
DataFrame.insert(loc=len(df.columns), column='e', value=e)

value can be a Series, an integer (in which case all cells get filled with this one value), or an array-like structure

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.insert.html

回答 15

在分配新列之前，如果您已建立索引数据，则需要对索引进行排序。至少就我而言，我必须：

data.set_index(['index_column'], inplace=True)
"if index is unsorted, assignment of a new column will fail"        
data.sort_index(inplace = True)
data.loc['index_value1', 'column_y'] = np.random.randn(data.loc['index_value1', 'column_x'].shape[0])

Before assigning a new column, if you have indexed data, you need to sort the index. At least in my case I had to:

data.set_index(['index_column'], inplace=True)
"if index is unsorted, assignment of a new column will fail"        
data.sort_index(inplace = True)
data.loc['index_value1', 'column_y'] = np.random.randn(data.loc['index_value1', 'column_x'].shape[0])

回答 16

但是要注意的一件事是，如果您这样做

df1['e'] = Series(np.random.randn(sLength), index=df1.index)

这实际上是df1.index上的左连接。因此，如果要产生外部联接效果，我可能不完善的解决方案是创建一个具有索引值的数据框，该索引值覆盖数据的整个范围，然后使用上面的代码。例如，

data = pd.DataFrame(index=all_possible_values)
df1['e'] = Series(np.random.randn(sLength), index=df1.index)

One thing to note, though, is that if you do

df1['e'] = Series(np.random.randn(sLength), index=df1.index)

this will effectively be a left join on the df1.index. So if you want to have an outer join effect, my probably imperfect solution is to create a dataframe with index values covering the universe of your data, and then use the code above. For example,

data = pd.DataFrame(index=all_possible_values)
df1['e'] = Series(np.random.randn(sLength), index=df1.index)

回答 17

我一直在寻找一种通用方法，将numpy.nans 的列添加到数据框而不会变得愚蠢SettingWithCopyWarning。

从以下内容：

答案在这里
关于将变量作为关键字参数传递的问题
这种在线生成numpyNaN数组的方法

我想出了这个：

col = 'column_name'
df = df.assign(**{col:numpy.full(len(df), numpy.nan)})

I was looking for a general way of adding a column of numpy.nans to a dataframe without getting the dumb SettingWithCopyWarning.

From the following:

the answers here
this question about passing a variable as a keyword argument
this method for generating a numpy array of NaNs in-line

I came up with this:

col = 'column_name'
df = df.assign(**{col:numpy.full(len(df), numpy.nan)})

回答 18

要将新列“ e”添加到现有数据框中

 df1.loc[:,'e'] = Series(np.random.randn(sLength))

To add a new column, ‘e’, to the existing data frame

 df1.loc[:,'e'] = Series(np.random.randn(sLength))

回答 19

为了完整性-使用DataFrame.eval（）方法的另一种解决方案：

数据：

In [44]: e
Out[44]:
0    1.225506
1   -1.033944
2   -0.498953
3   -0.373332
4    0.615030
5   -0.622436
dtype: float64

In [45]: df1
Out[45]:
          a         b         c         d
0 -0.634222 -0.103264  0.745069  0.801288
4  0.782387 -0.090279  0.757662 -0.602408
5 -0.117456  2.124496  1.057301  0.765466
7  0.767532  0.104304 -0.586850  1.051297
8 -0.103272  0.958334  1.163092  1.182315
9 -0.616254  0.296678 -0.112027  0.679112

解：

In [46]: df1.eval("e = @e.values", inplace=True)

In [47]: df1
Out[47]:
          a         b         c         d         e
0 -0.634222 -0.103264  0.745069  0.801288  1.225506
4  0.782387 -0.090279  0.757662 -0.602408 -1.033944
5 -0.117456  2.124496  1.057301  0.765466 -0.498953
7  0.767532  0.104304 -0.586850  1.051297 -0.373332
8 -0.103272  0.958334  1.163092  1.182315  0.615030
9 -0.616254  0.296678 -0.112027  0.679112 -0.622436

For the sake of completeness – yet another solution using DataFrame.eval() method:

Data:

In [44]: e
Out[44]:
0    1.225506
1   -1.033944
2   -0.498953
3   -0.373332
4    0.615030
5   -0.622436
dtype: float64

In [45]: df1
Out[45]:
          a         b         c         d
0 -0.634222 -0.103264  0.745069  0.801288
4  0.782387 -0.090279  0.757662 -0.602408
5 -0.117456  2.124496  1.057301  0.765466
7  0.767532  0.104304 -0.586850  1.051297
8 -0.103272  0.958334  1.163092  1.182315
9 -0.616254  0.296678 -0.112027  0.679112

Solution:

In [46]: df1.eval("e = @e.values", inplace=True)

In [47]: df1
Out[47]:
          a         b         c         d         e
0 -0.634222 -0.103264  0.745069  0.801288  1.225506
4  0.782387 -0.090279  0.757662 -0.602408 -1.033944
5 -0.117456  2.124496  1.057301  0.765466 -0.498953
7  0.767532  0.104304 -0.586850  1.051297 -0.373332
8 -0.103272  0.958334  1.163092  1.182315  0.615030
9 -0.616254  0.296678 -0.112027  0.679112 -0.622436

回答 20

创建一个空列

df['i'] = None

To create an empty column

df['i'] = None

回答 21

以下是我的工作…但是，我对熊猫和Python真的很陌生，所以没有什么承诺。

df = pd.DataFrame([[1, 2], [3, 4], [5,6]], columns=list('AB'))

newCol = [3,5,7]
newName = 'C'

values = np.insert(df.values,df.shape[1],newCol,axis=1)
header = df.columns.values.tolist()
header.append(newName)

df = pd.DataFrame(values,columns=header)

The following is what I did… But I’m pretty new to pandas and really Python in general, so no promises.

df = pd.DataFrame([[1, 2], [3, 4], [5,6]], columns=list('AB'))

newCol = [3,5,7]
newName = 'C'

values = np.insert(df.values,df.shape[1],newCol,axis=1)
header = df.columns.values.tolist()
header.append(newName)

df = pd.DataFrame(values,columns=header)

回答 22

如果得到SettingWithCopyWarning，一个简单的解决方法是复制您要向其中添加列的DataFrame。

df = df.copy()
df['col_name'] = values

If you get the SettingWithCopyWarning, an easy fix is to copy the DataFrame you are trying to add a column to.

df = df.copy()
df['col_name'] = values

问题：熊猫使用什么规则生成视图与副本？

回答 0

问题：我为什么要在熊猫中复制数据框

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

问题：将特定的选定列提取到新DataFrame中作为副本

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

问题：如何处理熊猫中的SettingWithCopyWarning？

背景

产生错误的功能

更多错误讯息

Background

The function that gives errors

More error messages

回答 0

回答 1

SettingWithCopyWarning熊猫如何应对？

什么是SettingWithCopyWarning？

告诉我如何抑制警告！

“ XY问题”：我在做什么错？

How to deal with SettingWithCopyWarning in Pandas?

What is the SettingWithCopyWarning?

Just tell me how to suppress the warning!

The “XY Problem”: What am I doing wrong?

回答 2

回答 3

熊猫数据框复制警告

为什么 .ix是个坏主意

使用.loc替代

解决方案

Pandas dataframe copy warning

Why .ix is a bad idea

Use .loc instead

The solution

回答 4

回答 5

回答 6

插图

示例1：在原件上放置一列会影响复印

示例2：在副本上放置一列可能会影响原始

Illustration

Example 1: dropping a column on the original affects the copy

Example 2: dropping a column on the copy may affect the original

回答 7

回答 8

回答 9

回答 10

回答 11

回答 12

回答 13

回答 14

问题：在Python Pandas中向现有DataFrame添加新列

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

超简单的列分配

[] =是要走的路，但要当心！

实际发生了什么。

边注

解决问题

显式方式

另一种方式 df.reset_index()

注意 df.assign

Super simple column assignment

[]= is the way to go, but watch out!

What actually is going on.

Side note

`SettingWithCopyWarning`熊猫如何应对？

什么是`SettingWithCopyWarning`？

How to deal with `SettingWithCopyWarning` in Pandas?

What is the `SettingWithCopyWarning`?

为什么 `.ix`是个坏主意

使用`.loc`替代

Why `.ix` is a bad idea

Use `.loc` instead

另一种方式 `df.reset_index()`

注意 `df.assign`

Alternative way with `df.reset_index()`

Note on `df.assign`