知识问答

熊猫：索引数据框时出现多种情况-意外行为

内容 隐藏
问题：熊猫：索引数据框时出现多种情况-意外行为
回答 0
回答 1
回答 2

问题：熊猫：索引数据框时出现多种情况-意外行为

我正在按两列中的值过滤数据框中的行。

出于某种原因，OR运算符的行为类似于我期望AND运算符的行为，反之亦然。

我的测试代码：

import pandas as pd

df = pd.DataFrame({'a': range(5), 'b': range(5) })

# let's insert some -1 values
df['a'][1] = -1
df['b'][1] = -1
df['a'][3] = -1
df['b'][4] = -1

df1 = df[(df.a != -1) & (df.b != -1)]
df2 = df[(df.a != -1) | (df.b != -1)]

print pd.concat([df, df1, df2], axis=1,
                keys = [ 'original df', 'using AND (&)', 'using OR (|)',])

结果：

      original df      using AND (&)      using OR (|)    
             a  b              a   b             a   b
0            0  0              0   0             0   0
1           -1 -1            NaN NaN           NaN NaN
2            2  2              2   2             2   2
3           -1  3            NaN NaN            -1   3
4            4 -1            NaN NaN             4  -1

[5 rows x 6 columns]

如您所见，AND运算符将删除其中至少一个等于的每一行-1。另一方面，OR运算符要求两个值相等-1才能删除它们。我期望结果恰好相反。任何人都可以解释这种行为吗？

我正在使用熊猫0.13.1。

点击查看英文原文

I am filtering rows in a dataframe by values in two columns.

For some reason the OR operator behaves like I would expect AND operator to behave and vice versa.

My test code:

import pandas as pd

df = pd.DataFrame({'a': range(5), 'b': range(5) })

# let's insert some -1 values
df['a'][1] = -1
df['b'][1] = -1
df['a'][3] = -1
df['b'][4] = -1

df1 = df[(df.a != -1) & (df.b != -1)]
df2 = df[(df.a != -1) | (df.b != -1)]

print pd.concat([df, df1, df2], axis=1,
                keys = [ 'original df', 'using AND (&)', 'using OR (|)',])

And the result:

      original df      using AND (&)      using OR (|)    
             a  b              a   b             a   b
0            0  0              0   0             0   0
1           -1 -1            NaN NaN           NaN NaN
2            2  2              2   2             2   2
3           -1  3            NaN NaN            -1   3
4            4 -1            NaN NaN             4  -1

[5 rows x 6 columns]

As you can see, the AND operator drops every row in which at least one value equals -1. On the other hand, the OR operator requires both values to be equal to -1 to drop them. I would expect exactly the opposite result. Could anyone explain this behavior, please?

I am using pandas 0.13.1.

回答 0

如您所见，AND运算符会删除每一行中至少有一个等于-1的值。另一方面，OR运算符要求两个值都等于-1才能删除它们。

那就对了。请记住，您是根据要保留的内容而不是要丢弃的内容来写条件。对于df1：

df1 = df[(df.a != -1) & (df.b != -1)]

您说的是“保留其中df.a不是-1且df.b不是-1的行”，这与删除其中至少一个值为-1的每一行相同。

对于df2：

df2 = df[(df.a != -1) | (df.b != -1)]

您说的是“保留其中任一行df.a或df.b都不为-1的行”，这与删除两个值均为-1的行相同。

PS：连锁访问会给df['a'][1] = -1您带来麻烦。最好养成使用.loc和的习惯.iloc。

点击查看英文原文

As you can see, the AND operator drops every row in which at least one value equals -1. On the other hand, the OR operator requires both values to be equal to -1 to drop them.

That’s right. Remember that you’re writing the condition in terms of what you want to keep, not in terms of what you want to drop. For df1:

df1 = df[(df.a != -1) & (df.b != -1)]

You’re saying “keep the rows in which df.a isn’t -1 and df.b isn’t -1″, which is the same as dropping every row in which at least one value is -1.

For df2:

df2 = df[(df.a != -1) | (df.b != -1)]

You’re saying “keep the rows in which either df.a or df.b is not -1″, which is the same as dropping rows where both values are -1.

PS: chained access like df['a'][1] = -1 can get you into trouble. It’s better to get into the habit of using .loc and .iloc.

回答 1

您可以使用query（），即：

df_filtered = df.query('a == 4 & b != 2')
点击查看英文原文

You can use query(), i.e.:

df_filtered = df.query('a == 4 & b != 2')

回答 2

这里有一些数学逻辑理论：

“ NOT a AND NOT b”与“ NOT（a OR b）”相同，因此：

“ a NOT -1 AND b NOT -1” 等同于 “ NOT（a为-1 OR b为-1）”，与“（a is -1 OR b为-1）”的（补数）相反。

因此，如果您想要完全相反的结果，则df1和df2应如下所示：

df1 = df[(df.a != -1) & (df.b != -1)]
df2 = df[(df.a == -1) | (df.b == -1)]
点击查看英文原文

A little mathematical logic theory here:

“NOT a AND NOT b” is the same as “NOT (a OR b)”, so:

“a NOT -1 AND b NOT -1” is equivalent of “NOT (a is -1 OR b is -1)”, which is opposite (Complement) of “(a is -1 OR b is -1)”.

So if you want exact opposite result, df1 and df2 should be as below:

df1 = df[(df.a != -1) & (df.b != -1)]
df2 = df[(df.a == -1) | (df.b == -1)]