标签归档:bit-manipulation

按位操作和用法

问题:按位操作和用法

考虑以下代码:

x = 1        # 0001
x << 2       # Shift left 2 bits: 0100
# Result: 4

x | 2        # Bitwise OR: 0011
# Result: 3

x & 1        # Bitwise AND: 0001
# Result: 1

我可以用Python(和其他语言)理解算术运算符,但是我对“按位”运算符却不太了解。在上面的示例(来自Python书)中,我了解了左移功能,但不了解其他两个。

另外,按位运算符实际上是做什么用的?我会喜欢一些例子。

Consider this code:

x = 1        # 0001
x << 2       # Shift left 2 bits: 0100
# Result: 4

x | 2        # Bitwise OR: 0011
# Result: 3

x & 1        # Bitwise AND: 0001
# Result: 1

I can understand the arithmetic operators in Python (and other languages), but I never understood ‘bitwise’ operators quite well. In the above example (from a Python book), I understand the left-shift but not the other two.

Also, what are bitwise operators actually used for? I’d appreciate some examples.


回答 0

按位运算符是对多位值进行运算的运算符,但从概念上讲一次只能运算一位。

  • AND仅当其两个输入均为1时,才为1;否则为0。
  • OR如果其输入之一或全部为1,则为1,否则为0。
  • XOR仅当其输入之一恰好为1时为1,否则为0。
  • NOT 仅当其输入为0时为1,否则为0。

这些通常可以最好地显示为真值表。输入的可能性在顶部和左侧,结果位是在输入交点处显示的四个值之一(如果不是,则为两个,因为它只有一个输入)。

AND | 0 1     OR | 0 1     XOR | 0 1    NOT | 0 1
----+-----    ---+----     ----+----    ----+----
 0  | 0 0      0 | 0 1       0 | 0 1        | 1 0
 1  | 0 1      1 | 1 1       1 | 1 0

一个示例是,如果您只想要整数的低4位,则将其与15(二进制1111)进行与,则:

    201: 1100 1001
AND  15: 0000 1111
------------------
 IS   9  0000 1001

在那种情况下,15中的零位有效地充当了过滤器,迫使结果中的位也为零。

此外,>><<通常作为按位运算符包含在内,它们分别将值“左”右移和左移一定数量的比特,丢掉将要移向的端点的比特,并在该比特处输入零。另一端。

因此,例如:

1001 0101 >> 2 gives 0010 0101
1111 1111 << 4 gives 1111 0000

请注意,Python中的左移是不寻常的,因为它没有使用固定的宽度来丢弃位-虽然许多语言根据数据类型使用固定的宽度,但是Python只是扩展宽度以迎合额外的位。为了获得Python中的丢弃行为,您可以按位向左移动,and例如在8位值中向左移动四位:

bits8 = (bits8 << 4) & 255

考虑到这一点,位运算符的另一个例子是,如果你有两个4位值要打包成一个8位的一个,你可以使用所有这三个操作员的(left-shiftandor):

packed_val = ((val1 & 15) << 4) | (val2 & 15)
  • & 15操作将确保两个值仅具有低4位。
  • << 4是一个4位左移位移动val1进入前的8位值的4位。
  • |简单地结合了这两者结合起来。

如果val1为7且val2为4:

                val1            val2
                ====            ====
 & 15 (and)   xxxx-0111       xxxx-0100  & 15
 << 4 (left)  0111-0000           |
                  |               |
                  +-------+-------+
                          |
| (or)                0111-0100

Bitwise operators are operators that work on multi-bit values, but conceptually one bit at a time.

  • AND is 1 only if both of its inputs are 1, otherwise it’s 0.
  • OR is 1 if one or both of its inputs are 1, otherwise it’s 0.
  • XOR is 1 only if exactly one of its inputs are 1, otherwise it’s 0.
  • NOT is 1 only if its input is 0, otherwise it’s 0.

These can often be best shown as truth tables. Input possibilities are on the top and left, the resultant bit is one of the four (two in the case of NOT since it only has one input) values shown at the intersection of the inputs.

AND | 0 1     OR | 0 1     XOR | 0 1    NOT | 0 1
----+-----    ---+----     ----+----    ----+----
 0  | 0 0      0 | 0 1       0 | 0 1        | 1 0
 1  | 0 1      1 | 1 1       1 | 1 0

One example is if you only want the lower 4 bits of an integer, you AND it with 15 (binary 1111) so:

    201: 1100 1001
AND  15: 0000 1111
------------------
 IS   9  0000 1001

The zero bits in 15 in that case effectively act as a filter, forcing the bits in the result to be zero as well.

In addition, >> and << are often included as bitwise operators, and they “shift” a value respectively right and left by a certain number of bits, throwing away bits that roll of the end you’re shifting towards, and feeding in zero bits at the other end.

So, for example:

1001 0101 >> 2 gives 0010 0101
1111 1111 << 4 gives 1111 0000

Note that the left shift in Python is unusual in that it’s not using a fixed width where bits are discarded – while many languages use a fixed width based on the data type, Python simply expands the width to cater for extra bits. In order to get the discarding behaviour in Python, you can follow a left shift with a bitwise and such as in an 8-bit value shifting left four bits:

bits8 = (bits8 << 4) & 255

With that in mind, another example of bitwise operators is if you have two 4-bit values that you want to pack into an 8-bit one, you can use all three of your operators (left-shift, and and or):

packed_val = ((val1 & 15) << 4) | (val2 & 15)
  • The & 15 operation will make sure that both values only have the lower 4 bits.
  • The << 4 is a 4-bit shift left to move val1 into the top 4 bits of an 8-bit value.
  • The | simply combines these two together.

If val1 is 7 and val2 is 4:

                val1            val2
                ====            ====
 & 15 (and)   xxxx-0111       xxxx-0100  & 15
 << 4 (left)  0111-0000           |
                  |               |
                  +-------+-------+
                          |
| (or)                0111-0100

回答 1

一种典型用法:

| 用于将某个位设置为1

& 用于测试或清除特定位

  • 设置一个位(其中n是位数,0是最低有效位):

    unsigned char a |= (1 << n);

  • 清除一点:

    unsigned char b &= ~(1 << n);

  • 切换一下:

    unsigned char c ^= (1 << n);

  • 测试一下:

    unsigned char e = d & (1 << n);

以您的清单为例:

x | 2用于将第1位的设置x为1

x & 1用于测试第0位x是1还是0

One typical usage:

| is used to set a certain bit to 1

& is used to test or clear a certain bit

  • Set a bit (where n is the bit number, and 0 is the least significant bit):

    unsigned char a |= (1 << n);

  • Clear a bit:

    unsigned char b &= ~(1 << n);

  • Toggle a bit:

    unsigned char c ^= (1 << n);

  • Test a bit:

    unsigned char e = d & (1 << n);

Take the case of your list for example:

x | 2 is used to set bit 1 of x to 1

x & 1 is used to test if bit 0 of x is 1 or 0


回答 2

实际使用的按位运算符是什么?我会喜欢一些例子。

按位运算的最常见用途之一是解析十六进制颜色。

例如,这是一个Python函数,该函数接受类似于String的字符串,#FF09BE并返回其Red,Green和Blue值的元组。

def hexToRgb(value):
    # Convert string to hexadecimal number (base 16)
    num = (int(value.lstrip("#"), 16))

    # Shift 16 bits to the right, and then binary AND to obtain 8 bits representing red
    r = ((num >> 16) & 0xFF)

    # Shift 8 bits to the right, and then binary AND to obtain 8 bits representing green
    g = ((num >> 8) & 0xFF)

    # Simply binary AND to obtain 8 bits representing blue
    b = (num & 0xFF)
    return (r, g, b)

我知道有达到此目的的更有效方法,但是我相信这是一个非常简洁的示例,说明了移位和按位布尔运算。

what are bitwise operators actually used for? I’d appreciate some examples.

One of the most common uses of bitwise operations is for parsing hexadecimal colours.

For example, here’s a Python function that accepts a String like #FF09BE and returns a tuple of its Red, Green and Blue values.

def hexToRgb(value):
    # Convert string to hexadecimal number (base 16)
    num = (int(value.lstrip("#"), 16))

    # Shift 16 bits to the right, and then binary AND to obtain 8 bits representing red
    r = ((num >> 16) & 0xFF)

    # Shift 8 bits to the right, and then binary AND to obtain 8 bits representing green
    g = ((num >> 8) & 0xFF)

    # Simply binary AND to obtain 8 bits representing blue
    b = (num & 0xFF)
    return (r, g, b)

I know that there are more efficient ways to acheive this, but I believe that this is a really concise example illustrating both shifts and bitwise boolean operations.


回答 3

我认为问题的第二部分:

另外,按位运算符实际上是做什么用的?我会喜欢一些例子。

仅得到部分解决。这是我的两分钱。

在处理许多应用程序时,编程语言中的按位运算起着基本作用。几乎所有低层计算都必须使用此类操作来完成。

在所有需要在两个节点之间发送数据的应用程序中,例如:

  • 计算机网络;

  • 电信应用(蜂窝电话,卫星通信等)。

在通信的较低层,数据通常以所谓的frame发送。帧只是通过物理通道发送的字节字符串。该帧通常包含实际数据以及一些其他字段(以字节为单位),这些字段是标头的一部分。标头通常包含字节,这些字节编码一些与通信状态有关的信息(例如,带有标志(位)),帧计数器,校正和错误检测代码等。要在帧中获取传输的数据并构建帧帧发送数据,则需要确定按位操作。

通常,在处理此类应用程序时,可以使用API​​,因此您不必处理所有这些细节。例如,所有现代编程语言都提供用于套接字连接的库,因此您实际上不需要构建TCP / IP通信框架。但是,请考虑为您编程这些API的优秀人员,他们肯定必须处理框架构造;使用各种按位运算来从低级通信到高级通信。

举一个具体的例子,假设有人给您一个文件,其中包含直接由电信硬件捕获的原始数据。在这种情况下,为了找到帧,您将需要读取文件中的原始字节,并通过逐位扫描数据来尝试找到某种同步字。在识别了同步字之后,您将需要获取实际的帧,并在必要时(并只是故事的开始)按Shift键以获取正在传输的实际数据。

另一个非常不同的低层应用程序系列是当您需要使用某些(较旧的)端口(例如并行端口和串行端口)来控制硬件时。通过设置一些字节来控制此端口,就指令而言,该字节的每个位对该端口具有特定含义(例如,请参见http://en.wikipedia.org/wiki/Parallel_port)。如果要构建可以对该硬件执行某些操作的软件,则将需要按位操作以将要执行的指令转换为端口可以理解的字节。

例如,如果您有一些物理按钮连接到并行端口以控制其他设备,则可以在软件应用程序中找到以下代码行:

read = ((read ^ 0x80) >> 4) & 0x0f; 

希望这能有所作为。

I think that the second part of the question:

Also, what are bitwise operators actually used for? I’d appreciate some examples.

Has been only partially addressed. These are my two cents on that matter.

Bitwise operations in programming languages play a fundamental role when dealing with a lot of applications. Almost all low-level computing must be done using this kind of operations.

In all applications that need to send data between two nodes, such as:

  • computer networks;

  • telecommunication applications (cellular phones, satellite communications, etc).

In the lower level layer of communication, the data is usually sent in what is called frames. Frames are just strings of bytes that are sent through a physical channel. This frames usually contain the actual data plus some other fields (coded in bytes) that are part of what is called the header. The header usually contains bytes that encode some information related to the status of the communication (e.g, with flags (bits)), frame counters, correction and error detection codes, etc. To get the transmitted data in a frame, and to build the frames to send data, you will need for sure bitwise operations.

In general, when dealing with that kind of applications, an API is available so you don’t have to deal with all those details. For example, all modern programming languages provide libraries for socket connections, so you don’t actually need to build the TCP/IP communication frames. But think about the good people that programmed those APIs for you, they had to deal with frame construction for sure; using all kinds of bitwise operations to go back and forth from the low-level to the higher-level communication.

As a concrete example, imagine some one gives you a file that contains raw data that was captured directly by telecommunication hardware. In this case, in order to find the frames, you will need to read the raw bytes in the file and try to find some kind of synchronization words, by scanning the data bit by bit. After identifying the synchronization words, you will need to get the actual frames, and SHIFT them if necessary (and that is just the start of the story) to get the actual data that is being transmitted.

Another very different low level family of application is when you need to control hardware using some (kind of ancient) ports, such as parallel and serial ports. This ports are controlled by setting some bytes, and each bit of that bytes has a specific meaning, in terms of instructions, for that port (see for instance http://en.wikipedia.org/wiki/Parallel_port). If you want to build software that does something with that hardware you will need bitwise operations to translate the instructions you want to execute to the bytes that the port understand.

For example, if you have some physical buttons connected to the parallel port to control some other device, this is a line of code that you can find in the soft application:

read = ((read ^ 0x80) >> 4) & 0x0f; 

Hope this contributes.


回答 4

我希望这可以澄清这两个问题:

x | 2

0001 //x
0010 //2

0011 //result = 3

x & 1

0001 //x
0001 //1

0001 //result = 1

I hope this clarifies those two:

x | 2

0001 //x
0010 //2

0011 //result = 3

x & 1

0001 //x
0001 //1

0001 //result = 1

回答 5

将0视为假,将1视为真。然后按位的and(&)和or(|)就像常规的and和或或一样工作,只不过它们一次完成值中的所有位。通常,如果您可以设置30个选项(例如,在窗口上绘制样式),而又不想传递30个单独的布尔值来设置或取消设置每个布尔值,则可以将它们用作标志。将选项合并为一个值,然后使用&检查是否设置了每个选项。OpenGL大量使用这种标志传递方式。由于每个位都是一个单独的标志,因此您将获得以2(即仅设置了一位的数字)为幂的标志值1(2 ^ 0)2(2 ^ 1)4(2 ^ 2)8(2 ^ 3) 2的幂可以告诉您如果标志打开则将哪个位置1。

还要注意2 = 10,所以x | 2是110(6)而不是111(7)如果没有位重叠(在这种情况下为true)| 就像加法一样。

Think of 0 as false and 1 as true. Then bitwise and(&) and or(|) work just like regular and and or except they do all of the bits in the value at once. Typically you will see them used for flags if you have 30 options that can be set (say as draw styles on a window) you don’t want to have to pass in 30 separate boolean values to set or unset each one so you use | to combine options into a single value and then you use & to check if each option is set. This style of flag passing is heavily used by OpenGL. Since each bit is a separate flag you get flag values on powers of two(aka numbers that have only one bit set) 1(2^0) 2(2^1) 4(2^2) 8(2^3) the power of two tells you which bit is set if the flag is on.

Also note 2 = 10 so x|2 is 110(6) not 111(7) If none of the bits overlap(which is true in this case) | acts like addition.


回答 6

我没有看到上面提到的内容,但是您还会看到一些人使用左右移位进行算术运算。左移x等于乘以2 ^ x(只要它不会溢出),右移等同于除以2 ^ x。

最近,我看到人们使用x << 1和x >> 1来加倍和减半,尽管我不确定他们是否只是想变得聪明,还是真的比普通运算符有明显的优势。

I didn’t see it mentioned above but you will also see some people use left and right shift for arithmetic operations. A left shift by x is equivalent to multiplying by 2^x (as long as it doesn’t overflow) and a right shift is equivalent to dividing by 2^x.

Recently I’ve seen people using x << 1 and x >> 1 for doubling and halving, although I’m not sure if they are just trying to be clever or if there really is a distinct advantage over the normal operators.


回答 7

套装

可以使用数学运算来组合集合。

  • 联合运算符|将两个集合组合在一起,形成一个新集合,其中两个集合都包含项。
  • 交集运算符&仅在两个项中都获得项。
  • 差异运算符-在第一组中获得项目,但在第二组中则没有。
  • 对称差运算符^获取任一集合中的项目,但不能同时获取两者。

自己尝试:

first = {1, 2, 3, 4, 5, 6}
second = {4, 5, 6, 7, 8, 9}

print(first | second)

print(first & second)

print(first - second)

print(second - first)

print(first ^ second)

结果:

{1, 2, 3, 4, 5, 6, 7, 8, 9}

{4, 5, 6}

{1, 2, 3}

{8, 9, 7}

{1, 2, 3, 7, 8, 9}

Sets

Sets can be combined using mathematical operations.

  • The union operator | combines two sets to form a new one containing items in either.
  • The intersection operator & gets items only in both.
  • The difference operator - gets items in the first set but not in the second.
  • The symmetric difference operator ^ gets items in either set, but not both.

Try It Yourself:

first = {1, 2, 3, 4, 5, 6}
second = {4, 5, 6, 7, 8, 9}

print(first | second)

print(first & second)

print(first - second)

print(second - first)

print(first ^ second)

Result:

{1, 2, 3, 4, 5, 6, 7, 8, 9}

{4, 5, 6}

{1, 2, 3}

{8, 9, 7}

{1, 2, 3, 7, 8, 9}

回答 8

本示例将向您显示所有四个2位值的操作:

10 | 12

1010 #decimal 10
1100 #decimal 12

1110 #result = 14

10 & 12

1010 #decimal 10
1100 #decimal 12

1000 #result = 8

这是用法的一个示例:

x = raw_input('Enter a number:')
print 'x is %s.' % ('even', 'odd')[x&1]

This example will show you the operations for all four 2 bit values:

10 | 12

1010 #decimal 10
1100 #decimal 12

1110 #result = 14

10 & 12

1010 #decimal 10
1100 #decimal 12

1000 #result = 8

Here is one example of usage:

x = raw_input('Enter a number:')
print 'x is %s.' % ('even', 'odd')[x&1]

回答 9

另一个常见用例是操纵/测试文件权限。请参阅Python stat模块:http : //docs.python.org/library/stat.html

例如,要将文件的权限与所需的权限集进行比较,可以执行以下操作:

import os
import stat

#Get the actual mode of a file
mode = os.stat('file.txt').st_mode

#File should be a regular file, readable and writable by its owner
#Each permission value has a single 'on' bit.  Use bitwise or to combine 
#them.
desired_mode = stat.S_IFREG|stat.S_IRUSR|stat.S_IWUSR

#check for exact match:
mode == desired_mode
#check for at least one bit matching:
bool(mode & desired_mode)
#check for at least one bit 'on' in one, and not in the other:
bool(mode ^ desired_mode)
#check that all bits from desired_mode are set in mode, but I don't care about 
# other bits.
not bool((mode^desired_mode)&desired_mode)

我将结果转换为布尔值,因为我只关心真实性或虚假性,但是打印每个值的bin()值将是值得进行的练习。

Another common use-case is manipulating/testing file permissions. See the Python stat module: http://docs.python.org/library/stat.html.

For example, to compare a file’s permissions to a desired permission set, you could do something like:

import os
import stat

#Get the actual mode of a file
mode = os.stat('file.txt').st_mode

#File should be a regular file, readable and writable by its owner
#Each permission value has a single 'on' bit.  Use bitwise or to combine 
#them.
desired_mode = stat.S_IFREG|stat.S_IRUSR|stat.S_IWUSR

#check for exact match:
mode == desired_mode
#check for at least one bit matching:
bool(mode & desired_mode)
#check for at least one bit 'on' in one, and not in the other:
bool(mode ^ desired_mode)
#check that all bits from desired_mode are set in mode, but I don't care about 
# other bits.
not bool((mode^desired_mode)&desired_mode)

I cast the results as booleans, because I only care about the truth or falsehood, but it would be a worthwhile exercise to print out the bin() values for each one.


回答 10

在科学计算中,整数的位表示形式经常用于表示真假信息的数组,因为按位运算比迭代布尔数组快得多。(高级语言可能会使用位数组的概念。)

一个很好且相当简单的例子是Nim游戏的一般解决方案。看一下Wikipedia页面Python代码。它大量使用按位异或。^

Bit representations of integers are often used in scientific computing to represent arrays of true-false information because a bitwise operation is much faster than iterating through an array of booleans. (Higher level languages may use the idea of a bit array.)

A nice and fairly simple example of this is the general solution to the game of Nim. Take a look at the Python code on the Wikipedia page. It makes heavy use of bitwise exclusive or, ^.


回答 11

可能有更好的方法来查找数组元素在两个值之间的位置,但是如本示例所示,在此处有效,而and则无效。

import numpy as np
a=np.array([1.2, 2.3, 3.4])
np.where((a>2) and (a<3))      
#Result: Value Error
np.where((a>2) & (a<3))
#Result: (array([1]),)

There may be a better way to find where an array element is between two values, but as this example shows, the & works here, whereas and does not.

import numpy as np
a=np.array([1.2, 2.3, 3.4])
np.where((a>2) and (a<3))      
#Result: Value Error
np.where((a>2) & (a<3))
#Result: (array([1]),)

回答 12

我没有看到它,本示例将为您显示2位值的(-)十进制运算:AB(仅当A包含B时)

当我们在程序中持有表示位的动词时,需要执行此操作。有时我们需要添加位(如上),有时我们需要删除位(如果动词包含)

111 #decimal 7
-
100 #decimal 4
--------------
011 #decimal 3

使用python: 7&〜4 = 3(从7删除代表4的位)

001 #decimal 1
-
100 #decimal 4
--------------
001 #decimal 1

使用python: 1&〜4 = 1(从1删除代表4的位-在这种情况下1不是“包含” 4)。

i didnt see it mentioned, This example will show you the (-) decimal operation for 2 bit values: A-B (only if A contains B)

this operation is needed when we hold an verb in our program that represent bits. sometimes we need to add bits (like above) and sometimes we need to remove bits (if the verb contains then)

111 #decimal 7
-
100 #decimal 4
--------------
011 #decimal 3

with python: 7 & ~4 = 3 (remove from 7 the bits that represent 4)

001 #decimal 1
-
100 #decimal 4
--------------
001 #decimal 1

with python: 1 & ~4 = 1 (remove from 1 the bits that represent 4 – in this case 1 is not ‘contains’ 4)..


回答 13

操纵整数的位很有用,但对于网络协议(可能一直指定到位)通常是有用的,但是可能需要操纵更长的字节序列(不容易转换为一个整数)。在这种情况下,使用允许对数据按位进行操作的位库很有用-例如,可以将字符串“ ABCDEFGHIJKLMNOPQ”作为字符串或十六进制导入并对其进行位移(或执行其他按位操作):

>>> import bitstring
>>> bitstring.BitArray(bytes='ABCDEFGHIJKLMNOPQ') << 4
BitArray('0x142434445464748494a4b4c4d4e4f50510')
>>> bitstring.BitArray(hex='0x4142434445464748494a4b4c4d4e4f5051') << 4
BitArray('0x142434445464748494a4b4c4d4e4f50510')

Whilst manipulating bits of an integer is useful, often for network protocols, which may be specified down to the bit, one can require manipulation of longer byte sequences (which aren’t easily converted into one integer). In this case it is useful to employ the bitstring library which allows for bitwise operations on data – e.g. one can import the string ‘ABCDEFGHIJKLMNOPQ’ as a string or as hex and bit shift it (or perform other bitwise operations):

>>> import bitstring
>>> bitstring.BitArray(bytes='ABCDEFGHIJKLMNOPQ') << 4
BitArray('0x142434445464748494a4b4c4d4e4f50510')
>>> bitstring.BitArray(hex='0x4142434445464748494a4b4c4d4e4f5051') << 4
BitArray('0x142434445464748494a4b4c4d4e4f50510')

回答 14

以下按位运算符:| ^返回值(基于它们的输入),其逻辑门影响信号的方式相同。您可以使用它们来仿真电路。

the following bitwise operators: &, |, ^, and ~ return values (based on their input) in the same way logic gates affect signals. You could use them to emulate circuits.


回答 15

要翻转位(即1的补码/取反),可以执行以下操作:

由于ExORed与全1的值会导致取反,因此对于给定的位宽,您可以使用ExOR对其进行取反。

In Binary
a=1010 --> this is 0xA or decimal 10
then 
c = 1111 ^ a = 0101 --> this is 0xF or decimal 15
-----------------
In Python
a=10
b=15
c = a ^ b --> 0101
print(bin(c)) # gives '0b101'

To flip bits (i.e. 1’s complement/invert) you can do the following:

Since value ExORed with all 1s results into inversion, for a given bit width you can use ExOR to invert them.

In Binary
a=1010 --> this is 0xA or decimal 10
then 
c = 1111 ^ a = 0101 --> this is 0xF or decimal 15
-----------------
In Python
a=10
b=15
c = a ^ b --> 0101
print(bin(c)) # gives '0b101'

‘and’(布尔值)vs’&’(按位)-为什么列表与numpy数组在行为上有所不同?

问题:’and’(布尔值)vs’&’(按位)-为什么列表与numpy数组在行为上有所不同?

是什么解释了列表和NumPy数组上布尔运算和按位运算的行为差异?

&and在Python中正确使用vs 感到困惑,如以下示例所示。

mylist1 = [True,  True,  True, False,  True]
mylist2 = [False, True, False,  True, False]

>>> len(mylist1) == len(mylist2)
True

# ---- Example 1 ----
>>> mylist1 and mylist2
[False, True, False, True, False]
# I would have expected [False, True, False, False, False]

# ---- Example 2 ----
>>> mylist1 & mylist2
TypeError: unsupported operand type(s) for &: 'list' and 'list'
# Why not just like example 1?

>>> import numpy as np

# ---- Example 3 ----
>>> np.array(mylist1) and np.array(mylist2)
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
# Why not just like Example 4?

# ---- Example 4 ----
>>> np.array(mylist1) & np.array(mylist2)
array([False,  True, False, False, False], dtype=bool)
# This is the output I was expecting!

这个答案这个答案帮助我理解这and是一个布尔运算,但是&是按位运算。

我阅读了有关按位运算的信息,以更好地理解该概念,但是我正在努力使用该信息来理解我上面的四个示例。

示例4使我达到了期望的输出,这很好,但是我仍然对何时/如何/为什么使用andvs 感到困惑&。为什么列表和NumPy数组在这些运算符上的行为不同?

谁能帮助我了解布尔运算和按位运算之间的区别,以解释为什么它们对列表和NumPy数组的处理方式不同?

What explains the difference in behavior of boolean and bitwise operations on lists vs NumPy arrays?

I’m confused about the appropriate use of & vs and in Python, illustrated in the following examples.

mylist1 = [True,  True,  True, False,  True]
mylist2 = [False, True, False,  True, False]

>>> len(mylist1) == len(mylist2)
True

# ---- Example 1 ----
>>> mylist1 and mylist2
[False, True, False, True, False]
# I would have expected [False, True, False, False, False]

# ---- Example 2 ----
>>> mylist1 & mylist2
TypeError: unsupported operand type(s) for &: 'list' and 'list'
# Why not just like example 1?

>>> import numpy as np

# ---- Example 3 ----
>>> np.array(mylist1) and np.array(mylist2)
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
# Why not just like Example 4?

# ---- Example 4 ----
>>> np.array(mylist1) & np.array(mylist2)
array([False,  True, False, False, False], dtype=bool)
# This is the output I was expecting!

This answer and this answer helped me understand that and is a boolean operation but & is a bitwise operation.

I read about bitwise operations to better understand the concept, but I am struggling to use that information to make sense of my above 4 examples.

Example 4 led me to my desired output, so that is fine, but I am still confused about when/how/why I should use and vs &. Why do lists and NumPy arrays behave differently with these operators?

Can anyone help me understand the difference between boolean and bitwise operations to explain why they handle lists and NumPy arrays differently?


回答 0

and测试两个表达式在逻辑上是否相符,True&(当与True/ False值一起使用时)测试两个表达式是否均在逻辑上True

在Python中,通常将空的内置对象在逻辑上视为,False而将非空的内置对象在逻辑上视为True。这可以简化常见的用例,在这种情况下,如果列表为空,则要执行某项操作;如果列表不为空,则要执行其他操作。注意,这意味着列表[False]在逻辑上是True

>>> if [False]:
...    print 'True'
...
True

因此,在示例1中,第一个列表是非空的,因此在逻辑上是True,因此的真值and与第二个列表的真值相同。(在我们的例子中,第二个列表是非空的,因此从逻辑上讲是True,但要识别出该列表将需要不必要的计算步骤。)

例如,列表2不能以位方式有意义地组合,因为它们可以包含任意不同的元素。可以按位组合的事物包括:对与错,整数。

相反,NumPy对象支持矢量化计算。也就是说,它们使您可以对多个数据执行相同的操作。

示例3失败了,因为NumPy数组(长度> 1)没有真值,因为这防止了基于向量的逻辑混乱。

示例4只是一个向量化位and操作。

底线

  • 如果您不处理数组并且不执行整数的数学运算,则可能需要and

  • 如果你有真值的载体,你想结合,使用numpy&

and tests whether both expressions are logically True while & (when used with True/False values) tests if both are True.

In Python, empty built-in objects are typically treated as logically False while non-empty built-ins are logically True. This facilitates the common use case where you want to do something if a list is empty and something else if the list is not. Note that this means that the list [False] is logically True:

>>> if [False]:
...    print 'True'
...
True

So in Example 1, the first list is non-empty and therefore logically True, so the truth value of the and is the same as that of the second list. (In our case, the second list is non-empty and therefore logically True, but identifying that would require an unnecessary step of calculation.)

For example 2, lists cannot meaningfully be combined in a bitwise fashion because they can contain arbitrary unlike elements. Things that can be combined bitwise include: Trues and Falses, integers.

NumPy objects, by contrast, support vectorized calculations. That is, they let you perform the same operations on multiple pieces of data.

Example 3 fails because NumPy arrays (of length > 1) have no truth value as this prevents vector-based logic confusion.

Example 4 is simply a vectorized bit and operation.

Bottom Line

  • If you are not dealing with arrays and are not performing math manipulations of integers, you probably want and.

  • If you have vectors of truth values that you wish to combine, use numpy with &.


回答 1

关于 list

首先是非常重要的一点,一切都会随之而来(我希望)。

在普通的Python中,list它在任何方面都不是特殊的(除了具有可爱的构造语法,这主要是历史性的事故)。一旦创建了列表[3,2,6],它就具有所有意图和用途,只是一个普通的Python对象,例如number 3,set {3,7}或function lambda x: x+5

(是的,它支持更改其元素,并支持迭代和许多其他功能,但这只是类型的含义:它支持某些操作,而不支持其他一些操作。int支持提高幂,但不支持让它变得非常特别-这就是一个int是什么。lambda支持调用,但这并没有使其非常特别-这毕竟是lambda的目的:)。

关于 and

and不是运算符(您可以将其称为“运算符”,但也可以将其“用作”运算符:)。Python中的运算符是通过某种类型的对象(通常作为该类型的一部分编写)调用的方法(通过实现)。一种方法无法对其某些操作数进行求值,但是and可以(必须)这样做。

这样做的结果是and无法过载,就像for无法过载一样。它是完全通用的,并且通过指定的协议进行通信。您可以做的是自定义协议的一部分,但这并不意味着您可以and完全改变行为。该协议是:

想象一下,Python解释“ a和b”(这种方式在字面上并没有发生,但有助于理解)。当涉及“和”时,它查看刚刚评估过的对象(a),并询问:您是真的吗?(不是:是吗True?)如果您是Class的作者,则可以自定义此答案。如果a答案为“否”,and(完全跳过b,则完全不评估,并且)说:a是我的结果(不是:我的结果是False)。

如果a没有回答,and请问:您的长度是多少?(再次,您可以将其自定义为a的类的作者)。如果a答案为0,and则执行与上述相同的操作-认为它为false(NOT False),跳过b并给出a结果。

如果a第二个问题的答案不是0(“您的长度是多少”),或者根本不回答,或者第一个问题的答案为“是”(“您是真的”),则and计算b,然后说:b是我的结果。请注意,它b任何问题。

换句话说,这与a and b几乎相同b if a else a,除了a仅被评估一次。

现在,用笔和笔坐几分钟,使自己确信,当{a,b}是{True,False}的子集时,它的工作原理与您期望的布尔运算符完全相同。但是,我希望我已经说服了您,它更加通用,并且您将看到,这种方式更加有用。

将这两个放在一起

现在,我希望您理解您的示例1. and不管mylist1是数字,列表,lambda还是Argmhbl类的对象。它只关心mylist1对协议问题的回答。当然,mylist1回答有关长度的问题5,因此返回mylist2。就是这样。它与mylist1和mylist2的元素无关-它们不会在任何地方输入图片。

第二个例子:&list

另一方面,&例如,是与其他操作员一样的操作员+。可以通过在该类上定义特殊方法来为该类型定义它。int将其定义为按位“和”,而bool将其定义为逻辑“和”,但这只是一个选择:例如,集合和诸如dict键视图之类的其他对象将其定义为集合交集。list只是没有定义它,可能是因为Guido没想到任何定义它的明显方法。

麻木

另一方面:-D,numpy数组特殊的,或者至少是它们想要的。当然,numpy.array只是一个类,它不能and以任何方式覆盖,因此它做的第二件事是:当被问到“您是真的”时,numpy.array会引发ValueError,实际上是在说“请改正这个问题,我的真相视图不适合您的模型”。(请注意,ValueError消息并未and提及-因为numpy.array不知道是在问这个问题;它只是在谈论真相。)

对于&,这是完全不同的故事。numpy.array可以根据需要定义它,并且可以&与其他运算符一致地定义:逐点。这样您终于得到了想要的东西。

HTH,

About list

First a very important point, from which everything will follow (I hope).

In ordinary Python, list is not special in any way (except having cute syntax for constructing, which is mostly a historical accident). Once a list [3,2,6] is made, it is for all intents and purposes just an ordinary Python object, like a number 3, set {3,7}, or a function lambda x: x+5.

(Yes, it supports changing its elements, and it supports iteration, and many other things, but that’s just what a type is: it supports some operations, while not supporting some others. int supports raising to a power, but that doesn’t make it very special – it’s just what an int is. lambda supports calling, but that doesn’t make it very special – that’s what lambda is for, after all:).

About and

and is not an operator (you can call it “operator”, but you can call “for” an operator too:). Operators in Python are (implemented through) methods called on objects of some type, usually written as part of that type. There is no way for a method to hold an evaluation of some of its operands, but and can (and must) do that.

The consequence of that is that and cannot be overloaded, just like for cannot be overloaded. It is completely general, and communicates through a specified protocol. What you can do is customize your part of the protocol, but that doesn’t mean you can alter the behavior of and completely. The protocol is:

Imagine Python interpreting “a and b” (this doesn’t happen literally this way, but it helps understanding). When it comes to “and”, it looks at the object it has just evaluated (a), and asks it: are you true? (NOT: are you True?) If you are an author of a’s class, you can customize this answer. If a answers “no”, and (skips b completely, it is not evaluated at all, and) says: a is my result (NOT: False is my result).

If a doesn’t answer, and asks it: what is your length? (Again, you can customize this as an author of a‘s class). If a answers 0, and does the same as above – considers it false (NOT False), skips b, and gives a as result.

If a answers something other than 0 to the second question (“what is your length”), or it doesn’t answer at all, or it answers “yes” to the first one (“are you true”), and evaluates b, and says: b is my result. Note that it does NOT ask b any questions.

The other way to say all of this is that a and b is almost the same as b if a else a, except a is evaluated only once.

Now sit for a few minutes with a pen and paper, and convince yourself that when {a,b} is a subset of {True,False}, it works exactly as you would expect of Boolean operators. But I hope I have convinced you it is much more general, and as you’ll see, much more useful this way.

Putting those two together

Now I hope you understand your example 1. and doesn’t care if mylist1 is a number, list, lambda or an object of a class Argmhbl. It just cares about mylist1’s answer to the questions of the protocol. And of course, mylist1 answers 5 to the question about length, so and returns mylist2. And that’s it. It has nothing to do with elements of mylist1 and mylist2 – they don’t enter the picture anywhere.

Second example: & on list

On the other hand, & is an operator like any other, like + for example. It can be defined for a type by defining a special method on that class. int defines it as bitwise “and”, and bool defines it as logical “and”, but that’s just one option: for example, sets and some other objects like dict keys views define it as a set intersection. list just doesn’t define it, probably because Guido didn’t think of any obvious way of defining it.

numpy

On the other leg:-D, numpy arrays are special, or at least they are trying to be. Of course, numpy.array is just a class, it cannot override and in any way, so it does the next best thing: when asked “are you true”, numpy.array raises a ValueError, effectively saying “please rephrase the question, my view of truth doesn’t fit into your model”. (Note that the ValueError message doesn’t speak about and – because numpy.array doesn’t know who is asking it the question; it just speaks about truth.)

For &, it’s completely different story. numpy.array can define it as it wishes, and it defines & consistently with other operators: pointwise. So you finally get what you want.

HTH,


回答 2

不能覆盖短路布尔运算符()andor因为没有令人满意的方法就可以做到这一点,而又不会引入新的语言功能或牺牲短路。如您所知,您可能不知道,他们评估第一个操作数的真值,并根据该值评估或返回第二个参数,或者不评估第二个参数而返回第一个:

something_true and x -> x
something_false and x -> something_false
something_true or x -> something_true
something_false or x -> x

请注意,返回的是实际操作数(求值结果),而不是其真值。

定制其行为的唯一方法是重写__nonzero____bool__在Python 3中重命名为),因此您可以影响要返回的操作数,但不能返回其他值。当列表(和其他集合)根本不包含任何内容时,它们被定义为“真”,而当它们为空时,其定义为“假”。

NumPy数组拒绝该概念:对于它们针对的用例,两个不同的真理概念是相同的:(1)任何元素是否为真,以及(2)所有元素是否为真。由于这两个是完全(且无声地)不兼容的,而且都不是更正确或更普遍的,因此NumPy拒绝猜测并且要求您显式使用.any().all()

&|(和not,顺便说一句)可以完全覆盖,因为它们不会短路。当被重写时,它们可以返回任何东西,而NumPy则很好地利用了它来进行元素操作,就像它们几乎与其他任何标量操作一样。另一方面,列表不会在其元素之间广播操作。就像mylist1 - mylist2什么都没有,mylist1 + mylist2意味着完全不同一样,&列表也没有运算符。

The short-circuiting boolean operators (and, or) can’t be overriden because there is no satisfying way to do this without introducing new language features or sacrificing short circuiting. As you may or may not know, they evaluate the first operand for its truth value, and depending on that value, either evaluate and return the second argument, or don’t evaluate the second argument and return the first:

something_true and x -> x
something_false and x -> something_false
something_true or x -> something_true
something_false or x -> x

Note that the (result of evaluating the) actual operand is returned, not truth value thereof.

The only way to customize their behavior is to override __nonzero__ (renamed to __bool__ in Python 3), so you can affect which operand gets returned, but not return something different. Lists (and other collections) are defined to be “truthy” when they contain anything at all, and “falsey” when they are empty.

NumPy arrays reject that notion: For the use cases they aim at, two different notions of truth are common: (1) Whether any element is true, and (2) whether all elements are true. Since these two are completely (and silently) incompatible, and neither is clearly more correct or more common, NumPy refuses to guess and requires you to explicitly use .any() or .all().

& and | (and not, by the way) can be fully overriden, as they don’t short circuit. They can return anything at all when overriden, and NumPy makes good use of that to do element-wise operations, as they do with practically any other scalar operation. Lists, on the other hand, don’t broadcast operations across their elements. Just as mylist1 - mylist2 doesn’t mean anything and mylist1 + mylist2 means something completely different, there is no & operator for lists.


回答 3

范例1:

运算符就是这样工作的。

xy =>如果x为false,则为x,否则为y

因此,换句话说,由于mylist1不是False,因此表达式的结果为mylist2。(只有空列表的计算结果为False。)

范例2:

&操作是按位,正如你提到。按位运算仅适用于数字。ab的结果是一个由1组成的数字,而ab均为1 。例如:

>>> 3 & 1
1

使用二进制文字(上面的数字相同)更容易看到正在发生的事情:

>>> 0b0011 & 0b0001
0b0001

按位运算在概念上与布尔(真)运算相似,但是它们仅对位有效。

所以,给我一些关于我的车的陈述

  1. 我的车是红色的
  2. 我的车有轮子

这两个语句的逻辑“和”为:

(我的车是红色的吗?)和(车上有轮子吗?)=>假值的逻辑真

至少对于我的车来说,这两者都是正确的。因此,该语句的整体值在逻辑上是正确的。

这两个语句的按位“和”有点模糊:

(语句“我的车是红色的”的数值)和(语句“我的车有轮子的”的数值)=>数字

如果python知道如何将语句转换为数值,则它将这样做并计算两个值的按位和。这可能会使您相信与&可以互换and,但是与上述示例一样,它们是不同的东西。此外,对于无法转换的对象,您只会得到一个TypeError

示例3和4:

Numpy实现数组的算术运算

ndarray上的算术和比较运算被定义为逐元素运算,通常将ndarray对象作为结果。

但是不会为数组实现逻辑运算,因为您无法在python中重载逻辑运算符。这就是为什么示例三不起作用,但是示例四却不起作用的原因。

因此,回答您的andvs &问题:使用and

按位运算用于检查数字的结构(哪些位已设置,哪些位未设置)。这种信息主要用于底层操作系统接口(例如,unix许可位)。大多数python程序不需要知道这一点。

逻辑运算(andornot),然而,使用所有的时间。

Example 1:

This is how the and operator works.

x and y => if x is false, then x, else y

So in other words, since mylist1 is not False, the result of the expression is mylist2. (Only empty lists evaluate to False.)

Example 2:

The & operator is for a bitwise and, as you mention. Bitwise operations only work on numbers. The result of a & b is a number composed of 1s in bits that are 1 in both a and b. For example:

>>> 3 & 1
1

It’s easier to see what’s happening using a binary literal (same numbers as above):

>>> 0b0011 & 0b0001
0b0001

Bitwise operations are similar in concept to boolean (truth) operations, but they work only on bits.

So, given a couple statements about my car

  1. My car is red
  2. My car has wheels

The logical “and” of these two statements is:

(is my car red?) and (does car have wheels?) => logical true of false value

Both of which are true, for my car at least. So the value of the statement as a whole is logically true.

The bitwise “and” of these two statements is a little more nebulous:

(the numeric value of the statement ‘my car is red’) & (the numeric value of the statement ‘my car has wheels’) => number

If python knows how to convert the statements to numeric values, then it will do so and compute the bitwise-and of the two values. This may lead you to believe that & is interchangeable with and, but as with the above example they are different things. Also, for the objects that can’t be converted, you’ll just get a TypeError.

Example 3 and 4:

Numpy implements arithmetic operations for arrays:

Arithmetic and comparison operations on ndarrays are defined as element-wise operations, and generally yield ndarray objects as results.

But does not implement logical operations for arrays, because you can’t overload logical operators in python. That’s why example three doesn’t work, but example four does.

So to answer your and vs & question: Use and.

The bitwise operations are used for examining the structure of a number (which bits are set, which bits aren’t set). This kind of information is mostly used in low-level operating system interfaces (unix permission bits, for example). Most python programs won’t need to know that.

The logical operations (and, or, not), however, are used all the time.


回答 4

  1. 在Python的表达X and Y回报Y,鉴于bool(X) == True或任何的XY评估为False,例如:

    True and 20 
    >>> 20
    
    False and 20
    >>> False
    
    20 and []
    >>> []
  2. 根本没有为列表定义按位运算符。但是它是为整数定义的-在数字的二进制表示形式上进行操作。考虑16(01000)和31(11111):

    16 & 31
    >>> 16
  3. NumPy不是通灵的,它不知道您是否意味着例如在逻辑表达式中[False, False]应该等于True。在此方法中,它替代了标准的Python行为,即:“任何带有len(collection) == 0is的空集合False”。

  4. NumPy数组的&运算符的预期行为。

  1. In Python an expression of X and Y returns Y, given that bool(X) == True or any of X or Y evaluate to False, e.g.:

    True and 20 
    >>> 20
    
    False and 20
    >>> False
    
    20 and []
    >>> []
    
  2. Bitwise operator is simply not defined for lists. But it is defined for integers – operating over the binary representation of the numbers. Consider 16 (01000) and 31 (11111):

    16 & 31
    >>> 16
    
  3. NumPy is not a psychic, it does not know, whether you mean that e.g. [False, False] should be equal to True in a logical expression. In this it overrides a standard Python behaviour, which is: “Any empty collection with len(collection) == 0 is False“.

  4. Probably an expected behaviour of NumPy’s arrays’s & operator.


回答 5

对于第一个示例,基于Django的文档
,它将始终返回第二个列表,实际上非空列表被视为Python的True值,因此python返回“ last” True值,因此第二个列表

In [74]: mylist1 = [False]
In [75]: mylist2 = [False, True, False,  True, False]
In [76]: mylist1 and mylist2
Out[76]: [False, True, False, True, False]
In [77]: mylist2 and mylist1
Out[77]: [False]

For the first example and base on the django’s doc
It will always return the second list, indeed a non empty list is see as a True value for Python thus python return the ‘last’ True value so the second list

In [74]: mylist1 = [False]
In [75]: mylist2 = [False, True, False,  True, False]
In [76]: mylist1 and mylist2
Out[76]: [False, True, False, True, False]
In [77]: mylist2 and mylist1
Out[77]: [False]

回答 6

使用Python列表的操作在列表上进行list1 and list2将检查是否list1为空,并返回list1,如果它是,并且list2如果它不是。list1 + list2将附加list2list1,因此您将获得一个包含len(list1) + len(list2)元素的新列表。

仅在元素级应用时有意义的运算符(例如&)会引发一个TypeError,因为不支持在元素之间循环而不进行元素级操作。

Numpy数组支持按元素操作。array1 & array2将计算的按位或用于在每个相应元件array1array2array1 + array2将计算在每个相应元素的总和array1array2

这不适用于andor

array1 and array2 本质上是以下代码的简写:

if bool(array1):
    return array2
else:
    return array1

为此,您需要对进行良好的定义bool(array1)。对于像在Python列表上使用的全局操作,其定义是bool(list) == Trueif list不为空,并且False为空。对于numpy的逐元素运算,是否要检查是否有任何元素求值为True,还是要检查所有元素的取值为,都存在一些歧义True。因为两者都可以说是正确的,所以numpy不会猜测并引发(间接)在数组上调用ValueErrorwhen的bool()情况。

Operations with a Python list operate on the list. list1 and list2 will check if list1 is empty, and return list1 if it is, and list2 if it isn’t. list1 + list2 will append list2 to list1, so you get a new list with len(list1) + len(list2) elements.

Operators that only make sense when applied element-wise, such as &, raise a TypeError, as element-wise operations aren’t supported without looping through the elements.

Numpy arrays support element-wise operations. array1 & array2 will calculate the bitwise or for each corresponding element in array1 and array2. array1 + array2 will calculate the sum for each corresponding element in array1 and array2.

This does not work for and and or.

array1 and array2 is essentially a short-hand for the following code:

if bool(array1):
    return array2
else:
    return array1

For this you need a good definition of bool(array1). For global operations like used on Python lists, the definition is that bool(list) == True if list is not empty, and False if it is empty. For numpy’s element-wise operations, there is some disambiguity whether to check if any element evaluates to True, or all elements evaluate to True. Because both are arguably correct, numpy doesn’t guess and raises a ValueError when bool() is (indirectly) called on an array.


回答 7

好问题。与您在逻辑and按位运算&符上对示例1和4(或者我应该说1&4 :)的观察类似,我对运算符也有经验sum。numpy sum和py的sum行为也不同。例如:

假设“ mat”是一个numpy 5×5 2d数组,例如:

array([[ 1,  2,  3,  4,  5],
       [ 6,  7,  8,  9, 10],
       [11, 12, 13, 14, 15],
       [16, 17, 18, 19, 20],
       [21, 22, 23, 24, 25]])

然后numpy.sum(mat)给出整个矩阵的总和。而Python中的内置总和(例如sum(mat))仅沿轴求和。见下文:

np.sum(mat)  ## --> gives 325
sum(mat)     ## --> gives array([55, 60, 65, 70, 75])

Good question. Similar to the observation you have about examples 1 and 4 (or should I say 1 & 4 :) ) over logical and bitwise & operators, I experienced on sum operator. The numpy sum and py sum behave differently as well. For example:

Suppose “mat” is a numpy 5×5 2d array such as:

array([[ 1,  2,  3,  4,  5],
       [ 6,  7,  8,  9, 10],
       [11, 12, 13, 14, 15],
       [16, 17, 18, 19, 20],
       [21, 22, 23, 24, 25]])

Then numpy.sum(mat) gives total sum of the entire matrix. Whereas the built-in sum from Python such as sum(mat) totals along the axis only. See below:

np.sum(mat)  ## --> gives 325
sum(mat)     ## --> gives array([55, 60, 65, 70, 75])