Tensorflow大步向前

问题:Tensorflow大步向前

我想了解的进步在tf.nn.avg_pool,tf.nn.max_pool,tf.nn.conv2d说法。

文件反复说

步幅:长度大于等于4的整数的列表。输入张量每个维度的滑动窗口的步幅。

我的问题是:

  1. 4个以上的整数分别代表什么?
  2. 对于卷积网络,为什么必须要有stride [0] = strides [3] = 1?
  3. 此示例中,我们看到了tf.reshape(_X,shape=[-1, 28, 28, 1])。为什么是-1?

遗憾的是,文档中使用-1进行重塑的示例并不能很好地解释这种情况。

I am trying to understand the strides argument in tf.nn.avg_pool, tf.nn.max_pool, tf.nn.conv2d.

The documentation repeatedly says

strides: A list of ints that has length >= 4. The stride of the sliding window for each dimension of the input tensor.

My questions are:

  1. What do each of the 4+ integers represent?
  2. Why must they have strides[0] = strides[3] = 1 for convnets?
  3. In this example we see tf.reshape(_X,shape=[-1, 28, 28, 1]). Why -1?

Sadly the examples in the docs for reshape using -1 don’t translate too well to this scenario.


回答 0

池化和卷积运算在输入张量上滑动一个“窗口”。使用tf.nn.conv2d作为一个例子:如果输入张量有4个方面: [batch, height, width, channels],则卷积在二维窗口上操作height, width的尺寸。

strides确定窗口在每个维度上的移动量。典型用法是将第一个(批处理)和最后一个(深度)跨度设置为1。

让我们使用一个非常具体的示例:在32×32灰度输入图像上运行2-d卷积。我说灰度是因为输入图像的深度为1,这有助于使其简单。让该图像看起来像这样:

00 01 02 03 04 ...
10 11 12 13 14 ...
20 21 22 23 24 ...
30 31 32 33 34 ...
...

让我们在一个示例(批处理大小= 1)上运行2×2卷积窗口。我们给卷积的输出通道深度为8。

卷积的输入为shape=[1, 32, 32, 1]

如果指定strides=[1,1,1,1]padding=SAME,则滤波器的输出将是[1,32,32,8]。

过滤器将首先为以下内容创建输出:

F(00 01
  10 11)

然后针对:

F(01 02
  11 12)

等等。然后它将移至第二行,计算:

F(10, 11
  20, 21)

然后

F(11, 12
  21, 22)

如果将跨度指定为[1、2、2、1],则不会重叠窗口。它将计算:

F(00, 01
  10, 11)

然后

F(02, 03
  12, 13)

跨步操作对于池操作员类似。

问题2:为何大步走向[1,x,y,1]

第一个是批处理:您通常不想跳过批处理中的示例,否则您不应该首先将它们包括在内。:)

最后一个是卷积的深度:出于相同的原因,您通常不想跳过输入。

conv2d运算符比较笼统,因此您可以创建卷积以使窗口沿其他维度滑动,但这在卷积网络中并不常见。典型用途是在空间上使用它们。

为什么要重塑为-1 -1是一个占位符,它表示“根据需要进行调整以匹配整个张量所需的大小”。这是使代码独立于输入批处理大小的一种方法,因此您可以更改管道,而不必在代码中的任何地方调整批处理大小。

The pooling and convolutional ops slide a “window” across the input tensor. Using tf.nn.conv2d as an example: If the input tensor has 4 dimensions: [batch, height, width, channels], then the convolution operates on a 2D window on the height, width dimensions.

strides determines how much the window shifts by in each of the dimensions. The typical use sets the first (the batch) and last (the depth) stride to 1.

Let’s use a very concrete example: Running a 2-d convolution over a 32×32 greyscale input image. I say greyscale because then the input image has depth=1, which helps keep it simple. Let that image look like this:

00 01 02 03 04 ...
10 11 12 13 14 ...
20 21 22 23 24 ...
30 31 32 33 34 ...
...

Let’s run a 2×2 convolution window over a single example (batch size = 1). We’ll give the convolution an output channel depth of 8.

The input to the convolution has shape=[1, 32, 32, 1].

If you specify strides=[1,1,1,1] with padding=SAME, then the output of the filter will be [1, 32, 32, 8].

The filter will first create an output for:

F(00 01
  10 11)

And then for:

F(01 02
  11 12)

and so on. Then it will move to the second row, calculating:

F(10, 11
  20, 21)

then

F(11, 12
  21, 22)

If you specify a stride of [1, 2, 2, 1] it won’t do overlapping windows. It will compute:

F(00, 01
  10, 11)

and then

F(02, 03
  12, 13)

The stride operates similarly for the pooling operators.

Question 2: Why strides [1, x, y, 1] for convnets

The first 1 is the batch: You don’t usually want to skip over examples in your batch, or you shouldn’t have included them in the first place. :)

The last 1 is the depth of the convolution: You don’t usually want to skip inputs, for the same reason.

The conv2d operator is more general, so you could create convolutions that slide the window along other dimensions, but that’s not a typical use in convnets. The typical use is to use them spatially.

Why reshape to -1 -1 is a placeholder that says “adjust as necessary to match the size needed for the full tensor.” It’s a way of making the code be independent of the input batch size, so that you can change your pipeline and not have to adjust the batch size everywhere in the code.


回答 1

输入是4维的,格式为: [batch_size, image_rows, image_cols, number_of_colors]

通常,跨度定义了应用操作之间的重叠。对于conv2d,它指定卷积滤波器的连续应用之间的距离是多少。特定维度中的值1表示我们在每行/列应用运算符,值2表示每秒钟/以此类推。

关于1)对于卷积重要的值是2nd和3rd,它们表示卷积滤波器在沿行和列的应用中的重叠。值[1,2,2,1]表示我们要在每隔第二行和第二列上应用过滤器。

关于2)我不知道技术限制(可能是CuDNN要求),但通常人们会沿行或列尺寸使用步幅。在批处理大小上执行此操作不一定有意义。不确定最后一个尺寸。

关于3)为其中一个维设置-1表示“为第一维设置值,以使张量中的元素总数不变”。在我们的例子中,-1将等于batch_size。

The inputs are 4 dimensional and are of form: [batch_size, image_rows, image_cols, number_of_colors]

Strides, in general, define an overlap between applying operations. In the case of conv2d, it specifies what is the distance between consecutive applications of convolutional filters. The value of 1 in a specific dimension means that we apply the operator at every row/col, the value of 2 means every second, and so on.

Re 1) The values that matter for convolutions are 2nd and 3rd and they represent the overlap in the application of the convolutional filters along rows and columns. The value of [1, 2, 2, 1] says that we want to apply the filters on every second row and column.

Re 2) I don’t know the technical limitations (might be CuDNN requirement) but typically people use strides along the rows or columns dimensions. It doesn’t necessarily make sense to do it over batch size. Not sure of the last dimension.

Re 3) Setting -1 for one of the dimension means, “set the value for the first dimension so that the total number of elements in the tensor is unchanged”. In our case, the -1 will be equal to the batch_size.


回答 2

让我们从1-dim情况下的步幅开始。

让我们假设你input = [1, 0, 2, 3, 0, 1, 1]kernel = [2, 1, 3]卷积的结果是[8, 11, 7, 9, 4],它通过滑动你的内核在输入,进行逐元素乘法和求和计算出的一切。像这样

  • 8 = 1 * 2 + 0 * 1 + 2 * 3
  • 11 = 0 * 2 + 2 * 1 + 3 * 3
  • 7 = 2 * 2 + 3 * 1 + 0 * 3
  • 9 = 3 * 2 + 0 * 1 +1 * 3
  • 4 = 0 * 2 +1 * 1 +1 * 3

在这里,我们滑动了一个元素,但没有其他任何数字可以阻止您。这个数字是您的进步。您可以将其视为仅取第s个结果就对1步卷积的结果进行下采样。

知道输入大小i,内核大小k,步幅s和填充p,您可以轻松计算出卷积的输出大小为:

在这里|| 操作员表示天花板操作。对于池化层,s = 1。


N昏暗的情况。

一旦了解到每一个暗角都是独立的,就知道了1暗角情形的数学原理,n暗角情形很容易。因此,您只需分别滑动每个尺寸。这是2-d示例。请注意,您不必在所有尺寸上都具有相同的步幅。因此,对于N维输入/内核,您应提供N个跨度。


因此,现在很容易回答您的所有问题:

  1. 4个以上的整数分别代表什么?conv2dpool告诉您此列表表示每个维度之间的跨度。注意,步幅列表的长度与内核张量的秩相同。
  2. 为什么对于卷积网络,他们必须具有大步[0] =大步3 = 1?。第一个维度是批量大小,最后一个是渠道。既不跳过批处理也不跳过通道。因此,您将它们设置为1。对于宽度/高度,您可以跳过某些内容,因此它们可能不是1。
  3. tf.reshape(_X,shape = [-1,28,28,1])。为什么是-1? tf.reshape为您提供了帮助:

    如果形状的一个分量是特殊值-1,则将计算该尺寸的大小,以便总大小保持恒定。特别地,[-1]的形状变平为1-D。形状的最多一个分量可以是-1。

Let’s start with what stride does in 1-dim case.

Let’s assume your input = [1, 0, 2, 3, 0, 1, 1] and kernel = [2, 1, 3] the result of the convolution is [8, 11, 7, 9, 4], which is calculated by sliding your kernel over the input, performing element-wise multiplication and summing everything. Like this:

  • 8 = 1 * 2 + 0 * 1 + 2 * 3
  • 11 = 0 * 2 + 2 * 1 + 3 * 3
  • 7 = 2 * 2 + 3 * 1 + 0 * 3
  • 9 = 3 * 2 + 0 * 1 + 1 * 3
  • 4 = 0 * 2 + 1 * 1 + 1 * 3

Here we slide by one element, but nothing stops you by using any other number. This number is your stride. You can think about it as downsampling the result of the 1-strided convolution by just taking every s-th result.

Knowing the input size i, kernel size k, stride s and padding p you can easily calculate the output size of the convolution as:

Here || operator means ceiling operation. For a pooling layer s = 1.


N-dim case.

Knowing the math for a 1-dim case, n-dim case is easy once you see that each dim is independent. So you just slide each dimension separately. Here is an example for 2-d. Notice that you do not need to have the same stride at all the dimensions. So for an N-dim input/kernel you should provide N strides.


So now it is easy to answer all your questions:

  1. What do each of the 4+ integers represent?. conv2d, pool tells you that this list represents the strides among each dimension. Notice that the length of strides list is the same as the rank of kernel tensor.
  2. Why must they have strides[0] = strides3 = 1 for convnets?. The first dimension is batch size, the last is channels. There is no point of skipping neither batch nor channel. So you make them 1. For width/height you can skip something and that’s why they might be not 1.
  3. tf.reshape(_X,shape=[-1, 28, 28, 1]). Why -1? tf.reshape has it covered for you:

    If one component of shape is the special value -1, the size of that dimension is computed so that the total size remains constant. In particular, a shape of [-1] flattens into 1-D. At most one component of shape can be -1.


回答 3

@dga在解释方面做得非常出色,我无法感激它的帮助。我将以类似的方式分享我stride在3D卷积中如何工作的发现。

根据conv3d 上的TensorFlow文档,输入的形状必须按以下顺序排列:

[batch, in_depth, in_height, in_width, in_channels]

让我们使用一个示例从最右到左解释变量。假设输入形状为 input_shape = [1000,16,112,112,3]

input_shape[4] is the number of colour channels (RGB or whichever format it is extracted in)
input_shape[3] is the width of the image
input_shape[2] is the height of the image
input_shape[1] is the number of frames that have been lumped into 1 complete data
input_shape[0] is the number of lumped frames of images we have.

以下是有关如何使用步幅的摘要文档。

步幅:长度> = 5的整数列表。长度为5的1-D张量。每个输入维度的滑动窗口的步幅。一定有strides[0] = strides[4] = 1

正如许多作品所指出的那样,步幅仅表示窗口或内核距离最近的元素(无论是数据帧还是像素)有几步的距离(顺便说一句)。

从以上文档中可以看出,3D中的跨度应为以下跨度=(1,XYZ,1)。

文档强调了这一点strides[0] = strides[4] = 1

strides[0]=1 means that we do not want to skip any data in the batch 
strides[4]=1 means that we do not want to skip in the channel 

strides [X]表示我们应在集总帧中进行多少次跳过。因此,例如,如果我们有16帧,则X = 1表示使用每帧。X = 2表示每隔一帧使用一次,然后继续

strides [y]和strides [z]按照@dga的说明进行操作,因此我不会重做该部分。

但是,在keras中,您只需要指定3个整数的元组/列表,即可指定沿每个空间维度的卷积步幅,其中空间维度为stride [x],strides [y]和strides [z]。strides [0]和strides [4]已默认为1。

我希望有人觉得这有帮助!

@dga has done a wonderful job explaining and I can’t be thankful enough how helpful it has been. In the like manner, I will like to share my findings on how stride works in 3D convolution.

According to the TensorFlow documentation on conv3d, the shape of the input must be in this order:

[batch, in_depth, in_height, in_width, in_channels]

Let’s explain the variables from the extreme right to the left using an example. Assuming the input shape is input_shape = [1000,16,112,112,3]

input_shape[4] is the number of colour channels (RGB or whichever format it is extracted in)
input_shape[3] is the width of the image
input_shape[2] is the height of the image
input_shape[1] is the number of frames that have been lumped into 1 complete data
input_shape[0] is the number of lumped frames of images we have.

Below is a summary documentation for how stride is used.

strides: A list of ints that has length >= 5. 1-D tensor of length 5. The stride of the sliding window for each dimension of input. Must have strides[0] = strides[4] = 1

As indicated in many works, strides simply mean how many steps away a window or kernel jumps away from the closest element, be it a data frame or pixel (this is paraphrased by the way).

From the above documentation, a stride in 3D will look like this strides = (1,X,Y,Z,1).

The documentation emphasizes that strides[0] = strides[4] = 1.

strides[0]=1 means that we do not want to skip any data in the batch 
strides[4]=1 means that we do not want to skip in the channel 

strides[X] means how many skips we should make in the lumped frames. So for example, if we have 16 frames, X=1 means use every frame. X=2 means use every second frame and it goes and on

strides[y] and strides[z] follow the explanation by @dga so I will not redo that part.

In keras however, you only need to specify a tuple/list of 3 integers, specifying the strides of the convolution along each spatial dimension, where spatial dimension is stride[x], strides[y] and strides[z]. strides[0] and strides[4] is already defaulted to 1.

I hope someone finds this helpful!