问题：脾气暴躁：快速找到价值的第一指标
如何找到Numpy数组中数字首次出现的索引？速度对我很重要。我对以下答案不感兴趣，因为它们会扫描整个数组，并且在发现第一个匹配项时不会停止：
itemindex = numpy.where(array==item)[0][0]
nonzero(array == item)[0][0]
注1：这个问题的答案似乎都不相关。是否有一个Numpy函数返回数组中某个对象的第一个索引？
注意2：使用C编译方法优于Python循环。
How can I find the index of the first occurrence of a number in a Numpy array?
Speed is important to me. I am not interested in the following answers because they scan the whole array and don’t stop when they find the first occurrence:
itemindex = numpy.where(array==item)[0][0]
nonzero(array == item)[0][0]
Note 1: none of the answers from that question seem relevant Is there a Numpy function to return the first index of something in an array?
Note 2: using a C-compiled method is preferred to a Python loop.
回答 0
回答 1
尽管对您来说太晚了，但是供以后参考：在numpy实现之前，使用numba（1）是最简单的方法。如果您使用anaconda python发行版，则应该已经安装了它。该代码将被编译，因此将很快。
@jit(nopython=True)
def find_first(item, vec):
"""return the index of the first occurence of item in vec"""
for i in xrange(len(vec)):
if item == vec[i]:
return i
return -1
然后：
>>> a = array([1,7,8,32])
>>> find_first(8,a)
2
Although it is way too late for you, but for future reference:
Using numba (1) is the easiest way until numpy implements it. If you use anaconda python distribution it should already be installed.
The code will be compiled so it will be fast.
@jit(nopython=True)
def find_first(item, vec):
"""return the index of the first occurence of item in vec"""
for i in xrange(len(vec)):
if item == vec[i]:
return i
return -1
and then:
>>> a = array([1,7,8,32])
>>> find_first(8,a)
2
回答 2
我已经为几种方法设定了基准：
argwherenonzero 如问题.tostring() 就像@Rob Reilink的答案一样- python循环
- Fortran循环
在Python中和的Fortran代码是可用的。我跳过了那些没有希望的事情，例如转换为列表。
结果以对数刻度表示。X轴是针的位置（查找针是否在阵列的下方需要更长的时间）；最后一个值是不在数组中的指针。Y轴是找到它的时间。
该阵列具有100万个元素，并且运行了100次测试。结果仍然有些波动，但是定性趋势很明显：Python和f2py在第一个元素退出，因此它们的缩放比例不同。如果针头不在前1％，Python会变得太慢，而f2py速度会很快（但是您需要对其进行编译）。
总而言之，f2py是最快的解决方案，尤其是当针头出现得很早时。
它不是内置在其中的，但实际上只是2分钟的工作。将此添加到名为search.f90：
subroutine find_first(needle, haystack, haystack_length, index)
implicit none
integer, intent(in) :: needle
integer, intent(in) :: haystack_length
integer, intent(in), dimension(haystack_length) :: haystack
!f2py intent(inplace) haystack
integer, intent(out) :: index
integer :: k
index = -1
do k = 1, haystack_length
if (haystack(k)==needle) then
index = k - 1
exit
endif
enddo
end
如果您要查找的不是integer，请更改类型。然后使用以下命令进行编译：
f2py -c -m search search.f90
之后您可以执行（从Python）：
import search
print(search.find_first.__doc__)
a = search.find_first(your_int_needle, your_int_array)
I’ve made a benchmark for several methods:
argwherenonzero as in the question.tostring() as in @Rob Reilink’s answer- python loop
- Fortran loop
The Python and Fortran code are available. I skipped the unpromising ones like converting to a list.
The results on log scale. X-axis is the position of the needle (it takes longer to find if it’s further down the array); last value is a needle that’s not in the array. Y-axis is the time to find it.
The array had 1 million elements and tests were run 100 times. Results still fluctuate a bit, but the qualitative trend is clear: Python and f2py quit at the first element so they scale differently. Python gets too slow if the needle is not in the first 1%, whereas f2py is fast (but you need to compile it).
To summarize, f2py is the fastest solution, especially if the needle appears fairly early.
It’s not built in which is annoying, but it’s really just 2 minutes of work. Add this to a file called search.f90:
subroutine find_first(needle, haystack, haystack_length, index)
implicit none
integer, intent(in) :: needle
integer, intent(in) :: haystack_length
integer, intent(in), dimension(haystack_length) :: haystack
!f2py intent(inplace) haystack
integer, intent(out) :: index
integer :: k
index = -1
do k = 1, haystack_length
if (haystack(k)==needle) then
index = k - 1
exit
endif
enddo
end
If you’re looking for something other than integer, just change the type. Then compile using:
f2py -c -m search search.f90
after which you can do (from Python):
import search
print(search.find_first.__doc__)
a = search.find_first(your_int_needle, your_int_array)
回答 3
您可以使用array.tostring()，然后使用find（）方法将布尔数组转换为Python字符串：
(array==item).tostring().find('\x01')
但是，这确实涉及到复制数据，因为Python字符串需要是不可变的。一个优点是您还可以通过查找以下内容来搜索例如上升沿\x00\x01
You can convert a boolean array to a Python string using array.tostring() and then using the find() method:
(array==item).tostring().find('\x01')
This does involve copying the data, though, since Python strings need to be immutable. An advantage is that you can also search for e.g. a rising edge by finding \x00\x01
回答 4
在排序数组的情况下np.searchsorted工作。
In case of sorted arrays np.searchsorted works.
回答 5
我认为您遇到了一个问题，在此问题上，可以使用其他方法和一些先验知识来真正帮助您。您有X概率在数据的前Y个百分比中找到答案的情况。希望将问题分解为幸运的事物，然后在python中使用嵌套列表理解或类似的方法进行处理。
使用ctypes编写C函数来实现这种蛮力也并不难。
我一起砍的C代码（index.c）：
long index(long val, long *data, long length){
long ans, i;
for(i=0;i<length;i++){
if (data[i] == val)
return(i);
}
return(-999);
}
和python：
# to compile (mac)
# gcc -shared index.c -o index.dylib
import ctypes
lib = ctypes.CDLL('index.dylib')
lib.index.restype = ctypes.c_long
lib.index.argtypes = (ctypes.c_long, ctypes.POINTER(ctypes.c_long), ctypes.c_long)
import numpy as np
np.random.seed(8675309)
a = np.random.random_integers(0, 100, 10000)
print lib.index(57, a.ctypes.data_as(ctypes.POINTER(ctypes.c_long)), len(a))
我得到92。
将python包装成适当的函数，然后就可以了。
对于这个种子，C版本要快很多（〜20倍）（警告我使用timeit不好）
import timeit
t = timeit.Timer('np.where(a==57)[0][0]', 'import numpy as np; np.random.seed(1); a = np.random.random_integers(0, 1000000, 10000000)')
t.timeit(100)/100
# 0.09761879920959472
t2 = timeit.Timer('lib.index(57, a.ctypes.data_as(ctypes.POINTER(ctypes.c_long)), len(a))', 'import numpy as np; np.random.seed(1); a = np.random.random_integers(0, 1000000, 10000000); import ctypes; lib = ctypes.CDLL("index.dylib"); lib.index.restype = ctypes.c_long; lib.index.argtypes = (ctypes.c_long, ctypes.POINTER(ctypes.c_long), ctypes.c_long) ')
t2.timeit(100)/100
# 0.005288000106811523
I think you have hit a problem where a different method and some a priori knowledge of the array would really help. The kind of thing where you have a X probability of finding your answer in the first Y percent of the data. The splitting up the problem with the hope of getting lucky then doing this in python with a nested list comprehension or something.
Writing a C function to do this brute force isn’t too hard using ctypes either.
The C code I hacked together (index.c):
long index(long val, long *data, long length){
long ans, i;
for(i=0;i<length;i++){
if (data[i] == val)
return(i);
}
return(-999);
}
and the python:
# to compile (mac)
# gcc -shared index.c -o index.dylib
import ctypes
lib = ctypes.CDLL('index.dylib')
lib.index.restype = ctypes.c_long
lib.index.argtypes = (ctypes.c_long, ctypes.POINTER(ctypes.c_long), ctypes.c_long)
import numpy as np
np.random.seed(8675309)
a = np.random.random_integers(0, 100, 10000)
print lib.index(57, a.ctypes.data_as(ctypes.POINTER(ctypes.c_long)), len(a))
and I get 92.
Wrap up the python into a proper function and there you go.
The C version is a lot (~20x) faster for this seed (warning I am not good with timeit)
import timeit
t = timeit.Timer('np.where(a==57)[0][0]', 'import numpy as np; np.random.seed(1); a = np.random.random_integers(0, 1000000, 10000000)')
t.timeit(100)/100
# 0.09761879920959472
t2 = timeit.Timer('lib.index(57, a.ctypes.data_as(ctypes.POINTER(ctypes.c_long)), len(a))', 'import numpy as np; np.random.seed(1); a = np.random.random_integers(0, 1000000, 10000000); import ctypes; lib = ctypes.CDLL("index.dylib"); lib.index.restype = ctypes.c_long; lib.index.argtypes = (ctypes.c_long, ctypes.POINTER(ctypes.c_long), ctypes.c_long) ')
t2.timeit(100)/100
# 0.005288000106811523
回答 6
@tal已经提供了numba查找第一个索引的函数，但是仅适用于一维数组。使用，np.ndenumerate您还可以在任意维数组中找到第一个索引：
from numba import njit
import numpy as np
@njit
def index(array, item):
for idx, val in np.ndenumerate(array):
if val == item:
return idx
return None
样品盒：
>>> arr = np.arange(9).reshape(3,3)
>>> index(arr, 3)
(1, 0)
时间显示，其性能与tals解决方案相似：
arr = np.arange(100000)
%timeit index(arr, 5) # 1000000 loops, best of 3: 1.88 µs per loop
%timeit find_first(5, arr) # 1000000 loops, best of 3: 1.7 µs per loop
%timeit index(arr, 99999) # 10000 loops, best of 3: 118 µs per loop
%timeit find_first(99999, arr) # 10000 loops, best of 3: 96 µs per loop
@tal already presented a numba function to find the first index but that only works for 1D arrays. With np.ndenumerate you can also find the first index in an arbitarly dimensional array:
from numba import njit
import numpy as np
@njit
def index(array, item):
for idx, val in np.ndenumerate(array):
if val == item:
return idx
return None
Sample case:
>>> arr = np.arange(9).reshape(3,3)
>>> index(arr, 3)
(1, 0)
Timings show that it’s similar in performance to tals solution:
arr = np.arange(100000)
%timeit index(arr, 5) # 1000000 loops, best of 3: 1.88 µs per loop
%timeit find_first(5, arr) # 1000000 loops, best of 3: 1.7 µs per loop
%timeit index(arr, 99999) # 10000 loops, best of 3: 118 µs per loop
%timeit find_first(99999, arr) # 10000 loops, best of 3: 96 µs per loop
回答 7
如果您的列表已排序，则可以使用“ bisect”包非常快速地搜索索引。它是O（log（n））而不是O（n）。
bisect.bisect(a, x)
在数组a中找到x，在排序情况下肯定比通过所有第一个元素的C例程更快（对于足够长的列表）。
有时候很高兴知道。
If your list is sorted, you can achieve very quick search of index with the ‘bisect’ package.
It’s O(log(n)) instead of O(n).
bisect.bisect(a, x)
finds x in the array a, definitely quicker in the sorted case than any C-routine going through all the first elements (for long enough lists).
It’s good to know sometimes.
回答 8
据我所知，布尔数组上只有np.any和np.all是短路的。
在您的情况下，numpy必须遍历整个数组两次，一次创建布尔条件，第二次查找索引。
在这种情况下，我的建议是使用cython。我认为为这种情况调整示例应该很容易，尤其是对于不同的dtype和形状不需要很大的灵活性的情况下。
As far as I know only np.any and np.all on boolean arrays are short-circuited.
In your case, numpy has to go through the entire array twice, once to create the boolean condition and a second time to find the indices.
My recommendation in this case would be to use cython. I think it should be easy to adjust an example for this case, especially if you don’t need much flexibility for different dtypes and shapes.
回答 9
I needed this for my job so I taught myself Python and Numpy’s C interface and wrote my own. http://pastebin.com/GtcXuLyd It’s only for 1-D arrays, but works for most data types (int, float, or strings) and testing has shown it is again about 20 times faster than the expected approach in pure Python-numpy.
回答 10
通过分块处理数组，可以在纯numpy中有效解决此问题：
def find_first(x):
idx, step = 0, 32
while idx < x.size:
nz, = x[idx: idx + step].nonzero()
if len(nz): # found non-zero, return it
return nz[0] + idx
# move to the next chunk, increase step
idx += step
step = min(9600, step + step // 2)
return -1
数组以size的块进行处理step。的step时间越长，步骤是，更快的正在处理归零阵列（最坏情况）的。它越小，开始处理非零数组的速度就越快。诀窍是从小开始，step然后按指数增加。此外，由于收益有限，无需将其增加到某个阈值以上。
我已经将纯ndarary.nonzero和numba解决方案与1000万个浮点数组进行了比较。
import numpy as np
from numba import jit
from timeit import timeit
def find_first(x):
idx, step = 0, 32
while idx < x.size:
nz, = x[idx: idx + step].nonzero()
if len(nz):
return nz[0] + idx
idx += step
step = min(9600, step + step // 2)
return -1
@jit(nopython=True)
def find_first_numba(vec):
"""return the index of the first occurence of item in vec"""
for i in range(len(vec)):
if vec[i]:
return i
return -1
SIZE = 10_000_000
# First only
x = np.empty(SIZE)
find_first_numba(x[:10])
print('---- FIRST ----')
x[:] = 0
x[0] = 1
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0][0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=1000), 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=1000), 'ms')
print('---- LAST ----')
x[:] = 0
x[-1] = 1
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0][0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=100)*10, 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=100)*10, 'ms')
print('---- NONE ----')
x[:] = 0
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=100)*10, 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=100)*10, 'ms')
print('---- ALL ----')
x[:] = 1
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0][0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=100)*10, 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=100)*10, 'ms')
结果在我的机器上：
---- FIRST ----
ndarray.nonzero 54.733994480002366 ms
find_first 0.0013148509997336078 ms
find_first_numba 0.0002839310000126716 ms
---- LAST ----
ndarray.nonzero 54.56336712999928 ms
find_first 25.38929685000312 ms
find_first_numba 8.022820680002951 ms
---- NONE ----
ndarray.nonzero 24.13432420999925 ms
find_first 25.345200140000088 ms
find_first_numba 8.154927100003988 ms
---- ALL ----
ndarray.nonzero 55.753537260002304 ms
find_first 0.0014760300018679118 ms
find_first_numba 0.0004358099977253005 ms
纯ndarray.nonzero是绝对宽松的。在最佳情况下，numba解决方案的速度提高了约5倍。在最坏的情况下，速度快大约3倍。
This problem can be effectively solved in pure numpy by processing the array in chunks:
def find_first(x):
idx, step = 0, 32
while idx < x.size:
nz, = x[idx: idx + step].nonzero()
if len(nz): # found non-zero, return it
return nz[0] + idx
# move to the next chunk, increase step
idx += step
step = min(9600, step + step // 2)
return -1
The array is processed in chunk of size step. The step longer the step is, the faster is processing of zeroed-array (worst case). The smaller it is, the faster processing of array with non-zero at the start. The trick is to start with a small step and increase it exponentially. Moreover, there is no need to increment it above some threshold due to limited benefits.
I’ve compared the solution with pure ndarary.nonzero and numba solution against 10 million array of floats.
import numpy as np
from numba import jit
from timeit import timeit
def find_first(x):
idx, step = 0, 32
while idx < x.size:
nz, = x[idx: idx + step].nonzero()
if len(nz):
return nz[0] + idx
idx += step
step = min(9600, step + step // 2)
return -1
@jit(nopython=True)
def find_first_numba(vec):
"""return the index of the first occurence of item in vec"""
for i in range(len(vec)):
if vec[i]:
return i
return -1
SIZE = 10_000_000
# First only
x = np.empty(SIZE)
find_first_numba(x[:10])
print('---- FIRST ----')
x[:] = 0
x[0] = 1
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0][0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=1000), 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=1000), 'ms')
print('---- LAST ----')
x[:] = 0
x[-1] = 1
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0][0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=100)*10, 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=100)*10, 'ms')
print('---- NONE ----')
x[:] = 0
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=100)*10, 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=100)*10, 'ms')
print('---- ALL ----')
x[:] = 1
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0][0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=100)*10, 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=100)*10, 'ms')
And results on my machine:
---- FIRST ----
ndarray.nonzero 54.733994480002366 ms
find_first 0.0013148509997336078 ms
find_first_numba 0.0002839310000126716 ms
---- LAST ----
ndarray.nonzero 54.56336712999928 ms
find_first 25.38929685000312 ms
find_first_numba 8.022820680002951 ms
---- NONE ----
ndarray.nonzero 24.13432420999925 ms
find_first 25.345200140000088 ms
find_first_numba 8.154927100003988 ms
---- ALL ----
ndarray.nonzero 55.753537260002304 ms
find_first 0.0014760300018679118 ms
find_first_numba 0.0004358099977253005 ms
Pure ndarray.nonzero is definite looser. The numba solution is circa 5 times faster for the best case. It is circa 3 times faster in the worst case.
回答 11
如果您正在寻找第一个非零元素，则可以使用以下技巧：
idx = x.view(bool).argmax() // x.itemsize
idx = idx if x[idx] else -1
这是一个非常快速的 “ numpy-pure”解决方案，但在下面讨论的某些情况下失败。
该解决方案利用了以下事实：数字类型的几乎所有零表示都由0字节组成。它也适用于numpy bool。在最新版本的numpy中，argmax()函数在处理bool类型时使用短路逻辑。的大小bool为1个字节。
因此，需要：
- 创建数组的视图为
bool。没有创建副本
- 用于
argmax()使用短路逻辑查找第一个非零字节
- 通过将偏移量的整数除（运算符
//）除以以字节（x.itemsize）表示的单个元素的大小，来重新计算此字节与第一个非零元素的索引的偏移量
- 检查是否
x[idx]实际上非零，以识别不存在非零时的情况
我已经针对numba解决方案建立了一些基准，并进行了构建np.nonzero。
import numpy as np
from numba import jit
from timeit import timeit
def find_first(x):
idx = x.view(bool).argmax() // x.itemsize
return idx if x[idx] else -1
@jit(nopython=True)
def find_first_numba(vec):
"""return the index of the first occurence of item in vec"""
for i in range(len(vec)):
if vec[i]:
return i
return -1
SIZE = 10_000_000
# First only
x = np.empty(SIZE)
find_first_numba(x[:10])
print('---- FIRST ----')
x[:] = 0
x[0] = 1
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0][0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=1000), 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=1000), 'ms')
print('---- LAST ----')
x[:] = 0
x[-1] = 1
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0][0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=100)*10, 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=100)*10, 'ms')
print('---- NONE ----')
x[:] = 0
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=100)*10, 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=100)*10, 'ms')
print('---- ALL ----')
x[:] = 1
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0][0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=100)*10, 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=100)*10, 'ms')
我的机器上的结果是：
---- FIRST ----
ndarray.nonzero 57.63976670001284 ms
find_first 0.0010841979965334758 ms
find_first_numba 0.0002308919938514009 ms
---- LAST ----
ndarray.nonzero 58.96685277999495 ms
find_first 5.923203580023255 ms
find_first_numba 8.762269750004634 ms
---- NONE ----
ndarray.nonzero 25.13398071998381 ms
find_first 5.924289370013867 ms
find_first_numba 8.810063839919167 ms
---- ALL ----
ndarray.nonzero 55.181210660084616 ms
find_first 0.001246920000994578 ms
find_first_numba 0.00028766007744707167 ms
该解决方案比numba 快 33％，并且是“ numpy-pure”。
缺点：
- 不适用于numpy可接受的类型，例如
object
- 因偶尔出现在
float或double计算中的负零而失败
If you are looking for the first non-zero element you can use a following hack:
idx = x.view(bool).argmax() // x.itemsize
idx = idx if x[idx] else -1
It is a very fast “numpy-pure” solution but it fails for some cases discussed below.
The solution takes advantage from the fact that pretty much all representation of zero for numeric types consists of 0 bytes. It applies to numpy’s bool as well. In recent versions of numpy, argmax() function uses short-circuit logic when processing the bool type. The size of bool is 1 byte.
So one needs to:
- create a view of the array as
bool. No copy is created
- use
argmax() to find the first non-zero byte using short-circuit logic
- recalculate the offset of this byte to the index of the first non-zero element by integer division (operator
//) of the offset by a size of a single element expressed in bytes (x.itemsize)
- check if
x[idx] is actually non-zero to identify the case when no non-zero is present
I’ve made some benchmark against numba solution and build it np.nonzero.
import numpy as np
from numba import jit
from timeit import timeit
def find_first(x):
idx = x.view(bool).argmax() // x.itemsize
return idx if x[idx] else -1
@jit(nopython=True)
def find_first_numba(vec):
"""return the index of the first occurence of item in vec"""
for i in range(len(vec)):
if vec[i]:
return i
return -1
SIZE = 10_000_000
# First only
x = np.empty(SIZE)
find_first_numba(x[:10])
print('---- FIRST ----')
x[:] = 0
x[0] = 1
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0][0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=1000), 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=1000), 'ms')
print('---- LAST ----')
x[:] = 0
x[-1] = 1
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0][0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=100)*10, 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=100)*10, 'ms')
print('---- NONE ----')
x[:] = 0
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=100)*10, 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=100)*10, 'ms')
print('---- ALL ----')
x[:] = 1
print('ndarray.nonzero', timeit(lambda: x.nonzero()[0][0], number=100)*10, 'ms')
print('find_first', timeit(lambda: find_first(x), number=100)*10, 'ms')
print('find_first_numba', timeit(lambda: find_first_numba(x), number=100)*10, 'ms')
The result on my machine are:
---- FIRST ----
ndarray.nonzero 57.63976670001284 ms
find_first 0.0010841979965334758 ms
find_first_numba 0.0002308919938514009 ms
---- LAST ----
ndarray.nonzero 58.96685277999495 ms
find_first 5.923203580023255 ms
find_first_numba 8.762269750004634 ms
---- NONE ----
ndarray.nonzero 25.13398071998381 ms
find_first 5.924289370013867 ms
find_first_numba 8.810063839919167 ms
---- ALL ----
ndarray.nonzero 55.181210660084616 ms
find_first 0.001246920000994578 ms
find_first_numba 0.00028766007744707167 ms
The solution is 33% faster than numba and it is “numpy-pure”.
The disadvantages:
- does not work for numpy acceptable types like
object
- fails for negative zero that occasionally appears in
float or double computations
回答 12
作为matlab的长期用户，我很久以来一直在寻找有效解决此问题的方法。最后，在此讨论一个命题动机线程我试图拿出与正在实施类似建议什么的API的解决方案在这里，配套目前只有一维数组。
你会这样使用
import numpy as np
import utils_find_1st as utf1st
array = np.arange(100000)
item = 1000
ind = utf1st.find_1st(array, item, utf1st.cmp_larger_eq)
支持的条件运算符为：cmp_equal，cmp_not_equal，cmp_larger，cmp_smaller，cmp_larger_eq，cmp_smaller_eq。为了提高效率，扩展名用c编写。
您可以在此处找到来源，基准和其他详细信息：
https://pypi.python.org/pypi?name=py_find_1st&:action=display
为了供我们团队使用（在Linux和MacOS上使用anaconda），我制作了一个anaconda安装程序以简化安装，您可以按此处所述使用它
https://anaconda.org/roebel/py_find_1st
As a longtime matlab user I a have been searching for an efficient solution to this problem for quite a while. Finally, motivated by discussions a propositions in this thread I have tried to come up with a solution that is implementing an API similar to what was suggested here, supporting for the moment only 1D arrays.
You would use it like this
import numpy as np
import utils_find_1st as utf1st
array = np.arange(100000)
item = 1000
ind = utf1st.find_1st(array, item, utf1st.cmp_larger_eq)
The condition operators supported are: cmp_equal, cmp_not_equal, cmp_larger, cmp_smaller, cmp_larger_eq, cmp_smaller_eq. For efficiency the extension is written in c.
You find the source, benchmarks and other details here:
https://pypi.python.org/pypi?name=py_find_1st&:action=display
for the use in our team (anaconda on linux and macos) I have made an anaconda installer that simplifies installation, you may use it as described here
https://anaconda.org/roebel/py_find_1st
回答 13
请注意，如果您要执行一系列搜索，则如果搜索维不够大，则通过执行诸如转换为字符串之类的聪明操作而获得的性能提升可能会在外循环中丢失。了解使用上述提议的字符串转换技巧的find1和沿着内轴使用argmax的find2的性能（以及进行调整以确保不匹配返回-1的性能）
import numpy,time
def find1(arr,value):
return (arr==value).tostring().find('\x01')
def find2(arr,value): #find value over inner most axis, and return array of indices to the match
b = arr==value
return b.argmax(axis=-1) - ~(b.any())
for size in [(1,100000000),(10000,10000),(1000000,100),(10000000,10)]:
print(size)
values = numpy.random.choice([0,0,0,0,0,0,0,1],size=size)
v = values>0
t=time.time()
numpy.apply_along_axis(find1,-1,v,1)
print('find1',time.time()-t)
t=time.time()
find2(v,1)
print('find2',time.time()-t)
输出
(1, 100000000)
('find1', 0.25300002098083496)
('find2', 0.2780001163482666)
(10000, 10000)
('find1', 0.46200013160705566)
('find2', 0.27300000190734863)
(1000000, 100)
('find1', 20.98099994659424)
('find2', 0.3040001392364502)
(10000000, 10)
('find1', 206.7590000629425)
('find2', 0.4830000400543213)
也就是说，用C语言编写的查找至少比这两种方法都快一点
Just a note that if you are doing a sequence of searches, the performance gain from doing something clever like converting to string, might be lost in the outer loop if the search dimension isn’t big enough. See how the performance of iterating find1 that uses the string conversion trick proposed above and find2 that uses argmax along the inner axis (plus an adjustment to ensure a non-match returns as -1)
import numpy,time
def find1(arr,value):
return (arr==value).tostring().find('\x01')
def find2(arr,value): #find value over inner most axis, and return array of indices to the match
b = arr==value
return b.argmax(axis=-1) - ~(b.any())
for size in [(1,100000000),(10000,10000),(1000000,100),(10000000,10)]:
print(size)
values = numpy.random.choice([0,0,0,0,0,0,0,1],size=size)
v = values>0
t=time.time()
numpy.apply_along_axis(find1,-1,v,1)
print('find1',time.time()-t)
t=time.time()
find2(v,1)
print('find2',time.time()-t)
outputs
(1, 100000000)
('find1', 0.25300002098083496)
('find2', 0.2780001163482666)
(10000, 10000)
('find1', 0.46200013160705566)
('find2', 0.27300000190734863)
(1000000, 100)
('find1', 20.98099994659424)
('find2', 0.3040001392364502)
(10000000, 10)
('find1', 206.7590000629425)
('find2', 0.4830000400543213)
That said, a find written in C would be at least a little faster than either of these approaches
回答 14
这个怎么样
import numpy as np
np.amin(np.where(array==item))
how about this
import numpy as np
np.amin(np.where(array==item))
回答 15
您可以将数组隐藏为list并使用其index()方法：
i = list(array).index(item)
据我所知，这是C编译方法。
You can covert your array into a list and use it’s index() method:
i = list(array).index(item)
As far as I’m aware, this is a C compiled method.
![矩阵乘法（大小= [1000,2000,3000,5000,8000]）](https://i.stack.imgur.com/ZU7u4.png)
