问题:使用Python / NumPy对数组中的项目进行排名,而无需对数组进行两次排序
我有一个数字数组,我想创建另一个数组,该数组代表第一个数组中每个项目的等级。我正在使用Python和NumPy。
例如:
array = [4,2,7,1]
ranks = [2,1,3,0]
这是我想出的最好方法:
array = numpy.array([4,2,7,1])
temp = array.argsort()
ranks = numpy.arange(len(array))[temp.argsort()]
有没有更好/更快的方法可以避免对数组进行两次排序?
I have an array of numbers and I’d like to create another array that represents the rank of each item in the first array. I’m using Python and NumPy.
For example:
array = [4,2,7,1]
ranks = [2,1,3,0]
Here’s the best method I’ve come up with:
array = numpy.array([4,2,7,1])
temp = array.argsort()
ranks = numpy.arange(len(array))[temp.argsort()]
Are there any better/faster methods that avoid sorting the array twice?
回答 0
在最后一步中,在左侧使用切片:
array = numpy.array([4,2,7,1])
temp = array.argsort()
ranks = numpy.empty_like(temp)
ranks[temp] = numpy.arange(len(array))
通过在最后一步中反转排列,可以避免两次排序。
Use slicing on the left-hand side in the last step:
array = numpy.array([4,2,7,1])
temp = array.argsort()
ranks = numpy.empty_like(temp)
ranks[temp] = numpy.arange(len(array))
This avoids sorting twice by inverting the permutation in the last step.
回答 1
使用argsort两次,首先获取数组的顺序,然后获取排名:
array = numpy.array([4,2,7,1])
order = array.argsort()
ranks = order.argsort()
在处理2D(或更高维)数组时,请确保将轴参数传递给argsort以在正确的轴上排序。
Use argsort twice, first to obtain the order of the array, then to obtain ranking:
array = numpy.array([4,2,7,1])
order = array.argsort()
ranks = order.argsort()
When dealing with 2D (or higher dimensional) arrays, be sure to pass an axis argument to argsort to order over the correct axis.
回答 2
这个问题已有几年历史了,可以接受的答案很好,但是我认为以下仍然值得一提。如果您不介意对的依赖scipy
,则可以使用scipy.stats.rankdata
:
In [22]: from scipy.stats import rankdata
In [23]: a = [4, 2, 7, 1]
In [24]: rankdata(a)
Out[24]: array([ 3., 2., 4., 1.])
In [25]: (rankdata(a) - 1).astype(int)
Out[25]: array([2, 1, 3, 0])
的一个不错的功能rankdata
是,该method
参数提供了几种处理关系的选项。例如,在中有3次出现20次,两次出现40次b
:
In [26]: b = [40, 20, 70, 10, 20, 50, 30, 40, 20]
默认值将平均等级分配给绑定值:
In [27]: rankdata(b)
Out[27]: array([ 6.5, 3. , 9. , 1. , 3. , 8. , 5. , 6.5, 3. ])
method='ordinal'
分配连续等级:
In [28]: rankdata(b, method='ordinal')
Out[28]: array([6, 2, 9, 1, 3, 8, 5, 7, 4])
method='min'
将绑定值的最小等级分配给所有绑定值:
In [29]: rankdata(b, method='min')
Out[29]: array([6, 2, 9, 1, 2, 8, 5, 6, 2])
有关更多选项,请参见文档字符串。
This question is a few years old, and the accepted answer is great, but I think the following is still worth mentioning. If you don’t mind the dependency on scipy
, you can use scipy.stats.rankdata
:
In [22]: from scipy.stats import rankdata
In [23]: a = [4, 2, 7, 1]
In [24]: rankdata(a)
Out[24]: array([ 3., 2., 4., 1.])
In [25]: (rankdata(a) - 1).astype(int)
Out[25]: array([2, 1, 3, 0])
A nice feature of rankdata
is that the method
argument provides several options for handling ties. For example, there are three occurrences of 20 and two occurrences of 40 in b
:
In [26]: b = [40, 20, 70, 10, 20, 50, 30, 40, 20]
The default assigns the average rank to the tied values:
In [27]: rankdata(b)
Out[27]: array([ 6.5, 3. , 9. , 1. , 3. , 8. , 5. , 6.5, 3. ])
method='ordinal'
assigns consecutive ranks:
In [28]: rankdata(b, method='ordinal')
Out[28]: array([6, 2, 9, 1, 3, 8, 5, 7, 4])
method='min'
assigns the minimum rank of the tied values to all the tied values:
In [29]: rankdata(b, method='min')
Out[29]: array([6, 2, 9, 1, 2, 8, 5, 6, 2])
See the docstring for more options.
回答 3
我试图将两个以上的解决方案扩展到一个以上维度的数组A,假设您逐行处理数组(轴= 1)。
我用行循环扩展了第一个代码;可能可以改善
temp = A.argsort(axis=1)
rank = np.empty_like(temp)
rangeA = np.arange(temp.shape[1])
for iRow in xrange(temp.shape[0]):
rank[iRow, temp[iRow,:]] = rangeA
根据k.rooijers的建议,第二个变为:
temp = A.argsort(axis=1)
rank = temp.argsort(axis=1)
我随机生成400个形状为(1000,100)的数组;第一个代码大约是7.5,第二个代码是3.8。
I tried to extend both solution for arrays A of more than one dimension, supposing you process your array row-by-row (axis=1).
I extended the first code with a loop on rows; probably it can be improved
temp = A.argsort(axis=1)
rank = np.empty_like(temp)
rangeA = np.arange(temp.shape[1])
for iRow in xrange(temp.shape[0]):
rank[iRow, temp[iRow,:]] = rangeA
And the second one, following k.rooijers suggestion, becomes:
temp = A.argsort(axis=1)
rank = temp.argsort(axis=1)
I randomly generated 400 arrays with shape (1000,100); the first code took about 7.5, the second one 3.8.
回答 4
有关平均排名的矢量化版本,请参见下文。我喜欢np.unique,它确实扩大了可以有效地向量化代码的范围。除了避免python for循环外,这种方法还避免了对’a’的隐式双循环。
import numpy as np
a = np.array( [4,1,6,8,4,1,6])
a = np.array([4,2,7,2,1])
rank = a.argsort().argsort()
unique, inverse = np.unique(a, return_inverse = True)
unique_rank_sum = np.zeros_like(unique)
np.add.at(unique_rank_sum, inverse, rank)
unique_count = np.zeros_like(unique)
np.add.at(unique_count, inverse, 1)
unique_rank_mean = unique_rank_sum.astype(np.float) / unique_count
rank_mean = unique_rank_mean[inverse]
print rank_mean
For a vectorized version of an averaged rank, see below. I love np.unique, it really widens the scope of what code can and cannot be efficiently vectorized. Aside from avoiding python for-loops, this approach also avoids the implicit double loop over ‘a’.
import numpy as np
a = np.array( [4,1,6,8,4,1,6])
a = np.array([4,2,7,2,1])
rank = a.argsort().argsort()
unique, inverse = np.unique(a, return_inverse = True)
unique_rank_sum = np.zeros_like(unique)
np.add.at(unique_rank_sum, inverse, rank)
unique_count = np.zeros_like(unique)
np.add.at(unique_count, inverse, 1)
unique_rank_mean = unique_rank_sum.astype(np.float) / unique_count
rank_mean = unique_rank_mean[inverse]
print rank_mean
回答 5
除了解决方案的简洁和简短之外,还存在性能问题。这是一个小基准:
import numpy as np
from scipy.stats import rankdata
l = list(reversed(range(1000)))
%%timeit -n10000 -r5
x = (rankdata(l) - 1).astype(int)
>>> 128 µs ± 2.72 µs per loop (mean ± std. dev. of 5 runs, 10000 loops each)
%%timeit -n10000 -r5
a = np.array(l)
r = a.argsort().argsort()
>>> 69.1 µs ± 464 ns per loop (mean ± std. dev. of 5 runs, 10000 loops each)
%%timeit -n10000 -r5
a = np.array(l)
temp = a.argsort()
r = np.empty_like(temp)
r[temp] = np.arange(len(a))
>>> 63.7 µs ± 1.27 µs per loop (mean ± std. dev. of 5 runs, 10000 loops each)
Apart from the elegance and shortness of solutions, there is also the question of performance. Here is a little benchmark:
import numpy as np
from scipy.stats import rankdata
l = list(reversed(range(1000)))
%%timeit -n10000 -r5
x = (rankdata(l) - 1).astype(int)
>>> 128 µs ± 2.72 µs per loop (mean ± std. dev. of 5 runs, 10000 loops each)
%%timeit -n10000 -r5
a = np.array(l)
r = a.argsort().argsort()
>>> 69.1 µs ± 464 ns per loop (mean ± std. dev. of 5 runs, 10000 loops each)
%%timeit -n10000 -r5
a = np.array(l)
temp = a.argsort()
r = np.empty_like(temp)
r[temp] = np.arange(len(a))
>>> 63.7 µs ± 1.27 µs per loop (mean ± std. dev. of 5 runs, 10000 loops each)
回答 6
两次使用argsort()可以做到:
>>> array = [4,2,7,1]
>>> ranks = numpy.array(array).argsort().argsort()
>>> ranks
array([2, 1, 3, 0])
Use argsort() twice will do it:
>>> array = [4,2,7,1]
>>> ranks = numpy.array(array).argsort().argsort()
>>> ranks
array([2, 1, 3, 0])
回答 7
我尝试了上述方法,但失败了,因为我有很多zeore。是的,即使使用浮点数,重复项也可能很重要。
因此,我通过添加领带检查步骤编写了一个修改后的一维解决方案:
def ranks (v):
import numpy as np
t = np.argsort(v)
r = np.empty(len(v),int)
r[t] = np.arange(len(v))
for i in xrange(1, len(r)):
if v[t[i]] <= v[t[i-1]]: r[t[i]] = r[t[i-1]]
return r
# test it
print sorted(zip(ranks(v), v))
我相信它会尽可能高效。
I tried the above methods, but failed because I had many zeores. Yes, even with floats duplicate items may be important.
So I wrote a modified 1D solution by adding a tie-checking step:
def ranks (v):
import numpy as np
t = np.argsort(v)
r = np.empty(len(v),int)
r[t] = np.arange(len(v))
for i in xrange(1, len(r)):
if v[t[i]] <= v[t[i-1]]: r[t[i]] = r[t[i-1]]
return r
# test it
print sorted(zip(ranks(v), v))
I believe it’s as efficient as it can be.
回答 8
我喜欢k.rooijers的方法,但是正如rcoup所写,重复数字是根据数组位置进行排序的。这对我不利,因此我修改了版本以对等级进行后处理,并将所有重复的数字合并为合并的平均等级:
import numpy as np
a = np.array([4,2,7,2,1])
r = np.array(a.argsort().argsort(), dtype=float)
f = a==a
for i in xrange(len(a)):
if not f[i]: continue
s = a == a[i]
ls = np.sum(s)
if ls > 1:
tr = np.sum(r[s])
r[s] = float(tr)/ls
f[s] = False
print r # array([ 3. , 1.5, 4. , 1.5, 0. ])
我希望这也可以对其他人有所帮助,我试图找到其他解决方案,但是找不到任何解决方案…
I liked the method by k.rooijers, but as rcoup wrote, repeated numbers are ranked according to array position. This was no good for me, so I modified the version to postprocess the ranks and merge any repeated numbers into a combined average rank:
import numpy as np
a = np.array([4,2,7,2,1])
r = np.array(a.argsort().argsort(), dtype=float)
f = a==a
for i in xrange(len(a)):
if not f[i]: continue
s = a == a[i]
ls = np.sum(s)
if ls > 1:
tr = np.sum(r[s])
r[s] = float(tr)/ls
f[s] = False
print r # array([ 3. , 1.5, 4. , 1.5, 0. ])
I hope this might help others too, I tried to find anothers solution to this, but couldn’t find any…
回答 9
argsort和slice是对称操作。
尝试两次切片,而不是argsort两次。因为slice比argsort快
array = numpy.array([4,2,7,1])
order = array.argsort()
ranks = np.arange(array.shape[0])[order][order]
argsort and slice are symmetry operations.
try slice twice instead of argsort twice. since slice is faster than argsort
array = numpy.array([4,2,7,1])
order = array.argsort()
ranks = np.arange(array.shape[0])[order][order]
回答 10
更通用的版本之一:
In [140]: x = np.random.randn(10, 3)
In [141]: i = np.argsort(x, axis=0)
In [142]: ranks = np.empty_like(i)
In [143]: np.put_along_axis(ranks, i, np.repeat(np.arange(x.shape[0])[:,None], x.shape[1], axis=1), axis=0)
请参阅如何将numpy.argsort()用作两个以上维度的索引?泛化为更多的暗淡。
More general version of one of the answers:
In [140]: x = np.random.randn(10, 3)
In [141]: i = np.argsort(x, axis=0)
In [142]: ranks = np.empty_like(i)
In [143]: np.put_along_axis(ranks, i, np.repeat(np.arange(x.shape[0])[:,None], x.shape[1], axis=1), axis=0)
See How to use numpy.argsort() as indices in more than 2 dimensions? to generalize to more dims.