测试列表是否共享python中的任何项目

问题:测试列表是否共享python中的任何项目

我想检查一个列表中的任何项目是否存在于另一个列表中。我可以使用下面的代码简单地做到这一点,但是我怀疑可能有一个库函数可以做到这一点。如果没有,是否有更多的pythonic方法可以达到相同的结果。

In [78]: a = [1, 2, 3, 4, 5]

In [79]: b = [8, 7, 6]

In [80]: c = [8, 7, 6, 5]

In [81]: def lists_overlap(a, b):
   ....:     for i in a:
   ....:         if i in b:
   ....:             return True
   ....:     return False
   ....: 

In [82]: lists_overlap(a, b)
Out[82]: False

In [83]: lists_overlap(a, c)
Out[83]: True

In [84]: def lists_overlap2(a, b):
   ....:     return len(set(a).intersection(set(b))) > 0
   ....: 

I want to check if any of the items in one list are present in another list. I can do it simply with the code below, but I suspect there might be a library function to do this. If not, is there a more pythonic method of achieving the same result.

In [78]: a = [1, 2, 3, 4, 5]

In [79]: b = [8, 7, 6]

In [80]: c = [8, 7, 6, 5]

In [81]: def lists_overlap(a, b):
   ....:     for i in a:
   ....:         if i in b:
   ....:             return True
   ....:     return False
   ....: 

In [82]: lists_overlap(a, b)
Out[82]: False

In [83]: lists_overlap(a, c)
Out[83]: True

In [84]: def lists_overlap2(a, b):
   ....:     return len(set(a).intersection(set(b))) > 0
   ....: 

回答 0

简短答案:使用not set(a).isdisjoint(b),通常是最快的。

有测试四种常见的方式,如果两个列表ab共享任何项目。第一种选择是将两个都转换为集合并检查它们的交集,如下所示:

bool(set(a) & set(b))

由于集合是使用Python中的哈希表存储的,因此可以搜索它们O(1)(有关Python中运算符复杂性的更多信息,请参见此处)。从理论上讲,这是O(n+m)对平均nm在列表中的对象ab。但是1)它必须首先从列表中创建集合,这可能花费不可忽略的时间量; 2)它假定哈希冲突在您的数据中很少。

第二种方法是使用生成器表达式对列表执行迭代,例如:

any(i in a for i in b)

这允许就地搜索,因此不会为中间变量分配新的内存。它也可以在第一个发现上解决。但是in操作员始终O(n)在列表中(请参阅此处)。

另一个建议的选项是混合访问列表中的一个,转换另一个集合,然后测试该集合的成员资格,如下所示:

a = set(a); any(i in a for i in b)

第四种方法是利用isdisjoint()(冻结)集合的方法(请参阅此处),例如:

not set(a).isdisjoint(b)

如果您搜索的元素在数组的开头附近(例如已排序),则倾向于使用生成器表达式,因为集合交集方法必须为中间变量分配新的内存:

from timeit import timeit
>>> timeit('bool(set(a) & set(b))', setup="a=list(range(1000));b=list(range(1000))", number=100000)
26.077727576019242
>>> timeit('any(i in a for i in b)', setup="a=list(range(1000));b=list(range(1000))", number=100000)
0.16220548999262974

这是此示例的执行时间与列表大小的关系图:

请注意,两个轴都是对数的。这代表了生成器表达式的最佳情况。可以看出,该isdisjoint()方法对于非常小的列表大小更好,而生成器表达式对于更大的列表大小更好。

另一方面,由于搜索是从混合表达式和生成器表达式的开头开始的,因此,如果共享元素系统地位于数组的末尾(或者两个列表都不共享任何值),则不相交和集合交集方法比生成器表达式和混合方法快得多。

>>> timeit('any(i in a for i in b)', setup="a=list(range(1000));b=[x+998 for x in range(999,0,-1)]", number=1000))
13.739536046981812
>>> timeit('bool(set(a) & set(b))', setup="a=list(range(1000));b=[x+998 for x in range(999,0,-1)]", number=1000))
0.08102107048034668

有趣的是,对于较大的列表大小,生成器表达式要慢得多。这仅适用于1000次重复,而不是前一个数字的100000次。当没有共享任何元素时,此设置也很合适,并且是不相交和设置相交方法的最佳情况。

这是两个使用随机数的分析(而不是操纵设置以偏爱一种或多种技术):

分享的可能性很高:元素是从中随机抽取的[1, 2*len(a)]。分享机会低:元素是从中随机抽取的[1, 1000*len(a)]

到目前为止,该分析假设两个列表的大小相同。如果有两个不同大小的列表,例如a小得多,isdisjoint()总是更快:

确保a列表较小,否则性能会降低。在此实验中,a列表大小设置为常量5

综上所述:

  • 如果列表很小(<10个元素),not set(a).isdisjoint(b)则总是最快的。
  • 如果列表中的元素已排序或具有可以利用的规则结构,则生成器表达式any(i in a for i in b)在大列表大小时最快。
  • not set(a).isdisjoint(b)用来测试设置的交集,它总是比快bool(set(a) & set(b))
  • 混合“遍历列表,按条件测试” a = set(a); any(i in a for i in b)通常比其他方法慢。
  • 当涉及到不共享元素的列表时,生成器表达式和混合函数比其他两种方法要慢得多。

在大多数情况下,使用该isdisjoint()方法是最好的方法,因为生成器表达式的执行时间会更长,因为在没有共享任何元素时效率非常低。

Short answer: use not set(a).isdisjoint(b), it’s generally the fastest.

There are four common ways to test if two lists a and b share any items. The first option is to convert both to sets and check their intersection, as such:

bool(set(a) & set(b))

Because sets are stored using a hash table in Python, searching them is O(1) (see here for more information about complexity of operators in Python). Theoretically, this is O(n+m) on average for n and m objects in lists a and b. But 1) it must first create sets out of the lists, which can take a non-negligible amount of time, and 2) it supposes that hashing collisions are sparse among your data.

The second way to do it is using a generator expression performing iteration on the lists, such as:

any(i in a for i in b)

This allows to search in-place, so no new memory is allocated for intermediary variables. It also bails out on the first find. But the in operator is always O(n) on lists (see here).

Another proposed option is an hybridto iterate through one of the list, convert the other one in a set and test for membership on this set, like so:

a = set(a); any(i in a for i in b)

A fourth approach is to take advantage of the isdisjoint() method of the (frozen)sets (see here), for example:

not set(a).isdisjoint(b)

If the elements you search are near the beginning of an array (e.g. it is sorted), the generator expression is favored, as the sets intersection method have to allocate new memory for the intermediary variables:

from timeit import timeit
>>> timeit('bool(set(a) & set(b))', setup="a=list(range(1000));b=list(range(1000))", number=100000)
26.077727576019242
>>> timeit('any(i in a for i in b)', setup="a=list(range(1000));b=list(range(1000))", number=100000)
0.16220548999262974

Here’s a graph of the execution time for this example in function of list size:

Note that both axes are logarithmic. This represents the best case for the generator expression. As can be seen, the isdisjoint() method is better for very small list sizes, whereas the generator expression is better for larger list sizes.

On the other hand, as the search begins with the beginning for the hybrid and generator expression, if the shared element are systematically at the end of the array (or both lists does not share any values), the disjoint and set intersection approaches are then way faster than the generator expression and the hybrid approach.

>>> timeit('any(i in a for i in b)', setup="a=list(range(1000));b=[x+998 for x in range(999,0,-1)]", number=1000))
13.739536046981812
>>> timeit('bool(set(a) & set(b))', setup="a=list(range(1000));b=[x+998 for x in range(999,0,-1)]", number=1000))
0.08102107048034668

It is interesting to note that the generator expression is way slower for bigger list sizes. This is only for 1000 repetitions, instead of the 100000 for the previous figure. This setup also approximates well when when no elements are shared, and is the best case for the disjoint and set intersection approaches.

Here are two analysis using random numbers (instead of rigging the setup to favor one technique or another):

High chance of sharing: elements are randomly taken from [1, 2*len(a)]. Low chance of sharing: elements are randomly taken from [1, 1000*len(a)].

Up to now, this analysis supposed both lists are of the same size. In case of two lists of different sizes, for example a is much smaller, isdisjoint() is always faster:

Make sure that the a list is the smaller, otherwise the performance decreases. In this experiment, the a list size was set constant to 5.

In summary:

  • If the lists are very small (< 10 elements), not set(a).isdisjoint(b) is always the fastest.
  • If the elements in the lists are sorted or have a regular structure that you can take advantage of, the generator expression any(i in a for i in b) is the fastest on large list sizes;
  • Test the set intersection with not set(a).isdisjoint(b), which is always faster than bool(set(a) & set(b)).
  • The hybrid “iterate through list, test on set” a = set(a); any(i in a for i in b) is generally slower than other methods.
  • The generator expression and the hybrid are much slower than the two other approaches when it comes to lists without sharing elements.

In most cases, using the isdisjoint() method is the best approach as the generator expression will take much longer to execute, as it is very inefficient when no elements are shared.


回答 1

def lists_overlap3(a, b):
    return bool(set(a) & set(b))

注意:以上假设您想要布尔值作为答案。如果您只需要在if语句中使用表达式,则只需使用if set(a) & set(b):

def lists_overlap3(a, b):
    return bool(set(a) & set(b))

Note: the above assumes that you want a boolean as the answer. If all you need is an expression to use in an if statement, just use if set(a) & set(b):


回答 2

def lists_overlap(a, b):
  sb = set(b)
  return any(el in sb for el in a)

这是渐近最优的(最坏情况O(n + m)),并且由于any的短路,可能比交叉点方法更好。

例如:

lists_overlap([3,4,5], [1,2,3])

到达后将立即返回True 3 in sb

编辑:另一种变化(感谢Dave Kirby):

def lists_overlap(a, b):
  sb = set(b)
  return any(itertools.imap(sb.__contains__, a))

这依赖于imap用C实现的迭代器,而不是生成器理解。它还sb.__contains__用作映射功能。我不知道这会带来多少性能差异。它仍然会短路。

def lists_overlap(a, b):
  sb = set(b)
  return any(el in sb for el in a)

This is asymptotically optimal (worst case O(n + m)), and might be better than the intersection approach due to any‘s short-circuiting.

E.g.:

lists_overlap([3,4,5], [1,2,3])

will return True as soon as it gets to 3 in sb

EDIT: Another variation (with thanks to Dave Kirby):

def lists_overlap(a, b):
  sb = set(b)
  return any(itertools.imap(sb.__contains__, a))

This relies on imap‘s iterator, which is implemented in C, rather than a generator comprehension. It also uses sb.__contains__ as the mapping function. I don’t know how much performance difference this makes. It will still short-circuit.


回答 3

您还可以将其any与列表理解一起使用:

any([item in a for item in b])

You could also use any with list comprehension:

any([item in a for item in b])

回答 4

在python 2.6或更高版本中,您可以执行以下操作:

return not frozenset(a).isdisjoint(frozenset(b))

In python 2.6 or later you can do:

return not frozenset(a).isdisjoint(frozenset(b))

回答 5

您可以使用任何内置函数/ wa generator表达式:

def list_overlap(a,b): 
     return any(i for i in a if i in b)

正如John和Lie所指出的那样,当对于两个列表共享的每个i bool(i)== False时,这都会给出错误的结果。它应该是:

return any(i in b for i in a)

You can use the any built in function /w a generator expression:

def list_overlap(a,b): 
     return any(i for i in a if i in b)

As John and Lie have pointed out this gives incorrect results when for every i shared by the two lists bool(i) == False. It should be:

return any(i in b for i in a)

回答 6

这个问题已经很老了,但是我注意到当人们在参数集合与列表时,没有人想到将它们一起使用。按照Soravux的示例,

清单的最坏情况:

>>> timeit('bool(set(a) & set(b))',  setup="a=list(range(10000)); b=[x+9999 for x in range(10000)]", number=100000)
100.91506409645081
>>> timeit('any(i in a for i in b)', setup="a=list(range(10000)); b=[x+9999 for x in range(10000)]", number=100000)
19.746716022491455
>>> timeit('any(i in a for i in b)', setup="a= set(range(10000)); b=[x+9999 for x in range(10000)]", number=100000)
0.092626094818115234

列表的最佳情况是:

>>> timeit('bool(set(a) & set(b))',  setup="a=list(range(10000)); b=list(range(10000))", number=100000)
154.69790101051331
>>> timeit('any(i in a for i in b)', setup="a=list(range(10000)); b=list(range(10000))", number=100000)
0.082653045654296875
>>> timeit('any(i in a for i in b)', setup="a= set(range(10000)); b=list(range(10000))", number=100000)
0.08434605598449707

因此,遍历一个列表以查看它是否在集合中比遍历两个列表更快,这是有意义的,因为检查数字是否在集合中需要固定的时间,而通过遍历列表进行检查所花费的时间与长度成正比。名单。

因此,我的结论是遍历一个列表,并检查它是否在set中

This question is pretty old, but I noticed that while people were arguing sets vs. lists, that no one thought of using them together. Following Soravux’s example,

Worst case for lists:

>>> timeit('bool(set(a) & set(b))',  setup="a=list(range(10000)); b=[x+9999 for x in range(10000)]", number=100000)
100.91506409645081
>>> timeit('any(i in a for i in b)', setup="a=list(range(10000)); b=[x+9999 for x in range(10000)]", number=100000)
19.746716022491455
>>> timeit('any(i in a for i in b)', setup="a= set(range(10000)); b=[x+9999 for x in range(10000)]", number=100000)
0.092626094818115234

And the best case for lists:

>>> timeit('bool(set(a) & set(b))',  setup="a=list(range(10000)); b=list(range(10000))", number=100000)
154.69790101051331
>>> timeit('any(i in a for i in b)', setup="a=list(range(10000)); b=list(range(10000))", number=100000)
0.082653045654296875
>>> timeit('any(i in a for i in b)', setup="a= set(range(10000)); b=list(range(10000))", number=100000)
0.08434605598449707

So even faster than iterating through two lists is iterating though a list to see if it’s in a set, which makes sense since checking if a number is in a set takes constant time while checking by iterating through a list takes time proportional to the length of the list.

Thus, my conclusion is that iterate through a list, and check if it’s in a set.


回答 7

如果您不在乎重叠的元素是什么,则只需检查len合并列表与合并为一组的列表的即可。如果有重叠的元素,则集合将更短:

len(set(a+b+c))==len(a+b+c) 如果没有重叠,则返回True。

if you don’t care what the overlapping element might be, you can simply check the len of the combined list vs. the lists combined as a set. If there are overlapping elements, the set will be shorter:

len(set(a+b+c))==len(a+b+c) returns True, if there is no overlap.


回答 8

我将以一种功能性的编程风格来介绍另一个:

any(map(lambda x: x in a, b))

说明:

map(lambda x: x in a, b)

返回在其中的元素布尔值的列表b中找到a。然后any将该列表传递给,该列表仅返回True是否有任何元素True

I’ll throw another one in with a functional programming style:

any(map(lambda x: x in a, b))

Explanation:

map(lambda x: x in a, b)

returns a list of booleans where elements of b are found in a. That list is then passed to any, which simply returns True if any elements are True.