为什么Python3中没有xrange函数?

问题:为什么Python3中没有xrange函数?

最近,我开始使用Python3,它缺少xrange的好处。

简单的例子:

1) Python2:

from time import time as t
def count():
  st = t()
  [x for x in xrange(10000000) if x%4 == 0]
  et = t()
  print et-st
count()

2) Python3:

from time import time as t

def xrange(x):

    return iter(range(x))

def count():
    st = t()
    [x for x in xrange(10000000) if x%4 == 0]
    et = t()
    print (et-st)
count()

结果分别是:

1) 1.53888392448 2) 3.215819835662842

这是为什么?我的意思是,为什么xrange被删除了?这是学习的好工具。对于初学者来说,就像我自己一样,就像我们都处在某个时刻。为什么要删除它?有人可以指出我正确的PEP,我找不到它。

干杯。

Recently I started using Python3 and it’s lack of xrange hurts.

Simple example:

1) Python2:

from time import time as t
def count():
  st = t()
  [x for x in xrange(10000000) if x%4 == 0]
  et = t()
  print et-st
count()

2) Python3:

from time import time as t

def xrange(x):

    return iter(range(x))

def count():
    st = t()
    [x for x in xrange(10000000) if x%4 == 0]
    et = t()
    print (et-st)
count()

The results are, respectively:

1) 1.53888392448 2) 3.215819835662842

Why is that? I mean, why xrange’s been removed? It’s such a great tool to learn. For the beginners, just like myself, like we all were at some point. Why remove it? Can somebody point me to the proper PEP, I can’t find it.

Cheers.


回答 0

进行一些性能评估,timeit而不是尝试使用手动进行time

首先,Apple 2.7.2 64位:

In [37]: %timeit collections.deque((x for x in xrange(10000000) if x%4 == 0), maxlen=0)
1 loops, best of 3: 1.05 s per loop

现在,python.org 3.3.0 64位:

In [83]: %timeit collections.deque((x for x in range(10000000) if x%4 == 0), maxlen=0)
1 loops, best of 3: 1.32 s per loop

In [84]: %timeit collections.deque((x for x in xrange(10000000) if x%4 == 0), maxlen=0)
1 loops, best of 3: 1.31 s per loop

In [85]: %timeit collections.deque((x for x in iter(range(10000000)) if x%4 == 0), maxlen=0) 
1 loops, best of 3: 1.33 s per loop

显然,3.x range确实比2.x慢一些xrange。OP的xrange功能与此无关。(不足为奇,因为__iter__在循环中发生的任何事情的1000 万次调用中,对插槽的一次性调用不太可能可见,但有人提出来了。)

但这仅慢了30%。OP如何使速度慢2倍?好吧,如果我使用32位Python重复相同的测试,则得到1.58和3.12。因此,我的猜测是,这是3.x针对64位性能进行了优化(以损害32位的方式)的又一案例。

但这真的重要吗?再次使用3.3.0 64位进行检查:

In [86]: %timeit [x for x in range(10000000) if x%4 == 0]
1 loops, best of 3: 3.65 s per loop

因此,构建所需的list时间是整个迭代的两倍以上。

至于“比Python 2.6+消耗更多的资源”,在我的测试中,看起来3.x range的大小与2.x的大小完全相同xrange,即使它的大小是10x的大小,也可以构建不必要的列表问题仍然比范围迭代可能做的任何事情多出10000000x。

那么显式for循环而不是内部的C循环又deque如何呢?

In [87]: def consume(x):
   ....:     for i in x:
   ....:         pass
In [88]: %timeit consume(x for x in range(10000000) if x%4 == 0)
1 loops, best of 3: 1.85 s per loop

因此,在for语句中浪费的时间几乎与迭代的实际工作中所浪费的时间一样多range

如果您担心优化范围对象的迭代,则可能在错误的位置。


同时,xrange无论人们告诉您相同的内容多少次,您都会不断询问为什么要删除它,但是我会再次重复:它没有删除:它被重命名为range,而2.x range是被删除的东西。

这是3.3 range对象是2.x xrange对象(而不是2.x range函数)的直接后代的证明:3.3range2.7xrange的源。您甚至可以查看更改历史记录(我相信,该更改已链接到替换文件中任何位置的字符串“ xrange”的最后一个实例的更改)。

那么,为什么它变慢?

好吧,其中之一是,他们添加了许多新功能。另外,他们已经在整个地方(尤其是在迭代过程中)进行了各种具有较小副作用的更改。尽管有时有时会稍微低估不太重要的案例,但仍有大量工作可以极大地优化各种重要案例。将所有这些加起来,令迭代range速度现在变慢会令我感到惊讶。这是最重要的案例之一,没有人会足够关注。没有人会遇到现实中的用例,这种性能差异是他们代码中的热点。

Some performance measurements, using timeit instead of trying to do it manually with time.

First, Apple 2.7.2 64-bit:

In [37]: %timeit collections.deque((x for x in xrange(10000000) if x%4 == 0), maxlen=0)
1 loops, best of 3: 1.05 s per loop

Now, python.org 3.3.0 64-bit:

In [83]: %timeit collections.deque((x for x in range(10000000) if x%4 == 0), maxlen=0)
1 loops, best of 3: 1.32 s per loop

In [84]: %timeit collections.deque((x for x in xrange(10000000) if x%4 == 0), maxlen=0)
1 loops, best of 3: 1.31 s per loop

In [85]: %timeit collections.deque((x for x in iter(range(10000000)) if x%4 == 0), maxlen=0) 
1 loops, best of 3: 1.33 s per loop

Apparently, 3.x range really is a bit slower than 2.x xrange. And the OP’s xrange function has nothing to do with it. (Not surprising, as a one-time call to the __iter__ slot isn’t likely to be visible among 10000000 calls to whatever happens in the loop, but someone brought it up as a possibility.)

But it’s only 30% slower. How did the OP get 2x as slow? Well, if I repeat the same tests with 32-bit Python, I get 1.58 vs. 3.12. So my guess is that this is yet another of those cases where 3.x has been optimized for 64-bit performance in ways that hurt 32-bit.

But does it really matter? Check this out, with 3.3.0 64-bit again:

In [86]: %timeit [x for x in range(10000000) if x%4 == 0]
1 loops, best of 3: 3.65 s per loop

So, building the list takes more than twice as long than the entire iteration.

And as for “consumes much more resources than Python 2.6+”, from my tests, it looks like a 3.x range is exactly the same size as a 2.x xrange—and, even if it were 10x as big, building the unnecessary list is still about 10000000x more of a problem than anything the range iteration could possibly do.

And what about an explicit for loop instead of the C loop inside deque?

In [87]: def consume(x):
   ....:     for i in x:
   ....:         pass
In [88]: %timeit consume(x for x in range(10000000) if x%4 == 0)
1 loops, best of 3: 1.85 s per loop

So, almost as much time wasted in the for statement as in the actual work of iterating the range.

If you’re worried about optimizing the iteration of a range object, you’re probably looking in the wrong place.


Meanwhile, you keep asking why xrange was removed, no matter how many times people tell you the same thing, but I’ll repeat it again: It was not removed: it was renamed to range, and the 2.x range is what was removed.

Here’s some proof that the 3.3 range object is a direct descendant of the 2.x xrange object (and not of the 2.x range function): the source to 3.3 range and 2.7 xrange. You can even see the change history (linked to, I believe, the change that replaced the last instance of the string “xrange” anywhere in the file).

So, why is it slower?

Well, for one, they’ve added a lot of new features. For another, they’ve done all kinds of changes all over the place (especially inside iteration) that have minor side effects. And there’d been a lot of work to dramatically optimize various important cases, even if it sometimes slightly pessimizes less important cases. Add this all up, and I’m not surprised that iterating a range as fast as possible is now a bit slower. It’s one of those less-important cases that nobody would ever care enough to focus on. No one is likely to ever have a real-life use case where this performance difference is the hotspot in their code.


回答 1

Python3的范围 Python2的xrange。无需在其周围包装迭代器。要获得Python3中的实际列表,您需要使用list(range(...))

如果您想要适用于Python2和Python3的产品,请尝试以下操作

try:
    xrange
except NameError:
    xrange = range

Python3’s range is Python2’s xrange. There’s no need to wrap an iter around it. To get an actual list in Python3, you need to use list(range(...))

If you want something that works with Python2 and Python3, try this

try:
    xrange
except NameError:
    xrange = range

回答 2

Python 3的range类型与Python 2的类型一样xrange。我不确定为什么会看到速度变慢,因为函数直接返回的迭代器xrange正是您返回的迭代器range

我无法在系统上重现速度下降的情况。这是我的测试方式:

Python 2,具有xrange

Python 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import timeit
>>> timeit.timeit("[x for x in xrange(1000000) if x%4]",number=100)
18.631936646865853

Python 3,range速度稍微快一点:

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:57:17) [MSC v.1600 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import timeit
>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=100)
17.31399508687869

我最近了解到Python 3的range类型还具有其他一些简洁的功能,例如对切片的支持:range(10,100,2)[5:25:5]is range(20, 60, 10)

Python 3’s range type works just like Python 2’s xrange. I’m not sure why you’re seeing a slowdown, since the iterator returned by your xrange function is exactly what you’d get if you iterated over range directly.

I’m not able to reproduce the slowdown on my system. Here’s how I tested:

Python 2, with xrange:

Python 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import timeit
>>> timeit.timeit("[x for x in xrange(1000000) if x%4]",number=100)
18.631936646865853

Python 3, with range is a tiny bit faster:

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:57:17) [MSC v.1600 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import timeit
>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=100)
17.31399508687869

I recently learned that Python 3’s range type has some other neat features, such as support for slicing: range(10,100,2)[5:25:5] is range(20, 60, 10)!


回答 3

修复python2代码的一种方法是:

import sys

if sys.version_info >= (3, 0):
    def xrange(*args, **kwargs):
        return iter(range(*args, **kwargs))

One way to fix up your python2 code is:

import sys

if sys.version_info >= (3, 0):
    def xrange(*args, **kwargs):
        return iter(range(*args, **kwargs))

回答 4

Python 2中的xrange是一个生成器,它实现了迭代器,而range只是一个函数。在Python3中,我不知道为什么将其从xrange中删除。

xrange from Python 2 is a generator and implements iterator while range is just a function. In Python3 I don’t know why was dropped off the xrange.


回答 5

comp:〜$ python Python 2.7.6(默认,2015年6月22日,17:58:13)[gcc 4.8.2]在linux2上

>>> import timeit
>>> timeit.timeit("[x for x in xrange(1000000) if x%4]",number=100)

5.656799077987671

>>> timeit.timeit("[x for x in xrange(1000000) if x%4]",number=100)

5.579368829727173

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=100)

21.54827117919922

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=100)

22.014557123184204

当timeit number = 1参数时:

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=1)

0.2245171070098877

>>> timeit.timeit("[x for x in xrange(1000000) if x%4]",number=1)

0.10750913619995117

comp:〜$ python3 Python 3.4.3(默认,2015年10月14日,20:28:29)[GCC 4.8.4]在Linux上

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=100)

9.113872020003328

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=100)

9.07014398300089

timeit number = 1,2,3,4参数可以快速且线性地工作:

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=1)

0.09329321900440846

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=2)

0.18501482300052885

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=3)

0.2703447980020428

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=4)

0.36209142999723554

因此,看来如果我们测量1个运行循环周期,例如timeit.timeit(“ [x表示x在range(1000000)中,如果x%4]”,number = 1)(正如我们在实际代码中实际使用的那样),python3的运行速度足够快,但在重复循环中,python 2 xrange()在速度上胜过python 3中的range()。

comp:~$ python Python 2.7.6 (default, Jun 22 2015, 17:58:13) [GCC 4.8.2] on linux2

>>> import timeit
>>> timeit.timeit("[x for x in xrange(1000000) if x%4]",number=100)

5.656799077987671

>>> timeit.timeit("[x for x in xrange(1000000) if x%4]",number=100)

5.579368829727173

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=100)

21.54827117919922

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=100)

22.014557123184204

With timeit number=1 param:

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=1)

0.2245171070098877

>>> timeit.timeit("[x for x in xrange(1000000) if x%4]",number=1)

0.10750913619995117

comp:~$ python3 Python 3.4.3 (default, Oct 14 2015, 20:28:29) [GCC 4.8.4] on linux

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=100)

9.113872020003328

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=100)

9.07014398300089

With timeit number=1,2,3,4 param works quick and in linear way:

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=1)

0.09329321900440846

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=2)

0.18501482300052885

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=3)

0.2703447980020428

>>> timeit.timeit("[x for x in range(1000000) if x%4]",number=4)

0.36209142999723554

So it seems if we measure 1 running loop cycle like timeit.timeit(“[x for x in range(1000000) if x%4]”,number=1) (as we actually use in real code) python3 works quick enough, but in repeated loops python 2 xrange() wins in speed against range() from python 3.