构建一个基本的Python迭代器

问题:构建一个基本的Python迭代器

如何在python中创建一个迭代函数(或迭代器对象)?

How would one create an iterative function (or iterator object) in python?


回答 0

python中的迭代器对象符合迭代器协议,这基本上意味着它们提供了两种方法:__iter__()__next__()

  • __iter__返回迭代器对象,并在循环开始时隐式调用。

  • __next__()方法返回下一个值,并在每次循环增量时隐式调用。当没有更多值要返回时,此方法将引发StopIteration异常,该异常由循环构造以停止迭代的方式隐式捕获。

这是一个简单的计数器示例:

class Counter:
    def __init__(self, low, high):
        self.current = low - 1
        self.high = high

    def __iter__(self):
        return self

    def __next__(self): # Python 2: def next(self)
        self.current += 1
        if self.current < self.high:
            return self.current
        raise StopIteration


for c in Counter(3, 9):
    print(c)

这将打印:

3
4
5
6
7
8

如上一个答案所述,使用生成器编写起来更容易:

def counter(low, high):
    current = low
    while current < high:
        yield current
        current += 1

for c in counter(3, 9):
    print(c)

打印的输出将相同。在内部,生成器对象支持迭代器协议,并且执行与类Counter大致相似的操作。

David Mertz的文章Iterators和Simple Generators是很好的介绍。

Iterator objects in python conform to the iterator protocol, which basically means they provide two methods: __iter__() and __next__().

  • The __iter__ returns the iterator object and is implicitly called at the start of loops.

  • The __next__() method returns the next value and is implicitly called at each loop increment. This method raises a StopIteration exception when there are no more value to return, which is implicitly captured by looping constructs to stop iterating.

Here’s a simple example of a counter:

class Counter:
    def __init__(self, low, high):
        self.current = low - 1
        self.high = high

    def __iter__(self):
        return self

    def __next__(self): # Python 2: def next(self)
        self.current += 1
        if self.current < self.high:
            return self.current
        raise StopIteration


for c in Counter(3, 9):
    print(c)

This will print:

3
4
5
6
7
8

This is easier to write using a generator, as covered in a previous answer:

def counter(low, high):
    current = low
    while current < high:
        yield current
        current += 1

for c in counter(3, 9):
    print(c)

The printed output will be the same. Under the hood, the generator object supports the iterator protocol and does something roughly similar to the class Counter.

David Mertz’s article, Iterators and Simple Generators, is a pretty good introduction.


回答 1

有四种方法可以构建迭代函数:

例子:

# generator
def uc_gen(text):
    for char in text.upper():
        yield char

# generator expression
def uc_genexp(text):
    return (char for char in text.upper())

# iterator protocol
class uc_iter():
    def __init__(self, text):
        self.text = text.upper()
        self.index = 0
    def __iter__(self):
        return self
    def __next__(self):
        try:
            result = self.text[self.index]
        except IndexError:
            raise StopIteration
        self.index += 1
        return result

# getitem method
class uc_getitem():
    def __init__(self, text):
        self.text = text.upper()
    def __getitem__(self, index):
        return self.text[index]

要查看所有四种方法:

for iterator in uc_gen, uc_genexp, uc_iter, uc_getitem:
    for ch in iterator('abcde'):
        print(ch, end=' ')
    print()

结果是:

A B C D E
A B C D E
A B C D E
A B C D E

注意事项

两种生成器类型(uc_genuc_genexp)不能为reversed(); 普通的iterator(uc_iter)将需要__reversed__magic方法(根据docs,它必须返回一个新的iterator,但返回self工作结果(至少在CPython中));并且getitem iteratable(uc_getitem)必须具有__len__魔术方法:

    # for uc_iter we add __reversed__ and update __next__
    def __reversed__(self):
        self.index = -1
        return self
    def __next__(self):
        try:
            result = self.text[self.index]
        except IndexError:
            raise StopIteration
        self.index += -1 if self.index < 0 else +1
        return result

    # for uc_getitem
    def __len__(self)
        return len(self.text)

为了回答上校Panic关于无限懒惰求值的迭代器的第二个问题,以下是使用上述四种方法中的每一个的示例:

# generator
def even_gen():
    result = 0
    while True:
        yield result
        result += 2


# generator expression
def even_genexp():
    return (num for num in even_gen())  # or even_iter or even_getitem
                                        # not much value under these circumstances

# iterator protocol
class even_iter():
    def __init__(self):
        self.value = 0
    def __iter__(self):
        return self
    def __next__(self):
        next_value = self.value
        self.value += 2
        return next_value

# getitem method
class even_getitem():
    def __getitem__(self, index):
        return index * 2

import random
for iterator in even_gen, even_genexp, even_iter, even_getitem:
    limit = random.randint(15, 30)
    count = 0
    for even in iterator():
        print even,
        count += 1
        if count >= limit:
            break
    print

结果(至少在我的示例运行中):

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32

如何选择使用哪一个?这主要是一个品味问题。我最常看到的两种方法是生成器和迭代器协议,以及混合方法(__iter__返回生成器)。

生成器表达式可用于替换列表推导(它们很懒,因此可以节省资源)。

如果需要与早期的Python 2.x版本兼容,请使用__getitem__

There are four ways to build an iterative function:

Examples:

# generator
def uc_gen(text):
    for char in text.upper():
        yield char

# generator expression
def uc_genexp(text):
    return (char for char in text.upper())

# iterator protocol
class uc_iter():
    def __init__(self, text):
        self.text = text.upper()
        self.index = 0
    def __iter__(self):
        return self
    def __next__(self):
        try:
            result = self.text[self.index]
        except IndexError:
            raise StopIteration
        self.index += 1
        return result

# getitem method
class uc_getitem():
    def __init__(self, text):
        self.text = text.upper()
    def __getitem__(self, index):
        return self.text[index]

To see all four methods in action:

for iterator in uc_gen, uc_genexp, uc_iter, uc_getitem:
    for ch in iterator('abcde'):
        print(ch, end=' ')
    print()

Which results in:

A B C D E
A B C D E
A B C D E
A B C D E

Note:

The two generator types (uc_gen and uc_genexp) cannot be reversed(); the plain iterator (uc_iter) would need the __reversed__ magic method (which, according to the docs, must return a new iterator, but returning self works (at least in CPython)); and the getitem iteratable (uc_getitem) must have the __len__ magic method:

    # for uc_iter we add __reversed__ and update __next__
    def __reversed__(self):
        self.index = -1
        return self
    def __next__(self):
        try:
            result = self.text[self.index]
        except IndexError:
            raise StopIteration
        self.index += -1 if self.index < 0 else +1
        return result

    # for uc_getitem
    def __len__(self)
        return len(self.text)

To answer Colonel Panic’s secondary question about an infinite lazily evaluated iterator, here are those examples, using each of the four methods above:

# generator
def even_gen():
    result = 0
    while True:
        yield result
        result += 2


# generator expression
def even_genexp():
    return (num for num in even_gen())  # or even_iter or even_getitem
                                        # not much value under these circumstances

# iterator protocol
class even_iter():
    def __init__(self):
        self.value = 0
    def __iter__(self):
        return self
    def __next__(self):
        next_value = self.value
        self.value += 2
        return next_value

# getitem method
class even_getitem():
    def __getitem__(self, index):
        return index * 2

import random
for iterator in even_gen, even_genexp, even_iter, even_getitem:
    limit = random.randint(15, 30)
    count = 0
    for even in iterator():
        print even,
        count += 1
        if count >= limit:
            break
    print

Which results in (at least for my sample run):

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32

How to choose which one to use? This is mostly a matter of taste. The two methods I see most often are generators and the iterator protocol, as well as a hybrid (__iter__ returning a generator).

Generator expressions are useful for replacing list comprehensions (they are lazy and so can save on resources).

If one needs compatibility with earlier Python 2.x versions use __getitem__.


回答 2

首先,itertools模块在各种情况下都非常有用,在这种情况下,迭代器将是有用的,但是这里是在python中创建迭代器所需的全部:

那不是很酷吗?Yield可以用来代替函数中的正常收益。它返回的对象是相同的,但是它不会破坏状态并退出,而是为您要执行下一次迭代保存状态。这是直接从itertools函数列表中提取的示例:

def count(n=0):
    while True:
        yield n
        n += 1

如功能说明中所述(它是itertools模块中的count()函数…),它生成一个迭代器,该迭代器返回以n开头的连续整数。

生成器表达式是蠕虫(真棒蠕虫!)的另一种形式。可以使用它们代替列表推导来节省内存(列表推导会在内存中创建一个列表,如果未分配给变量,该列表在使用后会被销毁,但是生成器表达式可以创建一个Generator对象…说迭​​代器)。这是生成器表达式定义的示例:

gen = (n for n in xrange(0,11))

这与上面的迭代器定义非常相似,不同之处在于整个范围的预定范围是0到10。

我刚刚找到了xrange()(应该是我之前从未见过……)并将其添加到上述示例中。 xrange()range()的可迭代版本,其优点是不预先构建列表。如果您要遍历庞大的数据集并且只有那么多的内存可以进行访问,这将非常有用。

First of all the itertools module is incredibly useful for all sorts of cases in which an iterator would be useful, but here is all you need to create an iterator in python:

yield

Isn’t that cool? Yield can be used to replace a normal return in a function. It returns the object just the same, but instead of destroying state and exiting, it saves state for when you want to execute the next iteration. Here is an example of it in action pulled directly from the itertools function list:

def count(n=0):
    while True:
        yield n
        n += 1

As stated in the functions description (it’s the count() function from the itertools module…) , it produces an iterator that returns consecutive integers starting with n.

Generator expressions are a whole other can of worms (awesome worms!). They may be used in place of a List Comprehension to save memory (list comprehensions create a list in memory that is destroyed after use if not assigned to a variable, but generator expressions can create a Generator Object… which is a fancy way of saying Iterator). Here is an example of a generator expression definition:

gen = (n for n in xrange(0,11))

This is very similar to our iterator definition above except the full range is predetermined to be between 0 and 10.

I just found xrange() (suprised I hadn’t seen it before…) and added it to the above example. xrange() is an iterable version of range() which has the advantage of not prebuilding the list. It would be very useful if you had a giant corpus of data to iterate over and only had so much memory to do it in.


回答 3

我看到你们return self中有些人在做__iter__。我只想指出,__iter__它本身可以成为生成器(因此消除了对异常的需求__next__并引发了StopIteration异常)

class range:
  def __init__(self,a,b):
    self.a = a
    self.b = b
  def __iter__(self):
    i = self.a
    while i < self.b:
      yield i
      i+=1

当然,这里也可以直接生成一个生成器,但是对于更复杂的类,它可能很有用。

I see some of you doing return self in __iter__. I just wanted to note that __iter__ itself can be a generator (thus removing the need for __next__ and raising StopIteration exceptions)

class range:
  def __init__(self,a,b):
    self.a = a
    self.b = b
  def __iter__(self):
    i = self.a
    while i < self.b:
      yield i
      i+=1

Of course here one might as well directly make a generator, but for more complex classes it can be useful.


回答 4

这个问题是关于可迭代的对象,而不是关于迭代器。在Python中,序列也是可迭代的,因此制作可迭代类的一种方法是使其表现得像序列,即给它__getitem____len__方法。我已经在Python 2和3上对此进行了测试。

class CustomRange:

    def __init__(self, low, high):
        self.low = low
        self.high = high

    def __getitem__(self, item):
        if item >= len(self):
            raise IndexError("CustomRange index out of range")
        return self.low + item

    def __len__(self):
        return self.high - self.low


cr = CustomRange(0, 10)
for i in cr:
    print(i)

This question is about iterable objects, not about iterators. In Python, sequences are iterable too so one way to make an iterable class is to make it behave like a sequence, i.e. give it __getitem__ and __len__ methods. I have tested this on Python 2 and 3.

class CustomRange:

    def __init__(self, low, high):
        self.low = low
        self.high = high

    def __getitem__(self, item):
        if item >= len(self):
            raise IndexError("CustomRange index out of range")
        return self.low + item

    def __len__(self):
        return self.high - self.low


cr = CustomRange(0, 10)
for i in cr:
    print(i)

回答 5

对于复杂的对象,此页面上的所有答案都非常有用。但对于含有内置的迭代类型,属性那些像strlistsetdict,或任何实现collections.Iterable,你可以在你的类省略某些事情。

class Test(object):
    def __init__(self, string):
        self.string = string

    def __iter__(self):
        # since your string is already iterable
        return (ch for ch in self.string)
        # or simply
        return self.string.__iter__()
        # also
        return iter(self.string)

可以像这样使用:

for x in Test("abcde"):
    print(x)

# prints
# a
# b
# c
# d
# e

All answers on this page are really great for a complex object. But for those containing builtin iterable types as attributes, like str, list, set or dict, or any implementation of collections.Iterable, you can omit certain things in your class.

class Test(object):
    def __init__(self, string):
        self.string = string

    def __iter__(self):
        # since your string is already iterable
        return (ch for ch in self.string)
        # or simply
        return self.string.__iter__()
        # also
        return iter(self.string)

It can be used like:

for x in Test("abcde"):
    print(x)

# prints
# a
# b
# c
# d
# e

回答 6

如果没有,这是一个迭代函数yield。它利用iter函数和闭包将其状态保存在listpython 2的封闭范围内的可变()中。

def count(low, high):
    counter = [0]
    def tmp():
        val = low + counter[0]
        if val < high:
            counter[0] += 1
            return val
        return None
    return iter(tmp, None)

对于Python 3,封闭状态在封闭范围内保持不变,并nonlocal在局部范围内用于更新状态变量。

def count(low, high):
    counter = 0
    def tmp():
        nonlocal counter
        val = low + counter
        if val < high:
            counter += 1
            return val
        return None
    return iter(tmp, None)  

测试;

for i in count(1,10):
    print(i)
1
2
3
4
5
6
7
8
9

This is an iterable function without yield. It make use of the iter function and a closure which keeps it’s state in a mutable (list) in the enclosing scope for python 2.

def count(low, high):
    counter = [0]
    def tmp():
        val = low + counter[0]
        if val < high:
            counter[0] += 1
            return val
        return None
    return iter(tmp, None)

For Python 3, closure state is kept in an immutable in the enclosing scope and nonlocal is used in local scope to update the state variable.

def count(low, high):
    counter = 0
    def tmp():
        nonlocal counter
        val = low + counter
        if val < high:
            counter += 1
            return val
        return None
    return iter(tmp, None)  

Test;

for i in count(1,10):
    print(i)
1
2
3
4
5
6
7
8
9

回答 7

如果您想找简单明了的东西,也许对您来说已经足够了:

class A(object):
    def __init__(self, l):
        self.data = l

    def __iter__(self):
        return iter(self.data)

使用示例:

In [3]: a = A([2,3,4])

In [4]: [i for i in a]
Out[4]: [2, 3, 4]

If you looking for something short and simple, maybe it will be enough for you:

class A(object):
    def __init__(self, l):
        self.data = l

    def __iter__(self):
        return iter(self.data)

example of usage:

In [3]: a = A([2,3,4])

In [4]: [i for i in a]
Out[4]: [2, 3, 4]

回答 8

受Matt Gregory的回答启发,这里有一个更复杂的迭代器,它将返回a,b,…,z,aa,ab,…,zz,aaa,aab,…,zzy,zzz

    class AlphaCounter:
    def __init__(self, low, high):
        self.current = low
        self.high = high

    def __iter__(self):
        return self

    def __next__(self): # Python 3: def __next__(self)
        alpha = ' abcdefghijklmnopqrstuvwxyz'
        n_current = sum([(alpha.find(self.current[x])* 26**(len(self.current)-x-1)) for x in range(len(self.current))])
        n_high = sum([(alpha.find(self.high[x])* 26**(len(self.high)-x-1)) for x in range(len(self.high))])
        if n_current > n_high:
            raise StopIteration
        else:
            increment = True
            ret = ''
            for x in self.current[::-1]:
                if 'z' == x:
                    if increment:
                        ret += 'a'
                    else:
                        ret += 'z'
                else:
                    if increment:
                        ret += alpha[alpha.find(x)+1]
                        increment = False
                    else:
                        ret += x
            if increment:
                ret += 'a'
            tmp = self.current
            self.current = ret[::-1]
            return tmp

for c in AlphaCounter('a', 'zzz'):
    print(c)

Inspired by Matt Gregory’s answer here is a bit more complicated iterator that will return a,b,…,z,aa,ab,…,zz,aaa,aab,…,zzy,zzz

    class AlphaCounter:
    def __init__(self, low, high):
        self.current = low
        self.high = high

    def __iter__(self):
        return self

    def __next__(self): # Python 3: def __next__(self)
        alpha = ' abcdefghijklmnopqrstuvwxyz'
        n_current = sum([(alpha.find(self.current[x])* 26**(len(self.current)-x-1)) for x in range(len(self.current))])
        n_high = sum([(alpha.find(self.high[x])* 26**(len(self.high)-x-1)) for x in range(len(self.high))])
        if n_current > n_high:
            raise StopIteration
        else:
            increment = True
            ret = ''
            for x in self.current[::-1]:
                if 'z' == x:
                    if increment:
                        ret += 'a'
                    else:
                        ret += 'z'
                else:
                    if increment:
                        ret += alpha[alpha.find(x)+1]
                        increment = False
                    else:
                        ret += x
            if increment:
                ret += 'a'
            tmp = self.current
            self.current = ret[::-1]
            return tmp

for c in AlphaCounter('a', 'zzz'):
    print(c)