What is the difference between iterators and generators? Some examples for when you would use each case would be helpful.

iterator is a more general concept: any object whose class has a next method (__next__ in Python 3) and an __iter__ method that does return self.

Every generator is an iterator, but not vice versa. A generator is built by calling a function that has one or more yield expressions (yield statements, in Python 2.5 and earlier), and is an object that meets the previous paragraph’s definition of an iterator.

You may want to use a custom iterator, rather than a generator, when you need a class with somewhat complex state-maintaining behavior, or want to expose other methods besides next (and __iter__ and __init__). Most often, a generator (sometimes, for sufficiently simple needs, a generator expression) is sufficient, and it’s simpler to code because state maintenance (within reasonable limits) is basically “done for you” by the frame getting suspended and resumed.

For example, a generator such as:

def squares(start, stop):
    for i in range(start, stop):
        yield i * i

generator = squares(a, b)

or the equivalent generator expression (genexp)

generator = (i*i for i in range(a, b))

would take more code to build as a custom iterator:

class Squares(object):
    def __init__(self, start, stop):
       self.start = start
       self.stop = stop
    def __iter__(self): return self
    def next(self): # __next__ in Python 3
       if self.start >= self.stop:
           raise StopIteration
       current = self.start * self.start
       self.start += 1
       return current

iterator = Squares(a, b)

But, of course, with class Squares you could easily offer extra methods, i.e.

    def current(self):
       return self.start

if you have any actual need for such extra functionality in your application.

What is the difference between iterators and generators? Some examples for when you would use each case would be helpful.

In summary: Iterators are objects that have an __iter__ and a __next__ (next in Python 2) method. Generators provide an easy, built-in way to create instances of Iterators.

A function with yield in it is still a function, that, when called, returns an instance of a generator object:

def a_function():
    "when called, returns generator object"
    yield

A generator expression also returns a generator:

a_generator = (i for i in range(0))

For a more in-depth exposition and examples, keep reading.

A Generator is an Iterator

Specifically, generator is a subtype of iterator.

>>> import collections, types
>>> issubclass(types.GeneratorType, collections.Iterator)
True

We can create a generator several ways. A very common and simple way to do so is with a function.

Specifically, a function with yield in it is a function, that, when called, returns a generator:

>>> def a_function():
        "just a function definition with yield in it"
        yield
>>> type(a_function)
<class 'function'>
>>> a_generator = a_function()  # when called
>>> type(a_generator)           # returns a generator
<class 'generator'>

And a generator, again, is an Iterator:

>>> isinstance(a_generator, collections.Iterator)
True

An Iterator is an Iterable

An Iterator is an Iterable,

>>> issubclass(collections.Iterator, collections.Iterable)
True

which requires an __iter__ method that returns an Iterator:

>>> collections.Iterable()
Traceback (most recent call last):
  File "<pyshell#79>", line 1, in <module>
    collections.Iterable()
TypeError: Can't instantiate abstract class Iterable with abstract methods __iter__

Some examples of iterables are the built-in tuples, lists, dictionaries, sets, frozen sets, strings, byte strings, byte arrays, ranges and memoryviews:

>>> all(isinstance(element, collections.Iterable) for element in (
        (), [], {}, set(), frozenset(), '', b'', bytearray(), range(0), memoryview(b'')))
True

Iterators require a next or __next__ method

In Python 2:

>>> collections.Iterator()
Traceback (most recent call last):
  File "<pyshell#80>", line 1, in <module>
    collections.Iterator()
TypeError: Can't instantiate abstract class Iterator with abstract methods next

And in Python 3:

>>> collections.Iterator()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class Iterator with abstract methods __next__

We can get the iterators from the built-in objects (or custom objects) with the iter function:

>>> all(isinstance(iter(element), collections.Iterator) for element in (
        (), [], {}, set(), frozenset(), '', b'', bytearray(), range(0), memoryview(b'')))
True

The __iter__ method is called when you attempt to use an object with a for-loop. Then the __next__ method is called on the iterator object to get each item out for the loop. The iterator raises StopIteration when you have exhausted it, and it cannot be reused at that point.

From the documentation

From the Generator Types section of the Iterator Types section of the Built-in Types documentation:

Python’s generators provide a convenient way to implement the iterator protocol. If a container object’s __iter__() method is implemented as a generator, it will automatically return an iterator object (technically, a generator object) supplying the __iter__() and next() [__next__() in Python 3] methods. More information about generators can be found in the documentation for the yield expression.

(Emphasis added.)

So from this we learn that Generators are a (convenient) type of Iterator.

Example Iterator Objects

You might create object that implements the Iterator protocol by creating or extending your own object.

class Yes(collections.Iterator):

    def __init__(self, stop):
        self.x = 0
        self.stop = stop

    def __iter__(self):
        return self

    def next(self):
        if self.x < self.stop:
            self.x += 1
            return 'yes'
        else:
            # Iterators must raise when done, else considered broken
            raise StopIteration

    __next__ = next # Python 3 compatibility

But it’s easier to simply use a Generator to do this:

def yes(stop):
    for _ in range(stop):
        yield 'yes'

Or perhaps simpler, a Generator Expression (works similarly to list comprehensions):

yes_expr = ('yes' for _ in range(stop))

They can all be used in the same way:

>>> stop = 4             
>>> for i, y1, y2, y3 in zip(range(stop), Yes(stop), yes(stop), 
                             ('yes' for _ in range(stop))):
...     print('{0}: {1} == {2} == {3}'.format(i, y1, y2, y3))
...     
0: yes == yes == yes
1: yes == yes == yes
2: yes == yes == yes
3: yes == yes == yes

Conclusion

You can use the Iterator protocol directly when you need to extend a Python object as an object that can be iterated over.

However, in the vast majority of cases, you are best suited to use yield to define a function that returns a Generator Iterator or consider Generator Expressions.

Finally, note that generators provide even more functionality as coroutines. I explain Generators, along with the yield statement, in depth on my answer to “What does the “yield” keyword do?”.

Iterators:

Iterator are objects which uses next() method to get next value of sequence.

Generators:

A generator is a function that produces or yields a sequence of values using yield method.

Every next() method call on generator object(for ex: f as in below example) returned by generator function(for ex: foo() function in below example), generates next value in sequence.

When a generator function is called, it returns an generator object without even beginning execution of the function. When next() method is called for the first time, the function starts executing until it reaches yield statement which returns the yielded value. The yield keeps track of i.e. remembers last execution. And second next() call continues from previous value.

The following example demonstrates the interplay between yield and call to next method on generator object.

>>> def foo():
...     print "begin"
...     for i in range(3):
...         print "before yield", i
...         yield i
...         print "after yield", i
...     print "end"
...
>>> f = foo()
>>> f.next()
begin
before yield 0            # Control is in for loop
0
>>> f.next()
after yield 0             
before yield 1            # Continue for loop
1
>>> f.next()
after yield 1
before yield 2
2
>>> f.next()
after yield 2
end
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>>

Adding an answer because none of the existing answers specifically address the confusion in the official literature.

Generator functions are ordinary functions defined using yield instead of return. When called, a generator function returns a generator object, which is a kind of iterator – it has a next() method. When you call next(), the next value yielded by the generator function is returned.

Either the function or the object may be called the “generator” depending on which Python source document you read. The Python glossary says generator functions, while the Python wiki implies generator objects. The Python tutorial remarkably manages to imply both usages in the space of three sentences:

Generators are a simple and powerful tool for creating iterators. They are written like regular functions but use the yield statement whenever they want to return data. Each time next() is called on it, the generator resumes where it left off (it remembers all the data values and which statement was last executed).

The first two sentences identify generators with generator functions, while the third sentence identifies them with generator objects.

Despite all this confusion, one can seek out the Python language reference for the clear and final word:

The yield expression is only used when defining a generator function, and can only be used in the body of a function definition. Using a yield expression in a function definition is sufficient to cause that definition to create a generator function instead of a normal function.

When a generator function is called, it returns an iterator known as a generator. That generator then controls the execution of a generator function.

So, in formal and precise usage, “generator” unqualified means generator object, not generator function.

The above references are for Python 2 but Python 3 language reference says the same thing. However, the Python 3 glossary states that

generator … Usually refers to a generator function, but may refer to a generator iterator in some contexts. In cases where the intended meaning isn’t clear, using the full terms avoids ambiguity.

Everybody has a really nice and verbose answer with examples and I really appreciate it. I just wanted to give a short few lines answer for people who are still not quite clear conceptually:

If you create your own iterator, it is a little bit involved – you have to create a class and at least implement the iter and the next methods. But what if you don’t want to go through this hassle and want to quickly create an iterator. Fortunately, Python provides a short-cut way to defining an iterator. All you need to do is define a function with at least 1 call to yield and now when you call that function it will return “something” which will act like an iterator (you can call next method and use it in a for loop). This something has a name in Python called Generator

Hope that clarifies a bit.

Previous answers missed this addition: a generator has a close method, while typical iterators don’t. The close method triggers a StopIteration exception in the generator, which may be caught in a finally clause in that iterator, to get a chance to run some clean‑up. This abstraction makes it most usable in the large than simple iterators. One can close a generator as one could close a file, without having to bother about what’s underneath.

That said, my personal answer to the first question would be: iteratable has an __iter__ method only, typical iterators have a __next__ method only, generators has both an __iter__ and a __next__ and an additional close.

For the second question, my personal answer would be: in a public interface, I tend to favor generators a lot, since it’s more resilient: the close method an a greater composability with yield from. Locally, I may use iterators, but only if it’s a flat and simple structure (iterators does not compose easily) and if there are reasons to believe the sequence is rather short especially if it may be stopped before it reach the end. I tend to look at iterators as a low level primitive, except as literals.

For control flow matters, generators are an as much important concept as promises: both are abstract and composable.

Generator Function, Generator Object, Generator:

A Generator function is just like a regular function in Python but it contains one or more yield statements. Generator functions is a great tool to create Iterator objects as easy as possible. The Iterator object returend by generator function is also called Generator object or Generator.

In this example I have created a Generator function which returns a Generator object <generator object fib at 0x01342480>. Just like other iterators, Generator objects can be used in a for loop or with the built-in function next() which returns the next value from generator.

def fib(max):
    a, b = 0, 1
    for i in range(max):
        yield a
        a, b = b, a + b
print(fib(10))             #<generator object fib at 0x01342480>

for i in fib(10):
    print(i)               # 0 1 1 2 3 5 8 13 21 34


print(next(myfib))         #0
print(next(myfib))         #1
print(next(myfib))         #1
print(next(myfib))         #2

So a generator function is the easiest way to create an Iterator object.

Iterator:

Every generator object is an iterator but not vice versa. A custom iterator object can be created if its class implements __iter__ and __next__ method (also called iterator protocol).

However, it is much easier to use generators function to create iterators because they simplify their creation, but a custom Iterator gives you more freedom and you can also implement other methods according to your requirements as shown in the below example.

class Fib:
    def __init__(self,max):
        self.current=0
        self.next=1
        self.max=max
        self.count=0

    def __iter__(self):
        return self

    def __next__(self):
        if self.count>self.max:
            raise StopIteration
        else:
            self.current,self.next=self.next,(self.current+self.next)
            self.count+=1
            return self.next-self.current

    def __str__(self):
        return "Generator object"

itobj=Fib(4)
print(itobj)               #Generator object

for i in Fib(4):  
    print(i)               #0 1 1 2

print(next(itobj))         #0
print(next(itobj))         #1
print(next(itobj))         #1

Examples from Ned Batchelder highly recommended for iterators and generators

A method without generators that do something to even numbers

def evens(stream):
   them = []
   for n in stream:
      if n % 2 == 0:
         them.append(n)
   return them

while by using a generator

def evens(stream):
    for n in stream:
        if n % 2 == 0:
            yield n

• We don’t need any list nor a return statement
• Efficient for large/ infinite length stream … it just walks and yield the value

Calling the evens method (generator) is as usual

num = [...]
for n in evens(num):
   do_smth(n)

• Generator also used to Break double loop

Iterator

A book full of pages is an iterable, A bookmark is an iterator

and this bookmark has nothing to do except to move next

litr = iter([1,2,3])
next(litr) ## 1
next(litr) ## 2
next(litr) ## 3
next(litr) ## StopIteration  (Exception) as we got end of the iterator

To use Generator … we need a function

To use Iterator … we need next and iter

As been said:

A Generator function returns an iterator object

The Whole benefit of Iterator:

Store one element a time in memory

You can compare both approaches for the same data:

def myGeneratorList(n):
    for i in range(n):
        yield i

def myIterableList(n):
    ll = n*[None]
    for i in range(n):
        ll[i] = i
    return ll

# Same values
ll1 = myGeneratorList(10)
ll2 = myIterableList(10)
for i1, i2 in zip(ll1, ll2):
    print("{} {}".format(i1, i2))

# Generator can only be read once
ll1 = myGeneratorList(10)
ll2 = myIterableList(10)

print("{} {}".format(len(list(ll1)), len(ll2)))
print("{} {}".format(len(list(ll1)), len(ll2)))

# Generator can be read several times if converted into iterable
ll1 = list(myGeneratorList(10))
ll2 = myIterableList(10)

print("{} {}".format(len(list(ll1)), len(ll2)))
print("{} {}".format(len(list(ll1)), len(ll2)))

Besides, if you check the memory footprint, the generator takes much less memory as it doesn’t need to store all the values in memory at the same time.

I am writing specifically for Python newbies in a very simple way, though deep down Python does so many things.

Let’s start with the very basic:

Consider a list,

l = [1,2,3]

Let’s write an equivalent function:

def f():
    return [1,2,3]

o/p of print(l): [1,2,3] & o/p of print(f()) : [1,2,3]

Let’s make list l iterable: In python list is always iterable that means you can apply iterator whenever you want.

Let’s apply iterator on list:

iter_l = iter(l) # iterator applied explicitly

Let’s make a function iterable, i.e. write an equivalent generator function. In python as soon as you introduce the keyword yield; it becomes a generator function and iterator will be applied implicitly.

Note: Every generator is always iterable with implicit iterator applied and here implicit iterator is the crux So the generator function will be:

def f():
  yield 1 
  yield 2
  yield 3

iter_f = f() # which is iter(f) as iterator is already applied implicitly

So if you have observed, as soon as you made function f a generator, it is already iter(f)

Now,

l is the list, after applying iterator method “iter” it becomes, iter(l)

f is already iter(f), after applying iterator method “iter” it becomes, iter(iter(f)), which is again iter(f)

It’s kinda you are casting int to int(x) which is already int and it will remain int(x).

For example o/p of :

print(type(iter(iter(l))))

is

<class 'list_iterator'>

Never forget this is Python and not C or C++

Hence the conclusion from above explanation is:

list l ~= iter(l)

generator function f == iter(f)