Implementing Iteration

Agenda

  1. Review: Iteration
  2. Details: iterables, iterators, iter, and next
  3. Implementing iterators with classes
  4. Implementing iterators with generators and yield

1. Review: Iteration

Iteration simply refers to the process of accessing — one by one — the items stored in some container. The order of the items, and whether or not the iteration is comprehensive, depends on the container.

In Python, we typically perform iteration using the for loop.

In [43]:
# e.g., iterating over a list
l = [2**x for x in range(10)]

for n in l:
    print(n)
1
2
4
8
16
32
64
128
256
512
In [44]:
# e.g., iterating over the key-value pairs in a dictionary
d = {x:2**x for x in range(10)}

for k,v in d.items():
    print(k, '=>', v)
0 => 1
1 => 2
2 => 4
3 => 8
4 => 16
5 => 32
6 => 64
7 => 128
8 => 256
9 => 512

2. Details: iterables, iterators, iter, and next

We can iterate over anything that is iterable. Intuitively, if something can be used as the source of items in a for loop, it is iterable.

But how does a for loop really work? (Review time!)

In [45]:
l = [2**x for x in range(10)]
l
Out[45]:
[1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
In [46]:
type(l)
Out[46]:
list
In [47]:
it = iter(l) # or it = l.__iter__()
In [48]:
type(it)
Out[48]:
list_iterator
In [68]:
next(it) #repeat evaluating it multiple times
---------------------------------------------------------------------------
StopIteration                             Traceback (most recent call last)
<ipython-input-68-39f17e027fb2> in <module>()
----> 1 next(it) #repeat evaluting multiple times

StopIteration: 
In [70]:
it = iter(l)
while True:
    try:
        x = next(it) #try to get the next value from the interation
        print(x)
    except StopIteration: # this exception occurs if there is no more values
        break
1
2
4
8
16
32
64
128
256
512
In [81]:
for x in l:
    print(x)
1
2
4
8
16
32
64
128
256
512
In [82]:
it = iter(l)
In [98]:
for x in it:  #difference with "for x in l:" ? 
    print(x)

3. Implementing iterators with classes

In [ ]:
class MyIterator:
    def __init__(self, max):
        self.max = max
        self.curr = 0
        
    # the following methods are required for iterator objects
    
    def __next__(self):
        if self.curr == self.max:
            raise StopIteration
        else:
            ret = self.curr
            self.curr += 1
            return ret
            
    def __iter__(self):
        
        return self
In [121]:
it = MyIterator(10)
In [127]:
next(it)
Out[127]:
5
In [128]:
it = MyIterator(10)
while True:
    try:
        print(next(it))
    except StopIteration:
        break
0
1
2
3
4
5
6
7
8
9
In [120]:
it = MyIterator(10)
In [129]:
for i in it:  # def __iter__(self) is required for this statement
    print(i)

For a container type, we need to implement an __iter__ method that returns an iterator.

In [150]:
# now implement __iter__()

# Note that ArrayList is not an iterator, but an iterable; so no implementation of __next__()

class ArrayListIterator:
    def __init__(self, lst):
        self.lst = lst
        self.currIdx = 0
    
    def __next__(self):
        if self.currIdx < len(self.lst.data):
            ret = self.lst.data[self.currIdx]
            self.currIdx += 1
            return ret
        else:
            raise StopIteration
    
    def __iter__(self):
        return self
    
    
class ArrayList:
    def __init__(self):
        self.data = []
        
    def append(self, val):
        self.data.append(None)
        self.data[len(self.data)-1] = val
        
    def __iter__(self):
        ret = ArrayListIterator(self)
        return ret
In [151]:
l = ArrayList()

for x in range(10):
    l.append(2**x)
In [152]:
it = iter(l)
In [153]:
type(it)
Out[153]:
__main__.ArrayListIterator
In [154]:
next(it)
Out[154]:
1
In [155]:
for x in l:   #not one time thing, iterable, not iterator
    print(x)
1
2
4
8
16
32
64
128
256
512
In [156]:
#how to ensure that no one else can create an instance of ArrayListIterator?

class ArrayList:
    def __init__(self):
        self.data = []
        
    def append(self, val):
        self.data.append(None)
        self.data[len(self.data)-1] = val
        
    def __iter__(self):
        class ArrayListIterator:
            def __init__(self, lst):
                self.lst = lst
                self.currIdx = 0

            def __next__(self):
                if self.currIdx < len(self.lst.data):
                    ret = self.lst.data[self.currIdx]
                    self.currIdx += 1
                    return ret
                else:
                    raise StopIteration

            def __iter__(self):
                return self

        ret = ArrayListIterator(self)
        return ret
In [157]:
l = ArrayList()
it = iter(l)
type(it)
Out[157]:
__main__.ArrayList.__iter__.<locals>.ArrayListIterator

4. Implementing iterators with generators and yield

What's a "generator"?

In [205]:
l = [2**x for x in range(10)]
In [206]:
l
Out[206]:
[1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
In [207]:
len(l)
Out[207]:
10
In [213]:
#generator expression

g = (2**x for x in range(10))
In [209]:
g
Out[209]:
<generator object <genexpr> at 0x11236aaf0>
In [210]:
type(g)
Out[210]:
generator
In [212]:
for x in g:  #only once
    print(x)
In [225]:
next(g)
---------------------------------------------------------------------------
StopIteration                             Traceback (most recent call last)
<ipython-input-225-e734f8aca5ac> in <module>()
----> 1 next(g)

StopIteration: 
In [226]:
# generator
# a special type of iterator
# generate as you need it

g = ((a, b, c) for a in range(1, 100)
               for b in range(1, 100)
               for c in range(1, 100)
               if a*a + b*b == c*c)
In [243]:
next(g)
Out[243]:
(14, 48, 50)
In [244]:
l = [(a, b, c) for a in range(1, 100)
               for b in range(1, 100)
               for c in range(1, 100)
               if a*a + b*b == c*c]
In [245]:
l
Out[245]:
[(3, 4, 5),
 (4, 3, 5),
 (5, 12, 13),
 (6, 8, 10),
 (7, 24, 25),
 (8, 6, 10),
 (8, 15, 17),
 (9, 12, 15),
 (9, 40, 41),
 (10, 24, 26),
 (11, 60, 61),
 (12, 5, 13),
 (12, 9, 15),
 (12, 16, 20),
 (12, 35, 37),
 (13, 84, 85),
 (14, 48, 50),
 (15, 8, 17),
 (15, 20, 25),
 (15, 36, 39),
 (16, 12, 20),
 (16, 30, 34),
 (16, 63, 65),
 (18, 24, 30),
 (18, 80, 82),
 (20, 15, 25),
 (20, 21, 29),
 (20, 48, 52),
 (21, 20, 29),
 (21, 28, 35),
 (21, 72, 75),
 (24, 7, 25),
 (24, 10, 26),
 (24, 18, 30),
 (24, 32, 40),
 (24, 45, 51),
 (24, 70, 74),
 (25, 60, 65),
 (27, 36, 45),
 (28, 21, 35),
 (28, 45, 53),
 (30, 16, 34),
 (30, 40, 50),
 (30, 72, 78),
 (32, 24, 40),
 (32, 60, 68),
 (33, 44, 55),
 (33, 56, 65),
 (35, 12, 37),
 (35, 84, 91),
 (36, 15, 39),
 (36, 27, 45),
 (36, 48, 60),
 (36, 77, 85),
 (39, 52, 65),
 (39, 80, 89),
 (40, 9, 41),
 (40, 30, 50),
 (40, 42, 58),
 (40, 75, 85),
 (42, 40, 58),
 (42, 56, 70),
 (44, 33, 55),
 (45, 24, 51),
 (45, 28, 53),
 (45, 60, 75),
 (48, 14, 50),
 (48, 20, 52),
 (48, 36, 60),
 (48, 55, 73),
 (48, 64, 80),
 (51, 68, 85),
 (52, 39, 65),
 (54, 72, 90),
 (55, 48, 73),
 (56, 33, 65),
 (56, 42, 70),
 (57, 76, 95),
 (60, 11, 61),
 (60, 25, 65),
 (60, 32, 68),
 (60, 45, 75),
 (60, 63, 87),
 (63, 16, 65),
 (63, 60, 87),
 (64, 48, 80),
 (65, 72, 97),
 (68, 51, 85),
 (70, 24, 74),
 (72, 21, 75),
 (72, 30, 78),
 (72, 54, 90),
 (72, 65, 97),
 (75, 40, 85),
 (76, 57, 95),
 (77, 36, 85),
 (80, 18, 82),
 (80, 39, 89),
 (84, 13, 85),
 (84, 35, 91)]
In [247]:
l[1]
Out[247]:
(4, 3, 5)
In [191]:
next(g)
In [248]:
#generator function

def foo():
    yield
In [249]:
foo()
Out[249]:
<generator object foo at 0x11236bf68>
In [254]:
def foo():
    print("hello")
    yield
In [251]:
foo()
Out[251]:
<generator object foo at 0x11237c3b8>
In [252]:
def foo():
    print("hello")
    if 0 > 10:
        yield
In [253]:
foo()
Out[253]:
<generator object foo at 0x11237c518>
In [255]:
g = foo()
In [256]:
type(g)
Out[256]:
generator
In [257]:
next(g)
hello
In [258]:
def foo():
    print("hello")
    yield 1
In [259]:
g = foo()
In [260]:
next(g)
hello
Out[260]:
1
In [261]:
def foo():
    print('L1')
    yield 1
    print("L2")
    yield 5
    print('L3')
    yield 'ouch'
In [262]:
g = foo()
In [266]:
next(g)
---------------------------------------------------------------------------
StopIteration                             Traceback (most recent call last)
<ipython-input-266-e734f8aca5ac> in <module>()
----> 1 next(g)

StopIteration: 
In [386]:
def range_gen(max):
    i = 0
    while True:
        if i == max:
            break
        else:
            yield i
            i += 1
In [387]:
g = range_gen(10)
In [389]:
next(g)
Out[389]:
1
In [292]:
# implement our own range 

for i in range_gen(10):
    print(i)
0
1
2
3
4
5
6
7
8
9
In [293]:
# one more example of generator function

def infinitely(val):
    while True:
        yield val
In [294]:
g = infinitely('hello')
In [329]:
next(g)
Out[329]:
'hello'
In [330]:
# one more example of generator function

def fibonacci_series():
    i, j = 1, 1
    while True:
        yield i
        i, j = j, i+j
In [331]:
g = fibonacci_series()
In [381]:
next(g)
Out[381]:
12586269025
In [382]:
class ArrayList:
    def __init__(self):
        self.data = []
        
    def append(self, val):
        self.data.append(None)
        self.data[len(self.data)-1] = val
        
    def __iter__(self):
        for i in range(len(self.data)):
            yield self.data[i]
In [383]:
l = ArrayList()

for x in range(10):
    l.append(2**x)
In [384]:
for y in l:
    print(y)
1
2
4
8
16
32
64
128
256
512