Commit 60eca933 authored by Raymond Hettinger's avatar Raymond Hettinger

C Code:

* Removed the ifilter flag wart by splitting it into two simpler functions.
* Fixed comment tabbing in C code.
* Factored module start-up code into a loop.

Documentation:
* Re-wrote introduction.
* Addede examples for quantifiers.
* Simplified python equivalent for islice().
* Documented split of ifilter().

Sets.py:
* Replace old ifilter() usage with new.
parent cb3319f6
...@@ -12,45 +12,43 @@ This module implements a number of iterator building blocks inspired ...@@ -12,45 +12,43 @@ This module implements a number of iterator building blocks inspired
by constructs from the Haskell and SML programming languages. Each by constructs from the Haskell and SML programming languages. Each
has been recast in a form suitable for Python. has been recast in a form suitable for Python.
With the advent of iterators and generators in Python 2.3, each of The module standardizes a core set of fast, memory efficient tools
these tools can be expressed easily and succinctly in pure python. that are useful by themselves or in combination. Standardization helps
Rather duplicating what can already be done, this module emphasizes avoid the readability and reliability problems which arise when many
providing value in other ways: different individuals create their own slightly varying implementations,
each with their own quirks and naming conventions.
\begin{itemize}
The tools are designed to combine readily with each another. This makes
\item Instead of constructing an over-specialized toolset, this module it easy to construct more specialized tools succinctly and efficiently
provides basic building blocks that can be readily combined. in pure Python.
For instance, SML provides a tabulation tool: \code{tabulate(\var{f})} For instance, SML provides a tabulation tool: \code{tabulate(\var{f})}
which produces a sequence \code{f(0), f(1), ...}. This toolbox which produces a sequence \code{f(0), f(1), ...}. This toolbox
takes a different approach of providing \function{imap()} and provides \function{imap()} and \function{count()} which can be combined
\function{count()} which can be combined to form to form \code{imap(\var{f}, count())} and produce an equivalent result.
\code{imap(\var{f}, count())} and produce an equivalent result.
Whether cast in pure python form or C code, tools that use iterators
\item Some tools were dropped because they offer no advantage over their are more memory efficient (and faster) than their list based counterparts.
pure python counterparts or because their behavior was too Adopting the principles of just-in-time manufacturing, they create
surprising. data when and where needed instead of consuming memory with the
computer equivalent of ``inventory''.
For instance, SML provides a tool: \code{cycle(\var{seq})} which
loops over the sequence elements and then starts again when the Some tools were omitted from the module because they offered no
sequence is exhausted. The surprising behavior is the need for advantage over their pure python counterparts or because their behavior
significant auxiliary storage (unusual for iterators). Also, it was too surprising.
is trivially implemented in python with almost no performance
penalty. For instance, SML provides a tool: \code{cycle(\var{seq})} which
loops over the sequence elements and then starts again when the
\item Another source of value comes from standardizing a core set of tools sequence is exhausted. The surprising behavior is the need for
to avoid the readability and reliability problems that arise when many significant auxiliary storage (which is unusual for an iterator).
different individuals create their own slightly varying implementations If needed, the tool is readily constructible using pure Python.
each with their own quirks and naming conventions.
Other tools are being considered for inclusion in future versions of the
\item Whether cast in pure python form or C code, tools that use iterators module. For instance, the function
are more memory efficient (and faster) than their list based counterparts. \function{chain(\var{it0}, \var{it1}, ...})} would return elements from
Adopting the principles of just-in-time manufacturing, they create the first iterator until it was exhausted and then move on to each
data when and where needed instead of consuming memory with the successive iterator. The module author welcomes suggestions for other
computer equivalent of ``inventory''. basic building blocks.
\end{itemize}
\begin{seealso} \begin{seealso}
\seetext{The Standard ML Basis Library, \seetext{The Standard ML Basis Library,
...@@ -107,24 +105,36 @@ by functions or loops that truncate the stream. ...@@ -107,24 +105,36 @@ by functions or loops that truncate the stream.
\end{verbatim} \end{verbatim}
\end{funcdesc} \end{funcdesc}
\begin{funcdesc}{ifilter}{predicate, iterable \optional{, invert}} \begin{funcdesc}{ifilter}{predicate, iterable}
Make an iterator that filters elements from iterable returning only Make an iterator that filters elements from iterable returning only
those for which the predicate is \code{True}. If those for which the predicate is \code{True}.
\var{invert} is \code{True}, then reverse the process and pass through If \var{predicate} is \code{None}, return the items that are true.
only those elements for which the predicate is \code{False}. Equivalent to:
If \var{predicate} is \code{None}, return the items that are true
(or false if \var{invert} has been set). Equivalent to:
\begin{verbatim} \begin{verbatim}
def ifilter(predicate, iterable, invert=False): def ifilter(predicate, iterable):
iterable = iter(iterable) if predicate is None:
while True: def predicate(x):
x = iterable.next() return x
if predicate is None: for x in iterable:
b = bool(x) if predicate(x):
else: yield x
b = bool(predicate(x)) \end{verbatim}
if not invert and b or invert and not b: \end{funcdesc}
\begin{funcdesc}{ifilterfalse}{predicate, iterable}
Make an iterator that filters elements from iterable returning only
those for which the predicate is \code{False}.
If \var{predicate} is \code{None}, return the items that are false.
Equivalent to:
\begin{verbatim}
def ifilterfalse(predicate, iterable):
if predicate is None:
def predicate(x):
return x
for x in iterable:
if not predicate(x):
yield x yield x
\end{verbatim} \end{verbatim}
\end{funcdesc} \end{funcdesc}
...@@ -169,21 +179,17 @@ by functions or loops that truncate the stream. ...@@ -169,21 +179,17 @@ by functions or loops that truncate the stream.
\begin{verbatim} \begin{verbatim}
def islice(iterable, *args): def islice(iterable, *args):
iterable = iter(iterable)
s = slice(*args) s = slice(*args)
next = s.start or 0 next = s.start or 0
stop = s.stop stop = s.stop
step = s.step or 1 step = s.step or 1
cnt = 0 for cnt, element in enumerate(iterable):
while True: if cnt < next:
while cnt < next: continue
dummy = iterable.next() if cnt >= stop:
cnt += 1 break
if cnt >= stop: yield element
break next += step
yield iterable.next()
cnt += 1
next += step
\end{verbatim} \end{verbatim}
\end{funcdesc} \end{funcdesc}
...@@ -324,6 +330,18 @@ from building blocks. ...@@ -324,6 +330,18 @@ from building blocks.
>>> def nth(iterable, n): >>> def nth(iterable, n):
... "Returns the nth item" ... "Returns the nth item"
... return islice(iterable, n, n+1).next() ... return list(islice(iterable, n, n+1))
>>> def all(pred, seq):
... "Returns True if pred(x) is True for every element in the iterable"
... return not nth(ifilterfalse(pred, seq), 0)
>>> def some(pred, seq):
... "Returns True if pred(x) is True at least one element in the iterable"
... return bool(nth(ifilter(pred, seq), 0))
>>> def no(pred, seq):
... "Returns True if pred(x) is False for every element in the iterable"
... return not nth(ifilter(pred, seq), 0)
\end{verbatim} \end{verbatim}
...@@ -57,7 +57,7 @@ what's tested is actually `z in y'. ...@@ -57,7 +57,7 @@ what's tested is actually `z in y'.
__all__ = ['BaseSet', 'Set', 'ImmutableSet'] __all__ = ['BaseSet', 'Set', 'ImmutableSet']
from itertools import ifilter from itertools import ifilter, ifilterfalse
class BaseSet(object): class BaseSet(object):
"""Common base class for mutable and immutable sets.""" """Common base class for mutable and immutable sets."""
...@@ -204,9 +204,9 @@ class BaseSet(object): ...@@ -204,9 +204,9 @@ class BaseSet(object):
value = True value = True
selfdata = self._data selfdata = self._data
otherdata = other._data otherdata = other._data
for elt in ifilter(otherdata.has_key, selfdata, True): for elt in ifilterfalse(otherdata.has_key, selfdata):
data[elt] = value data[elt] = value
for elt in ifilter(selfdata.has_key, otherdata, True): for elt in ifilterfalse(selfdata.has_key, otherdata):
data[elt] = value data[elt] = value
return result return result
...@@ -227,7 +227,7 @@ class BaseSet(object): ...@@ -227,7 +227,7 @@ class BaseSet(object):
result = self.__class__() result = self.__class__()
data = result._data data = result._data
value = True value = True
for elt in ifilter(other._data.has_key, self, True): for elt in ifilterfalse(other._data.has_key, self):
data[elt] = value data[elt] = value
return result return result
...@@ -260,7 +260,7 @@ class BaseSet(object): ...@@ -260,7 +260,7 @@ class BaseSet(object):
self._binary_sanity_check(other) self._binary_sanity_check(other)
if len(self) > len(other): # Fast check for obvious cases if len(self) > len(other): # Fast check for obvious cases
return False return False
for elt in ifilter(other._data.has_key, self, True): for elt in ifilterfalse(other._data.has_key, self):
return False return False
return True return True
...@@ -269,7 +269,7 @@ class BaseSet(object): ...@@ -269,7 +269,7 @@ class BaseSet(object):
self._binary_sanity_check(other) self._binary_sanity_check(other)
if len(self) < len(other): # Fast check for obvious cases if len(self) < len(other): # Fast check for obvious cases
return False return False
for elt in ifilter(self._data.has_key, other, True): for elt in ifilterfalse(self._data.has_key, other):
return False return False
return True return True
......
...@@ -13,12 +13,19 @@ class TestBasicOps(unittest.TestCase): ...@@ -13,12 +13,19 @@ class TestBasicOps(unittest.TestCase):
def isEven(x): def isEven(x):
return x%2==0 return x%2==0
self.assertEqual(list(ifilter(isEven, range(6))), [0,2,4]) self.assertEqual(list(ifilter(isEven, range(6))), [0,2,4])
self.assertEqual(list(ifilter(isEven, range(6), True)), [1,3,5])
self.assertEqual(list(ifilter(None, [0,1,0,2,0])), [1,2]) self.assertEqual(list(ifilter(None, [0,1,0,2,0])), [1,2])
self.assertRaises(TypeError, ifilter) self.assertRaises(TypeError, ifilter)
self.assertRaises(TypeError, ifilter, 3) self.assertRaises(TypeError, ifilter, 3)
self.assertRaises(TypeError, ifilter, isEven, 3) self.assertRaises(TypeError, ifilter, isEven, 3)
self.assertRaises(TypeError, ifilter, isEven, [3], True, 4)
def test_ifilterfalse(self):
def isEven(x):
return x%2==0
self.assertEqual(list(ifilterfalse(isEven, range(6))), [1,3,5])
self.assertEqual(list(ifilterfalse(None, [0,1,0,2,0])), [0,0,0])
self.assertRaises(TypeError, ifilterfalse)
self.assertRaises(TypeError, ifilterfalse, 3)
self.assertRaises(TypeError, ifilterfalse, isEven, 3)
def test_izip(self): def test_izip(self):
ans = [(x,y) for x, y in izip('abc',count())] ans = [(x,y) for x, y in izip('abc',count())]
...@@ -133,7 +140,19 @@ Samuele ...@@ -133,7 +140,19 @@ Samuele
>>> def nth(iterable, n): >>> def nth(iterable, n):
... "Returns the nth item" ... "Returns the nth item"
... return islice(iterable, n, n+1).next() ... return list(islice(iterable, n, n+1))
>>> def all(pred, seq):
... "Returns True if pred(x) is True for every element in the iterable"
... return not nth(ifilterfalse(pred, seq), 0)
>>> def some(pred, seq):
... "Returns True if pred(x) is True at least one element in the iterable"
... return bool(nth(ifilter(pred, seq), 0))
>>> def no(pred, seq):
... "Returns True if pred(x) is False for every element in the iterable"
... return not nth(ifilter(pred, seq), 0)
""" """
......
This diff is collapsed.
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment