Python Patterns 1

Python Patterns 1

• “The patterns are built in” to Python
• Concurrent patterns need more attention
• Missing patterns could be new features

The gist of this talk—

The gist of this talk

• Built-in language features
• Are provided by frameworks
• We avoid big applications

What is a pattern?

Two books…

Small procedural pattern

if 'owner' in roledict:
    owner = roledict['owner']
else:
    owner = admin

owner = roledict.get('owner', admin)

Bigger procedural pattern

for item in items:
    if is_for_sale(item):
        cost = compute_cost(item)
        if cost <= wallet.money:
            buy(item)

for item in items:
    if not is_for_sale(item):
        continue
    cost = compute_cost(item)
    if cost > wallet.money:
        continue
    buy(item)

• Classes
• Objects
• References

1st principle in Design Patterns (p18):

Why would you do this?

No

Why?

In other words—

In general

Opinion

So,

• 5 Creational Patterns
• 7 Structural Patterns
• 11 Behavioral Patterns

The 5 Creational Patterns

Python is brilliant

• Returning another class
• Always return the same instance

• Returning another class

  (Factory Method, Abstract Factory)
• Always return the same instance

  (Singleton)

Notes about Python singletons

1. Using a global variable is bad,

   because you can never switch in

   something dynamic: mymodule.foo

   can not be intercepted!

Notes about Python singletons

2. So you should always make callers

   actually invoke a function to get

   your singleton; minimally:

   # A kind-of singleton
   
   _singleton = MyClass()
   def get_singleton():
       return _singleton
   

Notes about Python singletons

3. But note that the real Singleton

   pattern is supposed to let the

   caller provide a subclass!

_singleton = None

def get_singleton(cls=MyClass):
    global _singleton
    if _singleton is None:
        _singleton = cls()
    return _singleton

• Hide _singleton inside a closure
• Hide _singleton inside a class

Prototype

Builder

Making XML without a builder

from lxml import etree

root = etree.Element('body')
h1 = etree.Element('h1')
h1.text = 'The Title'
root.append(h1)
p = etree.Element('p')
p.text = 'Always write Python'
root.append(p)

Making XML with a builder

from lxml.builder import E

doc = E('body',
        E('h1', 'The Title'),
        E('p', 'Always write Python'))

Builder

Creational Patterns in Python

Designed away by language:

Trivial:

Useful:

The 7 Structural Patterns

Structural Pattern: Adapter

Adapter

Adapter example

>>> import socket
>>> s = socket.socket()
>>> s.read
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: '_socketobject' object
 has no attribute 'read'

Structural Pattern: Bridge

Bridge

• Says not to use subclassing for

  two separate purposes in one class
• Then you wind up with

  MacPlainWindow   MacBorderedWindow
  WinPlainWindow   WinBorderedWindow
  LinuxPlainWindow LinuxBorderedWindow
  

• Partly rationalized by limitations

  of C++ (no mixins, much subclassing)

Solution: another layer!

Gary Bernhardt at PyCon 2012

class Person(Database_Model):
    def rename(self, new_name): ...
    def direct_deposit(self, amount): ...

Gary Bernhardt at PyCon 2012

This leads to a Brandon rule:

Structural Pattern: Composite

Composition is a common pattern

• email.Message can have messages inside
• lxml.etree.Element lists child elements

Two common mechanisms:

# Store children in attributes

obj.attr = subobject

# Store a list or dict of children

obj.children = [subobject, ...]

Structural Pattern: Facade

• An etree Element lets you root.find()

  and root.iter() which traverse the entire

  tree of nodes for you

Structural Pattern: Flyweight

Flyweight

Flyweight

• A small set of objects can each

  appear thousands of times inside

  of parent objects
• But context like the x and y passed

  to Character.draw(x, y) will now need

  to be passed in to flyweight methods
• Saves memory at the expense of noise

  and higher coupling

Flyweight

Structural Pattern: Proxy

Proxy

• A proxy object wraps another object called

  the subject and accepts method calls and

  attribute lookups on its behalf
• Proxying can be performed dynamically

  in Python with __getattr__() instead of

  having to write n proxying methods

Proxy

• weakref.proxy(obj) returns a proxy
• Remote procedure call libraries offer

  proxies for remote APIs, including the

  Standard Library xmlrpc.client.ServerProxy

  and also Pyro and rpyc
• The Zope web framework used proxies

  that enforced security

Structural Pattern: Decorator

Decorator

• Like the Proxy, a Decorator class

  offers the same attributes and methods

  as the subject that it wraps
• But instead of being completely

  transparent, it varies the behavior

  or edits the data passing in and

  out of the subject
• Decorators tend to appear in “glue”

  code in applications, not inside

  Python libraries themselves

So there you have it

Structural Patterns

• Adapter
• Bridge
• Composite
• Decorator
• Facade
• Flyweight
• Proxy

The 11 Behavioral Patterns

Behavioral Pattern: Chain of responsibility

• Pass a request along a chain of objects

  until one of them decides to handle it

  def processClick(self, x, y):
      if self.active:
          self.buttonPress(x, y)
      else:
          self.next.processClick(x, y)
  

• Used by GUIs and DOMs for mouse events
• Can be used by “help” feature: maybe button

  has no help, but enclosing form does

Behavioral Pattern: Command

Replaces immediate actions like:

paintLine(x1, y1, x2, y2)

Instead, you instantiate a command:

cmd = PaintLineCommand(x1, y1, x2, y2)
cmd.do()
command_history.append(cmd)

• Allows auditing and undo()
• Crucial to version control systems

Behavioral Pattern: Interpreter

• Data structure can be AST or bytecode
• Python itself is interpreted!
• Small domain-specific languages do sometimes

  get written in Python to ease customization

Behavioral Pattern: Iterator

• Built-in because Python is awesome
• Introduced in 2001 — the most important

  Python innovation of decade
• See Raymond Hettinger's PyCodeConf 2011

  talk “What Makes Python Awesome?”
• Lets you for x in obj: across your own

  user-defined obj, or loop by hand

  with i = iter(obj) and i.next()

1. What it is iterating across
2. Where it is in the sequence

Iterator

class Box(object):
    def __init__(self): self.things = [10, 20, 30]
    def __iter__(self): return BoxIterator(self)

class BoxIterator(object):
    def __init__(self, box):
        self.box = box
        self.index = -1

    def next(self):
        self.index += 1
        if self.index >= len(self.box.things):
            raise StopIteration()
        return self.box.things[self.index]

Iterator

• Very powerful pattern
• Eliminates ugly mechanics of iteration

  so code is more readable
• Enhances re-use because code is

  easy to lock together in new ways;

  again, see Raymond's talk!

# JavaScript is littered with:
for (var i=0; i < box.length; i++) ...

Behavioral Pattern: Mediator

• Parent object responds to child events

  and plays out their consequences for

  other child objects
• Child objects like buttons or form fields

  can remain generic instead of having to

  be subclassed to learn specific behavior
• Children have no references to each other

Behavioral Pattern: Memento

• A memento is a record of an object's

  internal state — might be a string, or

  file, or complex data structure
• Callers ask an object instance for a

  memento, then can hand it back later

  to ask the object to restore itself

  to its earlier state
• Similar to pickle but instead of

  creating a new object like un-pickling,

  it changes an existing object

Behavioral Pattern: Observer

• Very popular pattern in GUIs and DOMs
• Display elements let the framework

  know that they need to redraw when

  specific model attributes change
• Your models stay simple and have

  no knowledge of the big application

  you have build around them, so long

  as they signal a list of listeners

  when an attribute changes

Without Observer pub-sub, you get:

class MyModel(...):
    def set_total(self, number):
        self.total = number
        self.titlebar.update()
        self.graph.update()
        self.summary.update()

With Observer pub-sub, things are simpler:

class MyModel(...):
    def set_total(self, number):
        self.total = number

class MyTitlebar(...):
    def __init__(self, model):
        subscribe(model, 'total', self.redraw)

⋮

Behavioral Pattern: State

        open()    close()
TCPStart → TCPOpen → TCPClosed

What we want to avoid:

if state == 'start':
    if action == 'open':
        ⋮
    elif action == 'close':
        ⋮
elif state == 'open':
    if action == 'open':
        ⋮
    elif action == 'close':
        ⋮

Behavioral Pattern: State

        open()    close()
TCPStart → TCPOpen → TCPClosed

• Represent each state as a class

  like TCPOpen or TCPClosed
• Give each class one method for

  each transition like open()

  and close() that returns the

  next state
• Avoids huge nested if-then's, but

  simple state machines can be a dict

  in Python instead

Behavioral Pattern: Strategy

• Parametrize a big process by passing in

  an object that specifies custom behavior
• Python has first-class functions so we

  often just pass callbacks instead
• A simple example of the Strategy pattern

  is the key= function that we pass to

  sorted() and list.sort()

Behavioral Pattern: Template

• Give your classes empty methods like

  aboutToStart() and justStarted()

  that a subclass can customize
• The original author of threading in

  the Standard Library intended callers to

  subclass (!) Thread and provide their

  own, more interesting run() method
• In Python we often pass a callable instead,

  or — when there are many callbacks — pass

  a Strategy object instead

Behavioral Pattern: Visitor

• Replaces doThis() and doThat() methods on

  tree nodes, that act on the current node and

  then call the same method on their children,

  with a do(action) traversal method
• Not frequently encountered in Python
• We tend to turn this problem inside out:

  for node in lxml_root.walk():  # or os.walk!
      # do something to the node
  

Behavioral Patterns

• Are big solutions for big problems
• Most patterns work fine in Python
• Many of them turn up in big and

  popular libraries, or even in the

  Standard Library
• The Iterator pattern is now a

  foundation of how we write Python

Conclusions

• Creational patterns do tend to partly

  disappear in Python
• But the Structural patterns are often

  good ways to structure your code
• Most Behavioral patterns remain useful

  but some can be collapsed into callbacks

  or data structures that hold functions

Bonus Round

Dependency Injection

The Problem

def a(): ... b() ...
def b(): ... c() ...
def c(): pass

• Hard to re-use a() in situations

  where you want to do something else

  instead of doing b()
• Hard to test a() in isolation
• Expensive to test a() without

  Foord's mock.patch() if b() or c()

  does something expensive like talking

  to a database or writing a file

The Problem

A simple, concrete example:

def google_search(query_text):
    ⋮
    urllib.urlopen(url)
    ⋮

• Hard to test
• Hard to re-use

Dependency Injection

def google_search(query_text, urlopen):
    ⋮
    response = urlopen(url)
    ⋮

• In production, pass urllib.urlopen
• In tests, just pass a fake function
• Easy to re-purpose in the future

Problem

This can get annoying

big_function(
    query_text,
    urlopen=urllib.urlopen,
    listdir=os.listdir,
    stat=os.stat,
    link=os.link,
        ⋮
    ):
    ⋮

Another Approach

def google_search(query_text):
    ⋮
    return url

def google_parse(response):
    ⋮
    return result

def main():
    url = google_search(query_text)
    response = urlopen(url))
    result = google_parse(response)

Guidelines

• I do this to application subsystems,

  but I do not make the entire

  application a huge function that

  thinks dozens of pieces together!
• I often pass my own subsystems as

  parameters, but give myself permission

  to use Standard Library calls without

  insisting on DI — Foord mock FTW

End of Bonus Round

So, Patterns

My claim:

Reason #1

• Answering a web request
• Transforming a text file
• Searching through log files
• Operate on a database table
• Doing a Hadoop reduce() step

Reason #2

• Pyglet
• PyGame
• Tkinter
• PySide / PyQt
• Twisted

Again:

Thank you!

Some earlier work:

• Vespe Savikko (1997) (academic)
• Bruce Eckel (2001) (unfinished)
• Greg Sullivan (2002) (not Python)
• Alex Martelli (2007)