davidbau.com

Python Templates

Here is a cute lightweight python templating module that supports inline python, speedy compilation, subtemplates, and subclassing, all with 70 lines of code.

(Update: before reading on, also consider this @stringfunction template approach which is faster and better and simpler.)

Basic usage looks like this:

import templet

class PoemTemplate(templet.Template):
  template = "The $creature jumped over the $thing."

print PoemTemplate(creature='cow', thing='moon')

Template strings are compiled into methods through the magic of python metaclasses, so they run fast.

The class above actually becomes a class with a method called "template" that calls "write" several times. You can think of the class as being roughly equivalent to this:

class PoemTemplate(templet.Template):
  def template(self, creature=None, thing=None):
    self.write("The ")
    self.write(creature)
    self.write(" jumped over the ")
    self.write(thing)
    self.write(".")

Templates can contain ${...} and ${{...}} blocks with python code, so you are not limited to just variable substitution: you can use arbitrary python in your template. This straightforward approach of compiling a template into python is simple, yet powerful...

Templates in Python

Why another python templating system?

Python's built-in string interpolation is great for little messages, but its lack of inline code makes the % operator too clumsy for building larger strings like HTML files. There are more than a dozen "full" templating solutions aimed at making it easy to build HTML in python, but they are all too full-featured for my taste.

For my weekend hacking projects, I don't really want to have to introduce a whole new language and install a big package just to build some strings. Why is it that Django and Cheetah call functions "filters" and make you register and invoke them with contortions like {{ var|foo:"bar" }} or ${var, foo="bar"}? Why do template systems invent their own versions of "if" and "while" and "def"? Python is a beautiful language, and I do not want to embed esperanto. I just want my templates to be python.

A few people have made simple python templates that avoid overcomplexity. For example, James Tauber has played with overriding getitem to make % itself more powerful. I like his idea, and I like the power you get when making each template into a class, but I find the % technique too clumsy in practice.

Tomer Filiba's lightweight Templite class is the solution I have seen that comes closest to exposing the simplicity of python in a lightweight template system. Wherever one of Filiba's templates is not literal text, it is python. Although Filiba does have his own template parser, he does not invent a new "if" statement or a new concept for function calls. I especially like the fact that Templite weighs in at just 40 lines of code with no dependencies.

But maybe Templite is just slightly too simple. When building the not-very-complicated UI for the RPS arena, I ended up rolling a new solution, because found that I wanted some key things that Templite misses:

• Access to globals (like imported modules) from inline code.
• Object-orientation: access to "self", method calls, inheritance, etc.
• Nice handling of indented text and other help with long strings in code.
• Load-time compilation (and friendly compilation errors).
• Syntactic sugar for common simple $vars and ${expressions()}.

The python templet module here provides all these things by synthesizing Filiba's idea and Tauber's style, mixing in the magic of metaclasses.

Keeping it Simple with Python Templets

The template language understands the following forms:

$myvar	inserts the value of the variable 'myvar'
${...}	evaluates the expression and inserts the result
${{...}}	executes enclosed code; you can use 'self.write(text)' to insert text
$<sub_template>	shorthand for '${{self.sub_template(vars())}}'
$$	an escape for a single $
$ (at the end of a line)	a line continuation

Code in a template has access to 'self' and can take advantage of methods, members, base classes, etc., like normal code in a method. The method 'self.write(text)' inserts text into the template. And any class attribute ending with '_template' will be compiled into a subtemplate method that can be called from code or by using $<...>. Subtemplates are helpful for decomposing a template and when subclassing.

A longer example:

import templet, cgi

class RecipeTemplate(templet.Template):
  template = r'''
    <html><head><title>$dish</title></head>
    <body>
    $<header_template>
    $<body_template>
    </body></html>
    '''
  header_template = r'''
    <h1>${cgi.escape(dish)}</h1>
    '''
  body_template = r'''
    <ol>
    ${{
      for item in ingredients:
        self.write('<li>', item, '\n')
    }}
    </ol>
    '''

Notice that all the power of templets comes from python. To get HTML escaping, we just "import cgi" and call "cgi.escape()". To repeat things, we just use python's "for" loop. To make templates with holes that can be overridden, we just break up the template into multiple methods and use python inheritance. We could have added ordinary methods to the class and called them from the template as well. So writing a templet template is nothing special: we are just writing a python class.

RecipeTemplate can be expanded as follows:

print RecipeTemplate(dish='burger', ingredients=['bun', 'beef', 'lettuce'])

And it can be subclassed like this:

class RecipeWithPriceTemplate(RecipeTemplate):
  header_template = "<h1>${cgi.escape(dish)} - $$$price</h1>\n"

We get all the power and simplicity of python in our templates.

The templet module

How does it work? The idea is simple. When a templet.Template subclass is defined, a metaclass processes the class definition, and all the "template" class member strings are chopped up and expanded into pieces of straight-line code.

When a template is instantiated, the main template() method is run and the written results are accumulated as a list of strings in "self.output". This list is concatenated when the instance is converted to a string.

Here is the code for the templet module. The full version here includes documentation and test code.

import sys, re

class _TemplateMetaClass(type):
  __pattern = re.compile(r"""\$(        # Directives begin with a $
        \$                            | # $$ is an escape for $
        [^\S\n]*\n                    | # $\n is a line continuation
        [_a-z][_a-z0-9]*              | # $simple Python identifier
        \{(?!\{)[^\}]*\}              | # ${...} expression to eval
        \{\{.*?\}\}                   | # ${{...}} multiline code to exec
        <[_a-z][_a-z0-9]*>            | # $<sub_template> method call
      )(?:(?:(?<=\}\})|(?<=>))[^\S\n]*\n)? # eat some trailing newlines
    """, re.IGNORECASE | re.VERBOSE | re.DOTALL)

  def __realign(cls, str):
    """Removes any leading empty columns of spaces and an initial empty line"""
    lines = str.splitlines();
    if lines and not lines[0].strip(): del lines[0]
    lspace = [len(l) - len(l.lstrip()) for l in lines if l.lstrip()]
    margin = len(lspace) and min(lspace)
    return '\n'.join(l[margin:] for l in lines)

  def __filename(cls, tname, globals):
    """Returns 'filename.py <MyClass template>' for labeling python errors"""
    if '__file__' not in globals: return '<%s %s>' % (cls.__name__, tname)
    return '%s: <%s %s>' % (globals['__file__'], cls.__name__, tname)

  def __compile(cls, template, name):
    globals = sys.modules[cls.__module__].__dict__
    code = []
    for i, part in enumerate(cls.__pattern.split(cls.__realign(template))):
      if i % 2 == 0:
        if part: code.append('self.write(%s)' % repr(part))
      else:
        if part == '$': code.append('self.write("$")')
        elif part.endswith('\n'): continue
        elif part.startswith('{{'): code.append(cls.__realign(part[2:-2]))
        elif part.startswith('{'): code.append('self.write(%s)' % part[1:-1])
        elif part.startswith('<'): code.append('self.%s(vars())' % part[1:-1])
        elif part == '':
          raise SyntaxError('Unescaped $ in ' + cls.__filename(name, globals))
        else: code.append('self.write(%s)' % part)
    code = compile('\n'.join(code), cls.__filename(name, globals), 'exec')
    del cls, template, name, i, part
    def expand(self, _dict = None, **kw):
      if _dict: kw.update([i for i in _dict.iteritems() if i[0] not in kw])
      kw['self'] = self
      exec code in globals, kw
    return expand

  def __init__(cls, *args):
    for attr, val in cls.__dict__.items():
      if attr == 'template' or attr.endswith('_template'):
        if isinstance(val, basestring):
          setattr(cls, attr, cls.__compile(val, attr))
    type.__init__(cls, *args)

class Template(object):
  """A base class for string template classes."""
  __metaclass__ = _TemplateMetaClass

  def __init__(self, *args, **kw):
    self.output = []
    self.template(*args, **kw)

  def write(self, *args):
    self.output.extend([str(a) for a in args])

  def __str__(self):
    return ''.join(self.output)

Let me know if you find it helpful, or if you have any ideas for improvements.

Update: I have added support for much faster python string template functions to the templet module. Read about the idea here.

Posted by David at February 18, 2007 05:49 PM