davidbau.com Python Templating with @stringfunction

September 09, 2011

Python Templating with @stringfunction

I've posted a new version of the templet.stringfunction utility I have discussed before. I have been using it to make a bunch of web UI in programming projects with my son (actually pretty sophisticated little projects - http://js.dabbler.org). This version reflects what's needed for a 2011-era web app.

What's New

This version is still svelte, clocking in at 91 lines of python code (plus comments and tests). Version 3.3 changes the following from v2:

The old class-template idioms in the original version have been removed; they were slow and unwieldy. Now it's just @stringfunction and @unicodefunction.
Common uses of $ in javascript no longer need to be escaped in templates. Jquery $. and $( are very common, and regular expressions often have $/ and $' and $", so all these sequences now pass through the template without escape.
Line numbers are aligned exactly so that both syntax errors and runtime errors in exception traces are reported on the correct line of the template file in which the code appears.

Together these small changes - particularly accurate error line numbers - make @stringfunction much more usable for composing large templates for website development.

Usage

Usage is unchanged: just annotate a python function with @stringfunction or @unicodefunction, and then put the template text where the docstring would normally be. Leave the function body empty, and efficient code to concatenate the contents will be created.

  from templet import stringfunction
  
  @stringfunction
  def myTemplate(animal, body):
    "the $animal jumped over the $body."
  
  print myTemplate('cow', 'moon')

This is turned into something like this:

  def myTemplate(animal, body):
    out = []
    out.append("the ")
    out.append(str(animal))
    out.append(" jumped over the ")
    out.append(str(body))
    out.append(".")
    return ''.join(out)

There are just six constructs that are supported, all starting with $:

$myvar inserts the value of the variable 'myvar'
${...} evaluates the expression and inserts the result
${[...]} runs a list comprehension and concatenates the results
${{...}} executes enclosed code; use 'out.append(text)' to insert text
$$ an escape for a single $
$ (at the end of the line) is a line continuation

All ordinary uses of $ in the template need to be escaped by doubling the $$ - with the exception of (as mentioned above) $., $(, $/, $', and $".

Philosophy

The philosophy behind templet is to introduce only the concepts necessary to simplify the construction of long strings in python; and then to encourage all other logic to be expressed using ordinary python.

A @stringfunction function can do everything that you can do with any function that returns a string: it can be called recursively; it can have variable or keyword arguments; it can be a member of a package or a method of a class; and it can access global imports or invoke other packages. As a result, although the construct is extremely simple, it brings all the power of python to templates, and the @stringfunction idea scales very well.

Beyond simple interpolation, templet does not invent any new syntax for data formatting. If you want to format a floating-point number, you can write ${"%2.3f" % num}; if you want to escape HTML sequences, just write ${cgi.escape(message)}. Not as brief as a specialized syntax, but easy to remember, brief enough, and readable to any python programmer.

Similarly, templet does not invent any new control flow or looping structures. To loop a template, you need to use a python loop or list comprension and call the subtemplate as a function:

 @stringfunction
 def doc_template(table):
   """
   <body>
   <h1>${ table.name }</h1>
   <table>
   ${{
     for item in table:
       out.append(self.row_template(item))
   }}
   </table>
   </body>
   """

If you prefer list comprehensions, it is slightly more brief:

 @stringfunction
 def doc_template(table):
   """
   <body>
   <h1>${ table.name }</h1>
   <table>
   ${[self.row_template(item) for item in table]}
   </table>
   </body>
   """

The design encourages simple templates that read in straight-line fashion, an excellent practice in the long run. Although when invoking subtemplates you need to pass state, of course you can use @stringfunction to make methods and pass state on "self", or use object parameters.

Details and Style

Some tips/guidelines for using these annotations.

Whitespace can be important inside HTML, but for python readability you often want to indent things, so @unicodefunction / @stringfunction gives you a few tools:

It identifies the number of leading spaces that are uniformly used to the left of the template and strips them.
It strips the first line of the template, if empty.
It allows you to use a $ at the end of a line for a line continuation.

So my recommended style for multiline templates is:

indent template text in the function as if it were python code.
use a python triple-quote and put the opening quote on its own line.
never indent HTML tags - they just get too deep, so put them all at column 0.
when nesting gets confusing, for readability, just put one tag on each line.
liberally use $ continuations if layout demands no-whitespace.
indent code inside ${{ and then put }} on its own line (a newline right after a closing }} is eaten).

Relative indenting for python code inside ${{...}} is preserved using the same leading-space-stripping trick as is used for the templates themselves, so you can indent embedded python as normal, and you can start the indenting at whichever column feels natural. I usually indent embedded python by one more level.

In the unusual case where it is necessary to emit text that has leading spaces on every line, you can begin the template with a continuation line with the $ in the column that you want to treat as column zero.

One question is whether the opening """ should be on the same line as the def or its own line. Either style is supported - for line number purposes, the program source is just scanned to discover the position of the opening quote - but for clarity I usually put the opening quote on its own line.

For example, if you want to achieve all on one line the following:


<tr><td class="..."><a class="..." href="/foo/bar/...">....</a></td><td class="...">...</td></tr>

Then you could use:

@unicodefunction
def table_row(row_data):
  """
  <tr>$
  <td class="${col1_class} def">$
  <a class="${link_class}"$
   href="/foo/bar/${cgi.escape(filename, True)}">$
  ${cgi.escape(link_text})}$
  </a>$
  </td>$
  <td class="${col2_class}">$
  ${{
    if (enrolled): out.append('enrolled')
  }}
  ${cgi.escape(label_text)}$
  </td>$
  </tr>
  """

Posted by David at September 9, 2011 01:08 PM

Comments

David,

Excellent, elegant work! I love finding gems like this.

One thing though. First time users should beware testing templet in interactive mode. The following line (148) fails silently:

docline, (source, _) = 2, inspect.getsourcelines(func)

The reason is that a func, if defined in interactive mode, does not have source code to scan.

I added two lines to re-raise the IOError with a message warning against the use of functions defined interactively. I doubt it's worth the extra 2 lines when a warning in the documentation should suffice.

FWIW

Posted by: Rob at February 20, 2012 11:12 AM

Hi Rob, thanks for the bug report.

I have posted a version 3.3 which fixes this bug so that templet can be used in interactive mode (it only needs the sourcelines to align line numbers for error messages; if it can't find them, then it can just guess when there is an error).

Posted by: David at February 21, 2012 08:02 AM

Would be perfect if indentation was kept, e.g. in the following example:

@stringfunction
def templettemplate(lines):
____'''\
____def foo():
________$lines
____'''
print templettemplate('a = 5\nb = 7')

it prints:
def foo():
____a = 5
b = 7

instead of:
def toto():
____a = 5
____b = 7

PS: sorry for the underscores - indents are also eaten by the comment :-)

Posted by: maxime-esa at April 19, 2013 11:33 AM

Beautiful. Prototyping my concept is a real joy using this template tool!

Posted by: thouters at October 29, 2013 04:35 PM

thanks for sharing your nice ideas.

this template script unfortunately seems to not work with encoding/decoding utf-8, could you please have a look to correct it?

@stringfunction
def unicode_test(text):
"$text"

#run
print(u'voil\u00e0:')
print(unicode_test(u'voil\u00e0'))

outputs the following:

voilà:
Traceback (most recent call last):
File "./form2tex.py", line 76, in
print(unicode_test(u'voil\u00e0'))
File "./form2tex.py", line 72, in unicode_test
"$text"
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe0' in position 4: ordinal not in range(128)

Posted by: Berteh at March 23, 2017 07:14 PM