Markup Progress and Notes

23 August 2006

Yesterday the second release of Markup (version 0.2) saw the light of day, about three weeks after the first public release.

Progress has been solid, with quite a few bug fixes and improvements, primarily in the areas of XPath support, error reporting, and serialization to HTML/XHTML.

I'll take the opportunity to talk about some of the highlights of Markup from my point of view.

Update: Markup has been renamed to Genshi. The new web site is

Error Reporting

I think one of the major advantages of Markup—compared to Kid but also many other template engines—is the error reporting. All of the various types of errors are handled pretty well, meaning you get an error message that actually tells you what the problem is:

  • Well-formedness errors in the XML templates
  • Syntax errors in embedded Python expressions
  • Runtime errors when evaluating embedded Python expressions
  • Invalid syntax or unsupported constructs in XPath expressions
  • Invalid usage of template directives

For example, let's examine the third item in that list: an exception raised when evaluating a Python expression. Most template languages would probably throw an exception that points somewhere deep into the generated code or the evaluation engine. In Markup, the exception points to the line of your template that contains the error:

  File ".../markuptest/templates/master.html", line 9, in <Expression "select('@*', False)">
    <body py:match="body" py:attrs="select('@*', False)">
TypeError: select() takes exactly 1 argument (2 given)

Yeah, that's the end of an actual Python stack trace. It tells you what the problem is, the file name of the template, the line number where the expression was found, and the source of the expression itself, just in case there are multiple expressions on that line. If the exception was raised somewhere inside the called function, you'd still get all that information, only further up the stack.

Includes vs. Inheritance

One of the design decisions in Markup that people were initially skeptical about was the reliance on XInclude instead of using an inheritance mechanism. From the feedback we've gotten so far, though, using includes is the way to go.

For template design, includes provide a lot more flexibility than inheritance. Templates can be included conditionally… for example, if you're generating a partial for AJAH

or just putting content in a popup window, you can tell a Markup template to only include the full interface chrome if a complete page is being generated. And of course you can make the choice of which template to include dependent on the template context data…

And even though Markup uses includes, that doesn't mean you need have things like the page header and the page footer in separate includes files. Includes can be combined with match templates and macros so that you only need to include a single layout template which adds all the site chrome you need wherever you want it.

The best about this approach is that it enables a very clean design, but still allows all of the common (or not-so-common) use cases to be satisfied. One language construct, lot's of possibilities.

The “No XML” Argument

In the Python community, a healthy dose of skepticism towards anything using XML it quite widespread. And they're often right: XML is not the answer to every problem. Some think that using XML isn't even appropriate for web templating, even though what you're generating about 90 percent of the time is markup. Obviously, I disagree about that sentiment.


One often-heard argument is that that XML-based template languages are more verbose. Frankly, that is far from true. Compare the following two snippets, the first using the Django template syntax:

<select name="foo">
  {% for option in %}
     <option value="{{ option.value }}"{% ifequal option.value selection %} selected="selected"{% endifequal %}>{{|escape }}</option>
  {% endfor %}

And now the same snippet in Markup:

<select name="foo">
  <option py:for="name, value in" value="$value"
          selected="${value == selection or None}">$name</option>

How exactly is the latter more verbose? Sure, you'll have to declare the namespace for template directives at the top of every template, but other than that… it's even quite a bit more compact, because you can take advantage of the fact that the template engine sees the logical structure of the template, and not just a stream of characters.

Generating other kinds of output

Another supposed disadvantage of XML-based template languages is that they're not good at producing output that isn't markup, such as plain text, LaTeX, iCalendar, or other text-based formats. That is certainly true, and if you want templates for doing things like code generation, you should be looking at other options (such as Cheetah.)

However, in the context of web applications, the only case I've personally used templates outside of markup generation was for creating plain text mail messages. Even that, though, doesn't work that great: if you're generating plain text, chances are you care about things like wrapping lines at a certain column, keeping text in multiple lines aligned at some column, and avoiding extra blank lines.

Try using something like Django templates to generate a plain text message displaying an order confirmation where the ordered goods, their quantities and their prices are displayed in a tabular manner.

Another example would be the ticket notification emails that are sent out by Trac. At the top of every such mail is a table with two columns, showing the values of the current fields of the ticket. Sure enough, Trac uses a template for the mail body. But where is that table layed out? In Python code. It would be pretty much impossible to do that kind of text formatting in ClearSilver or most other text-based template languages.

My point is that if you're generating things like plain text mails, or iCalendar files, or vCards, chances are you'll learn it's not that easy using general-purpose template language.

Other advantages

The other benefits you get from an XML-based template engine, and Markup in particular, are:

  • Your output is guaranteed to be well-formed (although Markup also provides ways around that if you really need to include bad markup). Having well-formed output, even when using “old-school” HTML, is important to make sure that styles and scripts are applied correctly. If you try to make Markup process a template that isn't well-formed, it'll inform you instantly.

  • You can serialize your output to different output formats. Using the same basic set of templates, you can produces either HTML 4, XHTML, or true XML output. Markup ensures that your output adjusts to the subtle but important differences between the different output formats.

    Did you start out generating HTML, but later realized you needed to be using XHTML to be able to embed snippets of MathML? With Markup, you just flip a switch.


Feedback has been very positive so far. Take for example the following mailing list post by Ethan Fremen, who's been working on implementing a proper web interface for Mailman as his Summer of Code project this year:

Aside from the name that makes for bloody impossible searches, I think you have done an extravagantly good job with Markup. XPath for finding nodes is The Way Things Should be, and using xinclude is also great.

I've used DTML, Page Templates, PHP, server-side includes, XSLT, and various other templating approaches throughout the years, and this is finally a templating approach that does things right.

And Kevin Dangoor is considering making Markup a first-class citizen in TurboGears, maybe even replacing Kid as the default template engine:

SQLAlchemy is definitely in the future of the project, and Markup is looking quite positive. In many ways, though, those projects will be brought in as “2.0” sorts of substitutes for existing pieces.

(Of course, you can already use Markup in TurboGears, or any other framework using the template engine plugin API, today.)

There's both a mailing list and an IRC channel for discussing Markup. So if you're interested, please don't hesitate to come by and share your thoughts!