Evoque Templating v0.4 for Python 3.0

Version 0.4 of Evoque Templating is out. Highlights include:

• Support for Python 3.0
  The same Evoque code base now runs on Python 2.4, 2.5, 2.6 and 3.0 -- possibly making Evoque the first full-featured template engine available for Python 3.0. Behaviour is identical on all four Python versions.

• Tighter lockdown of the in-process sandbox (restricted mode)
  Primarily by an elaborated runtime scan of all expressions that are passed for evaluation. See the overview of how Evoque implements the in-process sandbox. And yes, do interpret this as an open challenge and invitation to try and break it , and, in the unlikely chance that you succeed, you may request anything you like as compensation, as long as it is beer ;-) Want a quick and dirty script to run devious templates?

• Revision of whitespace handling
  Notably, reworked the meaning of the slurpy_directives=True mode to be more what most people seem to expect.

• Continued speed improvements
  More minor performance optimizations. Evoque is fast, and is a real contender for the dubious but gratifying honour of overall fastest pure-python template engine. Included in the distribution are 3 benchmarks -- you can see the measured results on OS X 10.5 for Python 2.6.1 (as there is not much to benchmark against on Python 3.0) -- those numbers are a real tribute to python's conceptual integrity -- that rewards a straightforward implementation of a simple design with... amazing performance.
  

See full changelog. It is curious to note that all those changes added a whopping 20 SLOC to the 972 count of the preceeding release.

Fastest pure python text-based templating?

Mako is a mature full-featured templating engine for python. It is also pure-python i.e. it has no C-compiled extensions, and amongst pure-python systems it is currently considered to be the fastest.

Evoque is a new lightweight templating engine for python, also pure-python and full-featured. Its simple implementation is based on the eval python builtin, and weighs in at less than 1K lines of code, compared to about 2.5K for Mako, or about 4K for Genshi. One of Evoque's distinguishing features is automatic input quoting that, to my knowledge, Mako does not do, at least not by default.

It is difficult to make sensical performance comparisons between systems -- never do such comparisons give the entire picture. I will try to highlight the specific performance results for Evoque in manual quoting mode (called evoque_mq when used this way) and those for Mako -- two systems that come close to being equivalent. The Evoque distribution contains the following three benchmarks:

• subs - does only string substitutions
• basic - does a variety of standard templating tasks such as looping, calling sub-templates, escaping of data
• bigtable - a simple brute force generation of a 10 columns x 1000 rows table

These three benchmarks are run on two different platforms, as indicated below, under python 2.5.1 in each case. All times are in milliseconds (less is better).

OS X 10.3 on a 667 MHz G4 PowerBook

             subs    basic    bigtable
evoque_mq    0.27    2.26     409.71
mako         0.72    2.73     445.47

Ubuntu 7.04 on a 2.0 GHz Intel Celeron

             subs    basic    bigtable
evoque_mq    0.13    1.25     152.12
mako         0.38    1.28     142.53

Evoque is not always faster. It seems to be consistently faster for simpler templates, while Mako seems to be quicker on loop-intensive templates, such as the 1000 row table benchmark. This is probably because of additional work that Evoque has to do for the runtime evaluation of loop variables.

Evoque - managed eval-based templating

Released the first version 0.1 of the lightweight, capable and fast Evoque templating engine on Saturday 7 March -- see comp.lang.python.announce announcement.

Evoque's distinguishing factors are:

• automatic input quoting, i.e. XSS protection. To my knowledge no other text-based templating system has this feature. This is achieved thanks to Qpy's quoted-string h8 class, although any other quoted-string class may be used instead.
• a restricted execution mode, to be able to safely expose templates for editing to untrusted clients. To facilitate this, only python expressions are allowed, i.e. nothing is ever exec'ed.
• small footprint -- under 1K lines of code -- yet full-featured, e.g. elaborate cache management, template inheritance, versatile template addressing.
• a contender for the fastest pure python templating engine currently out there. See some comparative performance benchmarks -- in particular compared with Mako that is currently considered to be the fastest text-based templating system.

Home page for Evoque is: http://evoque.gizmojo.org/

2 Million Page Views a Day with Quixote

Bo Yang send this small note to the Quixote mailing list this past Thursday, that I repoduce almost entirely below:

Just to report-in the progress we're making with a real-world Quixote installation: yesterday douban.com celebrated its first 2 million- pageview day. Quixote generated 2,058,207 page views. In addition, there're about 640,000 search-engine requests. These put the combined requests at around 2.7 millions. All of our content pages are dynamic, including the help and about-us pages.

We're still wondering if we're the busiest one of all the python/ruby supported websites in the world.

Quixote runs on one dual-core home-made server (costed us US$1500). We have three additional servers dedicated to lighttpd and mysql. We use memcached extensively as well.

Douban.com is the most visible python establishment on the Chinese web, so there's been quite a few django vs. quixote threads in the Chinese language python user mailing lists.

So, the question "Is busiest site powered by Quixote?" is still a very valid one...

Putting lipstick on a Pig

The question of what kind of database one should use has triggered many a techno-emotional conclusion-less discussion. The juiciest bone of contention is whether to use a Relational Database coupled with an Object Relational Mapper, or to just simply use an Object Database.

Well, here's a nice pre-emptive image summarizing the situation rather succintly... Mike Orr, replying to a specific question about comparisons between an ORM such as SQLObject coupled with MySQL for example, and the Durus Object Database, drops this:

Object-relational mappers are like putting lipstick on a pig.

Going on to add:

If you think foreign keys, flat tables, etc are programmer friendly, you haven't used an object database. Literally 90% of the work disappears. The SQLObject and SQLAlchemy manuals are a hundred pages because even trying to put an OO front end on a SQL database is a complex task.

Pretty much sums it up for me. Still, if you need more argument fodder, here's some nicely digested material about Object Databases, and how they compare with Relational Databases, from Oleg Broytmann.

While I'm at it, I should probably also mention that performance is one of the strongest points typically held in favour of Relational databases. Well, that is definitely not always the case. Mike Orr reports how, on his application, Durus proved to be more performant than both MySQL and Sqlite. This while not forgetting of course how much easier Durus is to work with.

New QP release is more open

The newly released QP version 1.5 is good news! It is now a lot easier than before to create QP sites that (a) do not use the default Directory-based URL traversal, (b) use a database other than the Durus object database, that is the persistence system used by default, and (c) do not use a database at all!

Some thread-awareness has also been added to the Publisher, to make it easier to run QP sites in multi-threaded contexts. QP sites are generally single-thread multi-process deployments. In this same vein, a __call__ method has been added to the Publisher, to make it more WSGI friendly.

Mucking with the included demo sites, it was also a pleasant surprise to find the "quickie wiki" simple wiki module, that is a really clear, small but functional example of how one would go about building web applications with QP and Durus.

New storage backends for Durus?

Durus, the little python object db that could, seems to suddenly be getting two new storage backends. Jesus Cea announces that he is working on a BerkeleyDB storage backend, and Peter Wilkinson announces another one for SQLite.

Both are motivated by the desire to Durus more usable for large data collections, such as start-up speed and impact on memory usage. And, both agree that working with the small and effective Durus code base is a joy...