Gizmo Web

Evoque Templating v0.4 for Python 3.0

noreply@blogger.com (mario ruggier) — Tue, 20 Jan 2009 16:24:00 +0000

Version 0.4 of Evoque Templating is out. Highlights include:

Support for Python 3.0
The same Evoque code base now runs on Python 2.4, 2.5, 2.6 and 3.0 -- possibly making Evoque the first full-featured template engine available for Python 3.0. Behaviour is identical on all four Python versions.
Tighter lockdown of the in-process sandbox (restricted mode)
Primarily by an elaborated runtime scan of all expressions that are passed for evaluation. See the overview of how Evoque implements the in-process sandbox. And yes, do interpret this as an open challenge and invitation to try and break it , and, in the unlikely chance that you succeed, you may request anything you like as compensation, as long as it is beer ;-) Want a quick and dirty script to run devious templates?
Revision of whitespace handling
Notably, reworked the meaning of the slurpy_directives=True mode to be more what most people seem to expect.
Continued speed improvements
More minor performance optimizations. Evoque is fast, and is a real contender for the dubious but gratifying honour of overall fastest pure-python template engine. Included in the distribution are 3 benchmarks -- you can see the measured results on OS X 10.5 for Python 2.6.1 (as there is not much to benchmark against on Python 3.0) -- those numbers are a real tribute to python's conceptual integrity -- that rewards a straightforward implementation of a simple design with... amazing performance.

See full changelog. It is curious to note that all those changes added a whopping 20 SLOC to the 972 count of the preceeding release.

Fastest pure python text-based templating?

noreply@blogger.com (mario ruggier) — Wed, 12 Mar 2008 11:40:00 +0000

Mako is a mature full-featured templating engine for python. It is also pure-python i.e. it has no C-compiled extensions, and amongst pure-python systems it is currently considered to be the fastest.

Evoque is a new lightweight templating engine for python, also pure-python and full-featured. Its simple implementation is based on the eval python builtin, and weighs in at less than 1K lines of code, compared to about 2.5K for Mako, or about 4K for Genshi. One of Evoque's distinguishing features is automatic input quoting that, to my knowledge, Mako does not do, at least not by default.

It is difficult to make sensical performance comparisons between systems -- never do such comparisons give the entire picture. I will try to highlight the specific performance results for Evoque in manual quoting mode (called evoque_mq when used this way) and those for Mako -- two systems that come close to being equivalent. The Evoque distribution contains the following three benchmarks:

subs - does only string substitutions
basic - does a variety of standard templating tasks such as looping, calling sub-templates, escaping of data
bigtable - a simple brute force generation of a 10 columns x 1000 rows table

These three benchmarks are run on two different platforms, as indicated below, under python 2.5.1 in each case. All times are in milliseconds (less is better).

OS X 10.3 on a 667 MHz G4 PowerBook

             subs    basic    bigtable
evoque_mq    0.27    2.26     409.71
mako         0.72    2.73     445.47

Ubuntu 7.04 on a 2.0 GHz Intel Celeron

             subs    basic    bigtable
evoque_mq    0.13    1.25     152.12
mako         0.38    1.28     142.53

Evoque is not always faster. It seems to be consistently faster for simpler templates, while Mako seems to be quicker on loop-intensive templates, such as the 1000 row table benchmark. This is probably because of additional work that Evoque has to do for the runtime evaluation of loop variables.

Evoque - managed eval-based templating

noreply@blogger.com (mario ruggier) — Wed, 12 Mar 2008 10:25:00 +0000

Released the first version 0.1 of the lightweight, capable and fast Evoque templating engine on Saturday 7 March -- see comp.lang.python.announce announcement.

Evoque's distinguishing factors are:

automatic input quoting, i.e. XSS protection. To my knowledge no other text-based templating system has this feature. This is achieved thanks to Qpy's quoted-string h8 class, although any other quoted-string class may be used instead.
a restricted execution mode, to be able to safely expose templates for editing to untrusted clients. To facilitate this, only python expressions are allowed, i.e. nothing is ever exec'ed.
small footprint -- under 1K lines of code -- yet full-featured, e.g. elaborate cache management, template inheritance, versatile template addressing.
a contender for the fastest pure python templating engine currently out there. See some comparative performance benchmarks -- in particular compared with Mako that is currently considered to be the fastest text-based templating system.

Home page for Evoque is: http://evoque.gizmojo.org/

2 Million Page Views a Day with Quixote

noreply@blogger.com (mario ruggier) — Mon, 12 Feb 2007 07:44:00 +0000

Bo Yang send this small note to the Quixote mailing list this past Thursday, that I repoduce almost entirely below:

Just to report-in the progress we're making with a real-world Quixote installation: yesterday douban.com celebrated its first 2 million- pageview day. Quixote generated 2,058,207 page views. In addition, there're about 640,000 search-engine requests. These put the combined requests at around 2.7 millions. All of our content pages are dynamic, including the help and about-us pages.

We're still wondering if we're the busiest one of all the python/ruby supported websites in the world.

Quixote runs on one dual-core home-made server (costed us US$1500). We have three additional servers dedicated to lighttpd and mysql. We use memcached extensively as well.

Douban.com is the most visible python establishment on the Chinese web, so there's been quite a few django vs. quixote threads in the Chinese language python user mailing lists.

So, the question "Is busiest site powered by Quixote?" is still a very valid one...

Putting lipstick on a Pig

noreply@blogger.com (mario ruggier) — Sun, 22 Oct 2006 09:21:00 +0000

The question of what kind of database one should use has triggered many a techno-emotional conclusion-less discussion. The juiciest bone of contention is whether to use a Relational Database coupled with an Object Relational Mapper, or to just simply use an Object Database.

Well, here's a nice pre-emptive image summarizing the situation rather succintly... Mike Orr, replying to a specific question about comparisons between an ORM such as SQLObject coupled with MySQL for example, and the Durus Object Database, drops this:

Object-relational mappers are like putting lipstick on a pig.

Going on to add:

If you think foreign keys, flat tables, etc are programmer friendly, you haven't used an object database. Literally 90% of the work disappears. The SQLObject and SQLAlchemy manuals are a hundred pages because even trying to put an OO front end on a SQL database is a complex task.

Pretty much sums it up for me. Still, if you need more argument fodder, here's some nicely digested material about Object Databases, and how they compare with Relational Databases, from Oleg Broytmann.

While I'm at it, I should probably also mention that performance is one of the strongest points typically held in favour of Relational databases. Well, that is definitely not always the case. Mike Orr reports how, on his application, Durus proved to be more performant than both MySQL and Sqlite. This while not forgetting of course how much easier Durus is to work with.

New QP release is more open

noreply@blogger.com (mario ruggier) — Thu, 06 Apr 2006 17:59:00 +0000

The newly released QP version 1.5 is good news! It is now a lot easier than before to create QP sites that (a) do not use the default Directory-based URL traversal, (b) use a database other than the Durus object database, that is the persistence system used by default, and (c) do not use a database at all!

Some thread-awareness has also been added to the Publisher, to make it easier to run QP sites in multi-threaded contexts. QP sites are generally single-thread multi-process deployments. In this same vein, a __call__ method has been added to the Publisher, to make it more WSGI friendly.

Mucking with the included demo sites, it was also a pleasant surprise to find the "quickie wiki" simple wiki module, that is a really clear, small but functional example of how one would go about building web applications with QP and Durus.

New storage backends for Durus?

noreply@blogger.com (mario ruggier) — Wed, 15 Mar 2006 08:01:00 +0000

Durus, the little python object db that could, seems to suddenly be getting two new storage backends. Jesus Cea announces that he is working on a BerkeleyDB storage backend, and Peter Wilkinson announces another one for SQLite.

Both are motivated by the desire to Durus more usable for large data collections, such as start-up speed and impact on memory usage. And, both agree that working with the small and effective Durus code base is a joy...

Busiest site is powered by Quixote?

noreply@blogger.com (mario ruggier) — Sun, 12 Mar 2006 07:42:00 +0000

There was a flurry of activity last week on the quixote-users mailing list. Quixote is that other quiet Python Web Framework, that few people talk about but is well-thought out, simple, solid, and has been around for a long time. It's typically hype-shy users are extremely appreciative....

Well there has been a lot of talk lately about Python Web Frameworks, with the new kids on the block, i.e. Django and TurboGears, getting all the attention. Of the arguments in their favour, at least for Django, is that they have been proven on high-traffic sites.

Well, Bo Yang thinks that possibly his Quixote-powered site, douban.com, is busier than any Django, TurboGears or even RoR site in the world, getting about 450,000 page views a day, all dynamically generated by Quixote. Oh, and if your chinese is not so good, there is an english version at douban.net.

Scalability issues? It is all running on just one machine! Quoting Bo Yang from further down the email thread: "douban.com is currently running on a $1,500 home-made 1U server with an AMD athlon 64 dual core cpu, and SATA drives. It is hosted in Beijing. lighttpd, quixote, memcached, mysql are all running on one machine."

Compound specs, and error messages

noreply@blogger.com (mario ruggier) — Tue, 28 Feb 2006 12:23:00 +0000

The wonderfully simple and powerful spec module, that comes with the QP Web Application Framework, allows you to build arbitrarily compound specifications against which you can check attribute values, or whole objects even. Btw, I have just realised that David Binger has actually just given a lightning talk over the weekend at PyCon 2006 about it.

Spec offers a generous range of basic specs, such as string (str or unicode), pattern, instance, sequence, and so on. All very useful. It also offers what it calls ConnectiveSpecOperators, that are basically compound specs, examples being no (must not match any), both (must match all), and either (must match any). As a side note, the names both and either are used instead of all and any to avoid nameclashing with python 2.5 same-name builtins.

One little problem with these compound specs, though, is that you do not know which of the sub-specs caused them to fail, or to succeed for that matter. It would be nice if the first sub-spec instance that caused the match or the failure be somehow bubbled up to the root spec, that can then be queried for by client code if desired, and using that information for example to provide more appropriate error messages to the trying-to-not-get-annoyed-with-technology end user.