I gotta have my orange juice.

Jesu, Juva

Posts Tagged ‘Performance

My experience with Django and Rails

with 9 comments

I’ve had the opportunity to work on both Django and Rails frameworks recently as part of one project.  The core application for the church administrative tools that I am working on is written in Rails, while the church guest follow-up application that I am responsible for is written in Django.  Why two separate stacks?  GuestView began its life independently from Gospel Software, and I chose Django there because of my familiarity with Python.  Three developers are sharing responsibility for the core of Gospel Software, however, and we chose Rails as the most reasonable lingua franca.

What follows are my personal opinions and observations.  These are mostly aesthetic or other value judgments, and I offer them simply for your consideration.

Language

I’ve used the Python programming language for a number of years and like it a lot.  I particularly enjoy its functional aspects, although lately I’ve become more of a fan of using list comprehensions and generator expressions wherever possible compared to map() and filter() with lambdas.  Compared to Python, Ruby has much more powerful functional capabilities, although some things don’t feel natural to me (Ruby’s design choice to not require parentheses to denote function invocation means that you must use .call to call a lambda, which feels clunky).  There are also some cases in Ruby where choosing one of several alternative forms of an expression can have a significant impact on your performance.  Lambdas seem particularly costly in Ruby as of version 1.8.

Overall I think the languages are fairly on par.  Right now I prefer Python for aesthetic rather than technical reasons.  As I grow in familiarity with Ruby, and as it matures and its performance improves I think I may eventually grow to prefer it.

Object-Relational Mapping (ORM)

The Django ORM is very powerful and you can express complicated queries very efficiently using it.  Django queries are not executed until they are actually used, so you can construct your queries piecemeal, which helps in writing readable code.  Django also allows you some flexibility with adding custom SQL to your queries, but for anything complicated I’ve found that I have to break down and write my own SQL.

Rails 2.1 introduced the ActiveRecord named_scope functionality.  Prior to this Rails was significantly lacking compared to Django’s expressive power for query construction, but named_scope pretty much evens the playing field.  And for complicated queries, which you will surely face in any real-world project as you seek to tweak performance, ActiveRecord gives you a degree of control over your SQL that really puts Django to shame.

Both Django and Rails seem to have adequate support for PostgreSQL, my database of choice.

URLs

Django lets you express your URLs using regular expressions; Rails accomplishes this using routes.   I personally prefer Django’s method, but both work well enough.

Templates

While Rails’ Embedded Ruby allows you to include arbitrary code in your templates, Django’s template engine is much more spartan.  It provides ways of getting at variables passed to the template, including objects, dictionaries, lists, and even methods.  And it has some simle control structures, but not covering the full expressive power of Python.  I yearned for a more powerful template language in Django at first.  But I found over time that the discipline of  a simple template language was helpful to me, forcing me to move any complicated behaviors to the controller (or “view” as Django calls it) which was in most cases the right thing to do anyway.

There are still some areas where I think the Django template language is lacking.  However, there is an open-source alternative to the Django template engine that is similar but sufficiently more powerful to meet my needs: Jinja2.

For me it is a toss-up between Embedded Ruby and Jinja2.

Performance

I suspect it’s common knowledge that Rails has a little ways to go in performance.  For our own purposes, I didn’t find too much difference in time measurements between Django and Rails.  However, Rails clearly has a much larger memory footprint than Django.

I was surprised to learn that even with a FastCGI or WSGI model, Django still opens and closes a database connection for each request.  While there may be technical reasons that the Django architecture requires this, it was still a surprise to me.  Django performance still seems on par with Rails in spite of this.  Interestingly, having Django use pgpool to connect to PostgreSQL didn’t improve my performance at all, perhaps because my application and database are currently located on the same host.

Console

Both Django and Rails allow you to run a REPL session for your application.  The Rails script/console command beats out Django hands-down, because Rails’ internal magic automatically imports pretty much everything you need.  In Django you still need to import any models or framework modules before you can use them.

Debugging

The Rails built-in log is enabled out of the box and is very handy.  Django provides logging functionality but you have to do a little extra work to enable it.  Rails wins out on logging.  Django is better at in-browser rendering of exception tracebacks.  Overall the handiness its logging means a slight win for Rails here for me.

Admin Application

Django’s admin application is truly its crown jewel.  If you need a private admin interface to your web application, Django will give you a very attractive and powerful interface almost entirely for free.  I’m not aware of any equivalent for Rails that even comes close to this.

Deployment

I’ve deployed Django using FastCGI and Rails using Mongrel.  Right now I am using Nginx to proxy to Mongrel, and to connect directly to the Django FastCGI instance.  Neither Django nor Rails seems to have a unique advantage or disadvantage in deployment.

Summary

I’ve spent more cumulative time with Django than with Rails, so I feel subjectively more at home with Django.  If I were going to write a small toy project, I’d choose Django mainly for ease and efficiency.  In fact, I took this route for the meal and potluck scheduler application that I recently wrote for Google App Engine.  GAE has many similarities with Django, and even allows you to run much of the Django stack on it.

However, for larger projects my current framework of choice is Rails.  With the named_scope functionality in Rails 2.1, ActiveRecord is finally on par with Django’s ORM.   And for any complicated queries ActiveRecord is superior to Django’s ORM.  While Django’s admin application is handy, I don’t make much use of it.  And while Rails falls slightly behind in performance and storage characteristics, I believe that Ruby and Rails will both continue to improve in this regard.

Written by Scott Moonen

January 9, 2009 at 12:59 am

Automatically minify your Javascript and CSS

with 2 comments

For best performance, it is recommended that you minify the Javascript and CSS that your web application uses.  What this involves is removing all unnecessary whitespace and comments.  So, for example, the following CSS:

body
{
  margin: 5px 10px 10px 10px;
  font-family: arial;
}

would look like this after being minified:

body{margin:5px 10px 10px 10px;font-family:arial;}

And similarly for Javascript. It is common to configure your server to perform GZip compression on files that it serves, including Javascript and CSS, and this can significantly reduce the time that it takes for browsers to load your pages. But minification when used with GZip usually helps to compress the files just a little bit further. And unlike GZip, which only compresses the file only as it is sent over the internet, minification compresses the file as it is seen by a browser. This allows the browser to parse it faster; additionally, smaller files are more likely to be cached by the browser.

It is common to manually minify your Javascript and CSS as part of deploying your application, saving a minified copy on your server either manually or as part of an automatic deployment script. But it is also possible to create custom Apache output filters to perform the minification for you. This gives you the best of both worlds — you can edit your files directly without their being minified, but you don’t have to engineer a minification process for when you deploy your application. Here’s how to do it, first for Javascript and then for CSS.

Javascript

  1. Ensure you have the Apache mod_ext_filter extension installed.

  2. Download the jsmin.py Python script from Douglas Crockford’s website. (There are also other languages available.)  Save it in your Python installation’s site-packages folder (possibly /usr/lib/python2.x/site-packages/).

  3. Add the following lines to your main Apache config file (httpd.conf, apache2.conf, etc.):

    <IfModule mod_ext_filter.c>
      ExtFilterDefine jsmin \
                      mode=output \
                      intype=application/x-javascript \
                      outtype=application/x-javascript \
                      cmd="/usr/bin/python /usr/lib/python2.4/site-packages/jsmin.py"
    </IfModule>
    
  4. Add the following statement to the context where you would like to minify your Javascript files (you can place this in your server config, but also within a virtual host configuration, a directory directive, or even a .htaccess file if FileInfo overrides are allowed):

    AddOutputFilter jsmin js
    

    This will cause all files with extensions ending in .js to be run through the Javascript minify filter before being sent to a browser. If you have some Javascript without the .js extension, you can add additional extensions, or you can use the AddOutputFilterByType directive instead to apply the filter to any content with the application/javascript MIME type. With appropriate mod_expires directives you can cause these files to be cached for a long time by browsers, thereby ensuring that the minify filter is not run more than necessary.

For debugging purposes you should ensure that the minify filter is applied only to your production server and not to your development server. Until you have verified the correctness of your Javascript it will be harder to locate Javascript errors within minified code!

CSS

  1. Ensure you have the Apache mod_ext_filter extension installed, as above.

  2. Install the cssmin Ruby gem:

    gem install cssmin
  3. Add the following lines to your main Apache config file (httpd.conf, apache2.conf, etc.):

    <IfModule mod_ext_filter.c>
      ExtFilterDefine cssmin \
                      mode=output \
                      intype=text/css \
                      outtype=text/css \
                      cmd="/usr/bin/ruby -e 'require \"rubygems\"; require \"cssmin\"; puts CSSMin.minify(STDIN)'"
    </IfModule>
    
  4. Add the following statement to the context where you would like to minify your CSS files (you can place this in your server config, but also within a virtual host configuration, a directory directive, or even a .htaccess file if FileInfo overrides are allowed):

    AddOutputFilter cssmin css
    

    This will cause all files with extensions ending in .css to be run through the CSS minify filter before being sent to a browser. If you have some CSS without the .css extension, you can add additional extensions, or you can use the AddOutputFilterByType directive instead to apply the filter to any content with the text/css MIME type. With appropriate mod_expires directives you can cause these files to be cached for a long time by browsers, thereby ensuring that the minify filter is not run more than necessary.

Written by Scott Moonen

August 17, 2008 at 9:36 pm

A performance comparison of AF_UNIX with loopback on Linux

with 2 comments

[PostgreSQL]On various Linux hosts I use either MySQL or PostgreSQL for my back-end database.  Typically the database is running on the local host, and this raises the question of how to connect to the database.  For both MySQL and PostgreSQL it is possible to connect using a local AF_UNIX connection, or using an AF_INET connection over the loopback socket (address 127.0.0.1).

I had thought that it might actually be more efficient to connect over loopback; for my day job I work on the TCP/IP stack for the z/OS operating system, and I know firsthand that our TCP/IP implementation is heavily optimized, including specific optimizations for local traffic.

I decided to test this hypothesis by comparing both the throughput and latency of AF_UNIX and AF_INET connections.  I was running on a Celeron Dual-core system running Debian 4.0 with kernel version 2.6.18-4-686.  The results disproved my hypothesis, at least for Linux:

  • AF_UNIX:
    • 100GB transferred in 80.575200 s
    • 100 million 8-byte messages:
      • Average latency 14.463910 μs
      • Standard deviation 0.003376 μs
  • AF_INET, loopback address
    • 100GB transferred in 226.717520 s
    • 100 million 8-byte messages:
      • Average latency 1133.444424 μs
      • Standard deviation 0.067419 μs

So, for both throughput and latency, on the Linux platform AF_UNIX is a superior choice to AF_INET.  For local MySQL and PostgreSQL connections you should use a local socket rather than the loopback socket.

Written by Scott Moonen

April 5, 2008 at 6:03 pm