January 27, 2009

Christian Hammond
Improving browser performance in Review Board

This past Sunday, I landed a set of changes into Review Board that provide improved performance, such as aggressive browser-side caching of media and pages. It’s just a start, but has already significantly reduced page load times in all of my tests, in some case by several seconds. We implemented these methods for Review Board, but they’re methods that can be applied to any Django project out there.

There are several key things that Review Board now does to improve performance:

  • Tells browsers to cache all media for one year.
  • Only sends page data if new data is available.
  • Compresses all media files to reduce transfer time.
  • Parallelizes media downloads.
  • Loads CSS files as early as possible.
  • Loads JavaScript files as late as possible.
  • Progressively loads expensive data.

A lot of the performance improvements come straight from Yahoo!’s Best Practices for Speeding Up Your Site. We’re not doing everything there yet, but we’re working toward it. That’s a great resource, by the way, and I recommend that everyone who has even made a website before go and read it.

So what do the above techniques buy us, and how are we doing them? Let me go into more details…

Caching all media for a year

The average site has one or more CSS files, JavaScript files, and several images. This translates to a lot of requests to the server, which may leave the site feeling slow. On top of this, a browser only makes a few requests to a server at a time, in order to avoid swamping the server, which will further hinder load times. This happens every time a user visits a page on your site.

Aggressive caching makes a huge difference and can greatly reduce load times for users. Review Board now tells the browser to cache media files for a year. Once a user downloads a JavaScript or CSS file, they won’t have to download it again, meaning that in general the only requests the browser needs to make is for the page requests and AJAX requests.

The big pitfall with long-term caching is that the cached resources can go stale. For example, if a new version of an image was uploaded, the browser wouldn’t even know about it, since it was told it should keep its old version for a year before checking again.

We solve this by introducing “media serials,” timestamps that are appended to all media paths. Instead of caching /js/myscript.js, the browser would cache /js/myscript.js?1273618736.

These media serials are computed on the first page request by our djblets.util.context_processors.ajaxSerial context processor. This quickly scans all media files known to the program, finding out the latest modification timestamp. It then provides a {{MEDIA_SERIAL}} variable for templates to append to media URLs as part of the query string.

The benefit to this method is that we can cache media files for a year and not worry about users having stale cached resources the next time we upgrade a copy of Review Board. The filenames requested will be different, browsers will see that the new file is not in the cache, and make a request, caching the new file for a year.

Only send page data if new data is available

Aggressive caching of media files is great and saves a lot of time, but it doesn’t help for dynamically generated content. For this, we need a new strategy.

When a browser makes a request, it can send a If-Modified-Since header to the server containing the Last-Modified value it received the last time it downloaded that page. This is a very valuable header, and there’s some things we can do with it to save both the server and the browser a lot of trouble.

If the browser sends If-Modified-Since, and we know that no new data has been generated since the timestamp provided, we can send an HttpResponseNotModified (HTTP response code 304). This will tell the browser it already has the newest version of the page. The sooner we do this, the better, as it means we don’t have to waste time building templates or doing any expensive database queries.

Djblets, once again, provides some functions to help out here: djblets.util.http.set_last_modified and djblets.util.http.get_modified_since.

The general usage pattern is that we first build a timestamp representing the latest version of the page. This could be the timestamp for a particular object that the page represents. We then check if we can bail early by calling:

if get_modified_since(request, timestamp):
    return HttpResponseNotModified()

Further down, after building the page, we must set the Last-Modified timestamp, using the same timestamp as above, like so:

set_last_modified(response, timestamp)

We’re using this in only a few places right now, such as the review request details page, but it drastically improves load times. If the review request hasn’t changed and nobody’s commented on it since the browser last loaded the page, a reload of the page will be almost instant.

Compress all media files

Our Apache and lighttpd config files now enable compression by default. By compressing these files, we can turn a relatively large JavaScript file (such as the jquery and jquery-ui files) into a very small file before sending it over to the browser. This reduces transfer times at the expense of compression/decompression time (which is small enough to not worry for deployments of this size, and can be offset by caching of compressed files server-side).

Parallelize media downloads

It’s important to not mix loads of media files of different types. The browser parallelizes media downloads of the same type, in page load order, but if you load one CSS file, one JavaScript file, another CSS file, and then another JavaScript file, the browser will only attempt one load at a time. If you load all the CSS files before all JavaScript files, it will parallelize the CSS file download and then the JavaScript downloads. By enforcing the separation of loads, we can achieve faster page download/render times.

Load CSS files as soon as possible

Loading CSS files before the browser starts to display the page can make the page appear to load smoother. The browser will already know how things should look and will lay the page out accordingly, instead of laying the page out once and then updating that once the CSS files have loaded.

Load JavaScript files as late as possible

JavaScript loads block the browser, as the browser must parse and interpet the JavaScript before it can continue. Sometimes it’s necessary to load a JavaScript file early, but in many cases the files can be loaded late. When possible, we load JavaScript files at the very end of the document body so that they won’t even begin downloading until the page has rendered. This provides noticeable performance for script-heavy pages.

Progressively load expensive data

There are types of data that are just too expensive to load along with the rest of the page. For a long time, Review Board would parse and render fragments of a diff for display in the review request page, but that meant that before the page could load, Review Board would need to do the following:

  1. Query the list of all comments.
  2. Fetch every file commented on.
  3. Apply the stored patch to each file.
  4. Diff between the original and patched files.
  5. Render the portion of the diff commented on into the page.

This became very time-consuming, and if a server was down, the page wasn’t available until everything timed out. The solution to this was to lazily load each of these diff fragments in order.

We now display a placeholder table for each diff fragment in roughly the same size of the rendered fragment (to avoid excessive page scrolling on loads). The table contains a spinner showing that something is happening, and, one-by-one (to avoid dogpiling) we load each diff fragment.

The code to render the diff fragment, by the way, takes advantage of the If-Modified-Since header and is also cached for a year. We use an AJAX_SERIAL (same principal as the MEDIA_SERIAL above) to allow for changes in new deployments.

With these caching mechanisms in place, the review request page now loads in roughly a second in many cases (or less once cached), with diff fragments coming in lazily (and then almost immediately on future loads).

More to come…

This was a great first step, but there’s more we can do. Before we hit our 1.0 release, we’re going to batch together all our CSS files and JavaScript files into a couple of combined files and then “minify” them (basically compressing them in such a way to allow for smaller files and faster load times of the interpreted data).

Again, these are techniques we’re now making use of in Review Board, but they’re not in any way specific to Review Board. Anyone out there developing websites or web applications should seriously look into ways to improve performance. I hope this was a good starting point, but seriously, do read Yahoo!’s article as well.

January 20, 2009

Christian Hammond
Infect your application with Parasite!

Parasite

Ever find yourself stuck debugging an application because the UI is just doing something weird that you can’t track down? Maybe a widget isn’t appearing correctly, or you just need more information about the overall structure and logging statements aren’t doing you much good. Debugging complex UIs can be a pain.

We’ve had some real challenges at VMware, due to the complexity of our applications. It was enough to drive me mad one day, so rather than write more logging statements, I wrote Parasite.

Parasite is a debugging tool that David Trowbridge and I have been working on to give developers an interactive view of their entire application’s UI. It provides a number of really useful features, including:

  • See the entire widget hierarchy of your UI.
  • Watch properties update live.
  • Modify existing properties on a widget.
  • View all registered GtkActions.
  • Toggle GTK+’s debugging of graphic updates.
  • Inject custom code while the application is running.

Yes, you can inject new code into an application. With Python. Parasite runs in-process as a GTK+ module, so it has access to some internals of your application. We provide a Python shell equipped with PyGTK support for creating and modifying your UI on-the-fly, regardless of the language it was written in. Handy when you want to test out new concepts for a UI without writing new C code.

David has a nice screencast available showing some of what Parasite can do.

For more information on Parasite, including screenshots, a mailing list, and where to get the source code, see the Parasite homepage.

January 19, 2009

Christian Hammond
Review Board 1.0 alpha 1 released

Roughly two years ago, David Trowbridge and I began development of Review Board for use in our open source projects and our team at VMware. During that time, we’ve turned Review Board into a powerful code review tool that works with a variety of version control systems. Most of VMware has moved over to it, as have an estimated 50-100 companies world-wide. We’ve had over 100 contributors to the project, people providing volunteer support on the mailing list, and people have developed third party tools for integrating with Review Board.

After all this time in development, with this many people contributing, we decided it’s probably time to get a release out there. Sure, we could have done this a long time ago, but there’s a number of large things we were hoping to get in (a recently-committed UI rewrite, for instance). Now that we have most of the major features we want for our 1.0 release, we decided it was time for an alpha.

Over the coming months, we’ll be working on stabilizing the codebase, fixing a few large remaining usability quirks, enhancing performance, and writing some proper documentation (which is coming along nicely).

We’re eager to get a quality product out there and to begin development on the next release. There’s a lot of neat things planned:

  • Support for writing extensions to Review Board.
  • A fully-featured API covering every operation you’ll need to perform.
  • Some degree of policy support (specifying which users/groups can see which parts of a repository, for instance).
  • Reviews with statuses other than “Ship It”. This will probably be customizable to some degree.
  • Possibly some theme customization to allow Review Board to blend in better with corporate sites, Trac installs, etc.

Along with this, I plan to roll out a new website for the project that will have a browseable list of third party extensions, apps, Greasemonkey scripts, and more.

We have more information on our release on our release announcement.

December 26, 2008

Christian Hammond
Armed robbery in progress

Our Christmas this year was very nice, with lots of good presents, time spent with family, delicious food, and an opportunity to foil an armed robbery. Yes, an armed robbery. How do I continually get myself into these situations?

While heading back home from my stepdad’s parents’ house, I spotted a guy standing in a small field at a street corner near my parents’ house, holding what looked to be a gun. The guy was probably late teens/early adult, Caucasian, and dressed in all black. His actions looked instantly suspicious. He was pointing the gun to the ground and made some motion as if he was checking the ammo or something. He seemed pretty lost in his own world, apparently not even realizing he seemed very suspicious.

My Mom was driving, and decided to slowly drive away from home instead of toward it, and then turn around, giving us time to watch and see what was happening. We knew there was a gas station across the street and wanted to see if he was going to head in there. If so, there was a good chance we’d be witnessing a robbery.

Sure enough, he started walking across the street to the TowerMart, a gas station/convenience store. We parked the car and my Mom grabbed my phone and called 911 while my brothers and I carefully watched from behind a building, making sure we weren’t noticed. We saw the guy walking back and forth at the side of the building, looking very nervous. He was wearing long sleeves with his left hand exposed and his right hand (which was holding the gun) completely covered. As my Mom talked to the police, we continued to watch, and the guy eventually psyched himself into going into the store. He put the gun in his pocket and went in.

The store was pretty crowded, and we weren’t sure what to expect. Would people be running out screaming? Would people be locked in? Would we hear gun fire? Or would he just leave?

We never saw him actually leave, but we could only see the one side of the building. A couple minutes later, six police cars drove up, two right beside us. Two cop cars pulled up alongside us. One of them hopped out, grabbed a big ol’ semi-automatic, and approached the building, pointing the gun, ready to fire. Three other cops did the same, surrounding the building. A couple other cops went in and got people out of there, with another couple cops asking those people if they had seen anyone matching the description we gave, or saw other suspicious activity.

They spent some time going through the TowerMart and eventually came back out once they were sure he wasn’t hiding in there. It seems at some point, he had left the building. We weren’t able to see when. However, he was definitely in there. Practically everyone they talked to noticed him, as he was nervously walking in circles around the building, completely covered. He had either grown nervous with the number of people in there, or heard the sirens come. Either way, he got out of there before doing anything.

As most of the cops started to exit the building, the one who had pulled up closest to us walked up and asked what we saw. Then he said, “Oh, I talked to you before.” We all thought that was strange, since he hadn’t actually talked to us, but then he pointed out that he remembered us from the assault back in May. Good memory.

We walked with him to the field and pointed out roughly where we saw him standing. He found the footprints and was calling out the detectives to take pictures or whatever. We chit-chatted briefly before returning home. He told us we very well may have saved the store and a lot of people from experiencing a robbery on Christmas Day. I just hope the guy didn’t make another attempt elsewhere.

We found out that an hour or two later, the police were still all over that place. About an hour ago, they apparently had arrested someone not too far from here. Whether related or not, I don’t know. Given that they had video footage of the guy, testimonies and descriptions from a bunch of people in the store, and his footprints, it’s probably only a matter of time.

The rest of Christmas proceeded without police intervention. Merry Christmas!

December 25, 2008

Christian Hammond
Ubuntu and Desktop Notifications

This past Monday, Mark Shuttleworth wrote about their plans for an overhaul of desktop notifications (the little popup bubbles telling you someone IM’d you or there’s updates available). Many people have asked me about this, some concerned, and wanted to know what I thought. Being the maintainer of libnotify, notification-daemon and the Desktop Notifications specification, some people were concerned that this work would supersede my own.

The reality is, this isn’t a replacement of my project. This is a new joint effort between Ubuntu, KDE, and myself. What’s been written in Mark’s post may not end up being the end result. We’re still deciding how this will progress, and Ubuntu wants to experiment a bit. Aaron Seigo’s post on the subject sums up a lot of my thoughts on this, and I recommend reading it, though I’ll go further and discuss where this all started and how it’ll affect the project.

First of all, libnotify/notification-daemon isn’t going anywhere. It’s not being replaced, nor is an alternate spec being drafted. Ubuntu’s User Experience team has some new things they want to try, and that’s fine. The plan is to see how this stuff goes in their codebase, with the intention of migrating code back upstream.

A couple of weeks ago, during the Ubuntu Summit, I was invited to speak with some of the developers and User Experience guys from Ubuntu about their plans. They showed me the mockup that’s on Mark’s blog and told me about their plans. They have things they want to do that could definitely improve the experience. Some things are pretty controversial, such as the removal of actions on notifications. We sat down, discussed and debated various aspects of the proposals for a while, and I believe reached a general course of action for the project. The highlights include:

  • Actions will be removed for applications packaged in Ubuntu. The developers will try to replace them with something that they feel makes more sense, with the hope of pushing these changes upstream.
  • The released notification-daemon will, I believe, support actions, since many non-packaged applications do use them. They will, however, appear as old-style notifications and not appear in the new window stack demoed in Mark’s blog post.
  • We’ll be drafting new additions to the notification spec in order to address the needs of KDE and Mozilla.
  • The work will be done by their developers in either a fork or a whole new notification-daemon implementation, allowing them a greater ability to experiment. These changes (or parts of them) will make their way upstream. It will be spec-compatible so users won’t have to worry too much about losing features (aside from actions, perhaps).
  • In the end, it’ll likely be that the Ubuntu theme specifically works this new way, and that other themes will work differently (they may support actions more directly, for example).

Now I should point out that I don’t believe actions are a bad thing. There’s many use cases where actions are very much warranted, and it seems Aaron agrees. While the Ubuntu team has discussed the possibility of deprecating this in the spec, I believe the end result will be that actions will live on. I’m also pretty adamant that upstream notification-daemon will still support actions and some other features. Ubuntu can choose what experience to give their packaged applications, but that may differ a bit from what we decide upstream.

Time will tell how this all turns out. I personally think it’s great that there’s some momentum on this project. I know I haven’t had much time to work on it as of late, between VMware and Review Board, which is why getting some new people on board with fresh thoughts will probably be a good thing.

On a related note, I’d like to welcome Andrew Walton to the project. He’s going to be working as a co-maintainer and helping out when I’m busy (which will be a lot of the time for a while).

Over the next month or two, the project should start to pick up. We’re beginning to look at ways to improve the spec and at what work needs to be done in the near future for the project. These discussions will take place on xdg-list.

And with that, I wish everyone a Merry Christmas (or just a very good December 25th for those who don’t celebrate Christmas)!

1 2 3 4 5 6 7 8 pages