Working with webit! I recently built the dynamic parts of the new website of Fahrrad-XXL, a quite large group of bike dealers here in Germany. Besides the product catalog, which is maintained via a separate Rails app, there’s also lots of static content which is managed with the help of Bricolage, a Perl CMS generating static html pages.

I originally intended to name this post Find a new bike, powered by Ferret, because the full text search is one of the coolest features of the site. But maybe I’m a bit biased here ;-). In fact, I’ll delay the Ferret stuff to a later post and instead tell you about another interesting aspect of this project.

Integration of CMS driven static html with Rails

As long as the static pages are completely static that’s no big deal and we already did that in other projects: just have the CMS generate your Rails layouts to ensure visual consistency across the site, and let it publish it’s files into public/. But what do you do if your static pages aren’t really that static?

At www.fahrrad-xxl.de you have a watch list where you can remember bikes you want to revisit later on. The number of bikes currently on this list is shown on every page, regardless of whether it stems from the CMS or comes out of the Rails application.

So how can you do this? The first solution that comes to mind is of course to embed some ERb rendering the watch list status, and pipe each and every page through Rails. This might have worked but why take on the whole overhead of Rails compared to serving the file directly through the web server just for that tiny little number?

I wanted to do better, and remembered a blog post talking about nginx server side includes I stumbled across a while ago. The SSI feature of nginx is really cool because it allows you to include dynamically generated content retrieved via HTTP somewhere in your page.

So what we did was building an action that just rendered that tiny snippet showing the status of your watch list, and put an SSI include directive pointing to this action on every page:

Before serving such a page to the user, nginx will parse the page for any SSI directives, retrieve the content from the location specified there, and replace each directive with the content:

The good

So what have we gained? It’s still one request to the rails application per page, but it’s one that’s substantially faster because it only needs to output a few bytes of text instead of the whole page.

In fact we started using nginx’ SSI in several other places, too, which yielded another benefit of this approach: with pages containing multiple SSI directives you will experience multiple concurrent requests hitting the Rails app, which is a good thing because by splitting the task of rendering a page into several smaller tasks these may be distributed across multiple CPU cores and/or physical servers.

Having pages composed of multiple small snippets also eases caching: the decision whether a particular piece of content is eligible for caching and when it has to be expired is far easier to make for a small and focused snippet, than for a whole page potentially containing data with different life cycles. For example in our case we could easily page cache the watch list status snippet and expire it once the user modifies his watch list. Or, even better, use memcached to store the rendered snippet and have nginx retrieve it directly from there as shown in the blog post I mentioned above.

The ugly

There are some things you should be aware of when trying this approach out:

No setting of cookies

Your actions rendering stuff included via SSI cannot set cookies because the response you send is received by nginx, and not by the client. And since it wouldn’t make any sense for nginx to try and merge headers from multiple SSI responses into the single response that gets sent back to the client, it silently drops all those headers. So, no cookies, and especially no modification of session data if you use the cookie based session store. Which leads us to the next point:

Don’t use Rails’ stock session stores

At least if you intend to have multiple SSI-directives on a single page and the corresponding actions are session-aware (i.e. you don’t call session :off for them). After processing a request Rails by default writes back the session data to the configured session store, even if you didn’t touch the session at all. That doesn’t hurt much as long as you don’t have concurrent requests for the same session, or none of those requests modifies the session state.

With nginx and SSI you will have concurrent requests for the same session, so now if at least one of these requests changes session data, there’s a good chance it won’t end up stored correctly just because another request finished a bit later, overwriting the updated session data with stale data read from the session store before the change has been saved by the first request. Fun stuff, cost me a day to debug ;-)

To get around this issue we’re using the SmartSessionStore plugin. You can find out more about the issue and the plugin in this fine blog post at texperts.com.