As we all know, HTTP is a stateless protocol. We do all sort of hacks to add state, like ext/session in PHP. While such hacks work great for a lot of use cases, we should remind ourselves that they are hacks. There is a phenomenon of state creep: coupling unrelated HTTP requests. Think of a page that references a thumbnail in an <img/>-tag and the picture is generated as needed: it would be possible to generate that image in the context of the request that embeds that image. So the template calls a helper to generate the thumbnail and the thumbnail is generated in the file system.

While this works well for a single host, your personal weblog about cooking and cats, it won’t work for something serious. When you start load balancing between two webserver nodes you are set on fire as you can’t guarantee that the image is present on the correct node (beside you are generating the image n times where n is the number of nodes). The solution is not that hard: pregenerate all the images with a queuing system and display “This image is currently not available”-placeholders as long as they are not ready or – in case of little image uploads – generate them when uploading the image. The other option is to generate them on the fly when they are requested. If you do the latter, do it in the context of the request that tries to receive the image, not in the embedding context (the page that embeds the image). Generating on the fly means that you deliver your files through PHP or something similar: this is fine as long as you have an HTTP accelerator in place.

One of the systems that does it in the way described above is Drupal. I’ve implement MogileFS for image storage and retrieval for Drupal and let me say, it was not a pleasure.

On a side note: HTTP 1.1 allows resources to be fetched in parallel, which makes generating images in the wrong context even worse from a user experience point of view, as the page will not show up until each thumbnail is generated.