Images are everywhere on the web. Whether you’re building a simple blog, a CMS, or a true web application, chances are good that you’ll need to deal with images. Editors, authors, users, etc. expect to be able to upload images and have them resized and cropped on-the-fly. It seems like a mostly solved problem too. You can grab an open source image library in almost any language and get everything you need out-of-the-box… or so it would seem.
As with most things, the devil is in the details and when you grow your little website from one server to many, the water becomes even murkier. This post, however, will focus on the actual thumbnailing and all its complications.
Different Devices, Different Images
I used to be simple. If you wanted an image, you dropped in a single <img /> element which was served to all your visitors. Today, the image you serve is dependent on the screen it’s being served to. On a high pixel density desktop screen, you want to serve a large (in both size and resolution) image that will take full advantage of the extra pixels. On a small mobile screen, however, you want to save bytes and serve the smallest image that will still look good. Sending the desktop image to a small screen not only wastes bandwidth, but slows down the rendering time for your users. There is no single “right” solution to this problem, but you can learn more about it and some best practices via High DPI Images for Variable Pixel Densities on HTML5 Rocks.
Focal Points and Aspect Ratios
How do you make a portrait image fit in a landscape space? It’s easy, you just crop it, right? Not so fast, take this image of Ambrose Burnside.
Looks stunning as a portrait, now lets naively crop it to a landscape image by taking equal amounts off the top and bottom of the image.
Not so good. What we really want is to take more off the bottom than the top to center the image around his face.
Images like this have a focal point which either needs to be designated manually at upload time or dynamically via a detection algorithm.
Upscaling
Most thumbnailing implementations assume that the thumbnail is smaller than the source image. What do you do when it isn’t though? Upscaled images are ugly and “pixelated”. Here’s what our buddy Ambrose looks like upscaled from a height of 100px to 400px:
There are a few ways to handle this problem, none of them great:
- Don’t let users upload images that are too small. This assumes you know all the thumbnail sizes you will ever need at the time of upload.
- Upscale either via HTML
heightwidthattributes, CSS, or generate an upscaled image all of which will be pixelated. - Write special logic in your display code to handle smaller images differently.
Optimization
Images should be served as small as possible. It saves money on bandwidth and improves the experience for users (especially on mobile). You can’t, however, trust your to users upload images that have been optimized for the web. It’s not uncommon to see images that use twice as many bytes as necessary, even after being thumbnailed. To fix this, your thumbnailing solution will also need to pass them through a lossless image optimizer.
Color Profiles
Did you ever wonder what the “Save for web…” option in your image editor does? Among other things, it ensures images are exported with a sRGB color profile. This is the best profile for the web, but not the best for print. Using a different color profile can result in washed out or over-saturated images. The example below is the same image in two different color profiles resized by Python’s image manipulator (Pillow). The source for the image on the left was in sRGB while the source for the image on the right used a CMYK color profile. Notice the complete lack of definition on the lower-right side.
Prior to thumbnailing or during the initial upload, you’ll need to check the color profile and properly convert it to sRGB if necessary.
Rotation
How is it that however you hold your phone when you take a picture, the image always displays right-side up? It’s likely that the image isn’t actually rotated, but instead contains rotation instructions in the embedded EXIF data. If you don’t account for this additional orientation information, it’s very likely you’ll start seeing images that are rotated 90º or fully upside-down when you try to display them on your website. For more information on this topic see, EXIF Orientation Handling Is a Ghetto.
Conclusion
When dealing with images, edge-cases abound. Most of these problems will never show up in developer testing, but only when real users start uploading real images to your platform. When looking for a robust thumbnailing solution, be sure to you know how it will perform in these edge cases.
The technical “correctness” of the thumbnailing library you choose is only one part of the equation. On high-traffic sites, you also need to worry about how efficiently you can generate and serve the thumbnails. In part two we’ll look at performance and how different storage solutions (local filesystem, storage APIs like S3, etc.) affect the thumbnailing process.
For more, check out The Trouble with Thumbnails: Part 2.
