Visually checking a website is the number one way to see if things are still working. After an update you load the page in a browser and look to see if everything is the same as it was. That's great. Except it's not that great because it's actually quite hard to see what's changed. It's especially hard to see small changes.
There are visual checking tools that can help with this process. A visual checking tool works by taking a screenshot of the page before an update and then taking a second screenshot of the page after the change and compares the two. This can highlight where the differences are, which makes it very easy to see if anything has changed that really shouldn't have.
Fortunately, before we delve in to tools that can automate the process of visual testing, we can do a really quick and easy hack to enable a visual test without needing to install any software at all. Everything you need is right in your browser.
A feature of CSS3 in modern browsers is the ability to calculate the difference between two images using a filter. We can use this to overlay one image on top of another and see what's different - so if those two images are screenshots we can easily spot places where the rendering isn't quite the same.
By sliding the slider at the bottom left and right you can compare the screenshot of Github's homepage with a screenshot of Github's homepage where the font didn't load correctly. The left hand side is the original image. The right hand side is the original screenshot blended with a screenshot of the website without the correct font (eg a new version of the website that we're testing), which is made by using two background images and the CSS3 property of background-blend-mode: difference;. The resulting blended image is then inverted using the filter CSS property. It's not a brilliant method of testing, but as something you can implement in minutes it's quite a handy quick check.
There aren't a huge number of visual testing tools, and most of them are relatively tricky to set up as they're designed to be used as a part of an automated process. The main tools are;
- CSS. As shown in the example above, you can get an approximate visual diff using plain old CSS filters. It's not ideal but it does the job.
- Wraith. Made by the BBC, Wraith can crawl a list of URLs using a browser like PhantomJS and produce a set of screenshots at different resolutions. It can then use ImageMagick to create perceptual diffs between the screens (highlighting what looks different). It's based on Ruby and it's a very useful tool if you've not got a test automation server to use.
- pdiff aka Perceptual Diff is a command line tool that can be used as a screenshot diff tool in automation servers.
- dpxdt is a tool for comparing screenshots before a deployment and after to make sure they're the same (it's pdiff with some extras to make it easier to use).
In addition to tools that are specifically designed for finding visual differences in testing it's also possible to do things in a more manual way. pdiff works from the command line, but also imagemagick can do the same thing, and you can even use software like Photoshop too. The only difference is the amount of effort and time necessary.
Websites rendered in different browsers on different operating systems will rarely look identical right down to a pixel-perfect replica on every user's computer. Different computers will render fonts in subtly different ways. Users might have plugins installed that affect the way things are displayed. A user with some assistance software could remove your styling CSS code completely in order to read the bare text. There isn't any way to force a website to displayed precisely how you want (well, there is - you could just render the website as a JPEG image, but we'll ignore that.)
Consequently, if we have to accept that different computers will render a website slightly differently, how close a match should a test be if you're comparing a browser to a reference screenshot taken to show how the website should look?
In my opinion I think we should be pragmatic. A website should offer the best possible experience to the user. So long as the design is consistent and looks good then it doesn't matter if things are a little different when the site is viewed on different computers. If an element has an extra pixel of vertical space around it then that's fine. If a font is antialiased differently that's OK. If form elements have slightly more or less padding then the user isn't going to be put off. The overwhelming majority of visitors will only experience a website in one browser so they won't be aware of any differences.
That said though, sometimes small differences do matter because the page content isn't fixed. You need to test that subtle changes between browsers don't affect the user experience in every context - if an element is a little farther up the page when a validation message is hidden then you have to check the page looks good when that message is shown. A tiny layout bug can mean an element will move when a message is shown, and things moving very slightly is a jarring experience for the user.
Another aspect of visual testing is checking that colours are displayed properly. Colour can be an vital part of a brand so making sure CSS colours and images work correctly is something that you need to test. It's also important to check the level of contrast between text and backgrounds for users with visual impairment.
Something that some designers and developers are unaware of is the way that image formats that are understood be browsers can affect the image that's displayed. People have a good understanding that the format affects the filesize, and therefore the download speed, but there doesn't seem to be much beyond that.
The most common file format that's used on the web is JPEG. JPEG is excellent for encoding photographs and graphics that have a lot of 'noise'. The way the format works is by compressing the image data using an algorithm that throws away information, and bundles up data that looks similar. This means that colour in a JPEG is not exact. If you try to create a JPEG image to match a specific 24bit colour in your CSS then it often won't match perfect. JPEG's are about small filesizes that perceptually match the original image. If you need accurate colour representation then don't use a JPEG.
The next common format is GIF. GIFs are an format that uses a palette of 256 colours to display an image, meaning you can only have 256 colours in total displayed Footnote 1. The colours that are displayed are defined by the image though, and the palette in the image isn't compressed, so you have good colour representation. GIFs are good for logos, diagrams, etc that need to be displayed in an exact colour.
The last common file format is PNG. PNG is a more modern format, and actually houses two different ways of encoding an image using either 24bit colour like a JPEG or an 8bit colour palette like the previous format, GIF. The lossy compression that PNGs use means that a specific colour will be retained, so PNG is a good choice for images with important colour values. However, because things can always be a little more difficult than they might need to be, PNGs have a trick up their sleave that means you still might not get quite the right colour unless you're careful. The PNG format includes a gamma value that browsers use to calculate how the image data should be shown using the user's monitor calibration values. If this value is wrong then the image won't be displayed accurately.
There are other, more estoric image file formats, such JPEG200, WebP, BGP, and so on. Most of these are based on clever forms of 24bit image compression so they all display colours accurately.
The last thing to mention for visual testing is replacing a typographic style with a "testing font" in order to eliminate text from the visual test. Blocks of text can make it harder to see where differences lie on a page, so a testing font can be used to replace the letters with something else. A good example is Blokk - http://www.blokkfont.com/ - the letters are simple blocks, with kerning set so that words become blocks on a page. This way the layout is as it would be with a proper typographic font, but there isn't any distraction from the content.
If Blokk isn't ideal for your design then an alternative can be found in https://github.com/christiannaths/Redacted-Font
Strictly speaking GIF image files can have more than 256 colours at once. GIFs can have multiple frames, used for animation, and each frame can have its own palette. Pixels in an image that are transparent are left as the previous frame's colour. So, by defining an area's colour in one frame, and then leaving that area transparent in subsequent frames, it's possible to achieve a higher number of colours than you can define in a single frame's palette. Using this feature it's even possible to make a GIF display a true colour image. Although quite why you'd do that is beyond me.