So you created the coolest site on planet, and want to make sure it works on all web browsers and every configuration possible. Just download all the released versions of all the web browsers you know, spend some time installing and begin testing…

But hey, do I really need to install all those browsers/versions? And test with JavaScript on/off? Flash? Screen sizes? Well, if you absolutely want to make sure your site works in every possible browser configuration on planet, the answer is unfortunate yes. But if you are smart, you do a little research, create an appropriate browser testing setup, test your site and expect it to work 99.9% times.

Ingredients

The ingredients for the successful browser testing are:

• knowledge of layout engines and browsers
• official and unofficial browser packages
• de-facto rendering add-ons
• useful browser plug-ins
• web-based testing tools
• multiple operating systems
• a virtual machine player

Let’s have a look on each ingredient in detail.

Layout engines and browsers

Under the hoods of every browser is a layout engine that does the hard work of rendering your HTML/CSS/JavaScript into something visual. There are many more browsers than engines, and from the testing point of view, you should make sure all the major engines are covered in your tests.

Each engine has a long history and comes with number of versions. Is there any recommendations about what engines and versions I should pick for my testing setup? One of the most referenced one is Yahoo’s Graded Browser Support. Unfortunately, it does not say a word about layout engines, but you can easily catch up by checking out the List of layout engines. The point is; knowing browser engines give you confidence that if you test something with browser X, it will very likely work well in browser Y. In other words, there might be little reasons to test a site with both browsers X and Y, if they use the same layout engine.

You should also follow the browser statistics of your own site. Use a log analyzer such as AWStats or statistics collector such as Google Analytics. Needing someone else’s word on it, Browser Version Market Share or W3School’s Browser Statistics are good sources to refer.

If all this sounds too complex to dig into, here’s some details of the most popular engines and browser recommendations.

Trident

Internet Explorer uses a proprietary (Trident) engine. There are currently three major versions of IE on the market – all using heavily rewritten layout engine. Unfortunately IE versions tend to stay on the market for quite long time. My recommendation is to test carefully with IE 7 and keep IE 6 on your list – especially if you are providing a consumer service. As IE 8 hits the official release, be prepared to add it to your testing setup.

Gecko

Gecko is popular engine among open source web browsers, and used in Firefox. Firefox users tend to upgrade their browsers much quicker that Internet Explorer users. You probably play quite safe by testing with the latest major version number, and one generation backwards. Currently it means Firefox 3.1 and Firefox 2.0. Firefox is popular in all operating systems. Therefore, it is great for testing how differently Windows, OSX and Linux versions render your site.

WebKit

Safari is the most well known web browser using WebKit engine. WebKit is also becoming very popular among open source browser projects. The engine is surely popular enough to include in your testing setup, but testing with the latest version of Safari or Google Chrome is probably enough. iPhone, Nokia smartphones and many other handhelds use WebKit engine as well. If you definitely need to make sure your site renders perfectly on a device, you should test the site with a real thing or a device simulator.

Presto

Opera browser uses an engine called Presto, and the engine is used by some niche devices as well. I recommended doing occasional testing with the latest Opera version. Presto engine is very accurate and standard respecting. Therefore, do I dare to say, your site is likely to work well on Presto, if it works well on a Gecko or WebKit based browser.

Other engines

I also recommend occasional testing using one of the text-based browsers such as ELinks. It is an easy (but incomplete) accessibility simulator and reveals the missing HTML semantics.

Browser packages

Installing multiple browser brands into your OS isn’t usually a big hazzle. Perhaps you will witness a tiny non-bloody fight whether Firefox, Opera or Safari shall be the default browser in your system. On the other hand, running multiple versions of a single brand of browser is often tricky, sometimes terribly hard. It’s not too hard to guess that Internet Explorer is the enemy of web developer in this territory too, but other browsers, such as Firefox and Safari, aren’t really your friends either. But hey, let’s stop mocking and get the job done.

Internet Explorer

Internet Explorer is tightly tied to the Windows operating system. Don’t expect to install any version of IE without updating some “system DLLs” and a reboot. Therefore, running multiple versions of Internet Explorer is very hard. Even worse, you cannot officially test drive the forthcoming IE 8 without removing IE 7. The official Microsoft solution is to use special virtual machine images. The problem with these images is they are only provided in Virtual PC format, they expire within couple of months and they are sometimes buggy (the virtual machines, not the browsers). If this is your way to go, there are instructions on how to convert images to other virtualization formats.

Somewhat more convenient solution is to use unofficial “standalone” IE 7 and Multiple IE packages. Don’t expect these to work perfectly. They might e.g. have broken bookmarking or occasional crashing, but rendering engines should be accurate. An interesting alternative is the IETester in which you can test multiple IE versions at the same time on shared screen.

Microsoft offers the “normal” versions of Internet Explorer 6, Internet Explorer 7 and Internet Explorer 8 on their website. Older versions are found at Old App’s Internet Explorer archive.

Firefox

Installing multiple Firefoxes is not hard, but it is a bit cumbersome to keep those instances not messing up each other. There are many guides how to manage it. Mac users might be happier using MultiFirefox doing all the dirty work for you.

The official Firefox site provides downloads to the latest Firefox 3.0 version and officially supported Firefox 2.0. The “next generation” Firefox is always developed under a codename and 3.1 is called “Shiretoko”. The latest build should be found somewhere in Firefox Releases archive. Older versions are found at Old App’s Internet Explorer archive. The Mac archive misses quite many versions.

Safari

Forthcoming version of Safari (4.0) can be installed without conflicts side-by-side with the existing Safari browser via WebKit nightly builds. Unfortunately, this is not so easy with the older Safari releases. Multi-Safari is to rescue (Mac users) providing well working standalone versions of Safari.

Official releases on Apple’s site are Safari 3.1 and Safari 2.0, and the older versions are found at Old App’s Internet Safari archive.

Opera

It seems Opera has least problems running multiple versions side-by-side. I have had no problems installing any version of Opera and running them simultaneously on my Mac. Opera provides comprehensive collection of browser versions at Opera archive.

Mobile browsers

Despite most mobile browsers use the same engine than Safari does, the screen size limitations and smart zooming/scaling features almost enforce proper testing with a real device. If you don’t have one or testing is difficult due networking problems, you will manage with the device simulators.

iPhone simulator (on Mac) is bundled with iPhone SDK. Remember that you don’t need the install the full development kit. Just locate the simulator image, and install from there. A decent Nokia phone simulator (for Windows) can be found inside S60 SDK. Again, don’t install it all – just select the simulator from the installation options.

Text based browsers

There are a number of text based browsers out there and you’ll find good hints and links to actual browsers at Site Testing with Text Based Browsers.

De-facto rendering add-ons

Not everything you see on a web page is always rendered by the browser’s engine.

Flash

According to Adobe’s Flash penetration statistics, you could say almost all browsers now have a Flash player installed. On the other hand, most mobile devices do not play Flash at all. All the Windows versions of Flash player can be downloaded at Adobe’s Archived Flash Players.

Java

I could not find good penetration statistics for Java in browsers, but Java vs. Flash gives you a clear impression that Java is much less popular. But if your site has Java applets, you could test it with different Java versions. Sun provides comprehensive Java archive for all the versions ever released.

Browser plug-ins

There are tons of helpful plugins for Firefox and certainly good ones for other major browsers as well. From the testing point of view, let’s have a quick look on two essential plug-ins.

NoScript

NoScript is a popular Firefox plugin for blocking JavaScript, Java and/or Flash. You might not want to use it in your daily web browsing (or you do), but it certainly is an excellent tool for testing how your site behaves when users cannot view all your content. For instance, according to W3School’s Browser Statistics, in January 2008 JavaScript was not enabled for 5% of the web surfers.

Web Developer

If you are a developer, you are likely to use excellent Web Developer toolbar for Firefox. It is also great for testing your site e.g. without stylesheets or cookies, and testing the various screen sizes. Example resources of screen size stats are Market Share’s Screen Resolution and W3School’s Browser Display Statistics.

Web-based testing tools

No surprise, you can find interesting web browser testing tools in the web.

Browsershots

Browsershots is a wonderful tool to test rendering of your (public) site with a number of browsers in different operating systems. If you are in a hurry, you can pay for priority rendering.

BrowserCam

An advanced web-based tool for browser testing is BrowserCam. In addition to browser screenshots, with a paid plan you can get shots from mobile devices or have remote access to real browsers. I did not test this in practice, but it sounds very cool, if remoting works fast enough over the network.

Operating systems

Some popular browsers can be installed to multiple operating systems. For instance, Firefox is available to almost any platform. The rendering results, however, are never pixel-to-pixel same. The colors can look surprisingly different and text may have totally different typefaces. Even the operating system settings influence on the rendering results. For instance, some Windows fonts look really ugly without ClearType turned on.

An obvious solution for cross-platform testing is to have multiple testing computers or a single multi-booting computer with multiple operating systems installed. Using a virtual machine, however, might be far better solution.

Virtual machine players

With a virtual machine player you can e.g. run Windows while using Mac OSX. The major benefit of having a virtual machine instead of running a “real one” is that you can create multiple setups of a single OS. For instance, you could have one setup for running IE 6 and another for IE 7. Even greater, some players support “snapshots” providing an easy way to reset your system to a clean state, if you messed up something. In addition, testing becomes more streamlined when you use a virtual machine, if the option is to reboot your computer between the testing cycles.

These are the most well known virtual machine players:

• Virtual Box – A free VM player working in almost any OS
• Virtual PC – Microsoft’s free VM player for Windows
• VMware player – A very popular and free VM player for Linux and Windows
• VMware Fusion – A commercial VM player for Mac OSX
• Parallels Desktop – An alternative commercial VM player for OSX

An alternative VM technology for running Windows apps is Wine. With some work (and good luck) you may get a Windows app running “natively” in your Linux or OSX. There are instructions on running Internet Explorer on Wine and some helper scripts such as IEs 4 Linux.

Browser testing setups

So, how should I test my site? Well, it depends :-) If your needs are “light”, I would suggest a light solution. You could, for instance, simply use Windows with those unofficial “standalone” browser packages, and use a web-based service for checking the rendering under other operating systems. If you are building the next Facebook, I would certainly install a good virtual machine solution for each developer and a testing center with multiple PCs to the test-only guys.

My solution is somewhat “semi-hardcore” and requires quite a lot of preliminary work. Once you get it running, however, it is quite a pleasure to use. It consists of a VM player supporting snapshots. I have three Windows images titled “Legacy Browsers”, “Modern Browsers” and “Future Browsers”. Each image contains the most popular browser of each layout engine, but there are only one version of each browser brand. I also use older Flash and Java versions in the Legacy Browsers image, and beta versions in Future Browsers. WebKit testing I like to do with unofficial standalone Safari packages and I use multiple Firefox profiles for “feature X ON/OFF” testing.

I will post a follow-up about the details of my setup within my next post. In a meanwhile, happy testing!