This article covers the subject of testing your site/pages and the compatibility with different browsers, several different services and solutions exist. I aim to cover as many solutions as possible in this article.
The most expensive solution is to have different physical computers, each with a separate Operating System (OS) and different browsers. The expense can be reduced by having different versions of several browser types on a same operating system. Say having IE 5.5 and FireFox 1.0 and Opera 8.0 on a Windows 2000 Pro box then IE 6 and FireFox 1.5 and Opera 8.5 on Windows XP SP1 box.
You could also use a dual (or mutli-) boot system providing you with the ability to switch between the various OS. The different OS can be on different partitions or hard drives of the same computer, reducing the cost of physical hardware required.
You have most probably heard of, or used, what is called Virtualization Software; complete copies of physical computers or servers that run separately or simultaneously on one same machine, as virtual machines. This technology is now being used for complex systems allowing multiple virtual servers to run on one high-end server, but it is also available for desktop solutions.
You could for example, have a complete and separate installation of Windows 98, another for Windows 2000 and another for Windows 2000 Professional including separate installations corresponding to the different Service Packs, a RedHat installation... on the same PC. In this screenshot you can see Windows XP running on Linux !
Switching from one OS to another is slightly longer than launching a program; a fast computer will allow you to switch between the different flavours of OS pretty quickly.
I have personally tested VMWare and was pleased with the results. You can download the free VMWare Player to get an idea of the concept (Ubuntu OS with FireFox).
You can also take a look at Parallels a version of which can run on the latest Macs; that have an Intel processor and are running Mac OS X 10.4.6. Yes running XP is now possible and from comments I have seen about it, at a pretty nice speed as well !
Various sites explain how to obtain near perfect solutions for having all different versions of Internet Explorer running on the same machine. For over two years I have been using Evolts' standalone versions that can be found here. Installation packages for nearly all previous versions of IE can be found here.
These versions of IE are part of an archive of numerous browsers with different versions of each. Please note that the archive is not being updated as it used to be though. There is even a version of Mosaic from NCSA, the first browser I used at University to browse the Web.
Although standalone versions of IE will provide you with a pretty good idea of what you can expect, this can be perfected further with some tweaking. Also, remember that these solutions may not give you a perfect simulation compared to the real set-up.
Getting several standalone versions of other browsers (Netscape, FireFox) to work on the same machine can be complicated. I've tried and often failed to stop the installation packages from upgrading the previous versions installed on the machine. The article at A List Apart about installing various browsers on a Mac is outdated now. If you know of any documented solutions I'd be happy to post the references right here !
Although this may not be the holy grail, being able to access remote and preconfigured testing machines sure seems like it, if you are lucky enough to have used these systems. It is also an amazing solution if you are often travelling or use a portable computer :
Note: some friendly advice, make sure that you agree upon the degree of cross-browser compliance at the beginning of the project, from experience, nothing good comes from hiding things.
Remote testing machines are simply computers using VNC that allow you to control what the computer does from a remote location. A company like BrowserCam sets their machines specifically so that you can pick the configuration you want to test. You will log on to the remote machine, just like in the image below, with the specific Operating System you chose and various browsers available.
Safari launched from within an Internet Explorer browser (Browsercam).
BrowserCam
The BrowserCam solution provides numerous configurations to choose from. To see this in action, you can check out Browsercam's tutorial here.
BrowserPool
BrowserPool offers a similar service in that you can log-on to the remote machines and you can even test the system free of charge (depending on their client capacity load, which seems fair). You can compare their feature list here. I haven't had time to test the service myself.
These solutions allow you to, not only see what the page looks like in the screen capturing services described below, but also how pages react dynamically. Dynamics can have a dramatic effect on a page pushing content left, right and centre when you mouse over the content because of a DIV that is one pixel too long in IE5, for example. Some pages which use Javascript/AJAX will redefine dynamically the dimensions of a DIV and in so doing create havoc in the page layout with certain older browsers.
All these scenarios and many others are difficult to test without separate and distinct (virtual, hardware or remote) environments. The standalone solutions often miss this unless properly tweaked.
I have listed below the companies (I know of) that provide screen capturing services. I have explained which ones I have used and what I liked about them.
SiteVista
I have used SiteVista on a project and found the system to be fairly reliable and fast. When I used it they had just transferred to a new hosting company and still had a few issues. The presentation is easy to use and understand. I did regret not being able to resubmit several specific screenshots from the list in one go, but it's frankly a minor detail. Take a look at the demo video to see how easy the system is to use
Screenshot from the SiteVista demo video.
Browsershots
Well BrowserShots is a free service, simple to use but your results will depend upon the queues of previously submitted sites. For example on August 24th I submitted significant-media.com to Browsershots. I was told that Epiphany 1.4.8, Firefox 1.0.4, Firefox 1.5.0, Galeon 1.3.20, Konqueror 3.3, Mozilla 1.7.8 and Opera 8.50 were rejected because the queue for them was longer than 60 minutes. The IE 6.0 screenshot should work !
URL2Image
I haven't used URL2Image's system and cannot therefore compare it. The demo gives an idea of what to expect (the demo's audio was out of sync for me BTW) and I wasn't that impressed, but do check it out.
Browser Photo
I don't know too much about Browser Photo so any comments are welcome. I must say the presentation is weird when you think of what they are supposed to be offering, so I'm not too sure about it.
iCapture and ieCapture
- iCapture has been around for a while and as Daniel Vine, the creator of the system explains on his home page he is "Just some guy with a website, and a very high bandwidth bill". The system had a backlog of 2198 minutes today !!
- ieCapture, yes again the same guy, created a screenshot system that enables you to preview your site in IE7 and in this case the backlog is 172 minutes. Should I be slightly facetious and ask whether this is a sign ?
BrowserCamp
BrowserCamp seems to be pretty interesting in that it is offering both Screen captures and remote (VNC) connections to Mac OS systems. You can also get a free view of a url in Safari for Mac ! When testing it with significant-media.com I realised that an image on the home page was not being displayed in Safari.
BrowserCam
Again what BrowserCam provide in terms of screen capturing, is probably the most complete solution in the market as you can see. To see this in action, you can check out Browsercam's tutorial here. Browsercam also offers a free trial for you to test their system. One important thing you should know is that Browsercam provides username and password type connections (http connections that require authentication), most others don't. Images can also be saved in PSD and PDF format, as well as being able to organize project files and save them as one zip file.
Sreenshot from the Browsercam tutorial.
As you have seen above, systems like Browsercam are capable of offering more than just screen capturing tests. It's logical since sites are more and more dynamic. Not only is the content dynamic and automatically refreshes via technologies like AJAX, the content is also visually dynamic and can change with a mouse over. A single screenshot cannot tell you this.
In view of cross-browser compliancy standards the newer browsers benefit greatly from standards like XHTML and CSS. However roughly compliant browsers like IE 6 and earlier, still have their weight in the browser population as per the current statistics.
Standards are helpful; they will make designing web sites easier. However at the moment, until the non compliant browser base is nearly insignificant, an XHTML and CSS compliant site is not always the most cross browser compliant site. IE, will have needed 7 versions to properly render the W3C standards. This is one of the main reasons why the general population has only recently started to see the real advantage of these standards. But if every browser rendered pages following the W3C standards there wouldn’t really be such a market for the above products and services.
There is a new software solution that allows you to test multiple versions of IE called Multiple IEs.
The same solution exists for Safari: Multiple Safaris.
You may also want to check out what is called "load & stress test" systems/solutions which allow you to set-up tests either remotely (with hosted solutions) or with software. The tests may check how the server(s) respond to a query or a predefined and specific sets of queries.
Screenshot from from Neotys
Load and stress tests are often used during the pre-production process to analyse potential bottlenecks and find solutions prior to launching. Well designed and thorough tests can also help anticipate hardware upgrades, and pinpoint areas it will be interesting to monitor, once the project is launched. A company I worked with, called Witbe (in France), also had alarm systems that would check, not only that the site was up, but that specific queries were completed in a set time, otherwise an alert was sent to a designated person. This could be done from various geographic locations around the world for a more in depth check.
There are also 'local' software solutions for these more functional and technical tests instead of hosted solutions like the one above. A few examples are : WinRunner, QuickTest, e-Valid, QEngine, SilkTest, Rational, QA Wizard.
Other :
I have no links with any of the companies cited unless otherwise stated (ie I have used their services)
Please feel free to comment...