Manual website inspection is necessary but not sufficient
Websites must be inspected on a regular and ongoing basis to ensure they comply with standards, legal obligations and corporate branding. A failure to comply may result in problems with usability, accessibility and function.
These problems are likely to cause:
- Users leaving the site before they have completed a task.
- Loss of brand reputation.
- Loss of brand loyalty.
- Loss of custom.
- Legal challenges.
There are three ways to test the usability, accessibility and functionality of a website:
- User feedback.
- Manual inspection.
- Automated testing.
And they all have their purposes and benefits.
Recently I was looking at a well designed and tested website and tried to request a download of a white paper; when I hit the request button I got an error message. Although, in theory, this should never have happened, the dynamic nature of the content of the web and the continual changes in the technology means it will inevitably occasionally happen. It is therefore essential that web sites have the capability for the user to report problems within the site itself via a simple feedback mechanism. There is nothing more frustrating than hitting a problem like that and not being able to do anything about it; the user is very likely to go to another site and not return.
Putting real users with different requirements, level of web experience and accessibility needs in front of a test site is a crucial method of ensuring that the site is usable, accessible and provides the functions needed. To ensure the maximum benefit from this type of testing it essential that:
- The scenarios are well defined.
- The capabilities of the individual testers are recorded.
- The details of the interactions are recorded so that any difficulties the user had can be resolved as well as fixing any complete failures.
The problem with user testing is that it is time consuming, expensive and unlikely to test all functions and routes in the website. User testing should concentrate on ensuring the usability of the site by all users and not finding issues that could have been highlighted by automated tests.
Hence my insistence that manual inspection is necessary but not sufficient.
Automated testing tools should be used from the very first stages of developing a web site; they can be used to test individual pages and highlight errors such as missing alt tags, use of deprecated tags and options, and misuse of tables and headings. There are a number of free sites, such as Cynthia Says, that provide this level of testing; as well as developer tools, such as HiSoftware AccVerify. These automated tests are better than human testing, firstly because they will pick up all the errors efficiently and inexpensively, and secondly because they pick up errors that may not be visible to a human tester, for example a deprecated tag may work with the browsers being used but may cause inaccurate rendering in future version of a browser.
The second phase of developing a website is connecting many pages together. At this stage it is essential that all links work and are coded correctly. Automated tools, such as AccVerify, can check all the links to ensure that they all work; these tests could be done manually but it is time consuming, hence expensive, and worse, it is error prone as it is very easy for a human to miss testing a link. The other benefit of automated testing is that the results can be fed into a repair tool, such as HiSoftware AccRepair. These tools can automatically correct some types of errors but they can also ensure that errors that require manual fixes are attended to.
At this point in development, when the automated tools have run clean, human testing must be carried out to ensure the real usability and accessibility of the site. The test should concentrate on scenarios with the users being given tasks to complete. The ability of the different users to efficiently complete the tasks is the main measure of quality of the site. If a user fails to complete a task because of an error that an automated tool could have picked up this must be considered as a wasted user test, and hence a waste of time and money.
Having completed the automated and human testing the site is ready to go into production. Websites are dynamic and will be modified and will have new content added; these changes may cause errors to creep in even if the change management system includes automated and human testing. It is essential that sites are monitored on a regular basis to pick up errors before an user does. HiSoftware AccMonitor can be used to test sites on a regular basis and the errors can then be fed back into AccRepair for fixing. At this stage it is really not practical to run these comprehensive tests manually; the number of pages and the number of links means that the cost will be prohibitive, especially when this has to be multiplied by the number of different types of user. Running automated monitoring can provide:
- A complete list of new or modified pages.
- A list of pages that have been flagged as in error.
- A list of broken links.
The owners of the pages can then fix the errors and broken links. After that a selection of new pages should be manually inspected to see if there are any further problems. This method of testing has been shown to greatly improve the quality of the site whilst lowering the cost of testing.
The only issue with AccMonitor is that the software and the installation requirements mean that it is expensive for smaller sites, or sites with limited technical support. HiSoftware have recently addressed this by offering a managed service. The service runs AccMonitor against the client’s website and provides a complete report to the client, thus providing a cost effective monitoring facility.
Automated testing is a necessary part of web site development and production environments but is not sufficient, as user testing and feedback are also required. The correct balance of user feedback, user testing and automated testing will create the best site and at a reasonable cost.