Testing isn’t always the most humorous of subjects but we do have at least one professional joke.
A tester walks into a bar
Orders a beer;
Orders NULL beers;
Orders 9999999 beers;
Orders a bear;
Orders -1 beers;
Orders szdgskfhd;
With that covered the bar is good and ready to welcome their first real customer, who promptly asks where the bathroom is, at which point the whole place catches fire and burns down.
Like most jokes the humour revolves around a germ of truth, in this case that your end user will find a different way to use the system than you anticipated and this scenario may well not be covered.
User Testing in the World of Enterprise Software
When enterprise software is sourced for an organisation the end user should not only be consulted on what they need to do their job but also on whether what they get meets their needs and whether it fundamentally works.
One tool to ensure this is User Acceptance Testing (UAT), where a subset of the end users test the sytem before it is rolled out. This test phase should be run after the system has been created, configured and is ready to be delivered.
Why Do We Run UAT?
By the time we come to run UAT the system should have been thoroughly tested at various points. These include during development, together with the Product Owner reviewing work as it is delivered, and when integration testing is run. This begs the question why do we need an extra layer of test? What benefit does it bring?
The primary purpose of UAT is to give us confidence that the system we’re is as issue and defect free as possible. To put it simply, to find any bugs that might have slipped through and get them fixed, or at the very least establish a roadmap for fixing them. Having the end users running these tests brings real world data and experience that the test and development team will lack. The users know the pain points and exceptions and UAT allows us to leverage this knowledge.
A secondary purpose is that it gives us an opportunity to check that we’ve actually got what we paid for; running UAT allows us to have basic requirements coverage carried out by the business.
A tertiary purpose is that well run UAT is invaluable to the introduction of new or significantly changed systems within an organisation. It provides an informal training mechanism for end users and hopefully will create a cohort of evangelists who will help drive any roll out forward. I use the word hopefully on purpose because if you attempt to deliver a poor system you can end up creating resistance instead.
Who Does the Testing?
Generally there are two groups of people involved in the UAT phase; those who organise the testing and those who actually do the testing.
The organiser will be a test co-ordinator or a test lead. They will require strong people management skills, good soft skills, should be be well versed in the tooling being used and be able to aid in the creation of the tests to be run.
The people who actually conduct the testing need to be Subject Matter Experts (SMEs) in the area the system covers. These people will differ massively from project to project as different end users have different expectations and different levels of technical expertise. Two extreme examples I’ve worked on previously serve to illustrate this. Firstly, a new archiving and media ingest tool to be used by people who edit the media shot by filmmakers. These SMEs here were used to working with complex systems and cutting edge technology in their day to day job. Contrast this to another project aimed at creative directors, people who sometimes have an active disdain of technology.
Whilst it’s much easier to work with SMEs who are technically competent, more value is often gained with the less competent, as they need a system that is easy for them to use. This is why people management and soft skills are key. Running UAT often calls for an experienced cat herder.
When Do We Run UAT?
“Shifting left” is something of a buzz phrase in testing at the moment, the idea that the earlier you can run and catch issues the cheaper they are to fix. It might seem from what I’ve written that UAT doesn’t lend itself to shifting left as it is run as a final test when the system is already ready for delivery.
However good UAT shouldn’t begin at this point. It should started as early in the project as possible by identifying the SMEs and working with them to set up the tests that will be run on the system. Conducting this activity is a type of static analysis where testing is conducted without actually running any of code and it brings a number of benefits. The act of creating the tests helps highlight any issues there may be with incorrect or ill thought through requirements; issues stemming from requirements are much easier to fix before code is written than after. Engaging early with your SMEs is the first step in getting buy in with crucial people for the project.
Conclusion
UAT is vital in outsourced enterprise software to act as a final quality gate. Done well it allows the end user to bring their own unique expertise and ensure that they are getting the tools they need to perform their jobs. Further, when you do come to roll out new or improved system you have a ready team of enthusiasts at the grass roots level to help push out the new system. The sooner you engage in this activity the more value you’ll get out of it.
