In part 2 of this 3 part blog post we reviewed the first 5 suggestions of how you can improve your test automation strategy. In this final part, we conclude this post by reviewing 6 more suggestions that you can look at to start improving your test automation strategy.
The first 5 suggestions to improving your test automation strategy were from part 2 were:
Test Automation must be an integral part of your software development process.
If you are doing Agile or attempting to do Agile, then Test Automation is mandatory.
Test Automation must be executed on a regular and frequent basis.
Do the appropriate Test Automation in the different layers of the application.
Be ready to use the best tool for the type of Test Automation being done.
The remainder of the suggestions are:
6. Don’t get tied to a tool vendor. As you embark on test automation, make sure that you implement test automation methods that allow you to swap a tool out if a better one comes along or if your needs change. If you implement test automation in layers as discussed earlier in this post, and you choose common languages (ex: java, python), then your test automation scripts will be more transportable and you won’t be locked into continuing to use a tool just because you invested so much in test automation with that tool.
I have seen organizations continue to use a tool because they have been using it for so many years and have “so many scripts in the tool”. However, when I ask them, “how often do you run the scripts and how many of them are still good?” they quickly realize that they are not getting the ROI that they should from Test Automation. They are stuck in the Manual Test Automation rat hole!
7. Avoid End to End Test Automation – End to End testing is the most expensive type of testing that an organization can employ and it is usually too late in the process to be beneficial to the organization.
The challenge in “End to End Test Automation” is defining what is “End to End”. Is it from system A to system B or does it mean running a test from the start of a cycle and making sure that the process ends properly through the processing of 10 other systems?
The bigger and real challenge that the majority of organizations face is that they do not have a full test environment that truly represents how production works. Production usually has numerous interfaces to other systems (internal and third-party) and it is just too expensive to replicate the entire production environment.
I recommend avoiding End to End Test Automation as much as you can and if you really must do it, then keep it very minimal and focused on a very specific goal (ex: helping to test that a deployment was done properly in the organization’s environment).
8. Scriptless Automation – Don’t believe the hype….. Scriptless automation has been around for decades. In the early days it was the big promise of using a “Record and Playback” tool. It was supposed to make it easy to have your testers just “record” what they would normally test and then just “play it back”. Like anything that sounds too good to be true, record and playback was exactly that. Teams quickly realized that many of these tools had limits and had trouble playing back what was recorded. Teams quickly faced the reality that they spent more time administering the scripts than executing them. These were the early days of Manual Test Automation.
Move the clock forward to the 21st century and while tools have gotten more sophisticated, record and playback remains as fragile as ever. The only reason that some of these tools work better today is not because the tools themselves have become more sophisticated, but because the applications that they are trying to test are developed with better methods and tools.
Furthermore, anyone that uses so called scriptless automation tools, quickly hits limitations. Once people hit these limitations, the next logical move is to want to get access to the code that is generated. If you are going to access the generated code, then why not just cut out the middleman and do real test automation by coding it?
9. Start with a Test Automation Framework – When embarking on a test automation implementation, there is no need to start test automation from scratch. Applications have evolved and many of them have very common functionality that should allow you to find a library of ready to use code functions/snippets that can act as a base for your test automation efforts. This is often referred to as a Test Automation framework. This is one of the key pitches of tool vendors, you get a library of code to start. Aside from tool vendors, the open source community has evolved and matured to the point that some of the open source tools are either as robust or more robust than tools from a vendor. The open source tools also come with libraries of code that has been created by the community at large and is ready to use. These libraries are indispensable as a starting point for any test automation efforts and then your teams will tweak and evolve the framework to match the realities of the applications they are testing in the organization.
10. Test Automation & Business Process Testing- My first encounter with Business Process testing was in 1996 and I believe the tool was called “Certify”. It was produced by a small tool vendor at the time and it promised that you would be able to test your software by using a record and playback tool and just record all your business processes. The tool never caught on at the time and it disappeared but the concept of testing a software through the business processes has continued and is alive and well in today’s software industry.
There is no shortage of tools that make the same promise like the tool in 1996. This is especially true in the packaged software industry. Organizations buy off the shelf packages to run their business and they want to test to make sure that when they configure the package, it maps to their business process.
In this context of packaged software, it is enticing to think that you can just test by running through the business processes. This usually falls apart because most organizations buy packaged software and then customize large portions of the package to the point that there is more customized software running within the package, than the software from the original package. Why buy a package… but that is a topic for another day?
Therefore, if you are an organization that has bought a packaged software and you have kept the customizations to a minimum (I look for not more than 25% of the code base), then maybe Business Process testing could be helpful in these areas. With the remaining 25% of the code base, you should still look at employing some of the other test automation methods I mentioned in this post.
11. Test Automation cannot do it all – As many know when it comes to Test Automation, some things just don’t make sense to automate. Also, there are different types of testing that are required and more suited to help software teams deliver quality software. For example, during a sprint, using manual exploratory testing to help in the early stages of developing a Story is much more productive and cost effective than trying to automate it.
Also, I have found that while Test Automation is vital to an engineering’s team ability to deliver quality software, you lose that human touch which brings creativity in testing and in thinking like a human would. End users will always do the unexpected and having some manual exploratory testing as described by James Bach is integral to the process.
We think that if you follow these general guidelines, you will be able to implement a test automation strategy that will allow you to be truly agile. This will then allow you to use Continuous Integration strategies which will eventually lead you to the possibility of even doing Continuous Delivery if that is a goal that your organization wants to achieve.
If you would like to discuss how Avant can help you develop a Test Automation strategy that can work with your software delivery goals, whether you work in Agile or not, please contact us.