Menu
This 3 part blog post and the title for it is inspired by a colleague who is currently living the manual test automation nightmare on an engagement. He reminded me how common a pattern it is and one that I have seen way too often over various engagements. Based on recent engagements and stories I hear from other colleagues in the industry, it is clear that it will be a trend that will continue for some time to come..
When people talk about test automation it means different things to different people so let’s start this blog post by looking at the definition of the word automation.
According to Merriam Webster, automation is defined as follows:
the technique of making an apparatus, a process, or a system operate automatically
the state of being operated automatically
automatically controlled operation of an apparatus, process, or system by mechanical or electronic devices that take the place of human labor
The keywords in this definition are “automatically” and “take the place of human labor”. In my opinion, this is a clear and precise definition of what automation is.
When it comes to Test Automation in software, for some reason the definition of automation quickly gets skewed and/or adapted depending on the organization and the people that are implementing and executing software test automation.
Whenever I start an evaluation of a customer’s Test Automation efforts, I start by asking one basic question:
“Does your Test Automation execute on a regular basis without someone babysitting the automated scripts?”
With many customers the answer is usually; “Well, it is automated and sort of runs on its own but we can explain…”
This generally is the first sign that a customer is caught in the “Manual Test Automation” rat hole. Usually what follows is a whole bunch of “reasons” as to why their test automation needs manual intervention.
Here are but a few of the common patterns that could be an indication that you are running manual test automation:
The test environment is not reliable.
The test data keeps changing or you need to find data.
The scripts are End to End test automation scripts.
The End to End environment consists of many systems (Internal and External) that affect the system being automated.
The test automation tool is a “scriptless” one.
The solution is very complex.
If this is your reality then you may be stuck in the Manual Test Automation rat hole.
Why are you doing Test Automation?
Getting out of the Manual Test Automation rat hole starts by asking yourself, “Why are we doing Test Automation?” Do you have a clear and defined goal of what you expect out of Test Automation? If you cannot answer these questions quickly and with a clear answer, then chances are Test Automation has been implemented for the wrong reasons. Generally when the goal of software test automation is not clear, the implementation is not optimal.
Here is an example of the 3 most common patterns I have seen across engagements. Do you see any that describe your organization?
Reason 1 – Test Automation was implemented to “fix” the problem of bugs/defects slipping into production.
I have already covered the issue of defects/bugs slipping into production in another blog post. Trying to use test automation to “fix” the problem of bugs slipping into production is just another example of that same problem. In summary, the organization should be focusing on trying to avoid having defects/bugs getting created in the first place rather than trying to implement test automation at the end of the development cycle trying to “catch all the bugs” before they make it to production.
Reason 2 – Test Automation is being implemented as the “magic bullet” to fix all of the software delivery issues.
In my experience Test Automation that is “bolted on” after the software is completed, is usually expensive, doesn’t provide good test coverage and is being seen as the savior to fix deeper organizational issues.
If test automation was implemented in this context, then it is usually indicative of an organization that:
May have a management team that doesn’t fully comprehend software development.
Is weak at producing clear requirements for the teams to work with and that show clear Business Value.
May have an organizational structure where communication between various teams is not optimal.
Has developed/implemented a software system which is not optimal and requires constant “patching”. The technical debt that has been incurred over the years of development is just too great and is not being addressed.
It usually points to an organization whose managers look to test automation to try and save the day rather than deal with these deeper issues,.
Reason 3 – Test Automation is being implemented with the intent to replace manual testing which is seen as being a problem in the organization. The problems that usually exist in this case are:
Manual testing keeps taking longer and longer
Manual testing is seen as being ineffective and non comprehensive
Manual testing is conducted from a End user perspective only
In this case the organization has yet to realize that building quality software is a team effort and management is still thinking that the responsibility for quality lies solely with one team – the QA/QC team.
In part 2 of this blog post, we will review 5 suggestions you can start with to improve your test automation strategy.
If you would like to discuss how Avant can help you develop a Test Automation strategy that can work with your software delivery goals, whether you work in Agile or not, please contact us.
Read part 2 here.