There's no such thing as "Best Practice"
I don't usually write replies to someone else's articles or blog posts. But recently I stumbled upon an article, which I would like to reply to. The article named "10 Best Practices and Strategies for Test Automation" (original can be found here: https://www.softwaretestinghelp.com/automation-testing-tutorial-7/) states:
Simply put, if you're aligned with Context-driven school of Testing (created by James Bach, Brian Marick, Michael Bolton and Cem Kaner) the first thing you need to put into your head is that there's no practice that can be "Best" or even good in all possible contexts.
I strongly believe that there's no such thing as "Test Automation", but let's put it aside and dive into the article itself. The post is filled with rather useless and mostly subjective "advises" regarding how one should approach "Test Automation". I didn't find any specific logic why #1 is #1 and, not, for example, #10. "Advises" range from Captain Obvious statements like "You cannot automate everything" to cliched "Test automation is software development".
The first practice says "Hire a Dedicated Automation Engineer or Team". The reasoning behind this is that "Test automation is a full time job". It may (and may not) be a true statement. You might want to have a dedicated full time "Test Automation Engineer" or may want your Test Engineers to decide if they wish (or wish not) to have some of their routine activities automated. You may ask your development team to work on Test Tools. Or you may have a team of Craftspeople and everyone will do everything. Every option is a viable option, along with its benefits and drawbacks.
Fifth "advice" says "Good Automation means good manual test case". And the reasoning is a quote from Lessons Learned in Software Testing: "Automating without good test design may result in a lot of activity, but little value". Please note the word may here (which is different from the word will).
While I certainly agree with the good test design prerequisite, there're certain differences between a well-designed automated check and a manual "test case". Test tools do not do what people would do. Test tools would not find adjacent issues that human will find. Test tools simply perform predefined steps and check predefined parts of the results. In fact, I would probably even tell that most "Manual test cases" would not be a good candidate to be "automated" - and for more details why you may want to check this post: https://techbeacon.com/app-dev-testing/5-ways-simplify-your-automated-test-cases.
Eight's "practice" looks safe "Avoid GUI Automation when there's an alternate present" (the idea most reasonable engineers would agree with) until you dive into details:
Apart from all that, the article (and many other articles like this one) is based on one questionable and controversial premise, which is "Test automation is inherently a good thing". I put some of my thoughts regarding this topic in this blog post: http://aqaguy.blogspot.com/2019/03/a-problem-with-agile-automated-testing.html. Long story short, my viewpoint is that test automation is not a good (or bad) thing. It is a certain approach to achieve some particular benefits. And, there're other approaches that may yield the same result as well (check this out, for instance: https://www.youtube.com/watch?v=uSo8i1N18oc).
Quite similarly, there's no such thing as "Best Practice".
These strategies are taken from my own experience plus from the literature of testing gurus like Michael Bolton, James Bach and Cem Kaner. These practices should be followed in every automation project.Ironically, James Bach has a great post named "No Best Practice" (http://www.satisfice.com/blog/archives/27), in which James puts his view related to "Best Practice" idea, which can be briefly summarized - There's no such thing as "Best Practice". I wholeheartedly support this viewpoint. A similar idea is stated clearly on the very home page of Michael Bolton website (https://www.developsense.com/).
Simply put, if you're aligned with Context-driven school of Testing (created by James Bach, Brian Marick, Michael Bolton and Cem Kaner) the first thing you need to put into your head is that there's no practice that can be "Best" or even good in all possible contexts.
I strongly believe that there's no such thing as "Test Automation", but let's put it aside and dive into the article itself. The post is filled with rather useless and mostly subjective "advises" regarding how one should approach "Test Automation". I didn't find any specific logic why #1 is #1 and, not, for example, #10. "Advises" range from Captain Obvious statements like "You cannot automate everything" to cliched "Test automation is software development".
The first practice says "Hire a Dedicated Automation Engineer or Team". The reasoning behind this is that "Test automation is a full time job". It may (and may not) be a true statement. You might want to have a dedicated full time "Test Automation Engineer" or may want your Test Engineers to decide if they wish (or wish not) to have some of their routine activities automated. You may ask your development team to work on Test Tools. Or you may have a team of Craftspeople and everyone will do everything. Every option is a viable option, along with its benefits and drawbacks.
Fifth "advice" says "Good Automation means good manual test case". And the reasoning is a quote from Lessons Learned in Software Testing: "Automating without good test design may result in a lot of activity, but little value". Please note the word may here (which is different from the word will).
While I certainly agree with the good test design prerequisite, there're certain differences between a well-designed automated check and a manual "test case". Test tools do not do what people would do. Test tools would not find adjacent issues that human will find. Test tools simply perform predefined steps and check predefined parts of the results. In fact, I would probably even tell that most "Manual test cases" would not be a good candidate to be "automated" - and for more details why you may want to check this post: https://techbeacon.com/app-dev-testing/5-ways-simplify-your-automated-test-cases.
Eight's "practice" looks safe "Avoid GUI Automation when there's an alternate present" (the idea most reasonable engineers would agree with) until you dive into details:
For example, you want to test the installation of the application. The objective is to check whether application installed or not in a particular environment. One approach is to start the installation and click on “Next” button multiple times through your automation tool. It can be tricky, time-consuming and it is subject to maintenance if UI changes. The other approach is to initiate the application installation with a batch file giving silent arguments. The application will silently install showing no GUI. The objective will be achieved in less time and in a more reliable manner.That is simply not true. If test objective is to test UI-backed installation progress on different operating systems than by utilizing the suggested "approach" there're very good chances that we will test a wrong thing.
Apart from all that, the article (and many other articles like this one) is based on one questionable and controversial premise, which is "Test automation is inherently a good thing". I put some of my thoughts regarding this topic in this blog post: http://aqaguy.blogspot.com/2019/03/a-problem-with-agile-automated-testing.html. Long story short, my viewpoint is that test automation is not a good (or bad) thing. It is a certain approach to achieve some particular benefits. And, there're other approaches that may yield the same result as well (check this out, for instance: https://www.youtube.com/watch?v=uSo8i1N18oc).
Quite similarly, there's no such thing as "Best Practice".
Comments
Post a Comment