A QA-guy blog

While there're lots of information published on the internet about how one should develop/buy a Test Automation solution, there's an elephant in the room - which is the fundamental question "Do I need to do it at all?". That is a good question. While I think that there’re some legitimate contexts where product/project/team can benefit from having a sophisticated test automation solution/framework, my general rule of thumb is “If you need a framework - you do it wrong” . There’re at least several decent papers showing the benefit of unit-testing ([1],[2], [3]), while I could not find any research proving that system level tests could bring better results. Please let me know if you can prove me wrong. There’s at least one half-scientific paper showing the benefits of unit tests in comparison to integration/system level tests [4] There’s a good paper bringing a notion of running/writing and maintaining test vs. the benefit of having/running it at...

Some time ago I had an argument with another team member - he was claiming that Test-driven development (TDD) was not applicable to our application. He thought it would be more efficient to rely on manual or E2E testing instead. I disagreed. Without going too much into the details, the core of the application we were working on was a graph-traversing algorithm with some amount of mathematical calculations. It was painful to test on a system level as the number of possibilities was huge and it was not easy to put the system into a particular state to check specific conditions. I thought that thorough unit-test coverage, and, ideally, TDD would be more efficient, but was not able to persuade my colleague. At the end of the day, I started to doubt if I was right. Challenge was accepted, so I tried to do something more or less similar (though simplified) - implementing a Tic Tac Toe computer player logic via as pure TDD as possible. Long story short, after some time I managed to g...

Let's consider an example of what Layered (Tiered) architecture may look like for a test automation framework. As a system under test, we're going to use this simple and neat "Todo list" application. First, let's do a brief analysis of the application. From my brief exploration I came up with the following checklist: Area/Group Things to check Comments Todos CRUD 1 Create item Basic scenario, UI-level case 1.1 Create an item with the same name The case is not valid. There's no validation for that - nothing to check really. 1.2 Create an item with an empty/not valid/malicious name While it would be a nice thing to test manually, from test automation standpoint I would expect this to be filtered on service or repository level, so either integration or unit test . 2 Mark item complete Basic scenario, UI-level case 2.1 Un-mark item complete Basic scenario, UI-level case 3 Delete item Basic scenario, UI-level case 3.1 Delete item ...

Choosing between writing one UI tests and 100 Unit test I would always prefer the second option. Why? Well, I am actually lame with UI testing, but that not the main reason. The only thing I worry about is speed. How fast we can change software? Quality - I don't care about it much cause quality standards are contextual and can change. The only thing that is certain - there're going to be changes and we most likely don't know what those changes are about. So there's only one thing that matters - how fast we can adapt to this change. Internal quality is something that actually makes us able to change fast. UI tests do not contribute to internal quality. UI tests may even make things worse. That is the reason. And I invite you to Testcon Moscow to talk about this more - I am going to present some more thoughts on this topic there.

I have published several posts with the aim to deliver one message - sometimes it is more efficient (fast and convenient) to change the application under test (make it more testable or eliminate the need for testing at all) then invent or employ complicated test automation techniques to check the same functionality. Even though there was a lot of misunderstanding caused by badly forming those posts, I still think I stroke something deeper. For instance, this twitter post made me think that we speak two different languages: That's like asking a pharma company to self-certify that their drugs are safe without any independent approval! #softwaretesting #CIO — Ayush Trivedi (@ayushtrivedi) 4 September 2017 Now it started to seem to me that there're two different views on what test automation is. First, probably prevailing point of view is that test automation is a part of ages-old traditional QA process, where test automation specialist is just a test specialist usin...