A QA-guy blog

Intro I turn on my TV-set. I start my favourite TV application to watch a TV-show. It says there's a new version and insists on updating. Would I have access to new TV-shows or movies after this update? Not at all! Would this application work faster after that? Hardly. Would it be more stable? Hopefully, but no guarantees. What would this update give me? New UI (I was OK with the old one). Ability to choose which trailer I would like to watch (like I need more than one). It eats my internet traffic and time and gives me nothing of value in turn. I need to sort out my finances. I take my cell phone. I start an accounting application that works with my bank. It wouldn't start. Connectivity issue - it says. In reality - what I need is to go to Google Play and update the application. After the update, it looks slightly different, has some new feature I don't need and would hardly use and obfuscates previously learnt path to the features I need. The problem Both...

Some time ago I had an argument with another team member - he was claiming that Test-driven development (TDD) was not applicable to our application. He thought it would be more efficient to rely on manual or E2E testing instead. I disagreed. Without going too much into the details, the core of the application we were working on was a graph-traversing algorithm with some amount of mathematical calculations. It was painful to test on a system level as the number of possibilities was huge and it was not easy to put the system into a particular state to check specific conditions. I thought that thorough unit-test coverage, and, ideally, TDD would be more efficient, but was not able to persuade my colleague. At the end of the day, I started to doubt if I was right. Challenge was accepted, so I tried to do something more or less similar (though simplified) - implementing a Tic Tac Toe computer player logic via as pure TDD as possible. Long story short, after some time I managed to g...

Choosing between writing one UI tests and 100 Unit test I would always prefer the second option. Why? Well, I am actually lame with UI testing, but that not the main reason. The only thing I worry about is speed. How fast we can change software? Quality - I don't care about it much cause quality standards are contextual and can change. The only thing that is certain - there're going to be changes and we most likely don't know what those changes are about. So there's only one thing that matters - how fast we can adapt to this change. Internal quality is something that actually makes us able to change fast. UI tests do not contribute to internal quality. UI tests may even make things worse. That is the reason. And I invite you to Testcon Moscow to talk about this more - I am going to present some more thoughts on this topic there.

I have published several posts with the aim to deliver one message - sometimes it is more efficient (fast and convenient) to change the application under test (make it more testable or eliminate the need for testing at all) then invent or employ complicated test automation techniques to check the same functionality. Even though there was a lot of misunderstanding caused by badly forming those posts, I still think I stroke something deeper. For instance, this twitter post made me think that we speak two different languages: That's like asking a pharma company to self-certify that their drugs are safe without any independent approval! #softwaretesting #CIO — Ayush Trivedi (@ayushtrivedi) 4 September 2017 Now it started to seem to me that there're two different views on what test automation is. First, probably prevailing point of view is that test automation is a part of ages-old traditional QA process, where test automation specialist is just a test specialist usin...

Preface: I am in the middle of writing a series of posts about test automation frameworks architecture. I am still going to continue that series, even though this posts kind of devaluate the whole test automation framework concept a bit. Sorry for that, just can't stop ranting. Looking through the internet I spotted a couple of posts where some test automation specialists were talking about "page object" "pattern"/"model", as it was something special they had invented. Well, I also have something to say about "page object". “Java test automation engineers were told that there are different patterns than a PageObject” ( http://classicprogrammerpaintings.com/post/153817288474/java-test-automation-engineers-were-told-that ) "Page object" may be described as a pattern that allows us to decouple things you can do with the web page (external interface describing test/business logic) from the real implementation code you will h...

So what it means to have effective test automation? Let's say I can automate 10 000 tests a day. Impressive number, isn't it? Does it mean that my test automation effective? Hardly we can tell without knowing other things. Turns out that test automation effectiveness has little to do with how many tests you can automate a day. To judge effectiveness we need to think about what value test automation adds. That leads me to a thought that test automation may be really the wrong term. Test automation is an activity, not a product. Even worse, the product that test automation creates (the value it produces) may be created by other means, too. What value test automation creates? Faster execution of 70 000 UI test scenarios? Don't think so. Nice test result report? Well, maybe, but not necessarily. My favourite one, "elimination of human error in testing". I will not even bother to comment. So the value it creates? My best guess nowadays would be that testing is a...