Perceived quality level of a software may have dropped, but testing is not the answer.

-

"Modern software is of a lower quality that it was in a past".

Maybe. Perceived quality of software may have decreased, but I don't think that "more testing" is a proper solution.

More testing does not mean more quality

More testing may find more issues, but not necessarily. Simply spending more time on the activity does not mean results would be better. And somebody needs to fix bugs, test bug fixes.
So we can't tell that more testing means more quality but certainly means bigger costs.

Software quality != No bugs

Bugs matter, but that's hardly the only factor to measure quality level. There are a whole bunch of other things that matter: UX, number of features, documentation, price, delivery model, cute logos...
Spending more time on testing may mean spending less time on these things.

Adequate quality level

So it is obvious that one can't spend 100 years testing every possible case. I think that each product has some Adequate level of quality.
The adequate quality level is different for open-source/free/expensive-as-hell products. Test resources and budgets are (and should be) influenced a lot by the context. If bugs that drive you crazy are not getting fixed - that may be a conscious decision, not a lack of testing.


Software is getting cheaper and cheaper, prices dropped. Most of the software we use now is either cheap or free. It would be naive to expect that average quality level would be the same.

Instead of a conclusion

I would love to know your answers to these questions:
  • Would we prefer software with no bugs, which hasn't have features we need?
  • Or reasonably buggy and cheap/free software which has features we need?
  • Or maybe a software with no bugs, but which we cannot afford?

At the and of the day - it is all about money.




Comments

Post a Comment

Popular posts from this blog

Test automation framework - do I need it at all?

-
Image
While there're lots of information published on the internet about how one should develop/buy a Test Automation solution, there's an elephant in the room - which is the fundamental question "Do I need to do it at all?". That is a good question. While I think that there’re some legitimate contexts where product/project/team can benefit from having a sophisticated test automation solution/framework, my general rule of thumb is “If you need a framework - you do it wrong” . There’re at least several decent papers showing the benefit of unit-testing ([1],[2], [3]), while I could not find any research proving that system level tests could bring better results. Please let me know if you can prove me wrong. There’s at least one half-scientific paper showing the benefits of unit tests in comparison to integration/system level tests [4] There’s a good paper bringing a notion of running/writing and maintaining test vs. the benefit of having/running it at...
Read more

Efficient test automation - Secret #1 | Test Club

-
Image
It seems everyone talks about efficient test automation, but not so many people have actually seen it! To cover this I decided to make several videos regarding the secrets of efficient test automation. Today we will talk about the first (and, arguably, the main) secret - realistic expectations. Very often, test automation projects or initiatives are started with the following unrealistic expectations: "Test automation will replace 'manual' testing" "Test automation will provide fast results and benefits" "Test automation efficiency depends solely on test automation engineers" Unfortunately, all those expectations are not realistic! Watch the video to know more!
Read more

An alternative to ubiquitous UI-level checking - Subcutaneous tests

-
Image
Let's assume a hypothetical situation - you were assigned to a project to help with the "test automation" initiative. You have a huge "test plan" as an input, containing hundreds (if not thousands) of "test cases" and you need to do something about it, quick. Problem statement Your first urge (if it is a Web application) may be to write some UI-level automated checks using tools like, you know, Selenium. In fact, there's a huge demand for such "Selenium test automators" in the industry these days. But please, please, don't do this. There're lots of things which make UI-level automated checking the least desirable approach: UI-level automated checks will be slow. There's just no way to avoid it. You can parallel them or do some other tweaks to speed them up somehow, but they still will be slow. UI-level automated checks will be flaky. Partly - because they're slow. Partly because Web browser and UI interface w...
Read more