Six ways to fix flaky Web UI tests

Test automation at the Web UI level (you know, Selenium stuff) is usually a pain in the arse. It is usually also the only test-automation approach people are aware of/interested in (for whatever reasons). Below are several suggestions on how one can make her/his UI test automation at the Web UI level less painful and flaky.
1. Retry failed steps Each test consists of several steps. Because of various reasons, those steps from time to time fail with no apparent reason. Retrying failed step may save your test from failing.

Helps with:
Unresponsive, slow UIElements being shown with delayWrong moon phase [1]Drawbacks:
Sometimes it is just waist of time, especially when real issue is being "retried"Even worse: sometimes it may hide real issue that would go away after some time or page refresh 2. Retry failed tests From time to time - odds are just against you with a specific test run. Everyone, who ever had a misery to work with Web UI-test automation has seen this. Sometimes if yo…

Tic Tac Toe computer player algorithm via TDD

Sometime ago I had an argument with another team member - he was claiming that Test-driven development (TDD) was not applicable to our application. He thought it would be more efficient to rely on manual or E2E testing instead. I disagreed.

Without going too much into the details, the core of the application we were working on was a graph-traversing algorithm with some amount of mathematical calculations. It was painful to test on a system level as number of possibilities was huge and it was not easy to put system into a particular state to check specific conditions.

I thought that thorough unit-test coverage, and, ideally, TDD would be more efficient, but was not able to persuade my colleague. At the end of the day, I started to doubt if I was right.

Challenge was accepted, so I tried to do something more or less similar (though simplified) - implementing a Tic Tac Toe computer player logic via as pure TDD as possible.

Long story short, after some time I managed to get the result …

Test automation framework architecture. Part 2.2 - Two-layered architecture

As we've learned in previous posts of the series - Layered (also known as Tiered) architecture is a de-facto standard of the industry.

There're two major variations of this architectural pattern that one should be aware of: 3-layered architecture (described here) and 2-layered architecture. This post is going to concentrate on 2-layered architecture.

2-layered architecture is typically seen in UI-specific test automation frameworks. The key feature of this pattern is absence of Business-layer, so most of business-related logic are actually to be put into Page Objects.

In order to avoid "The God class" anti-pattern and still be able to contain business-related logic in Page Object, one usually uses libraries or frameworks that allow to simplify DOM-elements location, thus letting Page Objects to concentrate less on locating elements and more on business-logic itself.

The good thing in this architectural pattern is that it actually allows you to write less code (and le…

Agile, testing and more frequent deliveries.

Yet another time I think I see people using word Agile wrong. Yet another guy claimed "we need to have more predictability and stability so I think we would go for Agile methodologies, Scrum in particular".


I don't think that "predictability, stability, whatevershitability" has anything to do with Agile. If you start Agile transformation with such goal you've already failed.
Agile is about maximizing value delivery rate in a state of uncertainty, that's pretty much what it is.

If your business is uncertain as hell why do you think that "predictability" with engineering teams is going to fix things? It is not.
Worse, if you make engineering teams predictable while having unpredictable environment, you will miss the whole point and make things worse while still thinking you're doing fine.


Agile team is about removing management layers between engineering and customer, not making management happier.

Having an abi…

Test automation framework architecture. Part 2.1 - Layered architecture example

Let's consider an example of what Layered (Tiered) architecture may look like for a test automation framework. As a system under test we're going to use this simple and neat "Todo list" application.

First let's do a brief analysis of the application. From my brief exploration I came up with the following check list:

  Area/Group    Things to check    Comments  Todos CRUD1 Create itemBasic scenario, UI-level case1.1 Create item with same nameCase is not valid. There's no validation for that - nothing to check really.1.2 Create item with empty/not valid/malicious nameWhile it would be nice thing to test manually, from test automation stand point I would expect this to be filtered on service or repository level, so either integration or unit test.2 Mark item completeBasic scenario, UI-level case2.1 Un-mark item completeBasic scenario, UI-level case3 Delete itemBasic scenario, UI-level case3.1 Delete item when there're multiple items with the same nameKnowing …

Why I would prefer writing 100 unit test to one UI test

Choosing between writing one UI tests and 100 Unit test I would always prefer second option. Why?

Well, I am actually lame with UI testing, but that not the main reason.

The only thing I worry about is speed. How fast we can change a software? Quality - I don't care about it much cause quality standards are contextual and can change. The only thing that is certain - there're going to be changes and we mostlikely don't know what those changes are about. So there's only one thing that matters - how fast we can adapt to this change.

Internal quality is something that actually makes us able to change fast. UI tests do not contribute to internal quality. UI tests may even make things worse.

That is the reason.

And I invite you to Testcon Moscow to talk about this more - I am going to present some more thoughts on this topic there.

Manual and automated testing confusion

I am getting tired of post about how "automated testing" going to replace "manual testing".

Let me offer you a simple analogy. I have a car that have lots of useful self-check lights, like "check engine", "low gas", "low battery", etc. Automated tests are similar to those lights - if light is on, most likely something is wrong and I need mechanic to look at my car. Does having those self-check lights let me not to visit mechanic for human-driven check yearly? They don't. Fact that self-check light is not on does not mean my car is OK, does it?

Areas covered lots with self-check may require less time to check (as there lower risk they would be broken). However there are lots of areas that is not feasible to cover with self-checks. There're some car parts I am not even aware of, while mechanic knows they weak points and can find issue in a couple of minutes.

It is possible to have only automated checks and not have mechanics to loo…