Test automation framework architecture. Part 2.1 - Layered architecture example

Let's consider an example of what Layered (Tiered) architecture may look like for a test automation framework. As a system under test, we're going to use this simple and neat "Todo list" application.

First, let's do a brief analysis of the application. From my brief exploration I came up with the following checklist:

Area/GroupThings to checkComments
Todos CRUD 1 Create item Basic scenario, UI-level case
1.1 Create an item with the same name The case is not valid. There's no validation for that - nothing to check really.
1.2 Create an item with an empty/not valid/malicious name While it would be a nice thing to test manually, from test automation standpoint I would expect this to be filtered on service or repository level, so either integration or unit test.
2 Mark item complete Basic scenario, UI-level case
2.1 Un-mark item complete Basic scenario, UI-level case
3 Delete item Basic scenario, UI-level case
3.1 Delete item when there're multiple items with the same name Knowing that there's no validation this is a nice scenario to try. However, I don't think that it is a good scenario for UI verification.

Basically what we check there is that some other field (not name) is used for delete query generation (assuming it is stored in DB). Thus it is probably a great scenario for integration or even unit test, depending on the realisation.
Filtering 4 All Basic scenario, may be combined nicely with some other scenario, just to save time, UI-level case
5 Active Basic scenario, UI-level case
6 Completed Basic scenario, UI-level case
Bulk action 7 Delete several completed items Basic scenario, UI-level case
7.1 Delete one completed item Wonder what the difference from the first case? One interesting thing is that it seems the "bulk action" button should appear if we have at least one item selected.

Apart from that deleting one time or 100 items should be just slightly different SQL-queries generated (assuming DB is used), so I would rather not add this as a separate scenario.
8 Select all items Pretty simple pure UI thing to do. Depending on the UI implementation may be a good candidate for JS unit-test
Other 9 Valid item list shown Nothing special. Maybe combined nicely with some other scenario. Pretty minor thing anyway so I would rather not add a separate UI case for that.

Please notice, that  I did checklist first - thus I make test scenarios to drive my framework implementation, not vice versa. I think it is an important premise as our final product are tests. Let us call it "intent-driven test automation design" (or choose/suggest any other meaningful name).

Typical anti-pattern would be to start working on PageObjects first, then on Service/Step layers and only after that on test scenarios. Thus you usually get ugly and unreadable scenarios.

A typical layered framework is divided into three layers - Web (contains Page objects ), Business (contains business abstractions and exposing "steps" abstraction above) and Tests themselves (check this short post for additional info). For a record, I don't like thinking about Tests as a layer of a framework (rather it is a layer of a solution, please check this) but common vocabulary differs from mine.

Traditional three-layered framework/solution

Most test automation frameworks I worked with were "plain code" frameworks, where test scenarios are done using some XUnit library as pure test methods, resembling typical unit tests. The example I have created uses this approach as well, with a major difference - I inversed dependency so my lower level layers depend on a higher layer (this is inspired by Clean Architecture by Uncle Bob and Hexagonal Architecture pattern).


Mapping between layers and code itself:
Test layer/src/main/java/gmail/alexspush/test
/src/test/java/gmail/alexspush/test
Business (Steps/Service) Layer/src/main/java/gmail/alexspush/service
Core Layer/src/main/java/gmail/alexspush/utils
/src/main/java/gmail/alexspush/driver
/src/main/java/gmail/alexspush/page

All in all, this is a good enough solution. Tests are descriptive and tell the reader what they're testing (instead of how they do it). Of course, as you add test scenarios one should be very careful to preserve this communication, but still good enough already. The weakest point of this solution is a standard reporting, but it can be improved using external libraries (or ignored if you run tests often enough so you know a source of failure without reporting)


Github link: https://github.com/senpay/layered-test-framework-example

Traditional three-layered framework/solution + Serenity for reporting

A good compromise between "Vanilla BDD" and classical test automation would be using tools like Serenity. Without changing much of implementation I was able to add Serenity to the layered framework to produce readable and descriptive HTML-report.

The downside of this was that I had to use Serenity build in dependency injection mechanism, which killed my idea with interfaces. I am sure it can be worked around, but I decided that it was not necessary to prove a point - there's no much difference between frameworks disregard of what reporting/test layer specifics one choose.

Mapping between layers and code itself:
Test layer/src/main/java/gmail/alexspush/test
/src/test/java/gmail/alexspush/test
Business (Steps/Service) Layer/src/main/java/gmail/alexspush/service
Core Layer/src/main/java/gmail/alexspush/utils
/src/main/java/gmail/alexspush/driver
/src/main/java/gmail/alexspush/page

Github link: https://github.com/senpay/layered-test-framework-example-serenity

Vanilla BDD three-layered framework/solution powered by Serenity and Jbehave

The main difference of this in comparison to previous examples is that steps now have to worry about test state (while in two previous examples test state was handled by test code itself). Another drawback is the inability to run a single test instead of a full suite (not a problem if you have good CI though).

All in all steps in a Business Layer were changed only cosmetically and JBehave annotations were used to map Gherkin scenarios to steps.

The mapping between layers and code itself:
Test layer/src/test/resources/stories
Business (Steps/Service) Layer/src/main/java/gmail/alexspush/service
Core Layer/src/main/java/gmail/alexspush/utils
/src/main/java/gmail/alexspush/driver
/src/main/java/gmail/alexspush/page

Github link: https://github.com/senpay/layered-test-framework-example-serenity-jbehave

I hope that these were useful examples - I have a couple of other nice examples in my Github, feel free to check them as well. 

Comments

  1. Great info, thanks for sharing around and also keep me updated with upcoming ideas

    ReplyDelete
  2. Techsaga Corporations is an Operational automation strategy makers company. The benefits of automation create expectations for customers, stakeholders, and employees; the effectiveness of automation allows us to maintain them. As IT environments increase in complexity and the amount of data, devices, systems, and platforms expand.

    ReplyDelete

Post a Comment

Popular posts from this blog

Test automation framework architecture. Part 2 - Layered architecture

Efficient test automation - Secret #1 | Test Club