Is Automated Testing on the UI Worth the Effort?

Automated UI Testing

Is it really worth it to spend time and effort on Automated UI Testing?

I’m using Selenium WebDriver to automate my testing of a web application, which is an e-commerce application.

When I run my automated tests, there are always many failures related to UI changes, slowness of the application or problems with third-party services.

Our web application communicates with at least 7 different third-party web services. In most cases the, selenium automated scripts face intermittent failures.

It is very annoying as one time it passes and next time it fails. It is not possible to get a green build that I can rely on consistently.

So, given all the problems I face during automated UI testing, are they really worth the effort?

Automated Testing whether on UI or otherwise should provide value to be worthwhile.

If you find yourself spending more time fixing broken tests and maintaining scripts than writing meaningful tests, then you need to change your strategy in how you automate and run your tests.

Many agile teams seem to prefer UI level automation or think that such level of testing is necessary to prove the required business functionality.

However, within six to nine months after starting this effort, they will soon realize that the cost of maintaining UI level tests is higher than the benefit they bring.

Many have thrown away the tests at that point and effectively lost all the effort they put into them.

We should either avoid UI level test automation or at least keep them to the bare minimum. But if you really have to do significant Automated UI Testing, then here is how to do it right to keep the cost of maintenance low.

Three Levels of Automated UI Testing

A very good idea when designing UI level functional tests is to think about describing the test at these three levels:

  • Business rule/functionality level: what is this test demonstrating or exercising? For example, free delivery is offered to customers who order two or more books.
  • User interface workflow level: how does a user interact with the UI, on a higher activity level? For example, put two books in a shopping cart, enter address details, verify that delivery options include free delivery.
  • Technical activity level: what are the technical steps required to exercise the functionality? For example, open the shop homepage, log in with “test_user” and “test_password”, go to the “/book” page, click on the first image with the “book” CSS class, wait for the relative page to load, click on the “Buy now” link… and so on.

Why would you automate a test?

Test Automation Pyramid

I would use the “Inversion of the test automation pyramid” principle, where you focus the majority of your tests at the unit layer.

If you have web services you can test the application logic at that layer. Then, there would only be a handful of automated end-to-end tests on the UI.

That way you have the majority of your tests automated with minimum maintenance as there is no UI involved and they run much quicker.

The problem with running automated tests through the UI is you have to wait for the browser to launch.

Secondly, most modern web applications have third-party tracking tags which could slow down the page load.

The aim of automated UI tests is to find major application issues quickly. But because they take a long time to complete, we may not get quick feedback. In most cases, major issues are spotted straight away by someone manually testing. We don’t have to wait for automated tests to tell us something is wrong when we could see the issue straight away.

 

One Reply to “Is Automated Testing on the UI Worth the Effort?”

  1. Hi,
    automated tests need to stay in snyc with the development of production code. It makes no sense to automate the application after the production code has been finalised.

    Also the tool could be a problem: Only using Selenium because it is widely used in the industry and you do not have to pay for it is not a good selection. Looking for tools meeting the needs is much more effective: e.g. I have used Selenium many years and still using it but we also have introduced another one which costs a little but is able to get a better AJAX- and timeout-handling and does not react sensitive on fast GUI changes because it does NOT use the same mechanisms the developer use (CSS, XPATH) to get their DOMs build – it is more independent which is not a bad solution for a testing tool.

    Yours,
    Jogi

Leave a Reply