Automated Testing on the UI
I’m using selenium to automate my testing of a web application, which is an e-commerce application.
When I run my automated tests, there are always many failures either related to UI changes, but mostly related to slowness of the application and problems with third party services.
Our web application communicates with at least 7 different third party web services and in most cases when I execute the selenium automated scripts I get intermittent failures.
It is very annoying as one time it passes and next time it fails. It is not possible to get a green build that I can rely on consistently.
So, my question is, is it really worth the effort to automate tests that run on the UI given all the issues I face and the maintenance of the tests takes a long time.
Automated Testing whether on UI or otherwise should provide value to be worthwhile.
If you find yourself spending more time fixing broken tests and maintaining scripts than writing meaningful tests, then you need to change your strategy in how you automate and run your tests.
GUI Automation Testing does need more maintenance than say unit or API tests, due to the changes on the UI and locators.
So, the less tests that you have at UI layer the less of maintenance effort.
I would use the “Inversion of the test automation pyramid” principle, where you focus majority of your tests at unit layer, and if you have web services that’s where you can check and test the application logic, and then just have only a handful of automated end-to-end tests on the UI.
That way you have majority of your tests automated with minimum maintenance because no UI is involved and they run much quicker.
The problem with running automated tests through the UI is you first have to wait for browser to launch which can take few seconds. Secondly, most modern web applications have third party tracking tags, Facebook connectivity or communicate with affiliate networks which could slow down the page load.
The aim of automated UI tests is to find major application issues quickly. But because they take a long time to complete, we may not get quick feedback. In most cases, major issues are spotted straight away by someone manually testing, so we don’t have to wait for automated tests to tell us something has gone wrong when we could see the issue straight away.