Software Testing: When To Automate & When Not To

In the past, automated testing was difficult to set up, time-consuming to perform and inconsistent reports were generated.

But since then, tremendous advancements have been achieved, and now the decision NOT to automate testing can be seen as incredibly stupid. We assume this is due to two changes: first, the shift to review closer to the code using unit checks and evaluation of services, and secondly, the availability of more accurate testing methods.

This does not imply, though, that anything needs to be automated through QA Testing Tools. It can be a drawback to healthy tech growth to have so many tests. Tests that take ages to run longer decrease the input developers get when they submit their code, thus slowing down the whole phase of development.

And tests that are patchy or take too long to resolve lead to a loss of faith in the testing process, which implies that they will not be performed at all or that their findings will be overlooked.

Thus, understanding what tests to automate and what not to automate is the very first step in the progress of automation.

With that in mind, let’s dive into some of the do’s and don’ts of automation testing.


Automate Things Users do

What is your software’s main purpose? When using your app, what is the usual direction that a user would take? These are all the kinds of functions that automation tests can provide. You can schedule your automation test to do a particular task, instead of going along a manual user testing route each morning, and if there is an error, you will be alerted automatically.


Automate Routine Activities 

Some tests are so critical that they’ll need to be carried out repeatedly. The login test is a prime example of this: if your users are unable to log into the app/software, you have a serious customer service problem! Yet, again and again, no one wants to waste hours manually signing into an app/software. Automating the login assessments guarantees that a large spectrum of users and identities, both legitimate and invalid, are checked for authentication.


Automate activities as near as possible to the code

Unit tests are so critical because, without affecting any dependence, they exercise code functionality. If a software developer makes a modification that ends in a logic gap, it can find the hole before the change would make it to the tester, preserving precious time for everyone. TDD, or Test-Driven Development, is a common practice today. In this process, the developer writes the unit tests before they write the code, meaning that before addressing the technological problems of writing the code, they start by thinking about all potential use cases for the software.

Service testing is also highly useful because, without going into the UI, it can measure the software’s capabilities. Most apps of today use APIs that request the server to make REST requests. There are several limits on what a user can do by moving through the UI, but further test cases can be performed by going straight through the API. For instance, a UI could restrict the type of characters that can be inserted in a form area, but the input might not be checked on the server-side, which could be uncovered by API testing.


Avoid Automation With Tests That May Become Flaky

If you do not find a way to run an automatic test through QA Testing Tools on a component and make it pass reliably, you may want to manually run the test or find another way to assert the performance. For example, if you need to verify that a function sent an email and that the email was also received at the other end, then you’ll find it difficult to test email clients in the UI. Why? Because there’s no specific function to determine how long it takes until the email is sent. Rather, you have to ensure that the email company delivered the email, and for this purpose, it is better to perform a manual inbox search to ensure the email is received.


Don’t Automate Bugs you are Positive will Never be Seen Again

Some bugs are just temporary, and it is unlikely that they will appear again. The typo gives a clear example of this. Sometimes due to oversight if a developer actually put the text that said “Clal us” instead of “Call us. No developer will ever go into the code and return to the previous misspelling, so it is not appropriate to automate a test to validate the text.


Avoid Automating Test Cases That May Never Occur 

In tech, there will always be glitches and errors, but some are more likely to occur than others and thus be noticed by consumers. You may be intrigued by the error that is triggered by heading to a particular series of pages, inserting non-UTF-8 characters, and then pressing the Back button three times in a row, but as this is very unlikely to be triggered by an end-user, creating an automatic test for it is not worth your time. 


DON’T automate checking for functionality nobody cares about

Your software actually runs on a wide range of browsers, and on all of them, your instinct might be to run your tests. But it may be that on a certain browser, only 1 percent of users run your software.

Why go through all the hassle of having to run different forms of tests on this browser if that’s the case? Likewise, if there is a functionality that will be discontinued quickly in your framework and only 1% of your customers use it, your time will be best spent on the ability to automate another feature.


Final Words

When performed right, automated tests through QA Testing Tools provide developers with quick input, warn testers of issues well before they hit development, and open up testers to do more exploratory testing. But when automation is performed incorrectly, it results in evaluations that for all are not trusted and waste time.

  • About Me

  • Duke Brighton. Today I’ve got a great partner, a beautiful daughter, a stable job in finance and a fun side hustle in e-commerce. It wasn’t always like that though. I struggled for years and always seemed to make the wrong choices of what to do and whose advice to take. Late in my 20s, I found the right mentor and everything changed. I learned there are no shortcuts and if it sounds too good to be true, it probably is.

    I don’t know what your situation is like today, but I know there is someone out there who can guide you well. It’s my goal to help make that information accessible.