When To Use Automation Testing And When To Do Manual Testing

What’s Testing?

Software testing has become an essential part of software development, where reliability, quality and safety of a program are the core pillars of good output. The more programs integrate into our daily life the greater is the demand for quality assurance. Hence, quality assurance is the process of ensuring customers that the developed app has no defects or other issues. Unfortunately, there is no 100% certainty some errors will be spotted out when the testing is done with the help of a human, so there are automated testing tools to do the job and satisfy the customer. In order to ensure an application works accurately, QA engineers perform either manual or automation testing. Both manual testing and automation testing carry out the same mission but require different approaches towards testing itself. So, the question pops up – what is manual testing and automation testing, where and when is it used?

Manual Testing

Manual testing is a process of software testing that spots various defects and errors during a test suite. It is performed by a QA engineer, who runs test cases and analyzes the functionality of a program from the user’s perspective to make sure there are no bugs or unexpected program failures. Moreover, testers create test reports manually. During this kind of testing, QA engineers use no particular software automation tools.

Before applying to manual testing, it is necessary to understand how to do manual testing. Thus, it is important to analyze the requirements or software documentation, create a specific test plan, write test cases based on documentation requirements, review these test cases, execute them, report bugs (if there are some!) and re-run test cases to exclude bug repetitions. Generally, there are four stages of manual testing:

• unit testing is mostly done by developers to test each individual component within an app and ensure their correct functioning
• functional testing is executed to verify that the requirements of a client were met in the system functional
• integration testing that focuses on several different components of an app, normal functioning of these components, and their ability to operate together
• acceptance testing that is done internally by employees and externally by the end-users as the final stage before the app release

Based on these stages manual testing is divided into three types:

• white box testing, which requires familiarity with internal code and app structure (control flow testing, data flow testing, path testing, etc.)
• black-box testing that allows testers to test program functioning from the end-user perspective without understanding the code itself
• grey-box testing, which is a combination of the first two types

Some of the tools used in manual testing are Google Apps (Google Drive, Google Spreadsheets, etc.), JIRA and plugins (e.g. Zephyr), Test Link and Browser Dev Tools.
Usually, manual testing is carried out to find critical errors in the newly-developed apps before automating the testing process of these apps. Of course, everything, which is done manually, requires extra time, cost and effort and becomes boring when frequently repeated. It is not effective in the long run if to take ROI into account. In addition, not every app can be tested manually if multiple troubleshooting and test cases are needed. That’s when to automate testing.

Automation Testing: How to Automate Testing

Automation testing is the process of software testing with the help of software tools that is faster, more accurate and efficient. Here, test scripts are done by a computer that runs them on its own and at any time – even during the nights! Thus, QA engineers might concentrate on tasks that are more important at the moment and enjoy the testing process. For example, it is impossible to perform 10,000 checks of Facebook logins and activities on the platform at once manually. However, automation testing can do the job. If test scripts are automated, there is no need for additional manual check-ups. What is more, due to automation testing the ROI improves.

Before implementing automation testing and finding methods how to automate test cases it is also necessary to make up your mind what test cases should be automated, perform early and frequent testing to ensure better results, choose the right tool for automated testing of the app, make a smart division of the automated testing tendency, create test data of excellent quality, and develop automated tests that easily adjust to the UI changes.
So, when to automate tests? Here, the benefit of automation test cases lies in the number of test scripts to be run and repeated. Hence, automation is required when:

• multiple build tests require frequent repeating
• tests are prone to human-based error
• tests with numerous data sets
• app functionality of frequent usage has a high risk of error
• tests cannot be performed manually
• tests, located on different computers, run on different software platforms
• tests will be non-efficient to carry out manually

Numerous Automation Testing Tools exist and QA engineers use them to perform different kinds of testing activities, e.g. web application testing, API Testing Tools and Performance Testing Tools.
Despite the fact that automation testing decreases manual working hours, it does not substitute manual testing completely. It’s the type of test execution needed that really counts in the choice of a testing method. So, manual testing vs automation testing – what’s the difference?

Manual testing

• is done both by a human and a computer
• takes more time and employee input
• allows random testing
• is cheaper at the beginning and becomes costlier in the long run
• cannot be accurate due to the human factor
• is unchanged during the SDLC even if there are some changes in software
• is an investment in QA engineers
• is not cost-efficient when there is a big amount of regression tests
• is beneficial when you need to receive feedback about the user-friendliness of a system tested
• cannot test performance manually
• has no parallel execution
• cannot be batched
• requires no programming knowledge
• requires direct test execution setup
• is repetitive, boring and not without errors
• does not carry out Build Verification Testing (BVT)
• risks to miss deadlines often
• needs to work according to precise processes, checklists or guidelines
• has no training value
• is useful during often AUT changes, in Usability Testing, Exploratory testing and Adhoc Testing

Automation Testing

• is done merely by software tools
• is faster and less time-consuming
• has no random testing
• is costlier at the beginning and becomes beneficial with ROI increase within time
• is performed by tools and testing scripts, which have no specific needs
• needs to be adjusted towards sudden changes to carry out the expected analysis of software
• is an investment both in tools and QA engineers
• is not cost-effective when there is a little amount of regression tests
• provides test results whenever you want
• is inapplicable in case you want to find whether the system tested is user-friendly
• has performance testing via Spike testing or Load testing, etc
• allows parallel execution, which impacts time reduction
• can be batched
• requires initial programming knowledge
• has a relatively easy test execution setup
• is done by tools, which make it extra accurate
• can be applied to when there is a number of test cases that should be run frequently or occasionally
• carries out Build Verification Testing (BVT)
• does not miss deadlines and has no risks to miss them
• uses process accelerating frameworks such as Hybrid, Data Drive, and Keyword
• is valuable for automated unit test case training
• is used for Performance testing, Regression testing, Load testing or anywhere where test cases should be repeated multiple times.

What Is ROI and Why Is It So Important?

Return on Investment (ROI) is mostly used by businesses that desire to know their key performance indicator (KPI) and forecast the future profit of the current expenses and contributions. ROI shows statistical data of future successes based on the right business decisions. ROI calculation gives a better understanding of business solutions and whether your business decision-making is successful and brings profit. For example, if you would like to calculate the ROI of the automation process you should take into account:

• possibility of documenting the process
• wise decision-making
• how many full working hours are dedicated to the process by one employee?
• the total expense per one employee in USD
• quantity of steps the process requires
• what % of the time it takes to perform testing
• how many apps assist in the process completion?

Based on the given information, the biggest ROI will be achieved when the process is:

• documented
• has partial rational decision-making
• involves 5 or more QA engineers with high pay rates
• has 10 or fewer test steps
• takes up to 80% of testers’ time to complete the process
• uses a minimum of 1 additional app.

Here, your ROI will be 357% in a year, 379% in three years, and 383% in five years. In comparison, if the process is:

• documented
• has to be based on rational decision-making
• involves 1 QA engineer to do the job per average compensation
• requires to perform up to 50 steps
• has only 20% of the time dedicated to carrying out testing
• uses 5 or more additional apps

Your ROI will be 7% in a year, 15% in three years and 19% in five years, respectively. So, the choice is on you!

Best Practices in Our Company

At Inoxoft we understand what is best for our clients and plan application test execution thoroughly. Taking full responsibility for a successful app development, our QA team provides the best quality assurance services.
As the performance of an app is affected directly by the choice of the software testing methods, we suggest automation testing as an end-to-end testing because:

• it is cost-effective in the long run
• it speeds up the development and allows to do more work within less time
• it eliminates risks of human errors
• it is more efficient and more accurate

Having the knowledge of coding skills and working with automation frameworks and tools, our QA engineers know how to automation test cases and design test scripts. Therefore, the choice of applying Manual testing or Automation testing depends on the final goal you’d like to achieve.