Automation software test metrics are used to test the performance of the applied automated testing process. These metrics have visibly clear objectives of the automation work. It is somewhat that unswervingly relates to the performance of the effort. It is key to compose a metric that analyses the value of automation, particularly when the automated software testing approach is used for the first time in the project. The test team will need to calculate the time consumed on development and executing test scripts against the results that the scripts created. E.g., test team could match the number of hours to develop & execute the test procedures by the number of defects recognized that would not likely have been exposed during a manual test effort. There are some common test metrics and these metrics do not necessarily just apply to automation, but can be used in some cases, while some of them are most frequently used and are related to general software QA testing.
Software Test automation Metrics mainly covers: Test Coverage, System Coverage Analysis, Test Procedure Execution Status, Error Discovery Rate, Defect Aging, Defect Fix Retest, Current Quality Ratio, Quality of Fixes, Reporting Problems, Test Effectiveness as well as the Test Efficiency. Metric is a measurement criteria that helps to measure past and present performance or predict future performance which helps in improving the Organizational test processes, and consequently help to track their status.
Automated software testing metrics are the metrics used to measure the performance of the carried out automated testing process and related endeavors that are used to conduct this process. In real terms performance computing comprises the present, past and yes the future performances. Automated testing metrics should have clearly defined objective for the automation effort. A superior automated testing matrix should be uncomplicated, measurable, and worthwhile and should be able to pin down the areas of test automation improvement.
There are in-numerous metrics that are being widely used. Below are few of the major metrics:
1. Automation test metric: ATC % = Automation Coverage / Total Coverage
2. Automation progress: AP % = Number of test cases automated / Number of test cases automatable
3. Automation Index: AI or PA % = Number of test cases automatable / Total Number of test cases
4. Defect density: DD = No. Of Known Defects / Size of the Software/Application
5. Defect Aging: DA (in Time) = Defect Fix Date (OR Current Date) – Defect Detection Date
6. Defect trend analysis: DTA = No. Of Known Defects / No. Of test Procedures Executed
Test Metric is simple way to measure and keep track of Organizational test process and status. By using software test automation metric one can reverse the negative impact of complex projects if implemented properly.
No doubt, that automation test metric helps in evaluate and knowing whether the objectives such as quality as well as progress and coverage are being met. With a focused approach to complete these goals, we must have metrics in place which can set standard of measurement to keep track of our projects.
In most cases well-defined Test automation metrics comprises following features:
– It should be clearly measurable
– It should be meaningful
– It should be valid input criteria for Automation improvement.
– Last but not least it should be simple.
With Automation Software Test Metric, we measure relevant efforts and the performance of its process been put. In this we can differentiate unit test automation with integrated or system test.
Implementing Automation software testing with the help of software test metrics can increase quality and test coverage area details. It can drastically lower down the cost of delivery as well as time of QA testing. Organizational test processes can be implemented and improved by test metrics and can help tracking its status.
A clearly defined goal should be for automation software test metrics that defines the performance of the automation effort.
A metrics must following features:-
– Clearly measurable
– Purpose
– Graphical representation
– Time required to run the test
Broadly Test Automation Metrics can be of 3 types
1. Automation Scripting Productivity
It helps in productivity of scripting test automation scripts that can be calculated using following formula
Automation Scripting Productivity = [Operations Performed/Efforts(hours)]Operation(s)/hour
2. Automation Test Execution Productivity Auto. Execution Productivity = [(Total No. Of Automated TC executed (Ate)/Execution Efforts (hours))*8] Execution(s)/Day. Ate is calculated as ATe = Base Test Case + ((T(0.33)*0.33)+(T(0.66)*0.66)+(T(1)*1))
3. Automation Coverage: It tells number of test cases that are automated and how well you are doing to achieve goal. Automation Coverage = [(Total No. Of TC Automated/Total No. Of Manual TC)*100] %
Metrics therefore plays important role in health, progress and quality of an automated software testing effort. Automation Software test Metrics can also be used to track past performance, current and future trends.
So, to summarize Automation Software Test Metric: Automation testing metrics are those metrics which is used to measure the performance of the implemented automated testing process and track its status. As the time proceed, Adding new & improving existing features, bug fixes etc. make the software projects more complex due large coding. This complexity will have the tendency to take more time to test the product, decrease test coverage that influence the quality of the product. Automation Software Test Metric has the capacity to gain an accurate and deep understanding of tracking the automated testing process.
In broader terms: Different Types of Automation Software Test Metric: This can be divide into three types:
1) Coverage: To measure the testing success and scope.
2) Progress: These metrics are mainly concentrate on time i.e. time to test, time to fix the defect etc.
3) Quality: These metrics are mainly focus on measure of value, excellence, worth etc. of testing product.
