Test Effectiveness
Concept
Test effectiveness is based on the verdict of test cases which have been imported into the squore project. There is no difference between unit or integration tests. Both type "Unit" and "Integration" tests are handled in the same way.
Test Effectiveness represents the ratio of passed tests and should help the team to act on the acceptance of the test campaign which has been executed.
As test cases are linked to other artifacts (Requirements, source code functions), Test effectiveness should indirectly impact other work products and can be a strategic KPI to monitor.
In complex product architecture, the same test case could be executed more than one. For instance, deploying a test case on multiple OS, integrating a test within multiple ECU configuration etc… Thus, Squore will import Test Case Execution artifacts which will be used to provide the test effectiveness for each variant of the system.
In case the test executions are imported, Squore will provide a similar KPI called "Test Execution Effectiveness" based on the test execution verdicts.
Formula
Test Effectiveness = #Passed Tests / #Tests
Test Execution Effectiveness = #Passed Executions / #Executions
"Test Case Effectiveness" Vs "Test Execution Effectiveness"
Squore allows tracking multiple executions of the same test case. Rating of the test case is aggregated over the test execution verdicts.
In the following example, test case verdict is failed because at least one of its executions is failed. However, the test execution effectiveness is rather good (80% of passed executions).
Below, another sample where all test cases are failed while some executions are actually passed.
Test Verdict Overload
Squore offers an interactive mechanism to overload the test case verdict and/or a test case execution verdict. Overload can be tracked down in various way (highlight, graphic, filters and report). Thanks to the overload, the user can complete the review of his project prior to external communication.
