Requirement Test Coverage

Concept

This indicator allows to track the requirements test coverage. A requirement is "covered" if the 2 following conditions are true:

  • There is at least one test which is linked (imported via the ".vtc-tso")

  • All the linked tests have a successful verdict

Requirement Scope

Target the requirements which are in line with your current test execution plan.

SWAN req out of scope
Figure 1. Grey zone represents the requirement which are "out of the scope"

The "out of the scope" definition could be defined during project creation.

Main Requirements Metrics

  • Total Requirements: Raw import from ALM tools

  • Requirements "To Test" = Requirements which are taken into account for Rating System

  • Requirements "With Tests" = Requirements which are linked to at least 1 test

  • Requirement "Test Coverage" = Percent of Passed Tests linked to each Requirement

SWAN req metrics graph
Figure 2. Requirements trend chart sample

Setting Up the "To Test" Attribute for a Requirement

Objective

Reduce the scope of the requirements to rate in order to focus only on the requirements that matters in regard to the current state of the development/analysis.

How to Set?

There are different ways to define the requirements "To Test"

  • "All applicable": basically all requirements are in the "To Test" focus

  • "IADT": only requirement with IADT attribute define to "TEST" are in the focus

  • "With Test": Any requirement which is linked to a test is integrated in the "To Test" focus.

  • "With Executed Test": Any requirement which is linked to an executed test is part of the "To Test" focus.

  • "External Definition": Planned Requirements list is provided to Squore (=list is a subset of the vti-tso export from ALM tool)

SWAN req setting to test
Figure 3. Requirements settings

Track the Coverage Trend

SWAN req tests trend
Figure 4. Requirements Test Coverage trend

Visualizing Requirements Covered via Test User Overload

A test execution imported as "failed" can be manually overloaded by the end user. This allows to change the behavior of the rating system, turning red into light green.

SWAN req overload
Figure 5. Requirements overload

The overload mechanism is fully integrated in the workflow review. The user can change the test verdict but:

  • He has to provide a justification message

  • All justifications can be exported to ALM environment

SWAN req table
Figure 6. Requirements table

KPI: Review Progression

Squore tracks the elements (Requirement or Test) which shall be reviewed. The manual overloads are automatically integrated in this review progression.

SWAN review distribution
Figure 7. Review progress

Here is a sample of the review workflow.

SWAN review workflow
Figure 8. Workflow overview