mutate: view results based on a subset of the tests

[christoffer-nylen]

The user would like to see the mutation testing result from a specified set of tests, so that he/she can:

1. Verify tracing from software requirements -> test name -> covered source code.
2. Verify integration testing adequacy (data-flow and control flow) from high-level black-box tests.

Example:

• All tests that match the test name: 'test_suite.req_880.check_out_of_range' or 'test_suite.hlr_black_box_tests.*'

Suggestions

Approach 1: Using dextool mutate admin removeTestCase

cp my_mutation_db.sqlite3 my_mutation_db.sqlite3.bak
dextool mutate admin --operation removeTestCase --invert-match -e 'test_suite.foo*' -e 'test_suite.bar*'
dextool mutate report --style html --mutant lcr --mutant lcrb --mutant sdl

Approach 2: Using dextool mutate report

dextool mutate report --style html --mutant lcr --mutant sdl --testData 'test_suite.foo*' --testData 'test_suite.bar*'

Approach 3: Using the web gui

Add a box-item to activate/deactive contributions from a specific test.

[joakim-brannstrom]

should probably, actually implement both because they have a wider application than only this use case.

[christoffer-nylen]

1. Verify tracing from software requirements -> test name -> covered source code.

Example:

Assume we have:

• A sut that contains 6649 lines of code.
• A requirement Req-880: When a given parameter x is out-of-range, the SUT shall return BAD_DATA.
• A test test_suite.req_880.check_out_of_range that verifies this.

The user would like to check traceability between Req-880 and the source code where this is implemented.

Problem:

• Where in the code is the out-of-range calculation performed?
• Where does it set the return status to BAD_DATA?

Can mutation testing show the sut source code that is killed by test_suite.req_880.check_out_of_range ?