Example 2: Adverse Conditions
This web page presents details specific to running tests in adverse conditions. A general explanation of the screens is given on Example 1: Ordinary Execution.
To run a test in adverse conditions using the basic algorithm, select the algorithm Yes, all events in same, and one or more event types. This will trigger each of the selected event types at every injection site that is reached.
Note: some event types are experimental. For testing the life-cycle, use Pause-Resume, Pause-Stop-Restart, or Pause-Stop-Destroy-Create.
The image below shows the result of executing the test testCalculatorPlus in adverse conditions (specifically, the Pause-Stop-Destroy-Create event, which is often triggered during device orientations).
Contrary to in ordinary test executions (see Example 1: Ordinary Execution), the test fails.
By clicking on the test in the aggregation overview report (above), an individual test report incl. the specific failure is shown.
In this case, an assertion checking that 2+2=4 failed, because the Pause-Stop-Destroy-Create event caused the calculator display to be reset (as can be seen from the snapshot).
From the above image it can be seen that testCalculatorPlus lead to two test executions with ID's aggressive-0 and aggressive-4, respectively.
This is because Thor re-runs tests in case of failures, potentially revealing additional failures later in the same test (see the paper "Systematic Execution of Android Test Suites in Adverse Conditions").
Here, the ID's are interpreted as follows:
-
aggressive-0: adverse conditions were injected from injection site #0 -
aggressive-4: adverse conditions were injected from injection site #4
From this we can conclude that testCalculatorPlus fails due to an injection of Pause-Stop-Destroy-Create at injection site #0, #1, #2, or #3 (or multiple of them). A specific injection site that causes the problem can be detected by (Failure Isolation)[https://github.com/cs-au-dk/thor/wiki/Example-3:-Failure-Isolation].
