Frameworks and tools

For each type of output file of automated tests, Test Cases will be created in a dedicated manner. Below you will find a description of how Test Cases are created from interpreted test results.

Cucumber

1. For the Cucumber type, each test scenario from the feature file is represented as a Test Case in the Test Plan. Test Cases can have one of two statuses: Pass or Fail.
   Note

   The Fail status for the Test Case will appear when at least one of the steps in the Steps field is failed.

   

2. The Summary of the Test Case consists of the feature name and the scenario name separated by a colon.

   

3. In the Steps field, each Cucumber step is presented as a separate row (step). The keywords Given, When, Then, And are bolded.

   

4. Steps statuses are based on the Cucumber step status mapping in the project configuration.
   Info

   Learn more about status mapping on the Test Automation project configuration page.

   Note

   When changing step statuses is disabled, the Jenkins and Bamboo plugins can still update them when importing Cucumber results.

5. If the feature file contains parameters then each parameter set for a scenario becomes a new step group. The group name is the name of the scenario followed by a scenario index number coming from the Cucumber output file.

   

6. For better identification of failed tests, a error log appears in the given test step as an attached text file. The file name contains the name of the iteration in which the error occurred.
   Note

   If you rerun the same failed Test Case in the same iteration, the attached file in the step will be:

   • overwritten if the result of the next run fails again
   • deleted if it passes.

   If you rerun the same failed Test Case in the next iteration then all files (error logs) from previous iterations are preserved.

   

7. Test Case is linked to a given run/build on the CI Server.

JUnit

1. For the JUnit type, each test in the class is represented as a Test Case in the Test Plan. Test Cases can have one of two statuses: Pass or Fail.

   

2. The Summary of the Test Case consists of the class name and the test method name.

   

3. For better identification of failed tests, a stack trace for error appears in the Description field.
4. Test Case is linked to a given run/build on the CI Server.

TestNG

1. For the TestNG type, each test in the class is represented as a Test Case in the Test Plan. Test Cases can have one of two statuses: Pass or Fail.

   

2. The Summary of the Test Case consists of the class name and the test method name.

   

3. For better identification of failed tests, a stack trace for error appears in the Description field.
4. Test Case is linked to a given run/build on the CI Server.

NUnit

1. For the NUnit type, each test in the class is represented as a Test Case in the Test Plan. Test Cases can have one of two statuses: Pass or Fail.

   

2. The Summary of the Test Case consists of the class name and the test method name.

   

3. For better identification of failed tests, a stack trace for error appears in the Description field.
4. Test Case is linked to a given run/build on the CI Server.
5. If the tests are parameterized, each parameter in given test is imported as a separate Step. Step name consists of the last part of the test name with the parameter.
   Example

   For example a test with a full name attribute equal to DeduceTypeArgsFromArgs<Double,Double>(100.0d,42).TestMyArgTypes, Step name will contain only TestMyArgTypes(5,7)