Test Automation project configuration

The Test Automation tab appears in the project configuration when Jira Administrator adds a CI Server into the CI Servers configuration and sets the given project. On this page, you can decide if you want to create Test Case Templates along with the creation of Test Cases and set statuses mapping for Cucumber test results.

Steps

To get to the Test Automation tab:

1. Click on Projects in the main bar in Jira and select a project from the project list where you run the automated tests.
2. Go to Project Settings and click on Test Management from the left side menu.
3. Select the Test Automation tab.

Result

View of the Test Automation tab.

Creating Test Case Templates from test results

This option gives you the possibility of enabling the creation of Test Case Templates in the selected project. The Test Case Templates are created together with Test Cases which are based on automated tests. The benefits of this approach are:

• Created Test Case Templates can be reused - you can only add selected Test Case Templates to the Test Plan so you can limit which test results are to be imported.
• Test Case Templates and their executions in Test Cases can be linked with Requirements and tracked in TestFLO Reports.

To enable the creation of Test Case Templates select the Create in current project or Create in another project. Available options:

• Don’t create - Test Case Templates will not be created. This option is marked as default.
• Create in current project - Test Case Templates will be created in the current project.
• Create in another project - Test Case Templates will be created in selected the project from the list.

After you set the creation, when you run automated tests, the Test Case Templates will be created along with the Test Cases. If the Test Case Template has already been created from the source Test Case, they will not be created again if the Run automated tests operation is used again.

CI job custom parameters

This option is associated with the Run automated tests operation on the Test Plan and is used to send additional parameters to a build on the CI Server. By setting additional parameters you can:

• add more information to a given build,
• make the script execution on the CI Server side dependent on the needs of a given team or users, version, component, or other factors,
• select for rerun only those Test Cases which you are interested in (without importing all results of automated tests).

Add parameters

In this option, you can manually set the additional parameters whose values TestFLO will send to the CI Server during the Run automated tests operation on the Test Plan.

Steps

To add parameters:

1. Click the Add button.

   

2. In the Display name provide a clearer parameter name that the user will see when using the Run automated tests operation.

   

3. In the Parameter name provide the exact technical parameter name.

   

   Note

   In order to be able to transfer parameter values to the Jenkins CI Server, each parameter configured here must also be added to the given job in Jenkins. In the case of the Bamboo CI Server, there is no need for additional configuration.

4. In the Field type select type of the parameter. You can choose from three types:

   

   1. Text field - plain text field
   2. Select field - a field with a selection list
   3. Jira field - field from Jira. Supported fields: Number field, Text field (single line), Radio buttons, Select list (single choice and multiple choices)
5. In the Field mode decide in which mode the parameter will be used:

   

   1. Editable - allows users to edit the value of a parameter when using the Run automated tests operation
   2. Read-only - displays the default value of a parameter when using the Run automated tests operation
   3. Hidden - the set parameter value will not be displayed when using the Run automated tests operation. This is useful for purely technical parameters.
6. In the Value(s) enter the default value (for Editable field mode) or the target value (Read-only and Hidden field mode).

   

   Fill in the values for the given field types:
   1. Text field - fill the field with a text value.
   2. Select field - define the options that can be selected from the list. To configure options, click the pencil next to the field. On the Manage select field options screen, add a Label and Value for the option. Click the Add button. Add as many options as you need and click Save.
   3. Jira field - select a Jira field from the list. Remember that this must be a field that is configured for the Test Plan.
7. Save the changes by clicking the Checkmark.
8. To add more parameters, repeat steps 1 through 7.

Result

You have added a parameter/s. All parameters that you configured with the Editable option will appear for users to change when using the Run automated tests operation. Parameters that you have configured as Read-only will appear to users but they will not be able to change them. The Hidden option will not display the set parameters at all.

After configuring the parameters, you can manage them:

1. Use drag & drop to set the order in which the parameters are displayed.
2. To edit, click the Pencil icon.
3. To delete, click on the Bin icon.

Selected Test Cases Parameter

This parameter is responsible for the Select Test Cases to run option when using the Run automated tests operation. With this option, you can select from the existing Test Cases only those that you want to run again within the same or a new iteration.

The parameter stores the key values of selected Test Cases by the user during the Run automated tests operation. These values are the connection keys between each Test Case and the test on the CI Server, which are stored in the Automation Test Key field on the Test Case.

Steps

To configure the Selected Test Cases Parameter option:

1. Click slider next to Selected Test Cases Parameter.

   

2. Optional: In the Parameter name you can change the name of the parameter that stores connection keys.
3. Configure a string parameter on the Jenkins CI Server for the jobs on which you want to use the Select Test Cases to run option. The parameter name must be the same as in the Parameter name field.
   Note

   For the Bamboo CI Server you do not need to configure this parameter additionally.

Result

The configuration is complete. From now on, an additional option Select Test Cases to run will appear on the Run automated tests operation dialog, where you can decide which Test Cases to run again in the same or a new iteration.

Cucumber step status mapping

In this option, you can map the statuses that TestFLO receives in the Cucumber test results file to the statuses in the Steps field on the Test Case. There are three statuses from Cucumber to map Passed, Failed, and Skipped. You can assign each of these statuses a corresponding status in the Steps field. If you do not assign any status then when you import the test results TestFLO will assign the default steps status.

NUnit step status mapping

With this option, you can map the statuses that TestFLO receives in the test results file of the parameterized NUnit tests to the statuses in the Steps field in Test Case. There are three statuses from NUnit tests to map Passed, Failed, and Skipped. You can assign each of these statuses a corresponding status in the Steps field. If you do not assign any status then when you import the test results TestFLO will assign the default steps status.