Run automated tests

After configuring the CI server you can run automated tests in two ways:

• using the Run automated tests operation in the Test Plan
• directly from the CI Server using the appropriate parameters

Run automated tests operation in Test Plan

The Run automated tests operation lets you trigger automated tests from integrated CI Servers in the Test Plan. After choosing the operation, you can select the appropriate CI server from which the tests are to be run, and after all the tests are completed, the results will be pulled to the Test Plan as a Test Cases with the appropriate statuses.

Steps

To use the Run automated tests operation:

1. Click the Run automated tests operation from Test information panel or select from menu More.

   

2. Select appropriate CI Server from the list.

   

3. Select the automated tests you want to run. Depending on the CI server selected in the previous point, an option will be displayed:

   • Build plan for the Bamboo CI server build plans
   • Job for Jenkins CI server jobs

     

4. Select TC Creation Strategy:

   • Update and create new - Updates existing Test Cases and creates new Test Cases based on the results of the automated tests. If Test Plan doesn’t contain any Test Cases the strategy will only create new Test Cases.
   • Update existing - Updates existing Test Cases based on the results of the automated tests.

     

5. Select Import results to:

   • Current iteration - Creates Test Cases in the current iteration in Test Plan.
   • New iteration - Creates Test Cases in the new iteration in Test Plan. If in the Test Plan no Test Cases exist then new iteration will be not created.

6. Click Run.

7. You have started the process of creating Test Cases. In the Test Cases panel you will see the progress of Running CI Automated Tests. The status of running tests you can track on the Automated test runs page.

   

   Note

   You can execute the same test run twice or more within a given Test Plan without having to wait until the first test run is completed. All pending test runs are queued in the Activity tab on the Automated test runs page.

   Note

   While automated tests are running other operations in the Test Plan are blocked to avoid affecting the import of test results.

Result

After Run automated tests operation is complete the Test Cases will be created on the Test Plan with the appropriate test names and statuses. The connection between the created Test Cases and the automated tests is stored in the Automation Test Key. If the creation of Test Case Templates is enabled then for each Test Case a Test Case Template will be created as well. In the Test Case, the key of the Test Case Template will be visible in the TC Template field.

Running tests directly from the CI server as a part of the CI process

As an alternative to Run automated tests operation in Test Plan it is also possible to use automatic job execution, e.g. after changes in code repository are pushed. This approach however requires specifying default parameters in the plugin configuration and created Test Plan in Jira.

Jenkins

Steps

To run automated tests from Jenkins as a part of the CI process:

1. Go to the Configure option in the configured job in Jenkins.
2. In the configured parameters fill in the following values in Default Value:

   

   • testPlanKey - the key of the previously created Test Plan, on which Test Cases will be created based on tests from the selected job.
   • targetIteration - tells whether to add Test Cases to current iteration in Test Plan or to create new in Test Plan. You can use these options:
     • CURRENT_ITERATION
     • NEW_ITERATION
   • testCaseCreationStrategy - tells whether new Test Cases should be created with test results, or only limited to updating existing Test Cases in Test Plan. You can use these options:
     • CREATE_AND_UPDATE
     • UPDATE_EXISTING
3. Click Save.
4. Build a job with specified parameters.

Result

The Test Cases will be created in the Test Plan with the appropriate test names and statuses.

Bamboo

Steps

To run automated tests from Bamboo as a part of the CI process:

1. Go to the Configure plan option in the configured plan in Bamboo.
2. Select the job in the Stages tab where the TestFLO - Test results publisher task was added.
3. Select the task and fill the parameters with the following values:

   

   * testPlanKey - the key of the previously created Test Plan, on which Test Cases will be created based on tests from the selected job. * targetIteration - tells whether to add Test Cases to current iteration in Test Plan or to create new in Test Plan. You can use these options: * CURRENT_ITERATION * NEW_ITERATION * testCaseCreationStrategy - tells whether new Test Cases should be created with test results, or only limited to updating existing Test Cases in Test Plan. You can use these options: * CREATE_AND_UPDATE * UPDATE_EXISTING
4. Click Save.
5. Run plan with specified parameters.

Result

The Test Cases will be created in the Test Plan with the appropriate test names and statuses.

Running tests directly from CI Server manually

Another alternative to Run automated tests operation in Test Plan you can run tests directly from CI Server manually. This operation will require you to create a Test Plan in Jira first and provide the required parameters in the selected job/build plan.

Jenkins

Steps

To run automated tests directly from Jenkins manually:

1. Go to the configured job in Jenkins.
2. Click the Build with Parameters option.

   

3. Fill the parameters using the following options:
   • testPlanKey - the key of the created Test Plan, on which Test Cases will be created based on tests from the selected job.
   • targetIteration - tells whether to add Test Cases to current iteration in Test Plan or to create new in Test Plan. You can use these options:
     • CURRENT_ITERATION
     • NEW_ITERATION
   • testCaseCreationStrategy - tells whether new Test Cases should be created with test results, or only limited to updating existing Test Cases in Test Plan. You can use these options:
     • CREATE_AND_UPDATE
     • UPDATE_EXISTING
4. Click Build to build a job with specified parameters.

Result

The Test Cases will be created in the Test Plan with the appropriate test names and statuses.

Bamboo

Steps

To run automated tests directly from Bamboo manually:

1. Go to the Configure plan option in the configured plan in Bamboo.
2. Click the Variables tab.

   

3. Add Variable name and fill Value using the following options:
   • testPlanKey - the key of the created Test Plan, on which Test Cases will be created based on tests from the selected job.
   • targetIteration - tells whether to add Test Cases to current iteration in Test Plan or to create new in Test Plan. You can use these options:
     • CURRENT_ITERATION
     • NEW_ITERATION
   • testCaseCreationStrategy - tells whether new Test Cases should be created with test results, or only limited to updating existing Test Cases in Test Plan. You can use these options:
     • CREATE_AND_UPDATE
     • UPDATE_EXISTING
4. Click Run plan to build a plan with specified variables.

Result

The Test Cases will be created in the Test Plan with the appropriate test names and statuses.