Run automated tests from Test Plan

To run automated tests from Test Plan you need to use a dedicated operation called Run automated tests. This operation lets you trigger automated tests from integrated CI Servers in the Test Plan. After choosing the operation, you can select the appropriate CI server from which the tests are to be run, and after all the tests are completed, the results will be pulled to the Test Plan as Test Cases with the appropriate statuses. You can run automated tests without or with additional parameters.

Running tests without additional parameters configured

Steps

To use the Run automated tests operation:

1. Click the Run automated tests operation from the Test information panel or select from menu More. After selecting the operation, a dialog box will appear where you can select the tests to run.

   

2. In the Server name option select the appropriate CI Server from the list.

   

3. Select the automated tests you want to run. Depending on the selected CI Server, the name of this option will change to:
   • Build plan for the Bamboo CI server build plans
   • Job for Jenkins CI server jobs

     

4. In the Import results to option you can select which iteration to import the test results for.

   

   You have two options available:
   • Current iteration - Creates Test Cases in the current iteration in Test Plan. Select this option if you are running automated tests for the first time in the Test Plan or want to run them in the same iteration.
   • New iteration - Creates Test Cases in the new iteration in Test Plan. Select this option if you want to run automated tests in the next iteration. In this option, you can also set a different iteration name than the default one.
     Note

     If there are no Test Cases created for the Test Plan yet, selecting this option will not allow you to create a new iteration and the results of the automated tests will be imported into the current iteration. In the same case, if you change the iteration name, it will not be renamed.

     Info

     To be able to rename iterations during the Run automated tests operation, you must configure an additional testFloImportResultsParameters parameter on the Jenkins side. See more at Jenkins integration. For Bamboo CI Server no additional configuration is needed.

5. In the Processing test results you can choose how to process the Test Cases:

   

   • Update and create new Test Cases - Updates existing Test Cases and creates new Test Cases based on the results of the automated tests. Select this option:
     • If you are running tests for the first time on a Test Plan and you want to create Test Cases.
     • If you run the tests again and the scope of the tests on CI Server has changed and you want to update the existing Test Cases but also create new tests.
   • Update existing Test Cases - Updates existing Test Cases based on the results of the automated tests. Select this option:
     • If you are running the tests again and want to update only the existing Test Cases on the Test Plan.
6. After selecting the appropriate options, click Run.
7. You have started the process of creating Test Cases. In the Test Cases panel, you will see the progress of Running CI Automated Tests. The status of running tests you can track on the Automated test runs page.

   

   Note

   You can execute the same test run twice or more within a given Test Plan without having to wait until the first test run is completed. All pending test runs are queued in the Activity tab on the Automated test runs page.

   Note

   While automated tests are running other operations in the Test Plan are blocked to avoid affecting the import of test results.

Result

After Run automated tests operation is complete the Test Cases will be created on the Test Plan with the appropriate test names and statuses. The connection between the created Test Cases and the automated tests is stored in the Automation Test Key. If the creation of Test Case Templates is enabled then for each Test Case a Test Case Template will be created as well. In the Test Case, the key of the Test Case Template will be visible in the TC Template field.

Running tests with additional parameters configured

Running tests with additional parameters configured will give you the possibility to add more information to the build and parametrize it according to special requirements. Parameters can be set in the project settings on the Test Automation tab and later use while the Run automated tests operation is triggered.

Steps

To run automated tests with additional parameters:

1. Click the Run automated tests operation from the Test information panel or select from menu More. After selecting the operation, a dialog box will appear where you can select the tests to run.

   

2. In the CI server section select the Server name and job/plan that you want to run.

   

3. The configured parameters will appear in the CI server parameters section. If a parameter is configured as editable you will be able to change it. Parameters that cannot be edited are configured as read-only. Parameters that are configured as hidden are not shown on the screen but will be sent after running the operation.

   

4. Click Next.
5. In the Import results to select if you want to import results to Current iteration or New iteration. If you pick the New iteration option you can change the iteration name or leave the default one.

   

6. In the Processing test results you can choose how to process the Test Cases:
   • Update and create new Test Cases - Updates existing Test Cases and creates new Test Cases based on the results of the automated tests. Select this option:
     • If you are running tests for the first time on a Test Plan and you want to create Test Cases.
     • If you run the tests again and the scope of the tests on the CI Server has changed and you want to update the existing Test Cases but also create new tests.
   • Update existing Test Cases - Updates existing Test Cases based on the results of the automated tests. Select this option:
     • If you are running the tests again and want to update only the existing Test Cases on the Test Plan.
7. After selecting the appropriate options, click Run.
8. You have started the process of creating Test Cases. In the Test Cases panel, you will see the progress of Running CI Automated Tests. The status of running tests you can track on the Automated test runs page.

Result

After Run automated tests operation is complete the Test Cases will be created on the Test Plan with the appropriate test names and statuses. All values of the configured parameters TestFLO has sent to the given build of the selected job/plan.

Example

Example of parameters sent to the Jenkins job:

Running tests with Selected Test Cases Parameter configured

The configured Selected Test Cases Parameter will allow you to select only those Test Cases that you want to run again in the current or new iteration. This will save you a lot of time by avoiding the need to re-run the whole Test Plan when all you need is to re-run just a few failed Test Cases.

Steps

To run automated tests with a selection of Test Cases:

1. On a Test Plan on which you have run automated tests at least once, click the Run automated tests operation.

   

2. In the CI server section select the Server name and job/plan that was run on this Test Plan.

   

   Note

   Remember that the autocomplete suggests data from recently run tests. It does not necessarily have to be the same CI Server and job/plan, so you need to change these data to those that were run on this Test Plan. Otherwise, even if you select Test Cases and run the wrong job/plan, no Test Cases will be imported into the Test Plan.

3. Click the slider next to the Select Test Cases to run option.

   

4. In the Test Cases list, select the ones you want to run again. You can filter the Test Cases list using text, statuses and iteration.

   

5. Click Next.
6. In the Import results to select if you want to import results to Current iteration or New iteration. If you pick the New iteration option you can change the iteration name or leave the default one.

   

   Note

   When the Select Test Cases to run option is enabled, TestFLO does not allow the creation of new Test Cases, therefore the Update and create new Test Cases option is unavailable and the entire Processing test results option is inactive because TestFLO only updates existing Test Cases on the Test Plan.

7. Click Run
8. You have started the process of updating Test Cases. In the Test Cases panel you will see the progress of Running CI Automated Tests. The status of running tests you can also track on the Automated test runs page.

Result

The selected Test Cases were rerun on the CI server and their results were sent to the Test Plan and updated.