Apps documentation
TestFLO - Test Management for Jira
TestFLO - Test Management for Jira
8.8.x
8.7.x
8.6.x
8.5.x
8.3.x
8.2.x
8.1.x
8.0.x
FAQ
Release notes
Last updated Nov 17, 2020

Run automated tests

After configuring the CI server you can run automated tests in two ways:

  • using the Run automated tests operation in the Test Plan
  • directly from the CI Server using the appropriate parameters

Run automated tests operation in Test Plan

Note

The operation will be displayed in the Test Plan when the given project will be configured in the CI Servers configuration.

The Run automated tests operation lets you trigger automated tests from integrated CI Servers in the Test Plan. After choosing the operation, you can select the appropriate CI server from which the tests are to be run, and after all the tests are completed, the results will be pulled to the Test Plan as a Test Cases with the appropriate statuses.

Warning

To use the Run automated tests operation, the user must have Jira Create and Edit issues permissions.

Info

The Run automated tests operation in Test Plan can be restricted to certain users/groups/roles in the TestFLO Permissions.

Steps

To use the Run automated tests operation:

  1. Click the Run automated tests operation from Test information panel or select from menu More.

    Run automated tests operation in Test Plan in TestFLO - Test Management in Jira
    Run automated tests operation in Test Plan

  2. Select appropriate CI Server from the list.

    Run automated tests operation in Test Plan in TestFLO - Test Management in Jira
    CI Server selection in Run automated tests operation

  3. Select the automated tests you want to run. Depending on the CI server selected in the previous point, an option will be displayed:

    • Build plan for the Bamboo CI server build plans
    • Job for Jenkins CI server jobs
      Run automated tests operation in Test Plan in TestFLO - Test Management in Jira
      Tests selection in Run automated tests operation
  4. Select TC Creation Strategy:

    • Update and create new - Updates existing Test Cases and creates new Test Cases based on the results of the automated tests. If Test Plan doesn’t contain any Test Cases the strategy will only create new Test Cases.
    • Update existing - Updates existing Test Cases based on the results of the automated tests.
      Run automated tests operation in Test Plan in TestFLO - Test Management in Jira
      CI Server selection in Run automated tests operation
  5. Select Import results to:

    • Current iteration - Creates Test Cases in the current iteration in Test Plan.
    • New iteration - Creates Test Cases in the new iteration in Test Plan. If in the Test Plan no Test Cases exist then new iteration will be not created.
  6. Click Run.

  7. The process of creating Test Cases has started. The progress of Running CI Automated Tests will be displayed in the Test Cases panel.

    The progress of Running CI Automated Tests in Test Plan in TestFLO - Test Management in Jira
    The progress of Running CI Automated Tests
    Note

    While automated tests are running other operations in the Test Plan are blocked to avoid affecting the import of test results.

Result

Run automated tests operation in Test Plan in TestFLO - Test Management in Jira
Result of Run automated tests operation in Test Plan
After Run automated tests operation is complete the Test Cases will be created on the Test Plan with the appropriate test names and statuses. The connection between the created Test Cases and the automated tests is stored in the Automation Test Key. If the creation of Test Case Templates is enabled then for each Test Case a Test Case Template will be created as well. In the Test Case, the key of the Test Case Template will be visible in the TC Template field.

Info

By default, the Automation Test Key custom field is not present in the Test Case and Test Case Template view. For more information, see Automation Test Key.

Note

If the creation of Test Case Templates is enabled then you can reuse tests once again. You can only add selected Test Case Templates to the Test Plan to limit which test results are to be imported. You can also link the Test Case Templates and their executions in Test Cases with Requirements to track their connection in TestFLO Reports.

Note

You can also create a Test Case Template first and then add it to the Plan Test and use Run automated tests. This approach will require knowledge of the special key for the test from CI Server, which is stored in the field Automation Test Key in the Test Case Template and Test Case.

Example

Automation Test Key sample value:
[package name].[class name].[method name] com.deviniti.jira.plugins.steps.StepsServiceTest.shouldSetStepsComment

Running tests directly from the CI server as a part of the CI process

As an alternative to Run automated tests operation in Test Plan it is also possible to use automatic job execution, e.g. after changes in code repository are pushed. This approach however requires specifying default parameters in the plugin configuration and created Test Plan in Jira.

Jenkins

Steps

To run automated tests from Jenkins as a part of the CI process:

  1. Go to the Configure option in the configured job in Jenkins.
  2. In the configured parameters fill in the following values in Default Value:
    Example of completed parameters in Jenkins for run automated tests to TestFLO - Test Managemenet in Jira
    Example of completed parameters in Jenkins
    • testPlanKey - the key of the previously created Test Plan, on which Test Cases will be created based on tests from the selected job.
    • targetIteration - tells whether to add Test Cases to current iteration in Test Plan or to create new in Test Plan. You can use these options:
      • CURRENT_ITERATION
      • NEW_ITERATION
    • testCaseCreationStrategy - tells whether new Test Cases should be created with test results, or only limited to updating existing Test Cases in Test Plan. You can use these options:
      • CREATE_AND_UPDATE
      • UPDATE_EXISTING
  3. Click Save.
  4. Build a job with specified parameters.

Result

The Test Cases will be created in the Test Plan with the appropriate test names and statuses.

Bamboo

Steps

To run automated tests from Bamboo as a part of the CI process:

  1. Go to the Configure plan option in the configured plan in Bamboo.
  2. Select the job in the Stages tab where the TestFLO - Test results publisher task was added.
  3. Select the task and fill the parameters with the following values:
    Example of completed parameters in Bamboo for run automated tests to TestFLO - Test Managemenet in Jira
    Example of completed parameters in Bamboo
    * testPlanKey - the key of the previously created Test Plan, on which Test Cases will be created based on tests from the selected job. * targetIteration - tells whether to add Test Cases to current iteration in Test Plan or to create new in Test Plan. You can use these options: * CURRENT_ITERATION * NEW_ITERATION * testCaseCreationStrategy - tells whether new Test Cases should be created with test results, or only limited to updating existing Test Cases in Test Plan. You can use these options: * CREATE_AND_UPDATE * UPDATE_EXISTING
  4. Click Save.
  5. Run plan with specified parameters.

Result

The Test Cases will be created in the Test Plan with the appropriate test names and statuses.

Running tests directly from CI Server manually

Another alternative to Run automated tests operation in Test Plan you can run tests directly from CI Server manually. This operation will require you to create a Test Plan in Jira first and provide the required parameters in the selected job/build plan.

Jenkins

Steps

To run automated tests directly from Jenkins manually:

  1. Go to the configured job in Jenkins.
  2. Click the Build with Parameters option.
    Run automated tests directly from Jenkins manually to TestFLO - Test Managemnet in Jira
    Run automated tests directly from Jenkins manually
  3. Fill the parameters using the following options:
    • testPlanKey - the key of the created Test Plan, on which Test Cases will be created based on tests from the selected job.
    • targetIteration - tells whether to add Test Cases to current iteration in Test Plan or to create new in Test Plan. You can use these options:
      • CURRENT_ITERATION
      • NEW_ITERATION
    • testCaseCreationStrategy - tells whether new Test Cases should be created with test results, or only limited to updating existing Test Cases in Test Plan. You can use these options:
      • CREATE_AND_UPDATE
      • UPDATE_EXISTING
  4. Click Build to build a job with specified parameters.

Result

The Test Cases will be created in the Test Plan with the appropriate test names and statuses.

Bamboo

Steps

To run automated tests directly from Bamboo manually:

  1. Go to the Configure plan option in the configured plan in Bamboo.
  2. Click the Variables tab.
    Run automated tests directly from Bamboo manually to TestFLO - Test Managemnet in Jira
    Run automated tests directly from Bamboo manually
  3. Add Variable name and fill Value using the following options:
    • testPlanKey - the key of the created Test Plan, on which Test Cases will be created based on tests from the selected job.
    • targetIteration - tells whether to add Test Cases to current iteration in Test Plan or to create new in Test Plan. You can use these options:
      • CURRENT_ITERATION
      • NEW_ITERATION
    • testCaseCreationStrategy - tells whether new Test Cases should be created with test results, or only limited to updating existing Test Cases in Test Plan. You can use these options:
      • CREATE_AND_UPDATE
      • UPDATE_EXISTING
  4. Click Run plan to build a plan with specified variables.

Result

The Test Cases will be created in the Test Plan with the appropriate test names and statuses.