- Introduction
- About
- Onboarding
- Getting Started
- Demo backup
- User's guide
- Projects' organization
- Test design
- Organizing tests
- Test planning
- Test execution
- Requirements
- Operations
- Reporting and tracking
- Import/Export TCT
- Export test issues
- Issue search dialog
- TestFLO menu
- Admin's guide
- Project configuration
- TestFLO settings
- Test Automation configuration
- TestFLO Permissions
- Custom fields
- Advanced workflow configuration
- Panel Filter Result
- Upgrade reports
- Upgrade guide
- Integrations
- Confluence
- REST API
- Configuration Manager for Jira
Run automated tests
After configuring the CI server you can run automated tests in two ways:
- using the Run automated tests operation in the Test Plan
- directly from the CI Server using the appropriate parameters
Run automated tests operation in Test Plan
The operation will be displayed in the Test Plan when the given project will be configured in the CI Servers configuration.
The Run automated tests operation lets you trigger automated tests from integrated CI Servers in the Test Plan. After choosing the operation, you can select the appropriate CI server from which the tests are to be run, and after all the tests are completed, the results will be pulled to the Test Plan as a Test Cases with the appropriate statuses.
To use the Run automated tests operation, the user must have Jira Create and Edit issues permissions.
The Run automated tests operation in Test Plan can be restricted to certain users/groups/roles in the TestFLO Permissions.
Steps
To use the Run automated tests operation:
-
Click the Run automated tests operation from Test information panel or select from menu More.
-
Select the automated tests you want to run. Depending on the CI server selected in the previous point, an option will be displayed:
-
Select TC Creation Strategy:
- Update and create new - Updates existing Test Cases and creates new Test Cases based on the results of the automated tests. If Test Plan doesn’t contain any Test Cases the strategy will only create new Test Cases.
- Update existing - Updates existing Test Cases based on the results of the automated tests.
-
Select Import results to:
- Current iteration - Creates Test Cases in the current iteration in Test Plan.
- New iteration - Creates Test Cases in the new iteration in Test Plan. If in the Test Plan no Test Cases exist then new iteration will be not created.
-
Click Run.
-
The process of creating Test Cases has started. The progress of Running CI Automated Tests will be displayed in the Test Cases panel.
NoteWhile automated tests are running other operations in the Test Plan are blocked to avoid affecting the import of test results.
Result
By default, the Automation Test Key custom field is not present in the Test Case and Test Case Template view. For more information, see Automation Test Key.
If the creation of Test Case Templates is enabled then you can reuse tests once again. You can only add selected Test Case Templates to the Test Plan to limit which test results are to be imported. You can also link the Test Case Templates and their executions in Test Cases with Requirements to track their connection in TestFLO Reports.
You can also create a Test Case Template first and then add it to the Plan Test and use Run automated tests. This approach will require knowledge of the special key for the test from CI Server, which is stored in the field Automation Test Key in the Test Case Template and Test Case.
Automation Test Key sample value:
[package name].[class name].[method name]
com.deviniti.jira.plugins.steps.StepsServiceTest.shouldSetStepsComment
Running tests directly from the CI server as a part of the CI process
As an alternative to Run automated tests operation in Test Plan it is also possible to use automatic job execution, e.g. after changes in code repository are pushed. This approach however requires specifying default parameters in the plugin configuration and created Test Plan in Jira.
Jenkins
Steps
To run automated tests from Jenkins as a part of the CI process:
- Go to the Configure option in the configured job in Jenkins.
- In the configured parameters fill in the following values in Default Value:
- testPlanKey - the key of the previously created Test Plan, on which Test Cases will be created based on tests from the selected job.
- targetIteration - tells whether to add Test Cases to current iteration in Test Plan or to create new in Test Plan. You can use these options:
- CURRENT_ITERATION
- NEW_ITERATION
- testCaseCreationStrategy - tells whether new Test Cases should be created with test results, or only limited to updating existing Test Cases in Test Plan. You can use these options:
- CREATE_AND_UPDATE
- UPDATE_EXISTING
- Click Save.
- Build a job with specified parameters.
Result
The Test Cases will be created in the Test Plan with the appropriate test names and statuses.
Remember to refresh the Test Plan view to see Test Cases after you have finished running the tests on the CI Server.
Bamboo
Steps
To run automated tests from Bamboo as a part of the CI process:
- Go to the Configure plan option in the configured plan in Bamboo.
- Select the job in the Stages tab where the TestFLO - Test results publisher task was added.
- Select the task and fill the parameters with the following values:
- Click Save.
- Run plan with specified parameters.
Result
The Test Cases will be created in the Test Plan with the appropriate test names and statuses.
Remember to refresh the Test Plan view to see Test Cases after you have finished running the tests on the CI Server.
Running tests directly from CI Server manually
Another alternative to Run automated tests operation in Test Plan you can run tests directly from CI Server manually. This operation will require you to create a Test Plan in Jira first and provide the required parameters in the selected job/build plan.
Jenkins
Steps
To run automated tests directly from Jenkins manually:
- Go to the configured job in Jenkins.
- Click the Build with Parameters option.
- Fill the parameters using the following options:
- testPlanKey - the key of the created Test Plan, on which Test Cases will be created based on tests from the selected job.
- targetIteration - tells whether to add Test Cases to current iteration in Test Plan or to create new in Test Plan. You can use these options:
- CURRENT_ITERATION
- NEW_ITERATION
- testCaseCreationStrategy - tells whether new Test Cases should be created with test results, or only limited to updating existing Test Cases in Test Plan. You can use these options:
- CREATE_AND_UPDATE
- UPDATE_EXISTING
- Click Build to build a job with specified parameters.
Result
The Test Cases will be created in the Test Plan with the appropriate test names and statuses.
Remember to refresh the Test Plan view to see Test Cases after you have finished running the tests on the CI Server.
Bamboo
Steps
To run automated tests directly from Bamboo manually:
- Go to the Configure plan option in the configured plan in Bamboo.
- Click the Variables tab.
- Add Variable name and fill Value using the following options:
- testPlanKey - the key of the created Test Plan, on which Test Cases will be created based on tests from the selected job.
- targetIteration - tells whether to add Test Cases to current iteration in Test Plan or to create new in Test Plan. You can use these options:
- CURRENT_ITERATION
- NEW_ITERATION
- testCaseCreationStrategy - tells whether new Test Cases should be created with test results, or only limited to updating existing Test Cases in Test Plan. You can use these options:
- CREATE_AND_UPDATE
- UPDATE_EXISTING
- Click Run plan to build a plan with specified variables.
Result
The Test Cases will be created in the Test Plan with the appropriate test names and statuses.
Remember to refresh the Test Plan view to see Test Cases after you have finished running the tests on the CI Server.