App page
Requirements and Test Management for Jira
Cloud Server/Data Center
Requirements and Test Management for Jira

Cloud

Server/Data Center

Documentation
FAQ
Release notes
Migration to Cloud
Last updated Dec 27, 2024

Test Frameworks

Integrating your test frameworks with the RTM app guarantees that the test results are automatically imported and properly mapped. This section provides detailed instructions for integrating JUnit and NUnit test frameworks.

Mapping tests to Test Cases

When importing automated tests results RTM can map each test to an existing Test Case in the app. This connection allows to track how the Test Cases are executed by automated tests. Mapping is performed by searching for text patterns in test names or console output. RTM identifies Test Cases based on the steps:

  1. Checks if full test name contains pattern with Test Case issue key, for example given full test name com.rtm.example.CalculatorTest.shouldAddTwoNumbers adding an existing Test Case issue key in RTM anywhere like com.rtm.example.CalculatorTest.shouldAddTwoNumbers_PR_123will map created TCE to this TC, note that only issues in target project will be mapped.
  2. Checks if full test name contains pattern TC_KEY_ for example given full test name com.rtm.example.CalculatorTest.shouldAddTwoNumbers adding TC_KEY with an existing Test Case issue key in RTM anywhere like com.rtm.example.CalculatorTest.shouldAddTwoNumbers_TC_KEY_PR_123will map created TCE to this TC.
  3. Checks if console output contains patterns: TC_KEY=, RTM_AUT_TEST_KEY=.

In addition to embedding RTM keys in test names, you can include additional metadata in your test results by printing specially formatted strings to the console output (e.g., System.out). RTM can parse these outputs using regular expressions to extract metadata such as:

  • Test Case keys
  • Requirement keys
  • Test Case summaries

This approach allows you to enrich your test cases with more information without altering test names, which can be especially useful for parameterized tests or when additional context is needed.

How it works

RTM processes the console output of your tests and looks for specific patterns using regular expressions. When it finds a match, it extracts the value and associates it with the test case in RTM.

Supported patterns

Patterns are in the test name and in the system out. Test Case Key is checked first , then RTM Automated Test Key. In both cases system out is checked first, then test name if not found in system out.

Here are the patterns the RTM uses to extract information from the console output:

  • Test Case Key
TC_KEY=<test_case_key>
  • RTM Automated Test Key
RTM_AUT_TEST_KEY=<automated_test_key>
  • Test Case Summary (system out only)
TC_SUMMARY="<test_case_summary>"

The summary should be enclosed in double quotes.

  • Requirement Keys (system out only)
REQ_KEY=<requirement_key1>,<requirement_key2>,...

Behavior based on configuration and patterns

When you include these patterns in your test code, RTM behaves as follows:

  1. If TC_KE or RTM_AUT_TEST_KEY is present RTM searches for an existing Test Case in the project with the specified key.
  • If found, a Test Case Execution is created based on this Test Case.
  • If not found and Create Test Case option is enabled in the project configuration, RTM will create a new empty Test Case with the specified key and link it to the Test Case Execution.
  • If not found and Create Test Case option is disabled, RTM will create an empty Test Case Execution without linked Test Case.
  1. If the Create Test Case option is enabled and TC_SUMMARY is provided:
  • RTM will create a new Test Case with a given summary.
  • If REQ_KEY is also provided, a new Test Case will be linked to the specified requirements.

Project configuration impact

The project configuration significantly affects the mapping of tests in the app. Below are various examples you may find.

Configuration Result
Create Test Case option disabled No Test Cases containing the automation test key are found. RTM creates an empty Test Case Execution without linked Test Case.
Create Test Case option enabled No Test Cases containing the automation test key are found. RTM creates a new Test Case (empty or with provided summary) and links it to the Test Case Execution.
Searching for Test Case by parameter in test name or logs RTM searches for Test Cases using the TC_KEY parameter either in the test name or in the test logs (console output).
Multiple tests for one Test Case: When multiple automated tests reference the same TC_KEY, RTM associates all corresponding Test Case Executions with that single Test Case.
Need help?

If you can’t find the answer you need in our documentation, raise a support request. Include as much information as possible to help our support team resolve your issue faster.