Updated on May 7th, 2021 by Martyna Wiśnik

Functional testing consists of many phases which are strictly related to each other. None of the stages can be missed, starting from requirements gathering until the very end: final test results reporting. In order to maintain a continuous process, where all elements are well synchronized, it’s a smart idea to perform the whole project management inside a single tool. Atlassian’s Jira Software makes it possible – thanks to its multiple features, we can successfully track the progress of our software development, including tests, in a single environment. But what if Jira’s basic functionalities are not enough and don’t allow to deliver satisfying reports? This is where we come to help! 

What exactly do we mean by that? A complete and well-done test report should be detailed, clear, standard, andspecific. The ultimate goal of reporting relies on successful communication between the project manager, analysts, testing team members, and developers. Having that in mind, it must consist of information that is to-the-point, but exhaustive and above all transparent for all the stakeholders. It’s best if descriptions of particular objects are short and summarize test results. The included data should follow the standard pattern to preserve consistency. As test reports allow to monitor the progress of software development, they usually constitute their main object of interest and are the first thing they will ask for.

There are four essential elements that a perfect test report should contain:

But how to keep track of all the changes that are implemented during the project? It doesn’t have to be so hard to achieve. Performing your tests in Jira could be revolutionary for your whole software development project. Integrated with dedicated testing tools, Jira can become your number one test management solution. One of the reasons why is the possibility of tracking relations between all kinds of objects throughout the development process. If all data is safely stored in a single place, presenting connections is much easier to do and share with others. This is a facilitation you shouldn’t refuse, and you’ll see both your team members and customers will appreciate the results.

First of all, why should we bring your testing to Jira? There are some must-have features that only our favorite project management tool integrated with a testing app can provide us with. One of them is a great support of reporting and progress monitoring phases. Test management apps focus on making the whole process as intuitive and transparent for the team members as possible. Having in mind both testers’ and managers’ needs, these advantages will speed up the whole process, while preserving its quality. What’s more, if tests are executed in Jira, requirements, test cases, and defects can be seamlessly linked to Jira issues: Epics and user stories, as well as Confluence pages. That makes it possible for all the stakeholders to see the context of each object and understand particular stages better.

Both apps: Requirements and Test Management for Jira (RTM) and TestFLO will bring our testing processes to a new level. Thanks to their integrity with the Atlassian suite, we provide our teams with the familiar environment to work with. Both tools let us gather our requirements, tests and defects altogether. We can also set up test plans, execute them, and verify our progress every step of the way. There is a possibility to check if all the requirements are safely covered with test cases, so we can be sure that nothing’s missed and our company will release a complete, functional product in the end. Thanks to our app’s reporting features, we can track all possible relations between testing objects. At a glance, we will know whether your test cases are well included in test plans and executed during test executions. These options prevent possible bugs, but if they occur anyway – our team will find and fix them at the early stage of development, before they can seriously impact the project.

The Requirements Coverage report in RTM allows us to verify if all the Requirements are covered by related Test Cases, Test Plans, Test Executions, Test Case Executions, and Defects. It gives a possibility to filter the results by Project, Issue type, Fix version, Component, RTM Environment, and Assignee, which for example makes it easy to spot requirements that are the source of too many defects. The flexible view lets us choose which requirements we’d like to display, so we can select only covered or uncovered objects. If there’s some unnecessary information, it can be easily removed from the list – all we have to do is click on the icon in the upper left corner of the page and unselect the metrics we don’t want to have on our report. Requirements Coverage allows us to export all generated data to a .PDF or .CSV file, so we can send it to stakeholders.

The Requirement Test Coverage Report in TestFLO gives us the possibility to track whether the requirements are already covered or uncovered by Test Case Templates for a given version. This feature helps us determine if all the requirements are covered with tests. The report is available once the Jira issues have been selected from the specific project to be considered as requirements. Moreover, we can filter requirements using built-in filters or saved filters in Jira, thanks to the Quick filters function.

The report consists of two statistics: Summary and Coverage level. Summary displays information about the number of covered and not covered requirements. Coverage level section shows the percentage of covered and uncovered requirements. The Requirement Test Coverage Report can be exported to a PDF file. 

The Traceability Matrix displays all types of relations between two baselined issue types on a transparent view, using the many-to-many relationship comparison. In order to be fully functional, data shown on X and Y axes are customizable with JQL, so we can set more sophisticated filters if necessary. Thanks to JQL custom functions, we can display relations from multiple testing and development projects. RTM‘s Traceability report allows checking the total number and types of relationships for each issue. To be more specific, thanks to that functionality, we can:

Data from the Traceability Matrix panel can also be exported to a .PDF or .CSV file. All you have to do is click on the Export icon.

With the Requirement Traceability Report in TestFLO, we can track the relationship between requirements, Test Case Templates, Test Cases, and their execution in given iteration and created defects. This report will help us analyze the overall testing effort for requirements in a version. The report is available once the Jira issues have been selected from the specific project to be considered as requirements. The Requirement Traceability Report can be exported to a PDF file.

Moreover, we can filter the Requirements and Test Cases with built-in filters that refer to the Fix Version field and connection with a given requirement or use saved filters. Each filter list has its own searching categories. The filtering function makes us sure that an appropriate range of data is gathered. Thanks to the special switcher, we can also group requirements by epic.

Requirements and Test Management for Jira (RTM) gives us the possibility to track the progress of Test Case Executions with the dedicated Test Case Execution report. Before we get the final results, we can choose a specific Project, Test Plan, Execution, RTM Environment, TCE Created Date or TCE Assignee for which we’d like to display a pie chart. The generated chart will present to us all the TCE’s statuses. Below it, we’ll find the table with each Test Case and its Result, possible Defects, and other important elements. As in the case of the Traceability and the Requirements Coverage reports, we can export the final document to a .csv file.

Test Plan Execution Report in TestFLO shows the overall progress of Test Plans. Depending on the needs of granularity, we can check the progress based on Test Cases status or Test Case steps progress. Moreover, we have a possibility to filter the results by: Test Plans, Requirements for Test Plans, Test Cases, and Requirements for Test Cases. To check the progress of Test Plan execution based on Test Case status, we can select the Test Case status in the Report based on option. The Test Cases list includes the following data:

If we select the Test Case steps progress in the Report based on option, we can check the progress of Test Plan execution based on Test Case steps progress. The final results consists of the same parameters listed above, and is enriched with Steps progress and Executed steps. Thanks to this feature, TestFLO stands out among test management and test reporting tools.

The Test Execution report in RTM displays the overall progress of testing in selected Test Plans, Test Executions, RTM Environment, TCE Created Date or TCE Assignee, as well as the statuses of particular test cases. If necessary, all detailed information is listed in the table below. The feature allows estimating the time remaining until the whole testing process is finished. After clicking on the Preferences icon, we can choose exactly which status of Test Case Executions needs to be presented, so it will be easier to spot and fix the possible defects yet during tests. The report lets us specify which Test Executions’ progress we’d like to see in order to determine if each one is associated with improvements. With all the reports, TE also can be exported to .csv or .pdf.

The Test Plan Iterations Report in TestFLO shows all test executions from selected Test Plan. With this report, wecan easily analyze the actual state of each Test Case execution in given Iteration. The report consists of two sections: Summary and Test Execution Results. Summary section displays information about the number of Test Plans, Test Cases and its executions, also statuses of all executions. Test Execution Results section shows the percentage progress of test executions.

The columns in the report show the following elements: Test Plan, Test Case, Test Case Summary, Execution passed, and #Number (x%) that shows the iteration number and percentage of passed executions of all Test Cases in given iteration. Each row in the report presents one Test Case and its execution in a given iteration on Test Plan. Each cell represents the status color of the Test Case execution in the given iteration. If the Test Case is not included in the given iteration, the status color is not displayed and the N/A appears in its place.

RTM for Jira lets us display all our Test Execution and Test Case Execution reports on a single screen. The feature which makes it possible is Test Dashboard. To put on the view, after generating a TE or a TCE report, we have to click the Send to Test Dashboard button next to the Preferences icon. Now, we have two options to choose from. The first is sending the selected reports to our customized Test Dashboard (User), which is visible only for the person who created it. The reports added to Test Dashboard (Project) are, on the other hand, possible to see by everyone involved in the project, but can be modified exclusively by the team members with Project Admin permissions. The functionality of gathering the diagrams makes your testing even more transparent and helps you be prepared for all the questions from the customer or other team members in a blink of an eye. 

Although TestFLO doesn’t provide us with a built-in Dashboard, there’s still a possibility to view relevant information. First of all, we need to create a native Jira Dashboard and use features from TestFLO and Jira. TestFLO provides a built-in gadget that reflects the Test Plan Iterations Report, which we can also display in Confluence. This way Test Managers, Stakeholders, Testers, and other users who don’t have access to Jira, gain the ability to see test progress. To put on the view, we need to click the Add gadget button, find the TestFLO – Test Plan Iterations Report gadget and add it to the Dashboard. In the gadget configuration, we have to select the project from which we want to see test results, set the number of results, decide if we want to view the panel with Test Cases, and set refresh interval. After adding, we can filter Test Plans using the built-in filters and those saved in Jira. We can also use Jira gadgets along with data from TestFLO (issue types and custom fields). An example dashboard, apart from the TestFLO, can contain Jira features, such as:

There are many gadgets in Jira that we can use with TestFLO, those listed above show the usage of only a few of them.

When it comes to your customers, reporting can be considered the most important part of the testing process, as it constitutes the proof of effective test management. The evaluation of your team’s work will be certainly based on the reports delivered after each stage of their work. That’s why it’s so important to keep track of every change and be extremely careful about the details. As testing consists of many stages, having everything under control isn’t always easy. For that reason, bringing the whole software development process into Jira integrated with RTM or TestFLO is a smart way to go. You’ll have all your test results gathered in complete test reports, that can be presented to the stakeholders right away. With the support of dedicated reporting features, it won’t cost so much time and money to summarize tests anymore. Now, the significant part of step-by-step analysis can be done for you just in just a few clicks.

Deviniti is an Atlassian Platinum Solution Enterprise Partner and a Platinum Marketplace Partner. We’ve developed two Jira add-ons for requirements and test management, as well as dozens of applications for other purposes available on the Atlassian Marketplace. If you’re interested in gaining more knowledge about Jira Software as a test management solution, check out this video tutorial series on our YouTube.

Read more on bringing test management process inside Jira on our blog: