Functional testing consists of many phases which are strictly related to each other. None of the stages can be missed, starting from requirements gathering until the very end: final test results reporting. In order to maintain a continuous process, where all elements are well synchronized, it’s a smart idea to perform the whole project management inside a single tool. Atlassian’s Jira Software makes it possible – thanks to its multiple features, we can successfully track the progress of our software development, including tests, in a single environment. But what if Jira’s basic functionalities are not enough and don’t allow to deliver satisfying reports? This is where we come to help!
Importance of complete test reports
What exactly do we mean by that? Complete and a well-done test report should be detailed, clear, standard and specific. The ultimate goal of reporting relies on successful communication between the project manager, analysts, testing team members and developers. Having that in mind, it must consist of information that is to-the-point, but exhaustive and above all transparent for all the stakeholders. It’s best if descriptions of particular objects are short and summarize test results. The included data should follow the standard pattern to preserve consistency. As test reports allow to monitor the progress of software development, they usually constitute their main object of interest and are the first thing they will ask for.
There are four essential elements that a perfect test report should contain:
- Project Information containing basic data, like project name and description;
- Test Objective which includes information about test type and purpose;
- Test Summary defining passed, failed and blocked test cases;
- Defects described with priority and current status.
But how to keep track of all the changes that are implemented during the project? It doesn’t have to be so hard to achieve. Performing your tests in Jira could be revolutionary for your whole software development project. Integrated with dedicated testing tools, Jira can become your number one test management solution. One of the reasons why is the possibility of tracking relations between all kinds of objects throughout the development process. If all data is safely stored in a single place, presenting connections is much easier to do and share with others. This is a facilitation you shouldn’t refuse, and you’ll see both your team members and customers will appreciate the results.
Bringing test reporting into Jira
First of all, why should you bring your testing to Jira? There are some must have features that only your favourite project management tool integrated with a testing app can provide you with. One of them is great support of reporting and progress monitoring phases. Test management apps focus on making the whole process as intuitive and transparent for the team members as possibe. Having in mind both testers’ and managers’ needs, these advantages will speed up the whole process, while preserving its quality. What’s more, if tests are executed in Jira, requirements, test cases and defects can be seamlessly linked to Jira issues: Epics and user stories, as well as Confluence pages. That makes it possible for all the stakeholders to see the context of each object and understand particular stages better.
Our brand new app Requirements and Test Management for Jira (RTM) will bring your testing to the new level. Thanks to its integrity with Atlassian suite, you provide your team with the familiar environment to work with. RTM is an intuitive testing tool that lets you gather your requirements, tests and defects altogether. In RTM, you set up test plans as well as execute them. Our app’s features make it possible to verify your progress every step on the way. You can check if all the requirements are safely covered with test cases, so you can be sure that nothing’s missed and your company will release a complete, functional product in the end. Thanks to our app’s reporting features, you can track all possible relations between testing objects. At a glance, you will know whether your test cases are well included in test plans and executed during test executions. These options prevent possible bugs, but if they occur anyway – your team will find and fix them at the early stage of development, before they can seriously impact the project.
Requirements coverage report
The Requirements Coverage report allows to verify if all the Requirements are covered by related Test Cases, Test Plans, Test Executions, Test Case Executions and Defects. It gives a posibility to filter the results by Project, Issue type, Fix version, Component, RTM Environment and Assignee, which for example makes it easy to spot requirements are the source of too many defects. The flexible view lets us choose which requirements we’d like to display, so we can select only covered or uncovered objects. If there’s some unnecessary information, it can be easily removed from the list – all we have to do is click on the icon in the upper left corner of the page and unselect the metrics we don’t want to have on our report. Requirements Coverage allows to export all generated data to a .PDF or .CSV file, so we can send it to stakeholders.
The Requirements Test Coverage report displays your requirements and all the objects related to them
The Traceability Matrix displays all types of relations between two baselined issue types on a transparent view, using many-to-many relationship comparison. In order to be fully functional, data shown on X and Y axes is customizable with JQL, so you can set more sophisticated filters if necessary. Thanks to JQL custom functions, we can display relations from multiple testing and development projects. RTM‘s Traceability report allows to check the total number and types of relationships for each issue. Being more specific, thanks to that functionality, you can:
- track the coverage of Requirements by features and Test Cases,
- make sure that all Test Cases are included in Test Plans and Test Executions,
- verify the Defects linked to features, Test Cases and Executions,
- confirm that Test Plans are completed in Test Executions.
Data from the Traceability Matrix panel can also be exported to a .PDF or .CSV file. All you have to do is click on Export icon.
Traceability Matrix allows to see all the relations between testing objects at a glance
Test Case Execution report
Requirements and Test Management for Jira (RTM) gives you the possibility to track the progress of Test Case Executions with the dedicated Test Case Execution report. Before you get the final results, you can choose a specific Project, Test Plan, Execution, RTM Environment, TCE Created Date or TCE Assignee for which you’d like to display a pie chart. The generated chart will present to you all the TCE’s statuses. Below it, you’ll find the table with each Test Case and its Result, possible Defects, and other important elements. As in the case of the Traceability and the Requirements Coverage reports, you can export the final document to a .csv file.
Test Execution report
The Test Execution report in RTM displays the overall progress of testing in selected Test Plans, Test Executions, RTM Environment, TCE Created Date or TCE Assignee, as well as the statuses of particular test cases. If necessary, all detailed information is listed in a table below. The feature allows to estimate the time remaining until the whole testing process is finished. After clicking on the Preferences icon, we can choose exactly which status of Test Case Executions needs to be presented, so it will be easier to spot and fix the possible defects yet during tests. The report lets us specify which Test Executions’ progress we’d like to see in order to determine if each one is associated with improvements. As all the reports, TE also can be exported to .csv or .pdf.
Test Execution report shows overall testing progress and detailed information
Last but not least… the Test Dashboard
RTM for Jira lets us display all our Test Execution and Test Case Execution reports on a single screen. The feature which makes it possible is Test Dashboard. To put on the view, after generating a TE or a TCE report, we have to click the Send to Test Dashboard button next to the Preferences icon. Now, we have two options to choose from. The first is sending the selected reports to our customized Test Dashboard (User), which is visible only for the person who created it. The reports added to Test Dashboard (Project) are, on the other hand, possible to see by everyone involved in the project, but can be modified exclusively by the team members with Project Admin permissions. The functionality of gathering the diagrams makes your testing even more transparent and helps you be prepared for all the questions from the customer or other team members in a blink of an eye.
Test Dashboard put your Test Execution and Test Case Execution reports altogether
When it comes to your customers, reporting can be considered the most important part of the testing process, as it constitutes the proof of effective test management. The evaluation of your team’s work will be certainly based on the reports delivered after each stage of their work. That’s why it’s so important to keep track of every change and be extremely careful about the details. As testing consists of many stages, having everything under control isn’t always easy. For that reason, bringing the whole software development process into Jira integrated with RTM is a smart way to go. You’ll have all your test results gathered in complete test reports, that can be presented to the stakeholders right away. With the support of dedicated reporting features, it won’t cost so much time and money to summarize tests anymore. Now, the significant part of step-by-step analysis can be done for you just in just a few clicks.
If you’d like to test Requirements and Test Management for Jira (RTM) yourself, take a free 30-day trial from the Atlassian Marketplace. You can also book a live demo via Calendly, if you’d like to see the app in action, or read more on bringing test management process inside Jira on our blog:
- Why using Jira as a testing tool is a good choice for your project
- 4 reasons to use Jira apps for test management
- 5 steps to set up requirements management and testing in Jira
Kasia Kornaga is an Atlassian Apps Content Specialist, responsible for writing about requirements and test management on the Deviniti blog. She tries to discover and understand the mysteries of software testing process in order to share the knowledge with the readers interested in the subject. As an SEO enthusiast, most of all she values helpful, unique content where users can find answers to their questions, so she probably knows what you will look for before you even start. When she’s not writing, you can find her at the theatre, at home with a book or passing on city streets on her bike.