The main purpose of tests is the quality assurance of the final product. The best way to achieve that is by tracking defects throughout the testing process and sending them back to fix by the development team yet before releasing the software. In order to do that, it’s absolutely necessary to be up to date with every step and implemented changes, as well as the progress of test plan execution, especially if we’re dealing with more than one of them. It’s possible that each test execution will be performed by another person, what often makes the process difficult to follow. But there is one thing that, if used smartly, can be a big facilitation when it comes to tracking the overall progress by the manager. We’re talking here about the test execution report. Transparently displayed test results will help the stakeholders decide on further steps. But what makes for a good data presentation?
What makes for a good test execution report
Aiming for the quality assurance of our product, we should know some good practices. First of all, the reports should be clear to everyone. It may seem obvious, but specialists often tend to create reports that aren’t quite understandable for other team members or customers, because of the complicated form, too specialized vocabulary, too many or not enough details. This leads to communication problems and makes it impossible to settle on a plan of action, for example, if there’s a necessity of re-execution of any particular test plan or if the release date of a final project should be delayed. There are some things worth checking whilst creating a test execution report, so it will do its work:
- make sure your report is specific but delivers complete information;
- always double-check the data which is going to be placed on the report;
- imagine you’re looking at your results for the first time – verify if the report is readable;
- if the generated report is too big, think about implementing filters to it;
- when it comes to a meeting – remember to be prepared, on time and ready to answer all the questions.
The last point depends entirely on you, as to the rest of them – we might have something that can help. First of all, with the object of creating an accurate report, it’s smart to have all the elements of a process inside one place, so it can be well organized and consecutively executed. In the case of test management. Atlassians’ Jira gives us this possibility, but unfortunately, its’ basic functionalities often aren’t enough. That’s where dedicated testing apps come to rescue – and one of them is Requirements and Test Management for Jira (RTM).
How to keep track of your testing progress with RTM?
During the creation of RTM the fact that test management requires a collaboration of analysts, developers, testers, designers, and managers was taken into consideration. That’s why you can be sure the reports generated with RTM will be simple, but they will contain all important information. For now, there are four test reports in our app: Traceability, Requirement Coverage, Test Case Execution, and Test Execution. The first one displays the relations between any two baselined issue types, using many-to-many relationship comparison, in a special traceability matrix. The second allows you to verify if all your requirements are covered by related test cases, test plans, test case executions or defects. The Test Case Execution report gives the possibility of tracking a specified test case executions progress and is generated as a pie chart, which presents all TCEs statuses. The fourth report, Test Execution, also shows the overall progress of test case executions but in the wider context, which we describe more in detail in the present article.
The two last reports can also be displayed on the Test Dashboard. The feature lets us see the overall progress of our testing, as well as the statuses of particular test cases on a single screen. This functionality will surely be appreciated especially by Test Managers and IT Analysts, as it allows them to keep track and stay involved every step on the way. But of course, it’s not all of its benefits. The Test Dashboard supports the workflow of testers themselves, as using it they can immediately tell where they are in the whole process, estimate how much work they have to do, and come to many helpful conclusions.
Setting up the Test Execution report
Setting up the report is intuitive and doesn’t require gathering data from any external tools. This is one of the strongest advantages of bringing test management into Jira: having everything stored in one place. Thanks to that, in RTM reports are generated automatically, what prevents possible mistakes. Test Execution report presents the progress of each execution of a given test plan and the statuses of the test cases included in it. After selecting a specific Test Plan or/and Execution, you can filter displayed results by Test Plans/Executions (depending on what you consider your main indicator), RTM Environment, TCE Created Date and TCE Assignee. It’s not obligatory to complete all tabs, as it depends entirely on you how detailed you want the final report to be. The report also makes it possible to choose which Test Case Executions statuses should be presented, after clicking the Preferences icon in the upper right corner of the page. The results can be exported to .csv or .pdf file.
How the Test Execution report can help your project management?
With the Test Execution report, based on displayed progress, you can estimate how much time you need to complete your testing process. Thanks to the colorful bars on the chart along with the exact percentage of each status, it will take just one look to know the proportion of passed and failed test cases. The report also distinguishes objects that are in progress, yet to do, blocked or passed with restrictions. Planning your further work or presenting the current course of tests to other stakeholders can be really easy from now on. The transparent form of the Test Execution shows which test execution generates the majority of failed test cases. As the results can be filtered, the report simplifies observing how for example change of environment of the execution or person responsible influences the number of occurred defects. Last but not least, the functionality makes it possible to verify if a new release didn’t cause bugs in any of the previous ones. We can spot it by simply checking the statuses of the particular executions of test plans: if after the update we have less successful test cases in any of the versions of our product than before, this means something went wrong. That kind of information will certainly be useful not only for testers or developers but for product owners or managers as well.
In order to be sure that the communication inside your team is on the highest level, you need good test reports. Transparent charts and tables help the team members understand each other and make the exchange of information easier. When projects are as complex as testing, not to mention the whole software development, the risk of misunderstanding rises. That’s why in RTM for Jira there are a few test reports that allow each person involved to keep track of the process every step on the way.
Test Execution Report is intuitive to create, shows the progress of your test cases in successive executions, and what percentage of a specific execution constitutes each status. Its’ graphic form in which all the statuses have their own color is not too much, not too little, but just enough when it comes to data presentation.
If you’d like to test Requirements and Test Management for Jira (RTM) yourself, take a free 30-day trial from the Atlassian Marketplace. You can also book a live demo via Calendly, if you’d like to see the app in action, or read more on bringing test management process inside Jira on our blog:
Kasia Kornaga is an Atlassian Apps Content Specialist, responsible for writing about requirements and test management on the Deviniti blog. She tries to discover and understand the mysteries of software testing process in order to share the knowledge with the readers interested in the subject. As an SEO enthusiast, most of all she values helpful, unique content where users can find answers to their questions, so she probably knows what you will look for before you even start. When she’s not writing, you can find her at the theatre, at home with a book or passing on city streets on her bike.