More and more teams decide on bringing their tests into Jira Software. It’s not a surprise, as this solution turns out to be cheaper, faster, and also supports achieving the goal of each software development project: the best possible quality of the final release. On the Atlassian Marketplace, there are many test management apps integrated with Jira. We divide them into external and built-in apps. Selecting which one is the best choice for your project constitutes a significant part of success. There are some basic factors we should consider, like for example the number of team members involved or the complexity of our project. We’re here to analyze two of the most popular tools, keeping in mind the software development teams’ needs, and help you find out which fighter will help your project win the battle against bugs!
External vs built-in apps for Jira
Before you choose a specific tool, you have to decide on an external or a built-in extension. If you aim at full compatibility, we strongly recommend the second option, as external testing tools usually don’t have a similar interface as Jira Software. This is why bringing them into the process often requires additional training for the team. In consequence, an implementation of a new, separate app takes a lot of time yet before you start actual testing. And this is not the only one time-related obstacle. Using external testing software often involves the necessity of integrating and configuring supplementary tools for requirements, tests and bug tracking, which is not a problem in the case of Jira apps. Beyond wasting time, it also makes complete relations tracking impossible (or at least much more difficult). Testers often consider the race against time as one of their most common challenges. Managers, product owners and other stakeholders also want things to be done as soon as possible, so it’s better for everyone to manage tests in a well-known environment. The second reason why built-in addons should be your first choice is money saving. External tools are much more expensive, and still they often don’t meet all the usability expectations. Testing is a part of software development, which includes the work of many people, so the tool designed only for testers usually won’t be sufficient.
How a testing app for Jira can help you?
First of all, you’ll have your whole software development project inside one place. This opens up a lot of possibilities, as this solution allows all the stakeholders stay up-to-date. Communication between the team members gets much easier. Let’s imagine a situation where testers work on a specific case, confirming the functionality of a new feature, and suddenly the product owner decides that it would be more cost-effective not to implement this feature yet. Without a good tracking system, managers have difficulties with seeing the testing progress, which inevitably leads to wasting time of the whole QA team. Also, the lack of consistency automatically delays releasing the product. And this is exactly where built-in apps for Jira come to rescue. Two of the most commonly used are Zephyr for Jira and Requirements and Test Management for Jira (RTM). What are the main differences between the competitors? It’s time to find out!
Let the game begin
Mission no. 1: Requirements management
In this case, the name says it all. RTM for Jira has a transparent tree-structured view with folders and sub-folders for both requirements and tests. This way, you and your team can structure and prioritize requirements, assign related test cases to them, and track the changes implemented throughout the process. RTM introduces four dedicated issue types from which you can choose when creating requirements. Thanks to this functionality, you can keep an order yet in the initial part of the development process, which has a strong impact on the further stages of your work.
RTM’s tree structure of requirements helps organize the project in a single place right from the beginning
Zephyr focuses mostly on tests, so it has only one issue type for all kinds of objects. Lack of customization here practically excludes the possibility of including requirements sufficiently into the process. The second disadvantage is that if we, after all, decide to connect requirements gathered in Jira to our testing objects in Zephyr, we can’t do it, but through default Jira issue links. This solution substantially limits relations tracking options.
Mission no. 2: Integration and compatibility with Jira
Both apps are fully integrated with Jira, so you can seamlessly link testing objects to regular issues, Epics, and user stories. Your requirements, test cases, and defects are generated as Jira issues so you can easily search for them inside the Atlassian suite. But, it has to be admitted that when it comes to compatibility, RTM takes the lead. It features a well-known interface, built mostly with Atlaskit, which helps quickly adjust to the look and feel. Each user will be able to start working immediately after a plug and play configuration, no matter if we’re talking about a tester with many years of experience or a newcomer. Zephyr, on the other hand, has the front end of an external tool and requires navigating through the elements with the default Jira’s Issue Navigator, which can make the process really confusing.
Zephyr’s interface differs from the Atlassian product look
RTM for Jira makes each stage maximally intuitive for Jira users
Mission no. 3: High-grained relations between all objects
All testing tools assure better relations tracking than standalone Jira or even Jira integrated with an external tool. Both RTM and Zephyr have traceability-related features, which make it possible to verify how the objects are connected with each other. But considering the actual value of these functionalities, RTM wins again! Having only one issue type for each testing object, Zephyr isn’t capable of defining the actual types of relations, and we can only choose from two possible directions: Requirements → Tests → Executions → Defects or the other way around. Generating of the Traceability report inside the app is very unintuitive, and the connections aren’t described but just displayed as a plain list of elements. On the contrary, in Requirements and Test Management for Jira we have the Traceability Matrix, which enables us to see correlations between any two baselined issue types using many-to-many relationship comparison. We can define which types of issues will be displayed by customizing X-Axis and Y-Axis with JQL queries.
Deviniti’s app also provides the second relations tracking report – Requirement Coverage. Thanks to this feature, we can make sure that all the requirements are safely covered by related tests and other testing objects.
Thanks to Traceability Matrix in RTM, you can see testing objects’ relations at a glance
Mission no. 4: Automated testing support
Automated testing is without any doubt a great solution for testers, as above all, it frees them from doing the same things over and over again. Nevertheless, not all teams use this approach yet, because it surely needs additional preparation to be done well. Zephyr supports test automation, providing its users with the possibility to run scripts through REST API. Also, it has additional plugins to integrate with CI tools like Jenkins or Bamboo. This is a great advantage, as this solution speeds up the whole testing process and thereby saves a lot of time and money for the company. Although RTM focuses on manual testing at the moment, we know that automated testing support is a must-have of a good, fully functional tool, so stay tuned! The automation features are already on our roadmap.
Zephyr enhances test automation by integrating native Jira experience with CI loops
Mission no. 5: Flexible reports
This mission ends in a draw! RTM for Jira and Zephyr are well prepared for reporting test results. Both apps give you the possibility of generating charts and tables, during or right after you perform your executions. The reports, such as Test Execution report in RTM or Cycle Summary in Zephyr, allow to track the overall progress of your test plans. In Test Case Execution report in RTM, you’ll also find the status of each Test Case. The form of reports is clear and user-friendly, so all the stakeholders can catch up. This is very important, as there are always a lot of team members involved in the testing process. Consulting the current status and the previous results on the meeting gets much easier and to the point when you have the actual reports prepared, always ready to back up your words.
Test Execution and Test Case Execution reports in RTM for Jira
Test Metrics dashboard in Zephyr gather all the test results on one screen
Both tools have their pros and cons, we can even find some common qualities, like for example the fact that both are identical on Server and Cloud, so in case of the need for migration, it will save a lot of time. Having been the first ever test management tool on the Atlassian Marketplace, Zephyr boasts the highest number of active installations. Even though Requirements and Test Management for Jira is the newest one, it develops quickly, answers the most recent needs of software development teams, and comes up with the new features all the time.
To sum up, RTM for Jira should be your fighter, because it really lets you benefit from all the advantages of testing inside the Atlassian suite. With our app, you have requirements, features, tests, and defects tracked in a single place, whereas Zephyr mainly focuses on testing, which practically excludes requirements from the process. RTM allows you to keep track of the changes in even the most complex and demanding projects, where each team member can see the relations between the objects every step on the way. Our app also gives you the possibility to organize the elements in a tree-structured view. Thanks to this functionality, everyone involved sees the final product’s goals from the very beginning and understands the succeeding stages better.
If you’ve decided to test our app, RTM for Jira, take a free 30-day trial from the Atlassian Marketplace. You can also book a live demo via Calendly to see the app in action. We also invite you to read more on bringing test management process inside Jira on Deviniti blog:
- Why you should bring Requirements and Test Management right inside your Jira
- 6 steps to set up requirements management and testing in Jira
- 7 ways to structure software requirements and test cases in Jira
Kasia Kornaga is an Atlassian Apps Content Specialist, responsible for writing about requirements and test management on the Deviniti blog. She tries to discover and understand the mysteries of software testing process in order to share the knowledge with the readers interested in the subject. As an SEO enthusiast, most of all she values helpful, unique content where users can find answers to their questions, so she probably knows what you will look for before you even start. When she’s not writing, you can find her at the theatre, at home with a book or passing on city streets on her bike.