Why switch from TestRail to embedded testing with RTM for Jira?

When it comes to creating software, the objects and functionalities are related one way or another, so working on them is much easier when everything is transparently stored in one place. This solution has more advantages than just better chances of getting the right estimations. It guarantees a high level of communication inside and across the teams, which saves time and money for the whole company. If your team already uses Jira, you don’t have to look any further. In this article, we describe for you the two options that support bringing test management along with the rest of the development right inside the Atlassian suite.

Communication is the key 

Why are we bringing up the subject of communication between the team members to begin with? It’s because it can really make a difference for your software project. The communication issues were mentioned in the most common challenges which testers have to face during their work.

Of course, it doesn’t only concern testers. The more people are involved in a project, the bigger problem it becomes. If analysts, developers, designers, and managers work in a separate tool each, there’s a big chance that eventually someone will lose track. It may have serious consequences, like for example delays in releasing the final product or losing control of bugs which were already taken care of.

It takes time to set up meetings, attend them, and make sure that all the co-workers are up to date with the modifications. The unnecessary appointments constitute around 50% of all and can take up to 31 hours a month, which is 8 hours a week. Besides, there are always questions emerging throughout the project. It’s much easier to put them in the comments inside the tool than to always look for the person responsible for something in order to get an explanation.

Second of all, executing tasks in one place helps avoid duplicating work. The risk of doing something that’s already done by another team member else drops to the minimum, as the overall process and particular stages are easy to follow by each stakeholder. Bringing the whole software development process inside one place is a great solution for managers, too. Deciding on it, they not only integrate the workflow of the whole team, but also become able to track the progress, and estimate the time needed to complete the project.

The separate tools don’t provide this kind of consistency, although they can offer a wide range of functionalities as well. Let’s find out if this is enough to beat the possibilities of an extension for Jira, using the specific example for both kinds, in order to describe them in the most vivid way.

TestRail vs Requirements and Test Management for Jira 

TestRail as a representative of separate testing tools

TestRail is one of the most popular testing tools integrated with Jira. Despite the fact that it’s excluded from the Jira Software as such, many QA teams and project managers choose to buy it because of the user-friendly, lightweight interface and great functionality. The Test Cases are put in the clear tree structure on the right side of the screen. They’re executed in Test Runs, and it’s always possible to check the progress of testing the most important features on the Milestones tab.

page screen of Jira Software dashboard. Conditional formatting not applied with invalid range order and: details, decription, TestRail: Results, Activity and screen of a chart in Datahub, Milestones, Test Runs and Activiti.

The possibility of adding multiple projects is a strong plus of TestRail, which distinguishes it from the other test management tools available on the market. It also supports each type of software testing, as it can be used for both manual and automated processes.

What’s more, it integrates with other tools and includes an API which lets the teams create their own integrations, so the tool can fit the project’s needs best. But is it enough to boost your team’s efficiency to the level you expect it to?

RTM and the possibilities of embedded tests in Jira

Requirements and Test Management for Jira reflects a similar approach, but there are some basic differences, which need to be mentioned at the very beginning of our comparison. First of all, RTM is an extension for Jira, not a separate tool. This entails some functionalities that only this type of app can provide its users with.

First of all, its look is already familiar, so you won’t need any additional training for your team (which can also be quite costly). The apps for Jira are cheaper than dedicated tools, especially if you use the Atlassian suite for other parts of the software development project.

The functionalities are, in fact, similar to the ones present in TestRail. We have requirements and test cases together, test executions, defects management, and built-in reports. Last but not least, using RTM you get even more intuitive UX, transparency of all the steps, and most importantly, seamless linking between testing objects and other Jira issues.

page screen of RTM for Jira with requirements and test cases together, test executions, defects management, and built-in reports.

The most significant differences between the two solutions

Plug&play configuration

The first thing we do before we start working on the tool is its installation. It can flow smoothly or, on the contrary, discourage us from using the software. As it was mentioned earlier in this article, apps for Jira can be installed much quicker than separated tools. Right after finding and downloading our product from the Atlassian Marketplace, all that’s left to be done is to ask Jira Administrator to set up permissions. 

Then, we can move straight to the preconditions, screen, and Jira issue settings in order to choose and customize the fields that will be needed in the project later-on. All the testing elements provided by RTM for Jira can be seamlessly linked to other Jira issues, so the workflow can be performed in a way which is transparent for everyone in the process.

The procedure is slightly different in case of a separate testing tool. After the installation of TestRail itself, we have to go through a more complex configuration with Jira Software. In order to do that, we’re obligated to download a dedicated Jira integration from Atlassian Marketplace. Only then, we can proceed with the integration of all out-of-the-box Jira items, we may want to use in the future.

These possibilities give us quite impressive flexibility, but at the same time, they prolong the whole process before the work on a product actually starts, not to mention the time necessary to adapt to a completely new interface.

Requirements documentation 

There is a multitude of ways of gathering requirements. Different teams keep them in Excel, Notepad, paper sheets, or other separately stored documents. Moreover, there still are people who think that requirements documentation for tests is created independently from the requirements collected at the initial phase of the whole software development. This approach is related to high risk, especially in the case of bigger projects.

Just like many similar testing tools, TestRail provides its users with the possibility of integrating a document with requirements to our test cases written inside the plugin. Thanks to this functionality, the team members can see the connections between the two types of objects and track the progress of their realization.

Unfortunately, in this solution, all the modifications made in software requirements documentation won’t be visible for the testers right away, which may result in many misunderstandings. Despite the fact that there are lots of integrations between Jira and external testing tools, their main disadvantage is that they keep the people responsible for testing excluded from the rest of the software development team.

TestRetail formatting screen view with Estimated time, References and DH-1: Conditional formatting not applied with invalid range order.
Project, Component, Issue Type, Assignee, Status, Priority

In TestRail, the References functionality allows linking test cases with requirements and bugs. Source: Blog Gurock

Having this in mind, whilst creating RTM for Jira we took care of the requirements part, too. The app allows integrating the initial stage into the testing process, by including 4 preset requirement types available as Jira issues.

Additionally, in Requirements and Test Management you can structurize your requirements and group them as you see fit, because of a tree-structured view, which in most tools is available only for test cases. This feature lets everyone work together efficiently on a final product, and spot the possible changes as fast as possible. 

RTM lets us track all the relations between the objects at each step of the project. There’s the Requirement Coverage report which helps stay up to date with the requirements progress and connections. 

Requirement Test Coverage Report page screen where you can see: Requirements, Test Cases, Test Plans, Test Executions, Test Cases, Defects.

Detailed and complete relations tracking

Although all Jira integrated tools provide some ways of connecting the elements, only a built-in app can assure high-grained relations. It’s mostly because its objects are Jira issue types, so it’s easier to create and track custom links. The key is to be sure that everyone keeps track, avoid duplicating work or chaos inside and across the teams, and plan task executions better. RTM for Jira answers these needs.

The app has lists of relations visible on the dedicated tab, and also provides a special Traceability Matrix report. The report displays correlations between any two baselined testing objects, as well as user stories, and Epics (thanks to JQL), using many-to-many relationship comparison.

Due to this functionality, the stakeholders will be able to see at a glance if all the requirements are covered by test cases, test cases included by test plans and finally, check the executions of each one of them. It’ll also be much easier to track the features which generates most of the defects. On the Matrix all these links are automatically updated, so you won’t have to worry about unexpected modifications made by other members of the team.

Jira Test plans screen view of Treacebility matrix which you can generate. (TPJ) Functional Requirements / (TPJ) Test Cases

Although TestRail also gives a possibility of tracking relations, we have to remember that it doesn’t display indirect connections. The tool only shows test cases’ links to related requirements and bugs, so it doesn’t provide us with full traceability.

It also makes it much more difficult to link the objects with Jira issues. The tool can be integrated with separate files or documents, but it’s always additional work, which still won’t give the same results as if it all was performed in one place. In this case, the risk of omission is undoubtedly higher.

Jira dashboard demo milestones screen view: Milestones in M47 Release 2.4 raport: 36 defects its percanteges and its description

Source: Blog Gurock

Flexible and real-time reports

Whilst working on a project, we never know when we’ll be asked to present the progress. It usually equals putting the tasks we’re currently doing on hold and starting to prepare the reports in order to explain to the manager where the things are in the most understandable way possible.

If it gives you sleepless nights, we have good news: it doesn’t have to be like this. Most of the test management tools include the built-in testing progress reports, which are practically self-made and real-time. This is exactly the case in both presented tools: RTM and TestRail.

Within the built-in app, we get two easy-on-the eyes reports: Test Case Execution and Test Execution. The two have graphs and tables, which displays the overall or a particular test case’s status. We can observe (and present!) right away, how many issues are passed, failed, or yet to do.

The reports are customizable, so we can filter the results by date, assignee, priority, etc. This constitutes great support in estimating the time left to finish the project, or for example, compare the amount of time spent on test cases executed by different team members.

Jira Dashboard datahub screen view: Test Execution Report based on Demo-279: To do, in progress, pass, fail, blocked, passed with restrictions

Test Case Execution Report based on DEMO-279 chart. Chart shows percentage of tasks: to do, in progress, fail, blocked, passed with restrictions

TestRail offers a wide range of test reports as well. We can generate detailed reports for milestones, projects, and runs, and compare the results across multiple objects. There is also a report created per user, which can be used to track the workload of the entire team.

What distinguishes TestRail from RTM is that it has a full test history for each project with all test records saved. It turns out to be a great support when it comes to improving accountability.

A screen view of Datahub Milestones. Chart of the milestones percentage: 92% Passed, 29% Blocked, 25% Retest, 9% Failled

The most efficient tests are now within reach

We hope that we helped you see how bringing your testing into Jira can support the workflow of your software project. Nevertheless, the real question was: which tool is best for your individual project (and if you’re an Atlassian suite enthusiast, why it’s probably a built-in app)? If you’re still looking for the final answer, below you’ll find the comparison table with all the most important functionalities of TestRail and Requirements and Test Management for Jira (RTM). 

As you can see, the main features and assumptions are similar. The advantages of RTM for Jira that can outbalance the ones of TestRail are quicker configuration, more intuitiveness for Jira users, even if they’re beginners, full traceability of relations, and of course built-in requirements management. On the other hand, TestRail supports automated and cross-project testingwhich can be really useful when your team needs to work on many different products at a time.

RTM for Jira vs TestRail table of contents with a comparison of: Plug-n-Play configuration, Flexible test results reports, Intuitive look and feel, Requirements management, Transparent relations between the objects, Automated testing support

Software development includes multiple stages, and there usually are many people working on the final product. In order to achieve all the established goals, the workflow should be well-organized in a carefully selected tools, no matter which project management approach we choose. It’s not possible to predict all the changes that may occur during the process, but risk assessment is inevitable in every successful undertaking.

If you’d like to test Requirements and Test Management for Jira, take a free 30-day trial from the Atlassian Marketplace. You can also read more on bringing requirements and test management process inside Jira: 

Katarzyna Kornaga

Katarzyna is a Content Specialist, responsible for writing on the Deviniti blog. As an SEO enthusiast, most of all she values helpful, unique content where users can find answers to their questions. When not writing, you can find her at the theatre, at home with a good non-fiction book, or passing on city streets on her bike.

More from this author