Using TestFLO for Test Management in SAP Projects

Test management during an SAP implementation is always a challenge. In most part it is a manual process that involves a collaboration between consultants and Client’s users. It also requires a formal sign-off procedure.

Sometimes, you might also need to keep track of the original requirements or specific items agreed upon during a Business Design phase.

There are usually three or four test cycles such as unit testing, system integration testing and user acceptance testing to name a few. It was a pleasant surprise for me last year to discover TestFLO Jira add-on, which really helped our team in managing all this complexity.

In SAP projects, just like in any other large IT system implementation, there is never enough time to do everything perfectly right.

That’s why the testing phase often becomes the victim of quickly melting project budgets and time. If we want to save time during the testing phase, however, we will most probably have to give it back during the Go-Live and Support phase and it will turn out to be much more expensive.

In terms of SAP test management, there are two options you may choose from:

Option 1: Quick and Easy – use Excel or better, use Jira’s simple issue management functionality.

Option 2: Use a dedicated TestFLO add-on for Jira and enjoy its benefits.

When your Client is not interested in creating detailed test scenarios with business process test steps because it simply takes too much effort, your testing approach will have to be simple. This type of clients assume that testers have enough knowledge about the tested system and will smoothly carry out test procedures. I like to call that approach ‘flat testing’ – there are not prescribed steps to follow, just JIRA issue type Test with available test results Passed, Failed or Retest.

There are some advantages of this approach:

It might look great at first glance, but the solution has some serious drawbacks:

That’s why it’s smarter to name and describe process steps for your tests and use a tool for team collaboration during the testing phase which tool also offers a comprehensive reporting functionality. So, let’s see the Option 2.

TestFLO is a major component of the PROBE.Studio ERP Toolkit solution. It is our pre-configured solution for managing SAP projects, which also includes managing a testing process. We have already used it in a couple of SAP projects and users generally find it very useful and easy to learn.

All you need to do initially is to transfer the steps from your Business Design Documents (or Business Blueprints) to an Excel file and, once you’re done, you can easily import the data into Test FLO repository of Test Case Templates .

Have a look at this example file.

Note the column named Stream. That naming is part of the ASAP methodology that divides projects into Phases, Processes, and Workstreams. A Workstream basically shows that the test belongs to a particular team in TestFLO. In our pre-configured PROBE.Studio solution for managing SAP projects in Jira (see note below), we use one of the available fields for storing data about Workstreams.

After you have imported the Excel file into Jira, you created so called Test Case Templates (TCTs). These are a sort of containers with process steps that you will need to perform in each cycle to test a process.

The best thing about TCTs – they are reusable in all SAP test cycles. All Test Case Templates are stored in your Jira Test Repository – a project list of test cases that can be utilised in your project.

After the import, each Test Case Template looks like any other Jira task but with an additional section – Steps.

Now it’s time to import this and other Test Case Templates and create a new Test Plan. Here’s how to do it.

You need to start by creating a new task called Test Plan. This issue type is introduced to Jira during the TestFLO installation process.

Pick a Test Plan name and continue with filling the rest of the fields. Press the button Create and you will see its details on the screen: who owns the test plan (Assignee), to which Workstream it belongs, and during which ASAP Methodology phase this test will be performed.

See the Summary Panel, where you’ll be able to track man-days consumed by the test cases execution effort. That is not a standard feature and it’s part of the PROBE.Studio ERP Toolkit solution.

If the Test Plan status was inactive, it would mean that the plan is not ready for execution yet. However, by setting the status TO DO, I am communicating that the Test Plan is prepared and ready for testing.

Now I can add new issues to the JQL query I prepared earlier. The query enables me to add only the issues I want to have in this Test Plan.

That’s where the Test Plans naming convention helps a lot. Alternatively, you can also use JQL queries for this purpose.

Here are all my Test Cases imported to the previously created Test Plan.

Note that I used TestFLO’s flexibility and customised the Test Cases section of the Test Plan to display additional columns: Test Case Status, Defects and Steps Progress. That gives me a direct look into what’s happening in Test Cases without the need to open them.

Now I’m ready to start the execution of my Test Plan and monitor the outcome of my testing process.

Here you have a couple of functionalities to choose from. It all depends on the level of details you are interested in:

1. Test Case Level

2. Test Plan Level

3. Test Cycle Level

Let’s go through each of them:

Test Case Level – available when you open the Test Case

The section Test Information which nicely summarizes the progress in this Test Case.

Test Plan Level – available when you open the Test Plan

Test Cycle Level – visible in Issue Search view with appropriate columns displayed

And finally, you can build yourself one or more dashboards summarizing test cycle progress. In PROBE.Studio ERP Toolkit we have a few pre-configured testing dashboards available.

TestFLO dashboard displays the progress and status of every Test Plan, as well as test case execution statistics. Dashboards are very useful because they help to see the current picture of my testing cycle instantly.

And that’s in short how TestFLO helps to manage your testing phase in SAP project visually. But there is more.

If you use the Structure add-on, you can go further in visual reporting.

With this functionality you can get both visual and numerical representation of test progress. You’ll be able to see the budget time consumption and number of finished test steps. That will give you another perspective to judge the real progress of testing.

Also, it’s quite comfortable to expand and collapse Test Plans aligned in structure to see their content.

TestFLO offers a range visual representations of custom fields – like test case and test plan statuses, test plan progress, and steps progress. Additionally, the Structure add-on renders them perfectly, helping Jira users to quickly assess the state of their testing process.

If you intend to use Excel spreadsheets for SAP test management, I hope that this article will at least encourage you to reconsider that decision. Excel is quick to use but very expensive in terms of maintenance and not too effective in team collaboration (‘the file is now locked for editing by this user’ – do you remember that from your projects?)

Using Jira with TestFLO for test management in SAP projects will save you a lot of time that is usually wasted on face-to-face status monitoring, meetings, solving misunderstandings, and e-mail communication. It has been proven in practice that the tool is friendly to both consultants and Client users but at the same time useful for test managers with project managers.

The combination of TestFLO and the PROBE.Studio ERP Toolkit for Jira can save you even more time during the whole SAP project implementation process. All you have to do is to import your project plan with the list of planned business blueprints, configuration objects, data objects, test plans etc. into one integrated project management tool. The next step is the training in using Jira and the Toolkit together and you can put a tick at your Prepare Phase deliverable in  ASAP (or SAP Activate Methodology) – Project and Operational Standards.

Managing a SAP project in Jira ensures transparency, collaboration between consultants and client users, and enough oversight to ensure that SAP project is delivered cheaper, together with better organization, more effective communication, and clear responsibilities.

And all that will surely help save your resources. If you want to know more, reach out to our Sales Team.

*Note:

PROBE.Studio is a Project Management Information System based on Atlassian technology platform. The solution contains a series of best-practice project management processes combined into pre-configured toolkits, which are designed for boosting effective collaboration and process driven visual project and portfolio management. If you want to know more, contact us today.