3 proven uses of test automation + extra: Appium framework

Year after year, Jira, a project management tool enriched with issue tracking capabilities, becomes the most commonly used issue tracker tool in the world. In 2019, 68% of developers claimed to use it in their day-to-day work, and now the tool is used by over 75,000 specialists around the world. This Atlassian software, with a little help of a proper add-on, can be applied for both manual and automation testing. Automated testing is now on the rise – implemented by 44% of companies in at least 50% of all testing in the previous year. What stands behind this popularity of automation testing and what’s the best usage of this method?

Automation testing, contrary to manual testing performed by a human, is a software testing technique executed by a proper testing software tool. Although it requires some of the work done by a human specialist, nevertheless, it shoulders the burden of performing only manual tests from an employee. The main aim of automation testing is to minimize financial loss due to production errors. This way, we can increase the effectiveness and execution speed of testing. That doesn’t mean that each test should be automated, but it gives the testers the freedom to manually test only in cases when the human look is desirable.

In a glimpse of an eye, we can think of at least a couple of advantages of automation testing. One of the main advantages is a shorter software delivery time. Automated testing can be performed 24h/day, and the tool won’t get tired of it. When there is a large number of constituents to test, there is a place for automation testing.

The automation testing tools may seem not the cheapest at first, but in reality, they can save you tons of your spendings. Here are the reasons why.

Firstly, they work all day long, not 8 hours a day. Secondly, automation testing makes sense usually when developing complex projects – that’s why they don’t pay off in simpler cases. Thirdly, they are reusable. Once properly created test automation can be reused all over again, and you don’t have to hire new testers to design them. And lastly, the quicker you detect a bug, the fewer expenses you have to bear for it.

Automation testing won’t displace manual testing. But their cooperation is also beneficial for manual testers. Performing boring and repetitive tests is no tester’s dream, and after implementing automation, they can focus on more inspiring work (such as exploratory testing) – at least for some of the time. By relocating these tasks to automation testing software, you can benefit from your tester’s greatest advantage – creativity and the human factor.

Automated tests can’t run all types of tests. This one of the most common myths about automation testing may make a wrong impression that we don’t need manual tests. And that isn’t true. While testing is a neverending process, we can’t make provision for any kind of element that we want to analyze. Test automation framework (just like a framework of a manual test) is strictly specified – it has its beginning and its end. There is still a need for a specialist to plan it.

What’s more, there are some aspects that would be difficult to subject to automatic testing. Non-functional testing, including aspects such as usability, accessibility, or readability, have to be checked by the human. Our personal feeling about the visual side of the project is irreplaceable. In this case, we can’t predict a postcondition, as well as the end result, therefore automatic tests won’t always be put to good use here.

However, we can benefit from the methods of automation testing that allow us for:

This way, a human is involved in only one part of the process, and automation takes over the rest. If there are any deviations from the norm, the test is marked as failed.

Due to the fact that mobile applications have a different ecosystem (features and behavior) than web pages and applications, here, we have to implement a slightly different testing approach. The issues that should be considered during mobile testing:

While testing mobile apps, we can come across a problem with fragmentation. It’s associated with a multitude of devices available on the market. They differ with: sizes and density of a screen, capacities of internal memory and RAM, processor architecture (ARM vs. Intel), modules (GSM, GPS), sensors, operation systems, system overlays, and regional modifications of hardware and software.

How to deal with these kinds of issues? The solution is to:

Read about mobile application automation testing using Appium on our Medium blog.

As we’ve already determined, automation testing isn’t the answer to every problem. You should carefully pick your test automation strategy and implement it in these aspects of your test plan, where it will bring you the greatest profits. What is worth automating? We’ve selected 3 tests that are worth automating.

End-to-end (E2E) testing is a software testing strategy which goal is to test an application’s workflow from beginning to end. Performing E2E tests, we simulate a user’s behavior, using real scenarios, to provide a smooth experience while using the app. Because of the complexity of the E2E tests, they are often automated. Going through multiple steps can be not only time-consuming to do manually, but also makes a room for human errors through the process. While designing End-to-end automation tests, we should always keep an end-user perspective and focus on the application’s high-risk features. 

Regression tests, whose goal is to determine whether a code change in the software did or didn’t have an impact on an existing product, are the first great example of tests that can be subjected to automation. In this case, we test the parts of the software that haven’t been changed. In the best of circumstances, regression testing should be performed every time the application is being changed in any way – whether we’ve developed a new feature or fixed a bug. Because of the frequentness of regression testing, implementing the automation can save the resources (that can be used on other types of tests), as well as test iteratively and constantly.

Smoke testing allows for determining whether the build software build is stable or not. It’s a prerequisite to software testing – its results help the QA team decide if the software is ready to perform further testing. The importance of smoke tests is invaluable, however, to be efficient, smoke testing should be run really quickly. Because of that, it’s the perfect candidate to automate. Automating smoke tests will result in receiving return results automatically, and more consistently in comparison to manual testing.

Now, going back to testing in Jira. We, as Jira users, can use automated solutions that integrate with the Atlassian software. One of them is TestFLO, a flexible Jira testing tool, and today, we’ll show you its functionalities making automation testing easier.

TestFLO can be integrated with Bamboo and Jenkins CI Servers. Using the Automation Module in the tool, we can run automated tests from the Test Plan level in TestFLO on a CI Server. While the Server received the request from the tool, it runs the tests, and then, when all tests are completed, using our CI Server plugin, it gathers and sends the results to TestFLO in the shape of Test Cases. The Test Cases receive the status passed or failed.

How does automated testing look like inside the tool? We start from the Test Plan level: after configuring the server, we move to the Run automated tests function. Now, it’s time to configure the test. At this point, we choose the server (as we can have several servers connected). Then we choose the Job – here we also have a choice due to the many types of automated tests connected to different elements of the application.

Next, TC Creation Strategy stands for the effect we need to achieve after receiving the test results: update the existing Test Case and create a new one, or just update the existing Test Case and not create new ones. Finally, we choose the place we’d like to import the results: to the current iteration or a new iteration.

Moving on to the CI Server parameters, here, we configure the parameters that we can send from the TesTFLO level from Jira to the CI Server. Our request may concern, for example, a specific environment to which we want the result to be deployed, or run tests as a user with specific permissions (e.g. admin or QA manager, etc.).

In the end, we can select Test Cases to run. What does it exactly mean? At this point, we can choose which exact Test Cases we will run on a given server. This is a great convenience that allows us to bypass the need to run all the tests in the Test Plan, and run only those that, for example, failed in the previous iteration of tests.

Appium, an open-source tool for automating applications, can be used in combination with the TestFLO plugin. The Appium cross-platform feature enables writing tests on iOS and Android platforms using the same API. Taking into account the widespread use of selenium Webdriver, Appium has extended its API with specific methods of testing mobile applications.

The screenshot below shows, how scripts with automatic tests using the Appium framework to test mobile applications on phones can be embedded on CI Server.

The flow can be presented in the following steps:

In Appium, a web server using REST API receives connections from the client, waits for commands, and executes them on a given mobile device. The results of executing a given command are visible in the Test Scripts thanks to the reply to the HTTP responses.

As an example of using Appium with TestFLO, we can use adding an existing e-mail account in the Gmail mobile application.

Naturally, in a real-life scenario, you would be testing your own mobile application.

If you’d like to learn more about test management in Jira Software, check out this video tutorial series on our YouTube or read more about the TestFLO on our website.

Also, make sure to check out other articles available on the Deviniti blog: