STARWEST 2018 – where software testing is the star

The last quarter of 2018 started off with a trip to the USA. We went to Disneyland Hotel in Anaheim, CA, to attend StarWEST – the most established and renowned worldwide software testing conference. Over 40 top companies, 70+ empanelled speakers, and more than 1000 attendees assembled in the land of childhood heroes, each bringing valuable experiences and insights. For 6 days, experts from all over the world were sharing their knowledge, tips and tricks, as well as visions of the future of testing. Here are the key takeaways we’ve learned at this event.

These days, customers have more power on the market than ever. They look for the best user experience and have high expectations about functional quality. Moreover, as Wayne Ariola from Tricentis stated during his talk, ‘the interface of software alone has become synonymous with the consumers’ brand perception’, and this is more than true for other quality aspects – even not related directly to the product, like customer service or brand communication. Basically, as tap|QA’s Jennifer Bonine and Carnival Corp.’s Janna Loeffler stressed during the opening keynote, ‘software quality is everyone’s responsibility’, from architects to support teams. According to this approach, quality assurance has to be treated rather as a governance model applying to each stage of making, maintaining, and marketing a product, than a separate mechanism between development and release. For testers, it means that end-to-end traceability, UX, and continuous testing will come into play more and more often. For project managers, interconnecting different teams’ work is now a significant task in order to deliver the best possible customer experience with their software.

UX designers have already acknowledged this situation, as we’ve written in the review of Atlassian App Week 2018. Designing and rolling out the new experience was one of Atlassian’s priorities for this year, which can be well seen in their latest updates.

Next-gen project boards in Jira Software Cloud. Image source: Atlassian Blog

If a company can’t keep up with its customers’ changing needs, they simply turn to the competition. To meet those needs rapidly and efficiently, companies are adopting Agile and DevOps at a great scale, but there often is a question of how testing fits into the continuous delivery framework. To address it, Wayne Ariola presented the latest Forrester research and revealed Agile and DevOps testing best practices that help companies succeed on the market. They are:

The speaker, who is known for his efforts in promoting continuous testing, also stressed the importance of contextual metrics and end-to-end business process test cases to ensure consistency throughout the project. Applying these practices reportedly makes testing a secret weapon to gain speed in delivering high-quality products and services.

For quite a while now, AI has been perceived as a danger for many professions, software testers being among them. It turns out the devil isn’t so black as he is painted – as Jennifer Bonine and Janna Loeffler stated, AI and machine learning bring more power to software testing, as these technologies allow you to zero in on creative work. If you make the right use of these algorithms, they will do all the heavy-lifting job for you, like collecting and analyzing data, producing reports with insights on problem areas inside the product, or indicating potential pain points in test outcomes. What remains a human responsibility, though, is coming up with test design and architecture, as well as getting the best out of the insights collected by AI. While advanced algorithms can analyze current and past data and indicate problem sources, making the right decisions is still crucial to prevent potential failures in both development and testing and indicate the way forward to achieve the highest possible software quality.

That’s why these innovations won’t endanger software testing as a profession in the predictable future. There still is and always will be a place for manual testing – after all, algorithms and automated processes are better in collecting and analyzing data, and humans are in thinking out-of-the-box and solving problems. That’s where human testers come into the equation, and they won’t leave it any soon. So to make the technology our best friend, we could try to be more creative in planning and making the decisions, and let the machine do the repetitive work of gathering data and making comprehensive reports.

Here’s what humans are better at, according to Jennifer Bonine and Janna Loeffler. Image source: @Bria_Gangard on Twitter

As the DevOps framework matured, the concept of continuous testing emerged as a supplement to continuous delivery and accelerating release cycles. It is often misunderstood as just an implementation of automated tests, but during the panel discussion between Adam Auerbach, VP of Quality Engineering at EPAM Systems, Alex Martins and Stephen Feloney from CA Technologies, it was described as a process ‘done by everyone all the time, even before development starts’. Although there is no continuous testing without test automation, it is more than just that, and the cultural aspect of such a mindshift is one of the hardest obstacles to overcome.

A working solution to this was presented by Eran Kinsbruner of Perfecto. When it comes to testing, the right mindset is to settle into performing failure discovery in your production environments. The main focus should be on the feedback speed on the code change to the development team. The faster developers get the feedback, the sooner they’ll be able to implement the required changes and deliver the final product. According to Eran, to create a successful continuous testing environment, which reduces risk, improves response time, and boosts velocity, we need to:

As technology matures, AI and machine learning become more advanced, and automated testing improves over time as well. However, as all these are algorithms, they should also be tested – just like every other piece of code in the project. In the closing keynote, Max Saperstone from Coveros made a point that by applying development, testing, and deployment techniques that can be used to programmatically verify test correctness, we can achieve confidence in our applications when they go to production. These techniques can include constructing pipelines, staging and pre-production builds, or setting up container images of pre-production environments to mirror the dependencies.

At Deviniti, we’re making great use of Bitbucket Pipelines, which is one of the latest features shipped on Atlassian Cloud. Our Marketplace apps are rather small compared to the Atlassian tools we make them for, so we run mostly JavaScript unit tests through these pipelines, which then verify the logic of the code, but don’t contain logic themselves. This way we avoid excessive complication and actually don’t have to “test the tests”.

An example screenshot of Bitbucket Pipelines. Source: bitbucket.org

This one is repeated at every event we attend, but there’s never too much of it. At STARWEST, Jon Bach, who is a tester-turned-program manager at eBay, told that he wants his testers to expose risks that can threaten the brand, speak up when they’re worried, and communicate their needs with him. As our own 15-year experience shows, such communication is way more effective if all the teams involved into the project are connected through a fully integrated and unified working environment. For us, it’s the Atlassian Stack that allows everyone to always have a look at what’s going on at the peers’ yards, as well as quickly find and notify the right person if collaboration between teams is needed. Another big advantage of this setup is its customizability for every team’s particular needs – up to adding functionalities that can move the whole workflow to one big tool.

We’re developing such functionalities ourselves as an Atlassian Platinum Solution Partner and a Gold Marketplace Partner. At the event, we were showing the numerous attendees our TestFLO and ReqFLO apps, which allow to manage requirements and tests inside Jira and thus achieve end-to-end traceability easier, along with enabling teams to see the bigger picture of the project.

If you’re interested in moving your requirements and test management processes right inside your Jira, TestFLO and Requirements and Test Management for Jira are available for a free 1-month trial. You can also book a live demo via Calendly, if you’d like to see them in action.

Our next stop on the road was San Francisco, where we’ve hosted Jira Service Desk Day 2018. Read the event’s review if you’d like to learn the latest findings about modern ITSM practices and solutions.