Breathe easy, ladies and gentlemen, it’s Friday! As we, the collective workforce, slowly sign off into the aura of the weekend, it is important to note that Mondays are inevitable. Yes, we know, who dares dampen the Friday glow? We get it, Monday’s problems are for Monday. In today’s Feature Friday, we will be addressing a very Monday problem – test flakiness, brittleness, and fragmentation. Today’s feature we will be discussing revolves around Healer, an AI/ML-enabled Qyrus proprietary feature that may stand to save your Monday from ruin!
To take a deeper dive into the feature, we interviewed Tim and Suraj, part of our Client Development team based in Chicago, asking them a few questions about Healer and how it works.
Tell us about Healer, its use cases, and general impact on testing processes?
Tim: Healer is Qyrus’ AI tool that is used to help prevent against test flakiness and brittleness. What’s really neat about it is it saves testers a lot of time when it comes to refactoring/modifying and maintaining scripts.
Suraj: Imagine this scenario: I just put out a new version of my app but all of my regression tests fail during my login process. After checking the reports to see what the error was, I find out that the development team changed the ID value of the login button. Currently, I would have to manually update all of the locator values for that button in my scripts or go back and ask development to change the ID back. Healer takes care of it for you in that it will provide you the correct value to use.
Tim: Exactly. From there you would be able to have Healer take charge by monitoring and making changes to the scripts when necessary. It impacts test maintenance and execution, automating another time-consuming and costly process.
Murphy’s Law and Mondays, two concepts that seem to go hand in hand, as everything that can go wrong seemingly does on a Monday. If Healer does seem to offer a solution to the Monday blues, we should begin by noting the problem and prior solutions.
Does the same or similar functionality exist without Qyrus, and how do competitors address similar problems?
Tim: Simply put, competitors don’t have many features comparable to Healer. And for those that do, they operate differently in how they present the information back to the user. Most of the time, a few different locator values are given back and a confidence score is given for each. The system isn’t 100% sure on whether the value given would work.
Suraj: Alternatively, Healer works on 100% certainty. It doesn’t provide a value unless it establishes functionality. Healer goes a step further than anything that’s out there. To make something remotely similar to Healer would take an enormous amount of effort.
Tim: And without Healer, users have to manually deal with “object not found” errors. That means firstly identifying that it is an actual “object not found” error, then identifying which object it is on the page, and finally getting the correct locator value. After all that is done, tests need to be executed again and verifications need to take place.
Healer is a versatile feature that can bring value to multiple different types of users. Healer is not only helpful for traditional testers but can also expand across multiple roles.
How might Healer help testers, developers, and business technologists? What value can it bring?
Suraj: Obviously, Healer is most helpful to testers. Like we’ve mentioned before, it cuts maintenance time and saves effort and resources. Developers won’t have to worry about the integrity of element attributes. Healer will account for these changes that take place without breaking automation. Just like that, the clock keeps on ticking.
Tim: Furthermore, business technologists gain access to valuable reporting. A Healer report is given back to the user which will provide the locator information. In reality, BTs should be able to maintain test scripts on their own using Healer. Once it is set up, validating customizations to your SaaS applications becomes effortless.
Integrated into the testing space itself, it is important to note the major impact Healer has on the testing and QA (Quality Assurance) process.
What is Healer’s overall impact on the testing process?
Suraj: Overall, Healer has a huge impact in terms of effort reduction. This can snowball into other bonuses such as further cost reduction and reusability, and often prevents the need to create a whole new test script.
Tim: Healer also makes testing easier, faster, and smarter. Like Suraj said, it reduces effort and can help mitigate the need to modify and maintain test scripts. This automatically lends toward making testing easier and faster. Furthermore, by automatically taking care of errors for the user it makes testing smarter.
Suraj: Not to mention Healer works on both web automation and mobility automation!
We know a lot about Healer now including its capabilities, use cases, and the value it can bring to different types of users. Like anything digital, improvements and changes never stop coming.
What can we expect to see in Healer’s future roadmap?
Suraj: Healer is an ever-changing tool. We’re always looking for improvements and enhancements. One thing we hope to see in its future is an increase in speed.
Tim: The quicker we can get responses and reports back to the end user, the better. But we also hope to make every aspect of it automated. Not only will you get a report giving the correct locator value, but also Healer will implement the changes for you into the script. That way, it is truly a hands-off tool.
That concludes our interview with Tim and Suraj about Healer, an AI/ML-enabled solution against test flakiness and brittleness. We’ve discussed use cases and specifics, life without Healer, and Healer’s overall impact on the QA process. That should leave you with one question to conclude this week’s Feature Friday. In light of the madness surrounding March, would you consider Healer in the starting lineup against Mondays?
In the digital world, organizations look to achieve the trifecta of speed, quality, and cost to keep a competitive edge. Automating mobile application testing is key to speeding up testing, extending test coverage, and saving money. Mobile application automation is notoriously complex and often indecipherable to the untrained eye, we are here to shed some light on the topic.
What is Mobile Automation?
Mobile automation, as the name suggests, refers to the streamlining of basic processes that are executed on handheld devices such as phones and tablets. As a mobile application evolves, so too does the product flow, as well as the user interface requirements and specific features. More so, adding another layer of complexity, is the evolution of mobile devices themselves. In the past couple decades, people have gone from holding bricks in their hands to flipping devices and eventually small, thin touch screen cellular phones. As time moves forward, device designs change as well. Checking for backwards-compatibility and performing regression tests has never been more complex yet necessary.
Mobile Automation Testing Using Selenium
At its core, Selenium is a toolset for web browser automation. It remotely controls browser instances and emulates user interaction with elements on the browser’s page.
Some common activities that can be emulated:
Entering text into text fields
Selecting drop down selections
Checking boxes
Clicking buttons and links
Mouse movements and page scrolls
One of the guiding principles of Selenium is to support a common interface for all major browser technologies. Web browsers are incredibly complex and often different browsers perform operations in vastly different ways, even if on surface level it seems not. Selenium then takes these differences and abstracts them so that the same code written to be executed on Chrome also works on Firefox and other supported browsers. Furthermore, Selenium’s infrastructure provides users with the tools to create a grid of browsers to execute their tests on. This gives users the ability to test based on different browsers, browser versions, operating systems, and machines.
Graphical representation of how Selenium works:
A user executes a test on their local machine which then triggers hub, which is where all tests are sent. The hub is connected to nodes which are different instances that will execute tests on computer systems. The hub determines where this test is supposed to be sent and the nodes are where the individual executions happen.
Mobile Automation Testing Using Appium
Appium is a mobile test automation framework and tool for native, hybrid and mobile web applications for iOS and Android operating systems. It uses JSON Wire Protocol to interact with iOS and Android applications using Selenium’s WebDriver. These tools laid the foundation for mobile automation.
Appium follows four tenets to help it meet mobile automation needs:
1. It should not be required to recompile or modify the app in order to automate it.
This is ensured by Appium through their use of vendor-provided automation frameworks under the hood. No third-party frameworks are needed to be compiled, so this means that the same app being tested is the same app being shipped. Appium utilizes Apple’s XCUITest and UIAutomation, Android’s UiAutomator and UiAutomator2, and Microsoft’s WinAppDriver frameworks.
2. Specific languages or frameworks should not be required to write and run tests.
All of the aforementioned vendor-provided frameworks are then wrapped into a singular API, the Selenium WebDriver API. This API utilizes the JSON Wire Protocol which allows clients written in any language to be used to send the HTTP requests to the server. Popular languages already have clients written for them such as Java, JavaScript, C#, and Python.
3. A mobile automation framework should not reinvent the wheel when it comes to automation APIs.
WebDriver has become the standard when it comes to automating web browsers and is even a WC3 Recommendation. The idea is that just because it is mobile testing the same APIs in WebDriver should also be utilized. Appium extends the protocol with extra API methods specific to mobile automation.
4. A mobile automation framework should be open source, in spirit and practice as well as in name!
Appium is open source. This means that any developer can utilize the power of Appium in creating their own mobile test frameworks, applications, and scripts.
How Appium Works
At its heart, Appium is a webserver that exposes as REST API. Appium receives connections from clients which start sessions, listens for and executes commands on a mobile device, and responds with an HTTP response detailing the result of the execution.
Graphical representation of how Appium works:
A user executes some test on their machine which gets communicated to the Appium server. The Appium server determines where to send the execution based on the information given: Android vs iOS, version 10 vs version 9 OS, and so on and so forth. Once this is finished the results are communicated to the Appium server which then relays the information to the end user.
Because of how Appium incorporates a client-server architecture, many possibilities are introduced:
Test code can be written in any language, so long as that language has a HTTP client API
The server hosting Appium can be on an entirely different machine than the one tests are running on
Test code can be written and then executed on devices on the cloud or on local machines
To help with testing efforts, here are some strategies that can be worked towards:
Involve the testing team with business/product talks earlier in development
Testing does not start at the testing phase of the Software Development Life Cycle (SDLC). In actuality, it should be included from the start of the design phase. While designing what the final product will look like and how it will function, it is important for the testing team to take note of test cases and scenarios. Keeping teams aligned as to what needs to be tested and how ensures that all requirements are met during testing.
Plan ahead of time what OS types and versions, devices, and network speeds the application will run on
Paying attention to details such as what OS types and versions the application is meant to be run on is important. Studying the target market of the application can help with planning which OS versions to be tested.
Planning what devices the application runs on is next. This is important because the more devices that need to be tested on the longer the testing phase can become. Limiting which devices the application can be run on is an easy solution but can ostracize users of other devices. Utilizing emulators can be cost-effective but won’t perfectly replicate the behavior of a physical device. It is essential to set up a device testing strategy early on in the process. When possible, the use of a hybrid model is the most effective.
Lastly, paying attention to network connectivity is crucial. It is easy for developers to assume all is well while on their high-speed office networks, but in reality, not everyone has access to such network speeds. Ensuring the application works on all types of network speeds from 1G to 5G and Wi-Fi connections.
Think about security testing and keep in mind data being shared
As more data gets sent back and forth with applications data security starts to become of utmost concern. Mobile application security is typically terrible. It is very important that early on in the process security testing is taken into mind. Pay attention to any data leakages and make sure there are no vulnerabilities or security exploits.
Progressive application rollout
Choosing to rollout the application over phases can make a big difference when it comes to testing. Often times, not all of the bugs in an application can be caught during the testing phase. Considering using different rollout strategies such as regional or beta testing can help with catching bugs and inconsistencies. This way, if there are any glaring issues, they will only trouble a handful of users instead of all users.
Automate testing of the application
Automated testing allows users to write scripts to cover a wide range of test cases and scenarios. These scripts can be reused over and over again. A lot of time is saved automating versus manually testing every single scenario. Consistency comes to mind, as well. In automation, the same script is being ran every single time with no human error between tests. Initial investments may be somewhat costly, but in the long run it saves a lot of resources and manpower.
Automated Mobile Application Testing Benefits
Automated testing enables developers of applications to quickly and efficiently catch bugs in new features being developed. These bugs can range from smaller errors to large performance inefficiencies or security breaches. m
Here are a few more benefits of automated testing listed out:
Ability to execute tests quickly
Test scripts run at a very efficient rate, performing tasks that might take a significant time longer if done manually. Not only that, but human error is taken out of the equation.
Tests can be run on multiple devices simultaneously
Setting up a grid can speed up processes even faster. One script can be executed across dozens of devices all in parallel.
Tests are reusable and can be utilized to cover multiple scenarios
Scripts used in one scenario can also work to accomplish another. A script to test the login capabilities of an application can be used to both test for positive and negative outcomes.
Scripts can be directly linked to a CI/CD pipeline like Azure DevOps and Jenkins. This allows for thinks like scheduling test script executions, consolidating reporting, and getting analytical data about the application.
Lower manpower and resources
Automating tests require overall less manpower and a fewer amount of resources once everything is set up and in place. Now, a handful of people can handle the testing of a single application.
Faster application development and quicker go-to-market
The reality of automating is that applications are able to be developed at a higher speed and more efficiently and at a lower cost. Now that the SDLC is shortened, new doors open up for new features to be developed, old or outdated services to be refactored, and resources to be moved elsewhere.
Conclusion
In conclusion, we can see how automating mobile application testing provides a faster, more efficient testing process with a wider range of coverage and better test results. Test automation is inevitable if organizations wish to streamline workflows. As mobile applications continue to become a bigger and bigger part of our day-to-day lives whether it be personal or business-related, testing these applications becomes even more important. Those that choose to not at doomed to have slow development lifecycles and time-consuming testing practices.
The products and services identified on this page are third party products or services. Qyrus is not affiliated, sponsored, or endorsed by any of the owners of those third party products or services or the corresponding trademarks.
What is “The Great Resignation”?
Before we even begin, let’s look at some fascinating numbers: a record number of 4.5 million Americans quit their jobs in November 2021. Another 4.3 million Americans, or 2.9% of workers quit their jobs in December 2021 according to the U.S. Bureau of Labor Statistics. According to a study conducted by Harvard Business Review, resignations in the tech industry increased by 4.5%.
In the wake of the extraordinary crisis brought about by the Coronavirus pandemic, a new trend is sweeping across labor markets worldwide. This tidal wave of people quitting their jobs, which is being called the Great Resignation, is severely impairing business continuity forcing companies to rapidly adapt to more frequent and longer staffing shortages..
Negative Business Impact and Burden on Brand Performance
A Gallup report estimates that even before the pandemic, employee turnover was costing businesses more than $1 trillion per year. With the Great Resignation, the cost of recruiting, on boarding, and training new employees has have soared unprecedentedly and impacts are being felt in other critical areas of operations as well.
Customer Experience
The ramifications of the ‘big quit’ can be witnessed in multiple areas of customer experience, as staffing shortages often resulting in poor customer service and inconsistent digital experiences. Organizations that have created new digital experiences overnight just to offset the lack of resources are now facing the heat from customers who are increasingly frustrated with challenging interactions and inconsistent customer journeys.
With robust visual reporting including step by step screenshots and full length videos of test executions and your customer journey, discover how Qyrus test automation can play a pivotal role in enhancing customer experience in a holistic manner here.
Time to Market
However innovative a product may be, the failure to be first to market can provide competitors a winning advantage. The ability to hit the market hard and first requires optimal utilization of resources in the entire production pipeline. Employee shortages can be a substantial impediment leaving companies with little choice but to either push inferior products to customers or suffer crippling delays.
Check out more insights on how Qyrus features such as in-sprint test automationand automatic test scheduling capabilities can help businesses achieve faster time to market without sacrificing quality.
Cost Implications
The cost of replacing an employee can range anywhere between 50 to 200 percent of their salary. To avoid these substantial costs and to plug the leak, robust retention programs are essential. Without them, companies risk losing their best people which can mean losing your effective leaders, persistent innovators, and effective problem solvers.
While businesses will continue to rely on human intelligence, problem-solving skills, and tenacity, Qyrus test automation can assist businesses with more routine activities. Qyrus’ sophisticated recorders streamline test building for web and mobile applications ensuring your team can build more and build faster with fewer resources. Learn more!
Innovation Gap
Innovation is not an act of serendipity but an outcome of knowledge and purpose by dedicated individuals or teams to create something enterprising. More often than not, common innovation goals are pushed down to employees without equipping them with necessary tools to implement the ideas. Organizations embarking on all-encompassing digital transformation must align their spending strategy not only towards adopting new technology but also on closing the digital talent gap, a prerequisite for innovation and digital transformation programs.
A cloud based solution with browsers, Android, and iOS devices around the world, Qyrus provides on demand scalability with comprehensive codeless testing. Qyrus is an transformation ready platform that is bridging the innovation gapin software testing, enabling businesses to generate higher ROI in the short, and long run.
Employee Skill Mismatch
Last but not least, people quitting their jobs is leading to an substantial skill shortage. Employees leaving in record numbers and leaves companies battling to find replacements while having a negative impact on effective operations and the bottom line.
Qyrus’ test automation and machine learning algorithms are a solution to address this employee skill mismatch. The codeless platform provides Healer AI to reduce maintenance for web and mobile automated tests as your application changes. Read this blog to learn more about how Qyrus offers a solution!
Why is Automation Your Safe Harbor?
As companies continue to face staffing issues, automation has become the area of focus once again. Automation does not remove humans from the picture, it only means solutions and tools that enable a better allocation of resources in key business functions. Qyrus is a simple solution that provides codeless automation for testing that can be used to augment the skills of an expert automation engineer or empower a manual tester to start automating.
A few benefits of automation with Qyrus include:
It saves time and cost
It standardizes processes and promotes scalability
It empowers organizations with auditable records
It enables a better customer experience
It ensures better compliance
Automation in Testing is a No Brainer
The crux of the matter is anything that can be automated, should be automated and that’s true for testing as well. If your company’is pushing new software releases in rapid succession, testing must rely more heavily on automation tools. Let’s just put it out there even if it is an uncomfortable truth, artificial intelligence and robots can help businesses accomplish many things faster, more accurately, and more efficiently than with people alone.
Qyrus is a codeless, intuitive, and collaborative testing automation toolthat deploys innovative AI and ML to augment human ingenuity. By leveraging the power of Qyrus, your development and testing teams will produce higher-quality products with minimal human intervention delivering an exceptional user experience with each rollout while boosting your team’s productivity and efficiency.
What is Chaos Testing?
Chaos Testing or Chaos Engineering is the discipline of experimenting on a system in order to build confidence in the system’s capability to withstand turbulent conditions in production.
What does this mean for software testing? And how can chaos engineering be implemented?
Traditionally most of the research and principles of chaos engineering and testing have been on physical chaos – network disruptions, over-consumption of memory, CPU, etc.; we believe there is much more to chaos testing and engineering.
Qyrus has always focused on reusability and with many of our newer features, we are also focusing on repurposing. For example, repurposing a functional web test into a load-generating performance test, “repurpose a virtualized API into a chaos inducing agent.”
Chaos Testing with Qyrus API Process Testing
An ecommerce API process test on Qyrus
In the image above, you can see an ecommerce API process test on Qyrus which represents a set of orchestrated API calls that are made when an order is placed. As you can see, to successfully place an order after the user submits it from the UI, the system has to check for payment status, inventory, shipment dates, CRM, etc. All of these APIs must function as expected, in time, and in correct order for this transaction to be successful.
If you are responsible for testing the above system, here are just a few of the possible things that can go wrong:
The individual APIs don’t respond in time or in other words, they don’t stick to the SLA.
Any one or multiple APIs violate the contract. For example, the OrderID was supposed to be an integer which is 9 digits long and one of the APIs sends an alphanumeric 9 digits.
An API Schema is changed without notice.
In a well-orchestrated system with hundreds of APIs working in sync, one API going rogue by having any one or more of the 3 issues mentioned above can induce enough functionalchaos in the system. As a result, this affects many other processes than just placing an order. How well is this system designed to handle such conditions gracefully?
Qyrus SV + API Process Testing = Functional Chaos Testing
The critical question here is, how are testing teams equipped today to test for the above situations? At Qyrus, we have solved this exact problem through a combination of API Process Testing and Service Virtualization (test doubles).
Building an API process test is simple – just reuse functional tests and create connections to transfer data between endpoints to represent a real-world workflow; any of the APIs in this test can be virtualized by building and hosting it through our Service Virtualization module.
Service Virtualization in Qyrus, unlike a standard approach, gives freedom to teams using it to build their virtualized APIs with no code and have complete control of request – response pairs, persistence of data, delay induction, codeless schema design, and synthetic data generation.
The virtual API in the process test can be programmed to go rogue by delaying the response, violating the contract based on a certain request from other real APIs in the system, or passing incoherent data. By doing so, teams can perform a “What if?” on all the 3 conditions above and ensure the system handles the situations gracefully with proper error codes and statuses.
As traditional industry and business lines blur in the rapid pursuit of customer-centric ecosystems, there is mounting pressure on developers to release new capabilities, features, and changes quickly. The overall quality of the software is more critical than ever before. If these integrated, inter-dependent ecosystems are to survive and eventually thrive, they may want to look to automated software testing.
Common Misconceptions About Test Automation
Although software drives big ideas and massive changes across numerous business models, quality and testing continue to be the last innovators. Often, organizations view software testing as a necessary evil. Instead, they should embrace it as a strategic driver in a competitive world with limited second chances. In those instances, where organizations see testing and automated testing as an essential element of the overall solution and market strategy, they often succumb to continuous challenges that typically don’t lend themsleves to flexible and straightforward testing processes.
Consider this statement, “Well implemented automated software testing improves development output and speed to market.” 65% of organizations saw a reduction in test cycle time with automated testing and 62% saw reductions in test costs with automated testing (Sogeti, World Quality Report 2020-21). Speed to market through rapid product release cycles is a basic expectation in today’s market. Although this is true, users won’t forgive or, more importantly, are quick to punish buggy and poor-quality software. These issues caused by poor or inadequate automated testing and technical/skill challenges can have a debilitating impact on delivering software.
Key benefits of automated testing:
Shorten development cycles: Automated testing saves time through faster processes and increases the pace of high-quality development.
Improved testing efficiency: The reusability introduced by automation improves overall testing and development efficiencies. It also means more test coverage where required, which is impossible in manual mode.
Implement CI/CD and DevOps: Automation testing is an essential element of Continuous Delivery and DevOps practices. It shortens the loop between developers and test results, allowing for faster feedback on quality. Automated testing can eliminate the burden of automating routine, labor-intensive tests that take up a tester’s time. The time saved can be spent automating more tests, designing, and monitoring test execution.
Reduce cost: There is a prevalent view that automated testing is cost-intensive. On the contrary, unlike manual testing methods, automated tests can be executed more frequently, faster, and allow testers to increase coverage. Test automation implemented correctly will significantly reduce costs.
How to Select the Right Automation Platform?
The importance of platform selection cannot be over-emphasized. It must fit the environment, skills, and objectives of future product roadmaps and is the essential core of your automated testing practice. Furthermore, its value is further enhanced by coordinating test execution with other delivery tools to create a software delivery pipeline best suited for your organization’s needs.
Consider these points while selecting a platform:
Current Environment: Align the platform to your environment. How will it integrate with the ecosystem that’s already in place?
Type: Is the platform SaaS or on-premise? Will it test your future enterprise solution?
Integrated: Is the platform an integrated testing suite? Is it capable of testing web apps, APIs, business processes, and mobile device? Is there a need for an integrated device and browser farm?
No Code or Coding Model: Align with your team’s skillsets but also future-proof to the extent possible. Do you want to invest in a multitude of coding models?
Support: Is there product onboarding and testing process support?
Types of Automated Testing:
Modular Testing
Define and manage your tests as small, functional-based modules tested independently or efficiently grouped into functional or end-to-end business process test suites. Start automating small, simple test cases and increase the complexity of your scripts by identifying tests prone to human error, that use multiple data sets, and that require repetitive test runs on numerous builds.
Self-Healing Tests
Automation scripts are often flaky and brittle when the configurations, objects, and test data change since the last test execution. Even when scripts are maintained – which is often not the case – the changes are not continuously updated in the test script. This can cause erroneous test results. In many cases, it is impossible to fix scripts in a timely, cost-effective manner manually. Self-healing tests identify and remedy these issues, which often can be as simple as a button that has moved somewhere else on the page.
Mock and Stub
Reduce variability around test failures. Tests should fail when there is a bug. You can design and implement mocks and stubs until real system components are available for end-to-end testing.
Govern Automated Testing
Create a governance structure for automation testing. Make transparency a core element of the automation strategy. That way, all stakeholders – developers, database teams, business analysts – can contribute to the success and take advantage of the value provided by the platform.
What is Qyrus and how it can help you:
Qyrus is an on-demand SaaS codeless automated testing platform that deploys machine learning algorithms, an intuitive user experience, and a collaborative codeless approach to test automation. It not only empowers developers and testers with automation testing, but, just as important, the business team. When you partner with Qyrus, you can expect an exceptional user experience and greater productivity from your development and testing teams.
We are excited to share some test automation news – Qyrus is now officially SOC 2 Type 2 certified! We understand how our client’s tests, results, and other artifacts are vital to their business. This is one of the key reasons we’re thrilled to take our operational excellence and data security to the next level; Qyrus is now officially SOC 2 Type 2 certified!
We at Qyrus take customer data security seriously. This is why we have updated our security measures. This certification, which is facilitated by an independent third-party certified audit firm, is the standard for how we conduct our business presently, as well as into the future.
At Qyrus, we are committed to the highest security standards, including:
Regular vulnerability scans
Periodic third-party penetration testing
Strict code quality and mandatory security reviews
Stringent SOC 2 compliance vetting for all critical vendors
Enforcement of data security best practices across the Qyrus organization
