Shoes, clothing, and water bottles, with almost all consumer items, there is often a negotiation between quality and price. As brands flourish on being high quality and best in class, consumers can often find a similar money-conscious option available. But as with most decisions, it is important to be critical and note that if it sounds too good to be true, it just might be. These ideals remain true across a range of industries, including development and quality assurance. This week’s feature Friday discusses manual and automated testing using the Qyrus device farm, and some of the benefits of testing applications using real Android and iOS mobile phones and tablets located across the globe. Let’s dive in as we interviewed Jorell and Raoul to learn more.
Tell us more about the testing process using the Qyrus device farm, its use cases, and its impact on testing and QA processes?
Jorell: Though very powerful the Qyrus device farm is actually simple. Real physical Android and iOS devices hosted within the cloud to support manual and automated testing of native applications and web applications.
Raoul: Exactly, all the functionality, each test has videos, screenshots, performance profiling metrics, and device logs. This means that if a defect is found – the developer has as many pieces of evidence to comprehend, recreate, and remedy the issue – Qyrus has native configurations to Jira which means an issue can be filed directly from the Qyrus test report!
Does the same or similar functionality exist without Qyrus, and how do competitors address similar problems?
Raoul: Prior to device farm capabilities and during the testing process, testers would have to set up and maintain all required devices for testing purposes. Missing out on a range of features and functionalities, the process also requires more resources and time.
Jorell: And though other device farms exist, Qyrus differentiates itself through experience and capabilities. Feature-rich including the ability to toggle between network options, extensive reporting with screenshots and videos, all the way through offering beta and previous versions of Android and iOS to test on.
What is the overall impact on the testing process when using the Qyrus device farm?
Jorell: When considering the testing process as a whole, the device farms impacts are clear. From the beginning, device setup, and maintenance are mitigated. That’s a heap of time and resources, and as requirements grow, the cloud-native solution grows as well, offering custom and private device options.
Raoul: Furthermore, during the testing process devices are easily accessible at all points in time. Teams can share devices, monitor runs across devices, and test in parallel, with extensive reports and collaborative features making the entire testing and QA process more efficient.
How might the Qyrus device farm testing options help testers, developers, and business technologists? What value can this feature bring?
Raoul: Very broadly – you can test any scenarios, applications, and devices through this feature. Anything that the tester needs from manual testing to automated testing is both available, maintained, and ready to scale up or down as requirements change with no testing or developmental resources or overhead from the client side. This increases test case coverage allowing testers a range of devices on hand to test application functionality.
Jorell: We often find developers uploading beta versions of Android and iOS applications to test within the developmental process. Furthermore, with out-of-the-box Jira integrations, communication between developers and testers becomes seamless. Promoting high-quality applications.
Raoul: Furthermore, coupled with an amazing UI and rich visual reporting, Qyrus creates ease of access for business analysts to access application information. Providing screenshots, videos, and the ability to email and download reports, Qyrus provides the necessary application information and collaborative capabilities for business analysts to integrate themselves into the testing process and QA lifestyle.
How do you see the Qyrus device farm impacting day-to-day operations across the organization?
Jorell: The day-to-day impacts of this feature cannot be downplayed. Teams have access to any and all required devices for testing purposes. By enabling efficient testing utilizing parallel testing features, testers, and developers can establish functionality across a multitude of both Android and iOS devices simultaneously.
Raoul: Furthermore, by offering both private devices and a public device pool, clients can utilize devices across the globe. Test case coverage is no longer a concern. Furthermore, with options to scale up and down as seen fit, clients can ramp up devices during release and scale down accordingly to prevent excess spending as requirements change.
As applications dominate the forefront of business, there are certain things that are worth going the extra mile for. High-quality mobile applications are functional regardless of the device. But building out device-agnostic applications and testing them across a range of Android and iOS devices to promote the best user experience can be a daunting task. With a multitude of devices and mobile operating systems at your fingertips, and the ability to upload and test beta versions of applications, within a powerful solution providing robust visual and data-driven reporting, Qyrus’ Device farm aims to mitigate all aforementioned headaches by increasing testing efficiency, coverage, and enabling high-quality application production.
We’re excited to share that Quinnox has been recognized as a “strong performer” in the Forrester Wave™ for Modern Application Development, Q3 2022.
According to the report, Quinnox shines across most of the current offering criteria. These include modern development and delivery services with agile, product development, DevOps, SRE and automation, architecture API and microservices, continuous testing, cloud-native, AI and data, case history examples, metrics, methodologies, frameworks and software, and vertical industry knowledge” (All good things in one place)
Autumn is just around the corner, and with that the warm winds leave the Northern Hemisphere. Hopefully, if you’re from the Southern Hemisphere, you’re gearing up for the summer heat already! And hopefully, if you’re a tester, you’re thinking about gearing up for better and more efficient testing strategies! Today on this Feature Friday, we are joined by Daniel and Amy who are going to talk to us more about our exciting new feature, Test Data Management!
Tell us more about Test Data Management offered by Qyrus and its use cases.
Dan: Well, Test Data Management is a feature that manages your test data in one place. It helps eliminate the tedious steps of importing data from external sources like an Excel file. It also has an added benefit of allowing users to synthetically generate data within the Test Data Management system itself for usage during runtime.
Amy: So, when you import data from a database, you need only to establish a connection one time. Afterwards, Test Data Management will retrieve the data anytime a test is run that uses that data. Furthermore, you can even grab data from an API call for usage during runtime, as well.
Interesting… it seems like a tool that can cover a lot of use cases under one umbrella. This must mean it has a significant impact on testing processes throughout.
What is this feature’s overall impact on the testing process?
Dan: The areas in which we see the largest impact would probably by test building and execution. It has a direct benefit in terms of test coverage, allowing users to quickly build tests that use data dynamically and during runtime.
Amy: Instead of having to incorporate some 3rd party tools to generate data, it’s all done from our Test Data Management system. Connections to databases and pulling data from API calls during runtime helps enhance the testing capabilities that Qyrus already provides to its users.
Fragmented environments are the worst. They cause problems and if no problems exist today they will months or a couple years down the line. The ability for users to have a centralized system to handle all of their needs is important. How might other users, besides testers, utilize this feature?
How might Test Data Management help testers, developers, and business technologists? What value can this feature bring?
Amy: Well, a tester would utilize this feature heavily. Again, it has a large impact on the test building and execution process, and would empower testers to manage their test data more efficiently. This also helps in terms of maintenance from their perspective.
Dan: And developers can use the Test Data Management system to generate synthetic data to quickly build sample tests or unit tests even. This could help them then catch bugs before it even makes it to the testing phase.
Amy: And the cool thing about the system is that it can generate data similar to what you provide to it. So, in that aspect, a business technologist could provide a sample set of data and use the tool to expand that sample set.
So, with it being useful for a wide arrange of users and personas, we might see this making waves in the ebb and flow of normal business operations. Let’s learn more!
How do you see using Test Data Management impacting day-to-day operations across organizations?
Amy: Well, it helps increase test coverage at a faster pace, so day-to-day operations might be sped up to a degree. Data imports are faster, and AI is utilized to help generate sample data. All of these help speed up the testing process.
Dan: On top of that, from a maintenance standpoint, it all becomes manageable from one spot. It’s super easy to take care of. Connect to your database once, and never worry about it again.
Does the same or similar functionality exist without Qyrus, and how do competitors address similar problems?
Dan: From a competitor standpoint, not everything is integrated and combined in one place like we have in our Test Data Management system. From being able to connect and import data from databases, API calls, and synthetically generate similar data, we have it all in one spot.
Amy: And before Qyrus, testers would have to have studied and built up a large amount of coding knowledge. That’s if they want to replicate it from the ground up. However, they could use a fragmented suite of tools to help achieve the same functionality.
Enjoy the last remnants of summer while you can! Before it gets chilly up here for us northern folk, we’re out to enjoy one last hurrah before the bell tolls, signifying the end of our summer. We hope you learned a lot about our Test Data Management system and how it can bring value to a tester’s day-to-day life. Hopefully we’ve convinced you to give Qyrus out a try. Until next time, thanks for joining us for this Feature Friday!
Quinnox, a global provider of full-spectrum IT and digital solutions, is a material outsourcer to Shawbrook, and a key enabler of their technology transformation journey. For Shawbrook, this recent tie-up with Qyrus is another step towards testing at the speed of digital, re-invigorating their relationship with customers, improving customer satisfaction and loyalty and accelerating business outcomes.
Quinnox uses Qyrus, its proprietary AI-powered test automation platform, to develop new systems and change existing processes and systems for Shawbrook, a new type of financial services company in the UK that combines the relentless focus on customer service and innovation that would be expected from a fintech with the expertise and certainty of a bank.
As part of their ‘shift-left’ quality strategy, Qyrus supports Shawbrook’s ambitious delivery schedule of new product features to customers as well as puts its focus on automation, through automated regression testing across the breadth of the Shawbrook Platform.
Qyrus helped Shawbrook transform its Platform to seamlessly automate savings account opening & servicing, real estate & SME lending origination as well as customer due diligence onboarding processes.
Shawbrook was able to deliver releases in a much shorter time using the Web, API, and Business Process testing capabilities of Qyrus. The outcomes this drove were faster time to feedback, increased velocity of deployments, better quality of releases with significantly less effort.
Full details on Shawbrook’s strategy to develop its next generation banking platform and 2021 commercial performance can be found in their 2021 Annual Report.
Russ Thornton, CTO of Shawbrook
“Quinnox is a key partner of ours, not only supporting our technology delivery teams, but with tools like Qyrus. Test automation is a critical capability for any engineering team, and Qyrus supports the testing of our web, API and mobile components as a part of our CI/CD processes, but also in on-going regression testing across our partner ecosystem. The real power of Qyrus is that we have this extremely broad testing capability in one tool, run in the Cloud, and reusable across all our development teams”
Somnath Chakraborty, SVP, Head of Quinnox Europe
“We are delighted to have Shawbrook as one of our strategic clients here in the UK and this partnership has grown strength to strength, as together we intend to drive positive change in the Banking industry and are proud that Qyrus is able to help accelerate it.”
About Shawbrook Bank:
Banking for the real world: Over the last decade we have built a new type of bank that is designed to accommodate individuality, diversity and the dynamics of the modern world. Our ability to provide highly personalized finance solutions to help our customers achieve their immediate ambitions is enabled by our unique platform, which combines and integrates modular technology with human expertise, judgment and ingenuity. We power up ingenuity to create opportunity, every single day.
About Quinnox:
Quinnox is your agile, business-results-driven digital technology partner. With the power of human and applied intelligence, we simplify business processes, improve customer experiences and create exceptional business value for forward-thinking enterprises. Our data-driven digital solutions unlock the hidden potential of your business across your digital value chain, helping to accelerate success, today and tomorrow. We are headquartered in Chicago, Illinois, with a presence across geographies including UK, Germany, UAE and India.
About Qyrus:
Qyrus is a codeless, intelligent, and collaborative test automation platform for all things digital. It efficiently tests Web, Mobile, and APIs along with end-to-end business processes to drive quality, speed and frequency to market. As an on-demand AI-powered SaaS testing platform, it can scale with your needs enabling teams to deliver higher quality software faster and more efficiently thereby ensuring a seamless customer experience across all stages of the digital journey.
Over the years, we have seen many examples of large ecosystems and species harmed by changes in predator and prey relationships caused by natural selection.
Is it so naive to say today’s technological era is accustomed to rules that may not be so different? The consumer chose movement and reach, and thus laptops rendered towers and workstations near obsolete.
The Kindle Fire as a primary reading utensil was dominated by Surface Pros, iPads, and other powerful hybrid devices which have since transitioned into handheld mobile devices with hundreds of gigabytes of memory and multi-core processors capable of running laptop-equivalent functionality.
This continual necessity for simpler and more practical technology has led to an influx of devices running on an array of differing operating systems connecting to multiple browser versions amongst more.
The options seem limitless and, often, overwhelming for those who aim to create or maintain applications of any caliber.
When looking at these challenges in business processes, it’s important to ensure that the software works well on different browsers, devices, and operating systems. This might mean using a device farm.
What is a Device Farm?
Device farms are dedicated hosting locations for a multitude of devices, used for testing application functionality in real-time.
But, with industry leading names like Facebook, Instagram, Tinder, and Uber topping the list of companies with the highest percentages of personal data collected, it can be daunting choosing a platform or provider for your entire business’ devices and testing with the confidence that your data is secure.
Qyrus’ dedicated, cloud-based mobile device farm utilizes three prominent business fundamentals: power, performance, and privacy to maximize all facets of testing.
What Makes Qyrus’ Device Farm Different?
The Qyrus device farm is one of the prominent AWS device farm alternatives. It contains a multitude of different tablets and mobile devices to use for device farm testing including but not limited to iPads, various Android tablets, iPhones, and a multitude of Android-based mobile devices with device customizations available to fit specific client needs.
Alongside this, with the addition of parallel testing features, Qyrus allows multiple tests to be executed on multiple devices and operating systems simultaneously. The versatility provided by an option pool covering all facets of devices and testing synergized by the efficiency of parallel test execution comprise the power of the Qyrus platform, a platform that welcomes unique testing challenges.
The Qyrus device farm goes even a step further. With performance profiling metrics, further understand the application being tested. Learn how the app affects battery consumption, CPU and memory usage, and other device vitals. Screenshots can be captured during a session for reporting and device logs are also available for further debugging.
Qyrus’ device farm was built on a business centric interface that allows a user the freedom to connect and test from any device, in any location, and at any given time, within clicks. Emergencies are eminent and business never sleeps, therefore, your testing solution shouldn’t either.
Run suites of tests through the night using Qyrus’ scheduled run feature and wake up to a confirmation that your applications are properly functional. Furthermore, with a detailed and easily sharable analytics and reports tab, being in touch with your past and present business processes has never been easier.
Twenty-four seven coverage and testing capabilities with full scale reports and analytics, accessible from any location, built for a collaborative mentality, Qyrus specializes in increasing efficiency throughout all testing-based business, and development processes.
Qyrus’ device farm goes a step further in capturing useful data and metrics. With performance profiling metrics and
Qyrus Takes Your Data Security Very Seriously
Data security is becoming necessary at an alarming rate. As technology becomes more prevalent, the same can be said about cyber-attacks and data theft. Qyrus leverages private instances with each of their clients to allow maximum security through testing processes.
That’s to say, regardless of whether you are sharing, or have private devices, your instance of Qyrus and all company-related data hosted, by, or on Qyrus is private to you, and inaccessible by any other Qyrus users.
Qyrus also allows clients the option to essentially build their own device farm, providing private devices, instances, and specifications as required for optimal security.
Furthermore, Qyrus is empowered with many security features including single-sign-on options, and security testing features such as web token implementation and testing, alongside certificate-based authentication within API testing to ensure your web apps, mobile apps, and API’s are stable and secure.
As a recently certified SOC level II compliant company, Qyrus excels at all five pillars of the certification standard including security, availability, processing integrity, confidentiality, and privacy, standardizing a level of credibility and professionalism throughout client and business interactions.
Maximum coverage is the obvious combatant to ‘technological selection’. Keeping it simple, the more people you can reach the higher your chances of survival, and in this age of endless options, device farms are essential to ensure functionality and user retention effectively on arrays of different browsers, devices, and operating systems.
Technology can connect any corner of the world, and instead of fire or the wheel, it is device farms, and the simple, smart, and scalable solution in Qyrus, that stands as evolutionary assistance through unpredictable times.
Elevate Your Testing with Qyrus Device Farm
In today’s fast-paced digital landscape, ensuring that your applications perform seamlessly across various devices and platforms is crucial. The Qyrus Device Farm stands out as a premier solution that offers unparalleled testing capabilities, enhanced data security, and the flexibility to adapt to unique testing requirements.
With features like parallel testing and performance profiling, Qyrus not only simplifies the testing process but also empowers businesses to optimize their applications effectively.
Are you ready to elevate your testing strategy and experience the difference for yourself? Sign up today for a free trial of Qyrus Device Farm and transform the way you conduct application testing!
In this discussion’s simplest form, we pose a question, “If there was a tsunami en route, are you positioned on the shore?” The days of garage launches and the necessity of brick and mortar are becoming lost to the sands of time. The Technological Era through its comfort and practicality has become a silent killer. With Victoria’s Secret, Bed Bath & Beyond, Gamestop, and Men’s Warehouse topping the list of chains permanently closing the most stores in 2020, the movement inland must come with swift technological adaptations.
But this concept in and of itself is both foreign and daunting to most. Meanwhile, noticeably forcing the hand of industry leaders, it becomes clear that the transformation is imminent but how to go about it is less transparent. With differing standards including new and pre-existing business management, transitioning to online or handheld effectively and efficiently often requires a large checkbook and developmental nightmares on the road to functionality. But as the times indicate the only alternative seems to be going under.
Importance of Automation is Software Testing
As in every proverbial dark tunnel, the developments in testing, specifically, automated testing technology, stands to be the light. Providing the ability to build test cases around individual elements, pages, and sites while storing and running them at your convenience, testing technology has taken the initiative in paving a simpler path through the development and deployment cycle. Build, test, save, repeat: a simple process that allows you to manage functionality with constant feedback, regulation, and iteration.
Qyrus – Benefits of an All-in-One Testing Platform
In this realm of testing technology lies the answer to the question, “Why Qyrus?” Qyrus is an inclusive platform providing Web, Mobile, API, Artificial Intelligence, and Infrastructure testing. With a business-first structure, Qyrus allows you to begin with web testing and transition through the platform with your business progression. Meaning as your business goes handheld so does Qyrus, providing you with comfort through the troubling tides of various development and deployment cycles. Hosting international device farms based with the addition of client-specific private instances, the platform provides power and security, but even more so a solid foundation that as your business scales your testing solution remains reliable and comprehensive.
Beyond comfort and business progression there stands an area that Qyrus has re-defined; the testing market. Qyrus’ “low-code, no-code” platform fused with a user-first interface allows full-scale tests to be built in minutes using a series of clicks. Reports immediately provide screenshots and video feedback of the live test upon run completion. But we didn’t stop there, with platform-specific features and AI such as Hawkeye, Healer, QyrusBot, Sage, and Rover behind the simplicity, Qyrus has the power to meet a wide range of testing requirements and needs increasing in levels of complexity. All placed behind a series of clicks, Qyrus has the power to map and record every click of your mobile application, the intuition to fix brittle and defective test scripts through AI implementation, and future additions including the use of Native language commands to build and test. As a field dominated by software developers and engineers, testing has now been simplified to the likes of high school students thanks to Qyrus.
The Future of Software Testing
The motion towards web and mobile applications was once inevitable but is now upon us. With the global number of mobile contactless payment users growing to 760 million, including roughly 30% of the North American population shifting to mobile wallets, it becomes as simple as following the money. But with the synergy of power and simplicity, and the streamlining of extensive test coverage attainable by professionals of any background, Qyrus simplifies the web and mobile transition, paving the path inward for businesses in wake of the eminent technological tidal wave.
Ahhh, Spring. What a wonderful time to open our windows and doors and let the fresh air into our homes. As the cover of winter slowly lifts, trees begin to bud, birds begin to sing, and the scent of fresh flowers tickle the nose.
But let’s not forget the dangers that may lie just around the corner. Securing our applications, much like our houses, is of critical importance. Sensitive data and information can be accessed and stolen, similar to a burglar breaking into your house. Many applications require tokens or keys to access data. But, how exactly can those processes be tested?
This week’s Feature Friday covers just that. A prerequisite API test enables the testing of data-dependent APIs and API processes. To learn more, we interviewed Daniel and Steve on the Qyrus team:
Tell us more about the prerequisite API testing feature offered by Qyrus, its use cases, and its impact on testing and QA processes.
Steve: Just like the locks on a door, applications often work the same way, especially when they contain sensitive user and organization data. To test or even enter these applications, users require a unique key or token. This is a prominent use case for prerequisite API tests. Simply set up the prerequisite API, and it will execute prior to the API test, providing the required data to continue the testing process.
Daniel: More than just security tokens, prerequisite APIs come in handy anytime initialization is required. Often, a particular API is dependent upon input data from a different source, most likely another API. In order to truly test the behavior of the data-dependent API, it is nice to completely mimic this behavior. A simple prerequisite API can be used instead of building an API process of two items, saving time but accomplishing the required behavior and data transfer process.
Data-dependent APIs are fundamental in application creation. Therefore, it’s important to gain some knowledge and background on how these use cases are tested today. So, we asked:
Does the same or similar functionality exist without Qyrus, and how do competitors address similar problems?
Steve: Other competitors often require extensive setup, usually utilizing code, to run the prerequisite API, enable data transfer, and trigger the accompanying API test. Though, at the heart of it, the process boils down to calling concurrent API tests, Qyrus simplifies script building and management, mitigates coding requirements, and enables easy environment setup and dynamic data transfer all in one intuitive feature.
Daniel: Prerequisite APIs on Qyrus differ from our competitors because we provide rich reporting, including pass and fail indicators, execution time, and response headers and body. Furthermore, we offer the ability to dynamically use the responses, building them into dynamic variables that can be implemented as desired across test scripts in the project.
What is the overall impact on the testing process when using prerequisite API testing with Qyrus?
Daniel: We no longer need to manually enter in pre-request credentials for every pre-request API call. With the prerequisite API feature, the tester enters the pre-request credentials once, and the prerequisite API will always run before a request execution takes place.
Steve: Additionally, building these interdependent tests could be a little complicated, as the asynchronous behavior of APIs can be conceptually difficult for newer developers to understand. This feature makes testing easier by eliminating the coding barrier required to build these scripts.
Daniel: Furthermore, a user is able to dynamically use the response body from the prerequisite API into the request execution. In terms of data-dependent API testing, this is a simple, smart, and comprehensive option.
The prerequisite API test provides a lot of utility to users, that’s for sure. Newer developers have less of a hurdle to jump over, as just mentioned. The leap forward cannot be understated. However, does this mean that API testing will no longer be dependent on those with a high level of programming knowledge?
How might prerequisite API testing help testers, developers, and business technologists? What value can this feature bring?
Steve: A great question. We find a handful of personas are able to use this feature to test required data-dependent APIs. Testers use this feature to enable an efficient and high-level data transfer process. To be specific, testers build out comprehensive prerequisite API tests across complex use cases through simple form-filling functionality. They can quickly set required data as dynamic variables and implement them directly into the API test for steadfast test building, execution, and data transfer processes. Testers often use this feature to speed up the testing process of production-level APIs.
Daniel: That’s correct. Furthermore, we find developers using this feature to establish API functionality during development. Creating prerequisite APIs allows for the mapping and pre-development of API processes. Given the capability through prerequisite API testing, developers can pre-determine requirements and design their APIs to target specific use cases. Upon creation, a prerequisite API test can be created and conducted to validate the functionality of both the APIs and the data transfer process.
Steve: Also, the codeless nature of prerequisite API testing makes it easy for business analysts to build and execute prerequisite API tests. In fact, this feature could help provide more domain knowledge for business technologists. With form-filling functionality, business analysts could further understand the interdependent relationships between the APIs created by the business.
A unique feature for all personas within testing teams, we felt the need to further implore its usefulness to testing as a whole:
How do you see CI pipelines impacting day-to-day operations across organizations?
Daniel: The main objective of prerequisite API testing is to provide a functional solution to data-driven API tests where requirements are contained in a logical fashion without removing the modularity of the API tests.
Steve: Exactly, you can run functional tests on the prerequisite API. Once you have confirmed that it is working, you can then apply the same process across all APIs. Also, after a Prerequisite API is created and executed, a user is able to dynamically reuse the response body across the entire API project.
Daniel: And with a bright future roadmap, the prerequisite API feature aims to allow users to add logic or code snippets to the prerequisite request. This can be useful if a request header expects a unique and specific input.
That concludes this week’s Feature Friday! As the week concludes, let’s all take a breath of fresh air in anticipation of even better days ahead. Take a walk, stop and smell the flowers, and maybe have a picnic. Whatever it might be, we hope everyone enjoys the coming Spring!
They’re everywhere! Now that summer is rolling through the Northern Hemisphere, BUGS are everywhere! Yes, I’m talking about real, physical bugs, not the ones we find in our software. But, I suppose those are abundant everywhere, as well. Similarly, BOTS are now everywhere. From crawlers to transactional and informational bots and even Conversational Agents – or chatbots – the focus of this Feature Friday! Today, we’re here to introduce you to our new tool to test and train AI chatbots – BotMetrics.
What do you mean?
Chatbots are becoming more and more intelligent with advances in the area of Natural Language Processing and Understanding. But still, organizations and industries are hesitant and skeptical to adopt these agents. Not because chatbots are hard to build or develop — one can develop a chatbot easily using Google DialogFlow or Amazon Lex — but they are hard to test and maintain. Now, what if we say: we have a bot (we have named it – BotMetrics) that can connect to your Conversational Agent/chatbot, have a conversation with regard to your bot’s functionality, and rate your bot’s performance. Just like AI Gilfoyle having a chat with AI Dinesh, but without breaking anything. You might be thinking, “That’s sick!” It doesn’t end here; BotMetrics also helps your bot’s performance by providing it with data related to all the failed conversations they had.
What is BotMetrics?
BotMetrics is our Conversational Testing Platform, the very first of its kind. Currently, it’s under wraps, so we will try to divulge as much as we can to get you excited. We wanted to develop a Super Agent Generalized on Everything, and we started developing BotMetrics. Currently, it works with the English language, and there is a long way for it to go. BotMetrics can be seamlessly integrated with chatbot development platforms like Google’s DialogFlow, Meta’s Wit, and Amazon’s Lex to test and maintain your Conversational Agent/chatbot. SAGE tests the agent’s capability to understand and reply to phrases/utterances by creating hundreds to thousands of virtual conversations with your agent in just minutes.
Why BotMetrics?
We already have a lot of test automation functionalities to test APIs, Web Applications, and Native Mobile Applications in Qyrus, but we wanted to develop (still developing) something new which has never been developed or even imagined.
With the advancements in Natural Language Processing and Understanding, we thought, why don’t we create a Super Agent, which can ”test” other Conversational Agents/chatbots? Moreover, it’s hard to benchmark/test AI, and there is a scarcity of good platforms that can aide in testing and maintaining chatbots.
What does BotMetrics solve?
In the field of Machine Learning and AI, the models are benchmarked using a set of test data, and the model’s effectiveness is decided based on its performance on that data. Let’s take an example of a conversational agent who is trained to provide customers with details regarding their credit and debit accounts, transactions, purchases, and deposits. In this case, the agent needs to learn what type of sentences or phrases point to which of the above-mentioned category.
Moreover, the same thing can be asked in many different ways. Let’s say I want to know the outstanding amount on my credit card; I can ask the agent in a multitude of ways — what’s my credit card bill for this month, or what’s the due credit, or what’s the minimum due for this month? Still, these are normal phrases but if a non-native speaker is trying to interact with this bot and if they make a spelling mistake or the structure of the sentence is not correct — for example, “What account balance?” instead of, ”What’s my account balance?” — the bot won’t understand it and fail to provide an appropriate response. Not all of these test cases will be covered in the training, validation, or test datasets. Only when the user gets their hands on the agent, the developer comes to know about all of the shortcomings.
We have added different testing modes in BotMetrics which helps the developer check how their agent is performing with badly written phrases, phrases with spelling and grammatical errors, phrases with different slang, phrases with non-trivial words, etc. Plus, we provide a data upload integration, so the developer just needs to upload those phrases which their agent couldn’t understand and re-train in just a few clicks. These testing features and modes help the developers to add more and more features to their agents while continuously testing, benchmarking, and validating the new version using BotMetrics.
How does BotMetrics work?
Chatbots are everywhere, like it or not. We use chatbot assistants like Siri, Alexa, or Google to get different things done from time to time. Some might use it to set reminders or timers, some might use it to check facts, some might have a casual chat with it, or some might even use it to control household appliances and electronics.
Similar to these assistants, there can be a conversational agent for banks wherein a user can check their balance or transfer money by just chatting with the agent, or a conversational agent to track all your appointments and meetings, or a conversational agent for a restaurant support system. Applications like these, which are customer-facing for support and engagement are moving towards such conversational agents to minimize cost and human intervention for servicing and support. Platforms like DialogFlow and Wit are also boosting the adoption of Conversational Agents, but testing and maintenance remain an unsolved problem.
This is where BotMetrics comes in and alleviates the chatbot testing and maintenance issues.
With BotMetrics we offer two types of testing abilities for chatbots:
Intent Testing
Entity Testing
What is Intent Testing?
Every chatbot must understand what the user is saying and act accordingly. The process of understanding what the user is trying to say by typing or speaking is called Intent detection. The output of the actions (i.e. typing or speaking) is called utterance. The chatbot first needs to understand this utterance and map it to a pre-defined intent.
If the bot fails at this very first step (and trust us, many bots fail) the conversation won’t move forward, and the user might close the application and move on.
Using BotMetrics, you can avoid this. You need only to add your pre-defined intents into BotMetrics and provide your bot credentials, and it connects to your bot and generates hundreds to thousands of conversations with the bot based on the pre-defined intents. Thus, testing your bot’s ability to understand intent.
What is Entity Testing?
When you ask Siri to “Set a timer for ten minutes…” it understands the intent first and then extracts the words “ten” and “minutes” from the utterance and sets the timer based on that. These words are what we call entities. One could use different units of time to set an alarm or timer. Similarly, there can be other cases where different words can be used for the same entity type.Based on the intent and the type of entity the chatbot supports, BotMetrics creates a lot of words of the same type and converses with the chatbot by putting these generated words into the conversation to check the bot’s capability of detecting entities.
We hope you like what you learned about BotMetrics. Again, this is just a sneak peek at what’s to come! BotMetrics is still being developed, but we have seen amazing leaps and bounds thus far with what we already have. Join us again in future postings to learn more about the amazing abilities of BotMetrics. For now, close your laptops or shut down your machines and enjoy the weekend! Stay cool, stay safe!
Budgeting, a very interesting concept that some are privy to, comes down to the recording and maintenance of all monetary transactions. But all budgeters know and fear the moment that their payments don’t match their data logs. They are forced into hours on the phone with the bank, often to realize it was a relevant purchase that was improperly logged, leaving no one to blame but yourself. APIs and databases work almost identically, except instead of transaction history, the transferring and logging is of any essential application data. This week’s feature Friday is brought to you by Daniel and Joyal where we will dive into Qyrus’ API to database testing and validation feature.
Tell us more about the API to DB feature offered by Qyrus, its use cases, and its impact on testing and QA processes.
Daniel: API to DB is used to validate an API’s behavior whose job is to save or manipulate data to a database. The user maps values in their APIs request body to a column in their database and creates assertions to ensure that the data being sent by the API is properly saved or altered in their database.
Joyal: Similar to a logbook of transactions, your database is your central point for data logging and management. Therefore, it becomes essential to test API to DB processes, ensuring that the request body passed into an API is properly saved in the database and the right transformations, if required, have been done on the data before it was saved.
Daniel: In essence, this feature attempts to answer the question, “How can I be sure that my API properly saved the data into my database?”
A very interesting feature, API to DB, is sure to make an impact on the testing process. With a feature like this, testing databases becomes much easier and seamless. We asked Daniel and Joyal:
What is the API to DB feature’s overall impact on the testing process?
Daniel: API test can be a little more technical than things like Web and Mobile application testing. It is much easier for tests to fail simply because tests were not set up properly. As such, we have focused on simplifying the configurations required in this form of testing, as well as providing clear feedback and visualizations to assist the user in the building of these tests. One example would be our JSON tree mapping tool, which provides a visual representation of the API’s request body and is intractable. The user only needs to click on the node they are looking for in order to extract the JSON path they want to map to a database.
Joyal: Furthermore, we see it solving multiple problems, as you are ensuring any logic that is executed after the API is called is working properly, as well as the API itself. So, you are testing two different entities using the same test.
Daniel: Yes, Joyal, and that’s exactly the value, keep it simple but comprehensive.
As we’ve heard from Daniel, we want to keep it simple. Complex testing processes can lead to more errors. Having to switch from one application to another just to perform a simple task is ridiculous. The team at Qyrus sought to tie it all together.
Does the same or similar functionality exist without Qyrus, and how do competitors address similar problems?
Joyal: So, we have seen prerequisite API test options, but there are just a few differentiating factors. Tests on Qyrus include in-depth reporting and the ability to dynamically use responses across API tests upon execution of the prerequisite API.
Daniel: And it’s no secret that there are multiple ways to solve this problem without Qyrus. The automated way would require knowledge of a couple of different libraries and combining them to create a multi-step process. The manual way would be akin to something like calling in an API in Postman, opening up something like MySQL workbench to make a database query, and then visually confirming that what you entered into the API’s request body was saved into the database correctly. Believe it or not, but this manual workflow is what we often hear developers are doing today.
Database testing has historically been a complex task that only those with high knowledge of the database would be able to properly test it. With the introduction of API to DB testing in Qyrus, we have seen that the process has been simplified to the point where even non-technical people have a use for it!
How might the API to DB feature help testers, developers, and business technologists? What value can they bring?
Daniel: Great question… we often find that a tester would first use functional testing in the API service to ensure that the API is “behaving” the way they want it to by providing adequate assertions in their functional test. Once they know the API is behaving as expected, they then create database assertions to ensure that the API is properly storing the data in their database. Testers would use this to ensure proper communication between their APIs and their databases.
Joyal: Developers could use this to make sure the API itself is functioning and the logic that is executed after the API is called is working properly. Furthermore, verifying the final output in the database ensures database functionality. Developers can run these test executions during sprints and through CI pipelines to ensure existing data processing functionality upon new builds.
Daniel: With robust reporting and easy-to-make assertions, this feature stands as a great transition for business technologists and analysts to further understand the inner workings of their applications. Following the data transfer and storage process is made simple with robust reporting while staying in the loop is even easier with collaborative reporting options. With a codeless approach, non-technical specialists can go as far as building out test scripts to ensure application and database synchronization and functionality.
All of this being said, we know that this feature will have an impact on testing overall. The ramifications of this process being simplified mean that certain routines or testing processes will no longer be as lengthy or time-consuming. We asked:
How do you see Qyrus’ API to DB feature impacting day-to-day operations across organizations?
Joyal: The first thing we see is that the feature mitigates redundancies in testing operations. With a prerequisite API, the tester will no longer need to manually enter pre-request credentials for every pre-request-dependent API call. Within the prerequisite API, the tester enters the pre-request credentials once, and the API will always run before a request execution takes place.
Daniel: Correct, and after a prerequisite API is created and executed, a user is able to dynamically reuse the response body across the entire project.
Joyal: We also note that this feature offers you more depth in testing. You are not just testing whether you got the expected status code or response body from an API call anymore. You are making sure any transformation that is done on the request body is properly reflected on the data that is saved in the database.
Daniel: With further developments coming, including more database options and allowing users to add logic or code snippets into prerequisite requests for specific header input requirements, we truly see this feature making an impact on testing at a day-to-day, script-to-script level.
That’s all, folks! Thanks for joining us this week for Feature Friday. The team here at Qyrus hopes you enjoy the weekend and the weather, and stop to smell the roses every once and a while! As Spring rounds the corner, dreams of summer are just right behind it.
June is finally here! And with it, summer arrives! I’m sure everyone is excited for the warm days and fun-filled evenings to come. So, let’s leave Spring in the past, just like we here at Qyrus are leaving manual testing and traditional automated testing methods in the past! Put on your shades, break out the sunscreen, and get ready for the blinding brilliance that is API monitoring. This week we are joined again by Daniel and Joyal, two members from our Chicago Qyrus team, to give more insight on the various benefits it provides users. Without further to do, let’s jump right into it!
Tell us more about API monitoring offered by Qyrus and its use cases.
Daniel: Well, simply put, it is a service that checks the status of an API call by continuously calling that API and examining the results.
Joyal: Yup, it monitors the health of the API, making sure that the right people are notified whenever something goes wrong. This helps prevent long periods of service disruption where users might be dealing with interruptions for hours.
Daniel: Certain APIs are critical to an application’s ability to serve its users. API monitoring can potentially catch errors before users do. In a way you can say it’s testing the reliability of your API.
It’s very true that certain APIs can be quite literally “mission critical” when it comes to a functioning application. I mean, imagine if the search API for Google went down. The world might literally come close to burning down.
What is API monitoring’s overall impact on the testing process?
Joyal: Well, I’d say we see the most impact in terms of reporting. The whole premise is to provide users with constant feedback in the form of a report. Users can see a lot of different metrics here.
Daniel: And you can take things a step further, by allowing this reporting to notify you via email about the status of your API and its overall health such as when the average response time crosses a certain threshold and if the API goes down in general.
Monitoring the health of your API sounds complicated. However, Qyrus seeks to make testing simpler for all types of users – whether they be seasoned testing veterans or totally clueless about coding. In the past, testing was complicated. Today, it’s our job to make it more accessible for all.
How might API monitoring help testers, developers, and business technologists? What value can this feature bring?
Joyal: It can bring a wide range of benefits for different types of people. Regular testers wouldn’t use this service to “test” APIs, but rather to keep tabs on the overall health of their critical APIs.
Daniel: Yeah, and developers might use this service for the exact same reason. But also, they can use it to monitor any quirks or weird issues like random response time spikes that their API might be witnessing without their knowledge. And business technologists could use this service to share performance data with a client of theirs, or even internally. More importantly, I think, is that it’s so easy to set up that less technical people like business techs can set up and operate an API monitoring session.
Joyal: That’s right, Daniel. That’s really the essence of Qyrus, isn’t it? We try to make everything as intuitive as it can be – and code free!
Making testing simple takes a lot of work. It’s not easy for us to extrapolate the complexity of automated testing in a codeless and intuitive fashion. That being said – how complicated would this be to, say, try and do this on your own?
Does the same or similar functionality exist without Qyrus?
Daniel: In order to get anywhere near what the feature offers, it’d take a good amount of coding and prior knowledge on different 3rd party technologies to make everything work.
Joyal: Of course, it can be done. But, like Dan said, it would take a lot of effort. On top of that, we offer a good deal of reusability on Qyrus itself. The functional API tests you build on Qyrus can be imported into API monitoring and set up very quickly.
Daniel: And it’s also important to note that in functional API testing, you can import APIs from Swagger format or OpenAPI specifications. These additional features really help tie everything together.
The reusability and ease of use combined makes for a disruptive duo. That being said, it’s sure to make some people’s lives a lot easier. In a day-to-day scenario, what might take place?
How do you see this impacting day-to-day operations across organizations?
Joyal: Overall, it’ll make monitoring the health of APIs easier. You’ll probably see less performance testing overall and you won’t have to manually trigger APIs to check on health. The goal is to have the developer alerted about the issue before it becomes an issue for clients.
Daniel: Yup, and you only have to set up the API monitor once. It measures the reliability of the API, more accurately. And overall, it’ll make testing over prolonged periods of time much simpler.
API monitoring can bring a lot of benefits as we’ve seen. It represents a holistic and complete API testing experience – taking functional testing to another level – reliability testing. We won’t keep you any longer! Got your sunscreen applied? Sunglasses in hand? Get out there and enjoy the weather! Thanks for joining us for this week’s Feature Friday!
