Real test practitioners will show how test automation solved many of their quality issues when the Automated Software Testing and Quality one-day virtual event returns on May 16. Produced by software testing company Parasoft, among the topics to be discussed are metrics, how automation can significantly cut test time, shifting testing left, the use (or not) of generative AI, the synergy between automation and service virtualization, and more.

“We’ve worked really hard to make sure that most of the sessions are coming from the practitioner community,” said Arthur Hicken, chief evangelist at Parasoft. “So people are telling you how they solved their problem – what metrics they use to solve the problems, what the main challenge was, what kind of results they saw, you know what pitfalls they’ve hit.”

As for AI in testing, Hicken said Parasoft has created AI augmentations at every aspect of the testing pyramid, which he acknowledged is getting “kind of long in the tooth,” before adding that it still is useful, helpful advice. “Whether it’s static analysis, unit test, API testing, functional testing, performance testing, UX testing, we’ll talk about how these different things will help you in your day-to-day job.”

He went on to say that he doesn’t believe the things he’s talking about are job killers. “I think they’re just ways to help. I haven’t met any software engineer that says, I don’t have enough to do, I’ve got to pad my work with something. I think just being able to get their job done will make their life better.”

On the subject of generative AI, Hicken says it can be quite smart about some things but struggles with others. So, the more clearly you can draw the boundaries of what you expect it to be able to do, and the more narrow you can scope it down, AI just does a better job.

This, he said, is true of testing in general. “Service virtualization helps you decouple from real-world things that you can’t really control or can’t afford to play with,” he said. “Most people don’t have a spare mainframe. Some people interact with real-world objects. We see that in the healthcare space, where faxes are part of a normal workflow. And so testing becomes very, very difficult.”

Further, he said, “As we use AI to start to increase the amount of testing, we’re doing the permutations, we run into a data problem, we just don’t have enough real data. So it starts synthesizing virtual data. So the service virtualization is a way to synthesize data to get broader coverage. And because of that, there’s always a temptation to use real-world data as your starting point. But in many jurisdictions, real-world data is a pretty big no-no. GDPR doesn’t allow it.”

So, in the end, the question remains: How do you know it was worth it? What did you do to measure? Hicken said, “I don’t believe there’s a universal quality measure or ROI measure; I believe there are lots of fascinating different things that you can look at that might be interesting for you. So I would say look for that.”

Hicken also noted, humorously, that if test automation did not deliver value, the speakers he sought out for ASTQ would not have returned his calls. 

There is still time to register to learn more about automated software testing and Parasoft.

The post ASTQ Summit brings together test practitioners to discuss implementing automation appeared first on SD Times.

AI can help humans in a variety of ways, including reducing errors and automating repetitive tasks. Software test automation tools are maturing and have incorporated AI and machine learning (ML) technology. The key point that separates the hype of AI from reality is that AI is not magic, nor the silver bullet promised with every new generation of tools. However, AI and ML do offer impressive enhancements to software testing tools.

More Software, More Releases

Software test automation is increasing in demand just as the worldwide demand for software continues to surge and the demand for developers increases. A recent report by Statista corroborates this expectation with a projection that suggests that the global developer population is expected to increase from 24.5 million in 2020 to 28.7 million by 2024.

Since testing and development resources are finite, there’s a need to make testing more efficient while increasing coverage to do more with the same. Focusing testing on exactly what needs to be validated after each code change is critical to accelerating testing, enabling continuous testing, and meeting delivery goals.

AI and ML play a key role in providing the data needed by test automation tools to focus testing while removing many of the tedious, error-prone, and mundane tasks.

• Improve static analysis adoption.
• Improve unit test creation.
• Reduce test maintenance.
• Reduce test execution.
• Increase API test automation.
• Improve UI test automation.

Real examples 

Let’s look at some real-life examples of what happens when you apply AI and ML technology to software testing.

Improve Unit Testing Coverage and Efficiency

Creating unit tests is a difficult task since it can be time-consuming to create unique tests that fully test a unit. One way to alleviate this is by making it easier to create stubs and mocks with assisted test creation for better isolation of the code under test. AI can assist in analyzing the unit under test to determine its dependencies on other classes. Then suggest mocking them to create more isolated tests.

The capabilities of AI in producing tests from code are impressive. However, it’s up to the developers to continuously invest in and build their own tests. Again, using AI test creation assistance, developers can:

• Extend code coverage through clones and mutations.
• Create the mocks.
• Auto-generate assertions

Improve API Testing

The struggle to improve API testing has traditionally relied on the expertise and motivation of the development team because APIs are often outside the realm of QA. Moreover, APIs are sometimes poorly documented. Creating tests for them is difficult and time-consuming.

When it comes to API testing, AI and ML aim to accomplish the following:

• Increase functional coverage with API and service layer testing.
• Make it easier to automate and quicker to execute.
• Reuse the results for load and performance testing.

This technology creates API tests by analyzing the traffic observed and recorded during manual UI tests. It then creates a series of API calls that are collected into scenarios and represent the underlying interface calls made during the UI flow. An ML algorithm is used to study interactions between different API resources and store those interactions as templates in a proprietary data structure. The goal of AI here is to create more advanced parameterized tests, not just repeat what the user was doing, as you get with simple record-and-playback testing.

Automate UI Testing Efficiently

Validating the application’s functionality with UI testing is another critical component of your testing strategy. The Selenium UI test automation framework is widely adopted for UI testing, but users still struggle with the common Selenium testing challenges of maintainability and stability.

AI helps by providing self-healing capabilities during runtime execution to address the common maintainability problems associated with UI testing.  AI can learn about internal data structures during the regular execution of Selenium tests by monitoring each test run and capturing detailed information about the web UI content of the application under test. This opens the possibility of self-healing of tests, which is a critical time-saver in cases when UI elements of web pages are moved or modified, causing tests to fail.

Remove Redundant Work With Smart Test Execution

Test impact analysis (TIA) assesses the impact of changes made to production code. The analysis and test selection are available to optimize the execution of unit tests, API tests, and Selenium web UI tests.

To prioritize test activities, a correlation from tests to business requirements is required. However, more is required since it’s unclear how recent changes have impacted the code. To optimize test execution, it’s necessary to understand the code that each test covers and then determine the code that has changed. Test impact analysis allows testers to focus only on the tests that validate the changes.

Benefits of AI/ML in Software Testing

AI and ML provide benefits throughout the SDLC and among the various tools that assist at each of these levels. Most importantly, these new technologies amplify the effectiveness of tools by first and foremost delivering better quality software and helping testing be more efficient and productive while reducing cost and risk.

For development managers, achieving production schedules becomes a reality with no late- cycle defects crippling release timetables. For developers, integrating test automation into their workflow is seamless with automated test creation, assisted test modification, and self-healing application testing. Testers and QA get quick feedback on test execution, so they can be more strategic about where to prioritize testing resources.

The post AI in API and UI software test automation appeared first on SD Times.

People who converted to virtual interactions with their customers did well, and those who didn’t suffered. But to do so, and keep up with those virtual competitors, often meant exposing things before they were ready, or even fully thought out. That led to a lot of technical debt, yet still didn’t calm the need for speed.

If you look to the Facebook-type models of extremely rapid releases, you’d need a highly scalable infrastructure with a rigorous testing environment – which on its face seems anathema to digital transformation –  to give you the ability to rapidly stand things up to test, and to perform those tests.

So, with your business online and on the line, it’s almost impossible to keep testing at a pace the business needs to adhere to without employing automation.

So said Arthur Hicken, evangelist at test solutions provider Parasoft, in a discussion we had leading up to this week’s Improve: Testing conference – at which Hicken will be presenting a session.

“You have to do things right now, you’re testing, and you’ve got to have a high degree of automation,” Hicken told SD Times. “You’ve got to have a high degree of confidence in that automation. And you’ve got to make sure that you can do everything you can not just to find bugs, but you’ve got to stop creating so many bugs in the first place.”

Parasoft has supported rigorous testing efforts for years, in medical devices and other areas where safety is critical – what Hicken likes to call “planes, trains and automobiles.” And in that space, he said, organizations are kind of slow to adopt new practices.

“It’s interesting because what happens is that we see the enterprise market looking more at the rigor because they need that mission-critical reliability,” Hicken said. “And we see the safety-critical market where the volume of code is exploding, we see them adopting. I mean, agile is becoming the norm. It’s not the disruptors, it’s the market leaders. DevOps, containerization, lightweight tools, CI/CD have all become the norm.”

Parasoft this year has been positioned among the leaders in the Forrester Wave for Continuous Automation Testing platforms, showing both a strong product offering and a strong strategy. Hicken said Parasoft has been hyper-focused on AI augmentation – not looking to build an AI “silver bullet,” but looking at real problems people have with test creation, test execution and test maintenance. He calls these, “all the ways to reduce the effort on developers, especially tedious efforts. And to give them guidance for things that might not be obvious.”

It also involves “the ability to do self-healing, and the test impact analysis so that when you do make a change, and you’re worried about is this change that I made going to break my online infrastructure, that we can give you the exact correct set of tests that make sure that that functionality is working properly, no more, no less,” he explained.

At the end of the day, Hicken said, “when you’re looking for continuous testing tools, you’re looking for something that can solve an actual problem you have. You’re not looking for, say, a service virtualization tool; you’re looking for a tool that can help you test before other components are ready. You’re not looking for a UI automation tool; you’re looking to make sure that your tool isn’t going to break down when you release it.” 

The post Speed – and other stuff – drives the need for test automation appeared first on SD Times.

FEATURED PROVIDERS

mabl is the enterprise SaaS leader of intelligent, low-code test automation that empowers high-velocity software teams to embed automated end-to-end tests into the entire development lifecycle. Customer-centric brands rely on mabl’s unified platform for creating, managing, and running automated tests that result in faster delivery of high-quality, business critical applications. Learn more at https://www.mabl.com; follow @mablhq on Twitter and @mabl on LinkedIn.

Parasoft helps organizations continuously deliver quality software with its market-proven automated software testing solutions. Parasoft’s AI-enhanced technologies reduce the time, effort, and cost of delivering secure, reliable, compliant software with everything from deep code analysis and unit testing to web UI and API testing, plus service virtualization and merged code coverage. Bringing all this together, Parasoft’s award-winning reporting and analytics dashboard delivers a centralized view of application quality, enabling organizations to deliver with confidence.

testRigor helps organizations dramatically reduce time spent on test maintenance, improve test stability, and dramatically improve the speed of test creation. This is achieved through its support of “plain English” language that allows users to describe how to find elements on the screen and what to do with those elements from the end-user’s perspective. People creating tests on their system build 2,000+ tests per year per person. On top of it,  testRigor helps teams deploy their analytics library in production that will make systems automatically produce tests reflecting the most frequently used end-to-end flows from production.

OTHER PROVIDERS

Applitools is built to test all the elements that appear on a screen with just one line of code. Using Visual AI, you can automatically verify that your web or mobile app functions and appears correctly across all devices, all browsers and all screen sizes. It is designed to integrate with your existing tests. We support all major test automation frameworks and programming languages covering web, mobile, and desktop apps.

Appvance IQ can generate its own tests, surfacing critical bugs in minutes with limited human involvement in web and mobile applications. AIQ empowers enterprises to improve the quality, performance and security of their most critical applications, while transforming the efficiency and output of their testing teams and lowering QA costs.

Digital.ai Continuous Testing enables organizations to reduce risk and provide their customers satisfying, error-free experiences — across all devices and browsers. Digital.ai Continuous Testing provides expansive test coverage across 2000+ real mobile devices and web browsers, and seamlessly integrates with best-in-class tools throughout the DevOps/DevSecOps pipeline.

HCL Software develops, markets, sells, and supports over 20 product families with particular focus on Customer Experience, Digital Solutions, Secure DevOps, and Security & Automation. Its mission is to drive ultimate customer success of their IT investments through relentless innovation of our software products. 

HPE Software’s automated testing solutions simplify software testing within fast-moving agile teams and for continuous integration scenarios. Integrated with DevOps tools and ALM solutions, HPE automated testing solutions keep quality at the center of today’s modern applications and hybrid infrastructures. 

IBM: Quality is essential and the combination of automated testing and service virtualization from IBM Rational Test Workbench allows teams to assess their software throughout their delivery life cycle. IBM has a market leading solution for the continuous testing of end-to-end scenarios covering mobile, cloud, cognitive, mainframe and more. 

Keysight Technologies Digital Automation Intelligence (DAI) platform is the first AI-driven test automation solution with unique capabilities that make the testing process faster and easier. With DAI, you can automate 95% of activities, including test-case design, test execution, and results analysis. This enables teams to rapidly accelerate testing, improve the quality of software and integrate with DevOps at speed. The intelligent automation reduces time to market and ensures a consistent experience across all devices.

Micro Focus enables customers to accelerate test automation with one intelligent functional testing tool for web, mobile, API and enterprise apps. Users can test both the front-end functionality and back-end service parts of an application to increase test coverage across the UI and API.

Microsoft’s Visual Studio helps developers create, manage, and run unit tests by offering the Microsoft unit test framework or one of several third-party and open-source frameworks. The company provides a specialized tool set for testers that delivers an integrated experience starting from Agile planning to test and release management, on-premises or in the cloud. 

Kobiton offers its patented GigaFox on-premises or hosted, and solves mobile device sharing and management challenges during development, debugging, manual testing, and automated testing. A pre-installed and pre-configured Appium server provides “instant on” Appium test automation.

NowSecure identifies the broadest array of security threats, compliance gaps and privacy issues in custom-developed, commercial, and business-critical mobile apps. NowSecure customers can choose automated software on-premises or in the cloud, expert professional penetration testing and managed services, or a combination of all as needed. 

Orasi is a leading provider of software testing services, utilizing test management, test automation, enterprise testing, Continuous Delivery, monitoring, and mobile testing technology. 

Perfecto users can pair their favorite frameworks with Perfecto to automate advanced testing capabilities, like GPS, device conditions, audio injection, and more. It also includes full integration into the CI/CD pipeline, continuous testing improves efficiencies across all of DevOps.  

ProdPerfect is an autonomous, end-to-end (E2E) regression testing solution that continuously identifies, builds and evolves E2E test suites via data-driven, machine-led analysis of live user behavior data. It addresses critical test coverage gaps, eliminates long test suite runtimes and costly bugs in production, and removes the QA burden that consumes massive engineering resources.  

Progress Software’s Telerik Test Studio is a test automation solution that helps teams be more efficient in functional, performance and load testing, improving test coverage and reducing the number of bugs that slip into production. 

Sauce Labs provides a cloud-based platform for automated testing of web and mobile applications. Optimized for use in CI and CD environment, and built with an emphasis on security, reliability and scalability, users can run tests written in any language or framework using Selenium or Appium.

SmartBear tools are built to streamline your process while seamlessly working with your existing products. Whether it’s TestComplete, Swagger, Cucumber, ReadyAPI, Zephyr, or one of our other tools, we span test automation, API life cycle, collaboration, performance testing, test management, and more. 

Synopsys offers a powerful and highly configurable test automation flow that provides seamless integration of all Synopsys TestMAX capabilities. Early validation of complex DFT logic is supported through full RTL integration while maintaining physical, timing and power awareness through direct links into the Synopsys Fusion Design Platform.

SOASTA’s Digital Performance Management (DPM) Platform includes five technologies: TouchTest mobile functional test automation; mPulse real user monitoring (RUM); the CloudTest platform for continuous load testing; Digital Operation Center (DOC) for a unified view of contextual intelligence accessible from any device; and Data Science Workbench, simplifying analysis of current and historical web and mobile user performance data. 

Tricentis Tosca, the #1 continuous test automation platform, accelerates testing with a script-less, AI-based, no-code approach for end-to-end test automation. With support for over 160+ technologies and enterprise applications, Tosca provides resilient test automation for any use case. 

To read the full Buyers Guide, click here. To see how companies are helping with automated testing initiatives, click here.

The post A guide to automated testing tools appeared first on SD Times.

Darrel Farris, manager of solutions engineering at mabl

Software development teams are realizing that automated testing is key to accelerating product velocity and reaching the full potential of DevOps. When fully integrated into a company’s development pipeline, testing becomes an early alert system for short-term defects as well as long-term performance issues. The key to realizing this potential: simple test creation and rich reporting features. 

Mabl is low-code, intelligent test software that allows everyone to create automated tests covering web UIs, APIs, and mobile browsers with 80% less effort. Quality teams can extend the value of end-to-end tests even further with automated accessibility checks that help ensure every user has a delightful experience, regardless of access needs. Machine learning and AI features like auto-healing and Intelligent Wait help teams create more reliable tests and reduce test maintenance. Results from every test are tracked within mabl’s comprehensive suite of reporting features, making it easy to understand product quality trends. With test creation simplified and quality data at their fingertips, everyone can focus on resolving defects quickly and improving product quality. 

Mabl also includes native integrations with tools like Microsoft Teams, Slack, and Jira, so that testing information can be seamlessly integrated into workflows and everyone can benefit from mabl’s rich diagnostic data. Teams can monitor performance with speed indexes for all web pages, and manage API quality with data on the response time for each API endpoint. This allows teams to shift from reacting to failed tests and customer complaints to proactively managing product quality, improving the customer experience.

Arthur Hicken, chief evangelist at Parasoft

At Parasoft, we have various AI components and capabilities that augment the testers’ work at every layer of the testing pyramid. 

Our AI improves the static analysis experience with fewer false positives, better prioritization and understanding of risk models, and it has the necessary standards such as ISO 26262, PCI-DSS, OWASP, and CWE for compliance in certain industries. 

On top of that, we have advanced test creation with the generation of mocks and stubs to follow the best practices of unit testing in isolation and we have the tools that can help you determine how you can expand a test to provide additional code coverage. 

Test impact analysis helps you understand what tests you need to run when there are changes in code, tests, or requirements. 

We also have AI for API testing to record manual tester behavior and automatically convert that into API tests that are highly maintainable and execute quickly. We can apply AI to create test assets that not only perform functional testing, but you can automatically apply additional testing like security tests, or load and performance tests.

Further, we can use AI to capture a manual test and use it to create a test that can be run automatically because it can be automated and integrated in regression, have AI-based self-healing capabilities and perform security tests without additional tester effort or special training.

Parasoft’s solution can perform deep code analysis, which provides users with the ability to find structural problems. It also helps in functional testing, whether API testing, UI testing, or automated testing. We have a unique position in testing because our solutions cover both a white-box view at the code level as well as a black-box view at the functional and application level. Because we have both views, it enables us to make inferences that wouldn’t be possible otherwise. So, we can start to correlate literally what’s going on at the code analysis level and the unit and functional test level with what the external tests are doing and use this to provide better advice on where a problem exists in the code and how to repair it.

Parasoft’s capability of using AI to automate testing and having a full understanding from deep code analysis all the way through the external testing lets us provide a better experience to the end user. 

Artem Golubev, CEO at testRigor

testRigor empowers manual testers to build functional end-to-end test automation at any degree of complexity, without the need for engineering knowledge in the mix. If a user can express manual test case steps in English, they’ll be able to build tests on the platform. testRigor will then execute the test for you from a human’s standpoint, interacting with a web, native, or mobile application. 

Any person, including those that don’t necessarily have coding skills, will be able to edit, maintain, upgrade, in addition to creating those tests. Also, our tests were measured to be 200 times more stable than Selenium tests, and our customers are typically spending 95% less time managing these tests. 

The QA teams can then be freed from click-through manual regression testing and maintaining automated scripts because the issue of maintenance with testRigor is eliminated for good. 

Just ask Keith Powe, VP of Engineering at IDT Corporation. His team could automate only four test cases a week per person, but with testRigor, they have increased their testing coverage from less than 34% to more than 91% in under 9 months. Spending a maximum of 0.1% of the time in test maintenance, IDT has a 90% reduction in bugs and a more effective CI/CD. Many other companies such as Upgrade, DataHerald, and others have cited drastic improvements in their testing strategy with the benefits that testRigor offers. 

Be sure to visit our site https://testrigor.com/ to learn more about how testRigor can help solve the biggest challenges that you’re facing with automated testing today.

To read the full Buyers Guide, click here. To see the guide to automated testing tools, click here.

The post How these companies can help with your automated testing initiatives appeared first on SD Times.

However, while test automation coverage saw a notable dip during the pandemic, it has since rebounded last year, according to SmartBear’s State of Quality Testing 2022 report. 

Last year saw the amount of companies performing just manual tests at 11%, while that number dwindled to 7% this year, almost returning to pre-pandemic levels of 5% of all tests being performed completely manually. 

When looking at the different types of tests and how they are performed, over half of respondents reported using manual testing for usability and user acceptance tests.

Unit tests, performance tests, and BDD framework tests were highest among all automated testing. 

This year, the most time-consuming activity was performing manual and exploratory tests, jumping to 26% from 18% last year as the most time-consuming task. In the same time period, learning how to use test tools as the most time-consuming challenge with testing fell from 22% to just 8%.

In the Agile and DevOps realm, there are higher levels of automation versus those companies that are still in the waterfall stages, according to Diego Lo Giudice, VP, principal analyst at Forrester. This is inherent to DevOps because if most of the testing is manual, it’s just going to slow down the rest of the team. 

“With DevOps and all the automation going on around it, testing needs to be very high, it needs to be above 80%. You kind of see that only for a few companies or specific projects inside an organization, but if you look at the rest of the market, probably it’s less than 30%,” Lo Giudice said. “I would say we’ve made some progress, but there’s more automation that’s needed.”

In fact, some of the companies that are adopting agile or DevOps methods find that testing sometimes becomes the bottleneck to rapid delivery, according to Darrel Farris, manager of solutions engineering at mabl. Testing in DevOps must be integrated into the pipeline so developers aren’t throwing code over to QA that hasn’t been tested – especially if teams are deploying multiple times per week or month.

Some of the big challenges to implementing automated testing are that there’s a lack of skills and because test automation requires change within the organization. 

“So there are a number of changes regarding people, processes, and technology, it’s not just getting a tool. And automating tests, this is about organizing, testing completely in a different way,” Lo Giudice added. 

Challenges with getting automated testing just right 

“One of the challenges we see from people is that they’re fundamentally approaching this wrong. We’ve had some of our customers talk about this, how they had to change the way they were thinking and so that the kind of common obvious symptom that you see about this today is people saying ‘we had a whole bunch of manual testers and so we’ll build a whole strategy on recording what they do and playing it back and building from there. And this is just fundamentally the wrong approach,” said Arthur Hicken, chief evangelist at Parasoft. 

Another challenge is that automated tests can become incredibly time-consuming to maintain due to the sheer number of tests that are generated. 

“The largest issue is that once a person builds 300 tests, it becomes a full-time job to maintain those tests and you hit the ceiling,” Artem Golubev, CEO at testRigor said. “Coupled with the fact that budgets are limited, people just can’t build more automations.” 

Golubev added that this difficulty to maintain all automated tests is the main reason why the majority of tests are still executed manually today. Automating tests can also be futile if it’s focused on the wrong areas. 

“QA teams are spending 80% of their weeks maintaining scripts due to rapidly changing UIs, instead of focusing on growing functional test coverage or expanding the types of testing they are doing on their application, such as accessibility or performance testing,” mabl’s Farris said. 

“I believe the testing pyramid is built on false assumptions that have never been correct in the first place,” Golubev said. “In a perfect vacuum, of course this is how things work and there are maybe one or two companies which have done it that way. In a real scenario, it’s always been more of an hourglass shape of testing.” 

He explained that this is because engineers who mostly write unit tests are very unlikely to contribute to end-to-end tests, very few engineers would write integration tests since they are such a pain to maintain, and there would be a lot of end-to-end tests where you have people working on them full-time. 

While the integration test value is to make sure that the system integrates properly, it doesn’t matter if you enter and the system doesn’t work properly, Golubev continued. End-to-end tests are actually the ones covering integration because those tests are the test which will prove that your system is usable by your end users.

“Let’s say you’re logging into a banking application and they can’t transfer money from account A to account B, then it does not matter. Even if all your integration tests are green and all your unit tests pass through it, it’s completely useless,” Golubev said. “So the most important tests are end-to-end tests, only then can that system function as intended. And therefore end-to-end tests should be the bulk of the tests that are done.”

The best way to then optimize end-to-end tests to make them run faster is to prioritize because end-to-end tests will inherently be much slower than unit tests. 

“With every type of testing in the organization, people need to assess whether they need to really leverage automation? Is it worth it? Is it something that will be repeated over and over that changes continuously? If you have to run a test, the same test more than three, four times you start asking yourself, well, maybe I should automate this,” Forrester’s Lo Giudice said. “So I don’t think 100% is what customers will achieve and will keep it more towards 80% as I said.”

One of the most efficient ways to make sure that all testing resources are aligned correctly is to align as a team on a testing strategy by starting with the most critical test cases that will ensure a high quality application experience for users, according to mabl’s Farris. This can be done by taking on a few test cases at first, then layering in additional test cases over time.

One way to do this is to create a quality center of excellence or a “quality champion” in an organization. This person or group is a testing expert who can advise and coach everyone from developers to product owners on testing best practices, Farris explained. Some of the manual testing is changing too because of the increasing use of exploratory testing, Lo Giudice explained. This type of manual testing is where the tester sits down with the developer and they work out the issues together. The tester puts the application through certain scenarios, the developer sees the problems and tries to fix them, and they take about two hours a day like that. 

The structure around automated testing is shifting

Both companies’ attitudes towards testing and who gets involved have shifted. As testing becomes more federated, you no longer have a centralized team that does all the testing as an afterthought, according to Lo Giudice. 

Now, there are testers that are moving into the development teams and the product teams to get all of the testing done together. And so what remains in the central team is specialized testing resources that maybe choose the tools that define what the new practices would look like, whether that’s shifting testing to the left or suggesting test-driven development or behavior-driven development. 

The test center is now much smaller working in consulting with the teams but testers move into the team itself, Lo Giudice explained. 

“So the typical manual tester that used to put a test case in an Excel sheet and run it through the application looking at what the test case told him to do suddenly now finds himself with a tool that is quite technical where he needs to write code to automate what he was doing manually,” Lo Giudice said. To solve this, there’s a trend among vendors to raise the level of abstraction of the tools so that a manual tester or even a person on the business side can test using a low code testing tool. 

Then come the technologies, platforms, and tools because after all, an organization needs testing tools that are integrated into CI/CD pipelines with the rest of the development and delivery tools that integrate with CI servers effectively on the cloud. 

“The point really is that testing takes a village and it takes all these different personas in an organization: business tester, and a subject matter expert in testing who is technical but not a coder, and developers that also may be doing API testing, lower level infrastructure testing within their IDE at a very technical level,” Lo Giudice said. 

According to testRigor’s Golubev, the directors of QA will benefit the most from automated testing since they’ll be able to cover far more functionality faster than they ever could before. However, engineers, manual testers, and product management will also be able to benefit from automated testing tooling since they’ll be able to collaborate together on the same tool. 

Previously, it was companies in the banking and health sectors that have been getting automated testing right but now it’s organizations like Lenovo or Volkswagen that have these 

highly complex software test, build, and deploy systems that are the envy of anybody, Parasoft’s Hicken said. Ultimately, it’s one of the things companies are going to do because that is what they’re competitors are moving toward.

AI helps with various levels of testing 

When you send data of all the tests that passed: the log files, the bugs and feed them to AI it can start telling you what you need to test and how when there’s a change coming. It also helps to tell whether to run all of the tests or just to select the few ones that will be impacted by the change. 

There have been impressive improvements in the vision and computer vision space to enable visual testing, Lo Giudice said. There’s a tool out there that sees what the human eye does when looking at the application and will notice things that are going wrong. It can also do it on types of applications that move very fast that the human eye can’t capture. 

One can also teach AI to not fail tests in certain scenarios to help with self-healing. For example, tests can sometimes fail simply because an object moved on the screen differently on the same application on a browser, and then on a mobile device because the layout might change and it’s not necessarily a bug. And so one can now teach the algorithm to not fail the test even though it’s not in the same position because it can find the locator of that object in some other place, Lo Giudice explained. 

There are also AI models that help minimize tests to solve the maintenance problem.

“This is the idea of the AI guiding a person to create tests that are more stable. The Holy Grail is that you create a set of tests that maximize coverage, but minimize the number of tests so that you have less to maintain, and that they’re not brittle,” Hicken said. “You want tests that have proper levels of abstraction, so that you aren’t spending more on keeping them alive than you were in creating them in the first place.”

Also with error clustering, AI can help find and classify bugs in a way that a tester can quickly recognize the bug and can suggest the right developer to fix the bug to reduce mean time to repair. It can use data from production to find out what are the most frequently used features within that application. There’s even a tool that generates unit tests as you code, which Forrester refers to as the tester Turing bot. 

“AI can also support the execution of more stable tests. For example, tests running in the cloud can execute almost too fast, before your application is in a loaded state,” mabl’s Farris said. “It applies intelligence that can slow down or speed up the execution of your tests by automatically adjusting wait times.”

“So AI is infusing along the entire software development lifecycle. And testing is one of the stages where it’s actually more mature than any other stage of the development lifecycle,” Forrester’s Lo Giudice said. 

To read how providers are helping with automated testing initiatives, click here. To read the guide to automated testing tools, click here.

The post While automated testing has rebounded this year, it still has a long way to go appeared first on SD Times.

According to Parasoft, these updates work to optimize the solution with innovative AI/ML features that accelerate test creation, maintenance, and execution, enabling the company to leverage AI/ML and reach higher levels of software test automation. 

The ways in which Parasoft has expanded its AI/ML capabilities include the ability to automate unit test generation and parameterization for Java, automate API test generation and maintenance, and self-heal the execution of Selenium tests. 

Additionally, Parasoft leverages AI/ML to improve the adoption of static analysis efficiency through prioritization and risk models, and predict the impact of coding violations and patterns in analytics/reporting to determine the best developer match to fix a certain problem.

“AI/ML capabilities have greatly improved the challenges facing today’s testing organizations from creation through execution, maintenance, and optimization. Organizations that aren’t deploying testing with AI/ML capabilities are at a serious disadvantage,” said Arthur Hicken, evangelist at Parasoft.

Parasoft also utilizes AI to create tests automatically from the recording of manual UI testing. This simplifies the development of API test suites. These tests can then be used again for load, performance, and security testing.

According to the company, smart test execution with test impact analysis augments testing which leads to more intelligent and efficient testing that works to get better with each release. 

Visit the website to learn more about these updates.

The post Updates to Parasoft’s AI/ML capabilities appeared first on SD Times.

Virtualization enables organizations to put up a virtual service more easily than they can “yank a box on an Amazon server,” explained Shamim Ahmed, DevOps CTO and evangelist at Broadcom. Yet today, service virtualization (SV) can be seen as a life cycle technology, empowering what Ahmed calls continuous virtualization. This, he said, “enables even developers doing parallel development right now, just for testing. That’s on the left-hand side. And on the right-hand side, we’ve seen extremes, like customers using service virtualization for chaos testing.”

SV helped early-adopting organizations to decouple teams, said Diego Lo Giudice, vice president and principal analyst at Forrester, so that you could decouple customer with client. But, he noted, “with organizations being broken up into small teams, and parallelizing, the work with Agile became very hard. Project managers thought they could manage that. And there’s no way you can really manage a bunch of small agile teams working; making sure that you synchronize them through project management is impossible. And so service virtualization was kind of used a bit to decouple, at least from the testing perspective.”

Virtualization enables organizations to put up a virtual service more easily than they can “yank a box on an Amazon server,” Ahmed explained. 

So, where is service virtualization being used beyond testing?

Service virtualization use cases

Diego Lo Giudice, vice president and principal analyst at Forrester, said SV remains mainly a testing capability, though he said he is seeing an accelerated use of SV in the API world. “I haven’t really gotten, you know, beyond the typical use cases of testing unreachable or expensive third-party resources,” he said, noting that the biggest use case he keeps seeing is virtualizing mainframe environments. “I love the example a CEO gave me that he was saving a lot of money with service virtualization simply because one of his teams, for testing purposes, couldn’t access the mainframe. They only had a window of 30 minutes a month, and they had to wait every time for those 30 minutes. With service virtualization, they were able to virtualize that access to the mainframe, and therefore the team now kind of had the virtual access to the mainframe available all the time.”

Using service virtualization with APIs, Lo Giudice said, is “just one of the types of testing that needs to be done; integration tests, that activity that can be automated, software delivery pipelines. I see it a lot there.”

Among other areas where service virtualization is being seen is to create employee onboarding environments. Alaska Airlines uses Parasoft’s virtualization solution for its training, according to Ryan Papineau, a senior software engineer at the airline. With virtualization, he said, “we’re able to scale the amount of people that we have go through our training program.” While there are typically no test cases, Alaska can use the environment to see if the users can perform certain tasks, but none of that gets recorded or impacts the production environment. 

Service virtualization and test data management

But perhaps the biggest area of SV growth is in the test data management (TDM) testing space – a term that Papineau said is “kind of messy, because it can mean a lot of things.” It has become, in a word or two, a catch-all buzzword.

“We’ve been screening some new automation engineers, and they’ll put test data management on their resume. But you’ll never see any concept of any tools or techniques listed,” Papineau said. “What I believe that to be is they’re listing it, to say ‘Hey, I use data-driven tests and had Excel,’ and I’m like, that’s not what I’m looking for. I’m looking for data structures and relationships and databases. And that life cycle of creation to modification to deletion. And using an ETL tool, or custom scripts, which we use separately.” 

Papineau said that Parasoft’s solution essentially uses data and iterates it over APIs, records it and creates the relationships with the data. Papineau said, “You get this nice exploded, fancy UI that has all the relationships and you can drill down and do cloning and subsetting, so it has a lot of the old traditional test data management aspects to it, but all within their context.” 

Broadcom’s Ahmed added that his company, which acquired the Lisa SV software developed by iTKO through its purchase of CA, is seeing much more synergy between servers, virtualization and test data management. “When we acquired Lisa, TDM was not that big. But now with all this GDPR, and all the other regulations around data privacy, TDM is really hard. And it’s one of the biggest problems the customers are grappling with.”

Ahmed believes SV and TDM go hand-in-glove. “The way they work together, I think, is another key evolution of how the use of service virtualization has evolved,” he said. “Using SV is actually one of the easier ways to do test data management. Because, you know, you can actually record the test data by recording the back and forth between a client and a server. So that gives you an opportunity to create lightweight data, as opposed to using the more traditional test data mechanisms, particularly so for API-based systems.”

He noted that the use of SV reduces “the tedium burden,” because creating the test data for a live application versus creating the test data for an emulator is a much lower amount of TDM burden for the testers and everybody else.”

System integrations

While much about service virtualization has gone unchanged over the last years, much has changed, according to Lo Giudice. Developers are choosing open source more, deciding they don’t need all the sophistication vendors are providing.  “I’ve got data that shows the adoption of service virtualization has never really gone over 20%,” he said. “When you ask developers and testers, what is it that you’re automating around in 2022, I think the system integrators” are the only ones for whom this is key. 

“It’s actually very useful” in integration projects, Lo Giudice said. “If you think about Lloyds Banking, a customer that’s got a complex landscape of apps, and you’re doing integration work with good partnerships going on,” service virtualization can be quite beneficial. “If you’ve got an app and it interfaces another 10 big apps, you’d better use service virtualization to automate that integration,” he said.

Integration projects between assets held on-premises and those residing in the cloud caused some hardships for Alaska Airlines, Papineau said. The problem, he said, stemmed from internal permissions and controls into the cloud. One of their developers was taking older data repository methods and deploying the cloud, and struggled with the internal permissions between on-prem and the cloud.”

Papineau said organizations have to understand their firewalls and the access to servers. “Are your server and client both in local? Are they both in cloud order, and does one have to transverse between the other,” Papineau said. “So what we did there is we stumbled on getting the firewall rules exposed, because now all of these different clients are trying to talk to this virtual server. And so it’s like, ‘Oh, you got this one going up. Now you need to do another firewall request for this one?’ And I am not kidding you. When we did the Virgin (Atlantic) acquisition, viral requests were the largest nightmare in the longest time. So that’s why it’s an internal problem we struggled with and just gave up on it like, No, this is just taking too much time. This should not be this hard. This literally is a firewall overhead problem that we ran into.”.

Continuous virtualization

Virtualization is not something you do before you do testing any longer. From the time you start to do your backlog and your design, you have to think about what services you need, and how you design them correctly.

Then, according to Broadcom’s DevOps CTO and evangelist Shamim Ahmed, you have to think about how to evolve those services. “We think of service virtualization evolving and on the continuum,” he said. “You start with something simple we call a synthetic virtual service that can be created very easily – not using the traditional record-response mechanism.”

He noted that the old way of creating a virtual service relied on the fact that the endpoint already exists. That’s what enabled record and replay,  but in today’s development environment, the endpoint may not exist – all you might have is an API specification, and you might not even know whether the API has been implemented or not. “You need to have new ways of creating a virtual service, a very simple, lightweight service that can be created for something like a Swagger definition of an API. Developers need that when they’re doing unit testing, for example. The way we look at this is what we call progressive virtualization – that simple thing that we created can now evolve, as you move your application from left to right in the CI/CD life cycle.”

He offered the example once that application gets to the stage of integration testing, you perhaps need to enhance that synthetic virtual service with some more behavior. So more data is added, and then when you get to system testing, you need to replace that synthetic virtual service with the real recording, so it becomes progressively realistic as you go from left to right. 

“There’s a whole life cycle that we need to think about around continuous virtualization that talks about the kind of virtual servers needed to do integration testing, or build verification,” Ahmed said. “And of course, all the other kinds of tests – functional, performance and even security testing – virtual services are just as applicable for those things…  because if you think about the number of third-party systems that a typical application accesses in this API-driven world, you simply can’t run many of your tests end-to-end without running into some kind of external dependency that you do not control, from the perspective of functional, performance and security testing. So you can start to emulate all of those characteristics in a virtual service.”

The post Service virtualization: A continuous life cycle technology appeared first on SD Times.

Matt Klassen, CMO, Parasoft

Quality continues to be the primary metric for measuring the success of software deliveries. With the continued pressure to release software faster and with fewer defects, it’s not just about speed — it’s about delivering quality at speed. 

Managers must ask themselves if they are confident in the quality of the applications being delivered by their teams. Continuous quality is a must for every organization to efficiently reduce the risk of costly operational outages and to accelerate time-to-market.

Parasoft’s automated software testing solutions integrate quality into the software delivery process for early prevention, detection, and remediation of defects. From deep code analysis for security and reliability, through unit, API, and UI test automation, to performance testing and service virtualization, Parasoft helps you build quality into your software development process.

Parasoft leverages our deep understanding of DevOps to develop AI-enhanced technologies and strategies that solve complex software problems. Our testing solutions reduce the time, effort, and cost of delivering secure, reliable, and compliant software.

With over 30 years of making testing easier for our customers, we have the innovation you need and the experience you trust. Our extensive continuous quality solution spans every testing need and enables you to deliver with confidence. If you want to improve your software quality while achieving your business goals, partner with Parasoft.

RELATED CONTENT:
Targeting a key to automated testing
A guide to automated testing tools

Jonathon Wright, chief technology evangelist, test automation at Keysight 

Artificial Intelligence (AI) makes the process of designing, developing, and deploying software faster, better and cheaper. AI-powered tools enable project managers, business analysts, software coders and testers to be more productive and effective, allowing them to produce higher-quality software faster and at a lower cost.

At Keysight, our Eggplant intelligent automation platform allows citizen developers to easily use our no-code solution that draws on AI, machine learning, deep learning and analytics to automate test execution across the entire testing process. It empowers and enables domain experts to be automation engineers. The AI and ML take on scriptwriting and maintenance as a machine can create and execute thousands of tests in minutes, unlike a human tester. 

Keysight’s intelligent automation platform is a completely non-invasive testing tool, ensuring comprehensive test coverage without ever touching the source code or installing anything on the system-under-test. The technology sits outside of the application and reports on performance issues, bugs and other errors without the need to understand the underlying technology stack. This is critical for regulated industries such as healthcare, government and defense.

AI-powered automation can test any technology on any device, operating system or browser at any layer, from the UI to APIs to the database. This includes everything from the most modern, highly dynamic website to legacy back-office systems to point of sale, as well as command and control systems.

The overarching goal of Keysight’s intelligent automation is to understand how customer experiences and business outcomes are affected by the behavior of the application or software. More than this, though, it is about identifying opportunities for improvements and predicting the business impact of those changes.

Gev Hovsepyan, head of product, mabl

Software development teams are realizing that automated testing is key to accelerating product velocity and reaching the full potential of DevOps. When fully integrated into a company’s development pipeline, testing becomes an early alert system for short-term defects as well as long-term performance issues that could hurt the user experience. The key to realizing this potential: simple test creation and rich, accessible reporting features. 

Mabl is low-code, intelligent test software that allows everyone, regardless of coding experience, to create automated tests spanning web UIs, APIs, and mobile browsers with 80% less effort. Using machine learning and AI, features like auto-healing and Intelligent Wait help teams create more reliable tests and reduce overall test maintenance. Results from every test are tracked within mabl’s comprehensive suite of reporting features, making it easy to understand product quality trends. With test creation simplified and quality data at their fingertips, everyone can focus on resolving defects quickly and improving product quality. 

Mabl also includes native integrations with tools like Microsoft Teams, Slack, and Jira, so that testing information can be seamlessly integrated into workflows and everyone can benefit from mabl’s rich diagnostic data. These reporting features include immediate test results as well as long-term product trends so that quality engineering teams can support faster bug resolution and monitor their product’s overall performance and functionality. This allows software development teams to shift from reacting to failed tests and customer complaints to proactively managing product quality, enabling them to spend more time improving the customer experience.

The post How these solution providers support automated testing appeared first on SD Times.

Keysight Technologies Eggplant Digital Automation Intelligence (DAI) platform is the first AI-driven test automation solution with unique capabilities that make the testing process faster and easier. With DAI, you can automate 95% of activities, including test-case design, test execution, and results analysis. This enables teams to rapidly accelerate testing, improve the quality of software and integrate with DevOps at speed. The intelligent automation reduces time to market and ensures a consistent experience across all devices.

mabl is the enterprise SaaS leader of intelligent, low-code test automation that empowers high-velocity software teams to embed automated end-to-end tests into the entire development lifecycle. Customer-centric brands rely on mabl’s unified platform for creating, managing, and running automated tests that result in faster delivery of high-quality, business critical applications. Learn more at https://www.mabl.com; follow @mablhq on Twitter and @mabl on LinkedIn.

Parasoft: Parasoft helps organizations continuously deliver quality software with its market-proven automated software testing solutions. Parasoft’s AI-enhanced technologies reduce the time, effort, and cost of delivering secure, reliable, and compliant software with everything from deep code analysis and unit testing to web UI and API testing, plus service virtualization and merged code coverage. Bringing all this together, Parasoft’s award-winning reporting and analytics dashboard delivers a centralized view of application quality, enabling organizations to deliver with confidence.

RELATED CONTENT:
Targeting a key to automated testing
How these solution providers support automated testing

Appvance is the inventor of AI-driven autonomous testing, which is revolutionizing the $120B software QA industry. The company’s patented platform, Appvance IQ, can generate its own tests, surfacing critical bugs in minutes with limited human involvement in web and mobile applications. 

Applitools: Applitools is built to test all the elements that appear on a screen with just one line of code. Using Visual AI, you can automatically verify that your web or mobile app functions and appears correctly across all devices, all browsers and all screen sizes. Applitools automatically validates the look and feel and user experience of your apps and sites. 

Digital.ai Continuous Testing (formerly Experitest) enables organizations to reduce risk and provide their customers satisfying, error-free experiences — across all devices and browsers. Digital.ai Continuous Testing provides expansive test coverage across 2,000+ real mobile devices and web browsers, and seamlessly integrates with best-in-class tools throughout the DevOps/DevSecOps pipeline.

HPE Software’s automated testing solutions simplify software testing within fast-moving agile teams and for continuous integration scenarios. Integrated with DevOps tools and ALM solutions, HPE automated testing solutions keep quality at the center of today’s modern applications and hybrid infrastructures. 

IBM: Quality is essential and the combination of automated testing and service virtualization from IBM Rational Test Workbench allows teams to assess their software throughout their delivery life cycle. IBM has a market leading solution for the continuous testing of end-to-end scenarios covering mobile, cloud, cognitive, mainframe and more. 

Micro Focus: Accelerate test automation with one intelligent functional testing tool for web, mobile, API and enterprise apps. AI-powered intelligent test automation reduces functional test creation time and maintenance while boosting test coverage and resiliency. 

Mobile Labs (acquired by Kobiton)  Its patented GigaFox is offered on-premises or hosted, and solves mobile device sharing and management challenges during development, debugging, manual testing, and automated testing. A pre-installed and pre-configured Appium server provides “instant on” Appium test automation.

NowSecure identifies the broadest array of security threats, compliance gaps and privacy issues in custom-developed, commercial, and business-critical mobile apps. NowSecure customers can choose automated software on-premises or in the cloud, expert professional penetration testing and managed services, or a combination of all as needed.

Orasi is a leading provider of software testing services, utilizing test management, test automation, enterprise testing, Continuous Delivery, monitoring, and mobile testing technology. 

Perfecto: Users can pair their favorite frameworks with Perfecto to automate advanced testing capabilities, like GPS, device conditions, audio injection, and more. It also includes full integration into the CI/CD pipeline, continuous testing improves efficiencies across all of DevOps.  

ProdPerfect: ProdPerfect is an autonomous, end-to-end (E2E) regression testing solution that continuously identifies, builds and evolves E2E test suites via data-driven, machine-led analysis of live user behavior data. It addresses critical test coverage gaps, eliminates long test suite runtimes and costly bugs in production, and removes the QA burden that consumes massive engineering resources. 

Progress: Telerik Test Studio is a test automation solution that helps teams be more efficient in functional, performance and load testing, improving test coverage and reducing the number of bugs that slip into production. 

Sauce Labs provides the world’s largest cloud-based platform for automated testing of web and mobile applications. Optimized for use in CI and CD environments, and built with an emphasis on security, reliability and scalability, users can run tests written in any language or framework using Selenium or Appium.

SmartBear tools are built to streamline your process while seamlessly working with your existing products. Whether it’s TestComplete, Swagger, Cucumber, ReadyAPI, Zephyr, or one of our other tools, we span test automation, API life cycle, collaboration, performance testing, test management, and more. 

Synopsys: A powerful and highly configurable test automation flow provides seamless integration of all Synopsys TestMAX capabilities. Early validation of complex DFT logic is supported through full RTL integration while maintaining physical, timing and power awareness through direct links into the Synopsys Fusion Design Platform.

SOASTA’s Digital Performance Management (DPM) Platform enables measurement, testing and improvement of digital performance. It includes five technologies: TouchTest mobile functional test automation; mPulse real user monitoring (RUM); the CloudTest platform for continuous load testing; Digital Operation Center (DOC) for a unified view of contextual intelligence accessible from any device; and Data Science Workbench, simplifying analysis of current and historical web and mobile user performance data. 

Testmo: Tracking, reporting and monitoring test automation results become more important as teams invest in and scale their automation suites. The new unified test management tool Testmo was designed to manage automated, manual and exploratory testing all in one platform. To accomplish this, it also directly integrates with popular issue, DevOps and CI tools such as GitHub, GitLab and Jira. It supports submitting and collecting results from any automation tool and platform.

testRigor supports “plain English” language that allows users to describe how to find elements on the screen and what to do with those elements from the end user’s perspective.  testRigor helps teams deploy their analytics library in production that will make systems automatically produce tests reflecting the most frequently used end-to-end flows from production. 

Tricentis Tosca, the #1 continuous test automation platform, accelerates testing with a script-less, AI-based, no-code approach for end-to-end test automation. With support for over 160+ technologies and enterprise applications, Tosca provides resilient test automation for any use case. 

test

The post A guide to automated testing tools appeared first on SD Times.