Matt Klassen, CMO, Parasoft

Quality continues to be the primary metric for measuring the success of software deliveries. With the continued pressure to release software faster and with fewer defects, it’s not just about speed — it’s about delivering quality at speed. 

Managers must ask themselves if they are confident in the quality of the applications being delivered by their teams. Continuous quality is a must for every organization to efficiently reduce the risk of costly operational outages and to accelerate time-to-market.

Parasoft’s automated software testing solutions integrate quality into the software delivery process for early prevention, detection, and remediation of defects. From deep code analysis for security and reliability, through unit, API, and UI test automation, to performance testing and service virtualization, Parasoft helps you build quality into your software development process.

Parasoft leverages our deep understanding of DevOps to develop AI-enhanced technologies and strategies that solve complex software problems. Our testing solutions reduce the time, effort, and cost of delivering secure, reliable, and compliant software.

With over 30 years of making testing easier for our customers, we have the innovation you need and the experience you trust. Our extensive continuous quality solution spans every testing need and enables you to deliver with confidence. If you want to improve your software quality while achieving your business goals, partner with Parasoft.

RELATED CONTENT:
Targeting a key to automated testing
A guide to automated testing tools

Jonathon Wright, chief technology evangelist, test automation at Keysight 

Artificial Intelligence (AI) makes the process of designing, developing, and deploying software faster, better and cheaper. AI-powered tools enable project managers, business analysts, software coders and testers to be more productive and effective, allowing them to produce higher-quality software faster and at a lower cost.

At Keysight, our Eggplant intelligent automation platform allows citizen developers to easily use our no-code solution that draws on AI, machine learning, deep learning and analytics to automate test execution across the entire testing process. It empowers and enables domain experts to be automation engineers. The AI and ML take on scriptwriting and maintenance as a machine can create and execute thousands of tests in minutes, unlike a human tester. 

Keysight’s intelligent automation platform is a completely non-invasive testing tool, ensuring comprehensive test coverage without ever touching the source code or installing anything on the system-under-test. The technology sits outside of the application and reports on performance issues, bugs and other errors without the need to understand the underlying technology stack. This is critical for regulated industries such as healthcare, government and defense.

AI-powered automation can test any technology on any device, operating system or browser at any layer, from the UI to APIs to the database. This includes everything from the most modern, highly dynamic website to legacy back-office systems to point of sale, as well as command and control systems.

The overarching goal of Keysight’s intelligent automation is to understand how customer experiences and business outcomes are affected by the behavior of the application or software. More than this, though, it is about identifying opportunities for improvements and predicting the business impact of those changes.

Gev Hovsepyan, head of product, mabl

Software development teams are realizing that automated testing is key to accelerating product velocity and reaching the full potential of DevOps. When fully integrated into a company’s development pipeline, testing becomes an early alert system for short-term defects as well as long-term performance issues that could hurt the user experience. The key to realizing this potential: simple test creation and rich, accessible reporting features. 

Mabl is low-code, intelligent test software that allows everyone, regardless of coding experience, to create automated tests spanning web UIs, APIs, and mobile browsers with 80% less effort. Using machine learning and AI, features like auto-healing and Intelligent Wait help teams create more reliable tests and reduce overall test maintenance. Results from every test are tracked within mabl’s comprehensive suite of reporting features, making it easy to understand product quality trends. With test creation simplified and quality data at their fingertips, everyone can focus on resolving defects quickly and improving product quality. 

Mabl also includes native integrations with tools like Microsoft Teams, Slack, and Jira, so that testing information can be seamlessly integrated into workflows and everyone can benefit from mabl’s rich diagnostic data. These reporting features include immediate test results as well as long-term product trends so that quality engineering teams can support faster bug resolution and monitor their product’s overall performance and functionality. This allows software development teams to shift from reacting to failed tests and customer complaints to proactively managing product quality, enabling them to spend more time improving the customer experience.

The post How these solution providers support automated testing appeared first on SD Times.

Keysight Technologies Eggplant Digital Automation Intelligence (DAI) platform is the first AI-driven test automation solution with unique capabilities that make the testing process faster and easier. With DAI, you can automate 95% of activities, including test-case design, test execution, and results analysis. This enables teams to rapidly accelerate testing, improve the quality of software and integrate with DevOps at speed. The intelligent automation reduces time to market and ensures a consistent experience across all devices.

mabl is the enterprise SaaS leader of intelligent, low-code test automation that empowers high-velocity software teams to embed automated end-to-end tests into the entire development lifecycle. Customer-centric brands rely on mabl’s unified platform for creating, managing, and running automated tests that result in faster delivery of high-quality, business critical applications. Learn more at https://www.mabl.com; follow @mablhq on Twitter and @mabl on LinkedIn.

Parasoft: Parasoft helps organizations continuously deliver quality software with its market-proven automated software testing solutions. Parasoft’s AI-enhanced technologies reduce the time, effort, and cost of delivering secure, reliable, and compliant software with everything from deep code analysis and unit testing to web UI and API testing, plus service virtualization and merged code coverage. Bringing all this together, Parasoft’s award-winning reporting and analytics dashboard delivers a centralized view of application quality, enabling organizations to deliver with confidence.

RELATED CONTENT:
Targeting a key to automated testing
How these solution providers support automated testing

Appvance is the inventor of AI-driven autonomous testing, which is revolutionizing the $120B software QA industry. The company’s patented platform, Appvance IQ, can generate its own tests, surfacing critical bugs in minutes with limited human involvement in web and mobile applications. 

Applitools: Applitools is built to test all the elements that appear on a screen with just one line of code. Using Visual AI, you can automatically verify that your web or mobile app functions and appears correctly across all devices, all browsers and all screen sizes. Applitools automatically validates the look and feel and user experience of your apps and sites. 

Digital.ai Continuous Testing (formerly Experitest) enables organizations to reduce risk and provide their customers satisfying, error-free experiences — across all devices and browsers. Digital.ai Continuous Testing provides expansive test coverage across 2,000+ real mobile devices and web browsers, and seamlessly integrates with best-in-class tools throughout the DevOps/DevSecOps pipeline.

HPE Software’s automated testing solutions simplify software testing within fast-moving agile teams and for continuous integration scenarios. Integrated with DevOps tools and ALM solutions, HPE automated testing solutions keep quality at the center of today’s modern applications and hybrid infrastructures. 

IBM: Quality is essential and the combination of automated testing and service virtualization from IBM Rational Test Workbench allows teams to assess their software throughout their delivery life cycle. IBM has a market leading solution for the continuous testing of end-to-end scenarios covering mobile, cloud, cognitive, mainframe and more. 

Micro Focus: Accelerate test automation with one intelligent functional testing tool for web, mobile, API and enterprise apps. AI-powered intelligent test automation reduces functional test creation time and maintenance while boosting test coverage and resiliency. 

Mobile Labs (acquired by Kobiton)  Its patented GigaFox is offered on-premises or hosted, and solves mobile device sharing and management challenges during development, debugging, manual testing, and automated testing. A pre-installed and pre-configured Appium server provides “instant on” Appium test automation.

NowSecure identifies the broadest array of security threats, compliance gaps and privacy issues in custom-developed, commercial, and business-critical mobile apps. NowSecure customers can choose automated software on-premises or in the cloud, expert professional penetration testing and managed services, or a combination of all as needed.

Orasi is a leading provider of software testing services, utilizing test management, test automation, enterprise testing, Continuous Delivery, monitoring, and mobile testing technology. 

Perfecto: Users can pair their favorite frameworks with Perfecto to automate advanced testing capabilities, like GPS, device conditions, audio injection, and more. It also includes full integration into the CI/CD pipeline, continuous testing improves efficiencies across all of DevOps.  

ProdPerfect: ProdPerfect is an autonomous, end-to-end (E2E) regression testing solution that continuously identifies, builds and evolves E2E test suites via data-driven, machine-led analysis of live user behavior data. It addresses critical test coverage gaps, eliminates long test suite runtimes and costly bugs in production, and removes the QA burden that consumes massive engineering resources. 

Progress: Telerik Test Studio is a test automation solution that helps teams be more efficient in functional, performance and load testing, improving test coverage and reducing the number of bugs that slip into production. 

Sauce Labs provides the world’s largest cloud-based platform for automated testing of web and mobile applications. Optimized for use in CI and CD environments, and built with an emphasis on security, reliability and scalability, users can run tests written in any language or framework using Selenium or Appium.

SmartBear tools are built to streamline your process while seamlessly working with your existing products. Whether it’s TestComplete, Swagger, Cucumber, ReadyAPI, Zephyr, or one of our other tools, we span test automation, API life cycle, collaboration, performance testing, test management, and more. 

Synopsys: A powerful and highly configurable test automation flow provides seamless integration of all Synopsys TestMAX capabilities. Early validation of complex DFT logic is supported through full RTL integration while maintaining physical, timing and power awareness through direct links into the Synopsys Fusion Design Platform.

SOASTA’s Digital Performance Management (DPM) Platform enables measurement, testing and improvement of digital performance. It includes five technologies: TouchTest mobile functional test automation; mPulse real user monitoring (RUM); the CloudTest platform for continuous load testing; Digital Operation Center (DOC) for a unified view of contextual intelligence accessible from any device; and Data Science Workbench, simplifying analysis of current and historical web and mobile user performance data. 

Testmo: Tracking, reporting and monitoring test automation results become more important as teams invest in and scale their automation suites. The new unified test management tool Testmo was designed to manage automated, manual and exploratory testing all in one platform. To accomplish this, it also directly integrates with popular issue, DevOps and CI tools such as GitHub, GitLab and Jira. It supports submitting and collecting results from any automation tool and platform.

testRigor supports “plain English” language that allows users to describe how to find elements on the screen and what to do with those elements from the end user’s perspective.  testRigor helps teams deploy their analytics library in production that will make systems automatically produce tests reflecting the most frequently used end-to-end flows from production. 

Tricentis Tosca, the #1 continuous test automation platform, accelerates testing with a script-less, AI-based, no-code approach for end-to-end test automation. With support for over 160+ technologies and enterprise applications, Tosca provides resilient test automation for any use case. 

test

The post A guide to automated testing tools appeared first on SD Times.

Automated tests are fast, but only in the direction that you point them to. And having a lot of them can easily cause a traffic jam so it’s important to first make sure that they are applied in the right areas and in the right way. 

“What I want to achieve is not more and more tests. What I actually want is as few tests as I possibly can because that will minimize the maintenance effort, and still get the kind of risk coverage that I’m looking for,” said Gartner senior director Joachim Herschmann, who is on the App Design and Development team. 

RELATED CONTENT:
How these solution providers support automated testing
A guide to automated testing tools

To get started with automated testing, organizations need to first look at where their tests will deliver the most value to avoid test sprawl and to prevent high maintenance costs. 

“The warm, fuzzy feeling that you’ve got a thousand automated tests per week doesn’t really tell you anything from a risk perspective with risk-based testing,” said Arthur Hicken, the chief evangelist at Parasoft. “So I think this kind of approach to doing value-driven automation as to what’s got the most value and what kind of confidence we need, what kind of coverage we need is important.”

Organizations need to factor in what it costs to create a test and what it costs to maintain a test because often the maintenance winds up costing a lot more than the creation. 

One must also factor in what it costs to execute a test in terms of time. With Big Bang releases a couple of times a year, creating tests is not such a big issue, but if a company is used to rolling out  weekly updates such as with mobile apps it’s really critical to be able to narrow and focus the automation on exactly the right set of tests. 

With a value-driven test automation strategy, organizations can identify full-stack tests that only cover backend business logic and that can be tested more efficiently through API-level integration (or even unit) tests. They can also identify bottlenecks with dependencies that can be virtualized for more efficient testing and automation, according to Broadcom in a blog post. 

The testers might decide not to automate some tests that they thought were ideal for automation, because having them performed by testers turns out to be more efficient.

Test at the API level

One way to tackle the complexity that comes with automated testing is to test at the API level rather than the UI, according to Hicken.

UI testing, which ensures that an application is performing the right way from the user perspective is notoriously brittle. 

“[UI testing] is certainly the easiest way to get started in the sense that it’s easy to look at a UI and understand what you need to do like start poking things, but at some point, that becomes very hard to continue,” Hicken said. “It’s hard to make boundary cases happen or to simulate error conditions. Also, fundamentally UI testing is the hardest to debug, because you have too much context and it’s the most brittle to maintain.” 

Meanwhile at the unit level, the automated tests are pretty fast to execute and create and are easy to understand and maintain. After unit testing, one can add the simplest functional tests that they have and then go and backfill with the UI. Now, they can make sure that actual business cases and user stories occur and they can implement these tests against the business logic to get the proper blend of testing, Hicken explained.

“It’s not really that top down approach of if I see a system and automate that system, it’s actually now from a bottom up focus of well in which people are approaching automation at an enterprise scale and asking what’s the blueprint or pattern that we’re trying to do?,” Jonathon Wright, the chief technology evangelist of test automation at Keysight said. “It’s incredibly complex states and the devil’s in the details…they’re asking how do you test those things with realistic data rather than a happy path?” 

Wright explained that happy path testing just won’t cut it anymore because people are testing systems upstream and downstream with all the same kind of data and it all works out in the happy path kind of scenario. Even when people are doing contract testing where each one of the systems is tested end-to-end from an API perspective, people are just using one user with one account with one something and then, of course, it works. But this methodology misses the point, according to Wright. 

“Because people are testing in isolation, they’re also testing their shim or stub or their service virtualization component using Wireshark, so that they’re not actually testing against the real API. So they exclude a lot of things by just locking them out,” Wright added. 

Focus on real-user interactions

A good way to set up automated tests is to focus on how real users are interacting with the systems and how the behavior of those systems are being used. 

“It’s quite scary, because obviously, its perception of what the system does, but actually what the system is doing in the live environment and how the customers are using it. But you kind of assume that they’re going to use it in a particular way, when actually the behavior will change. And that will change weekly and monthly,” Wright said. 

That’s why testers can set up a digital twin of the system as it currently is, and then overlay that with what they thought the system was based on. 

“There’s a different type of behavior mapping; it’s learning from the right hand side this kind of shift right to inform the shift left blueprint model of the system which I think actually helps accelerate everything because you don’t need to create an activity,” Wright added. “You can create it all from real users. You just take their exact journey and then within a matter of minutes, we can actually generate all the automation artifacts with it.”

Teams must then slice the user journeys into smaller, more meaningful pieces and automate against those smaller journeys without going too deep. It’s important that they can automate every clique and not merge too many user journeys together in a single test resulting in multiple hundred step tests, according to Gev Hovsepyan, the head of product at mabl. 

That initial setup of the environment proves to be an interesting discussion between quality engineers and software engineers and in the organization as a whole. “I think that initial configuration, especially when onboarding the test automation platform, becomes an important discussion point, because the way you set it up, is going to define how scalable that approach is,” Hovsepyan said. 

The role of service virtualization

The key to unlocking continuous testing is having an available, stable, and controllable test environment. Service virtualization makes it possible to simulate a wide range of constraints in test environments, whether due to unavailability or uncontrollable dependencies. 

The behaviors of various components are mimicked with mock responses that can provide an environment almost identical to a live setting. 

“Service virtualization is an automation tester’s best friend. It can help to resolve roadblocks and allow teams to focus on the tests themselves instead of worrying about whether or not they can get access to a certain environment or third party service,” Amit Bhoraniya, the technical lead at Infostretch wrote in a blog post. 

Organizations can also prevent having too many automated tests by having a unified platform and by ensuring quality earlier on in the pipeline. 

Companies are looking for an approach that not only helps them with functional testing, but helps them with non-functional testing and scaling across different teams on a single platform, and having visibility across the quality of their product across different teams across different testing domains, according to mabl’s Hovsepyan. 

A unified approach helps because the responsibilities for testing and quality assurance are often shared within an organization, and that varies based on their DevOps maturity. 

At more mature organizations in terms of DevOps adoption, there is often a center of excellence of quality engineering, where they deploy the practices and then everyone in the organization participates in assuring the quality, including engineers, or developers.

Organizations that are still somewhere early or in the middle of their journey of DevOps adoption have a significant amount of ownership of quality assurance and quality automation at the team level. And these teams have added quality engineers, and they are responsible for ensuring the quality through automation as well as for manual testing. 

This collaborative effort to test automation can help ensure that the developers and testers both know how these tests should be created and maintained. 

“Test automation is one of those things that when it’s done it’s a huge enabler and can really give your business a boost,’ Hicken said. “And when it’s done wrong, it’s an absolute nightmare.”

AI can help with test creation and maintenance

The introduction of AI and ML assisting into automated testing makes it easier to shift quality left by providing earlier defect remediation and reducing risk for deliveries. 

By collecting and incorporating test data, machine learning can effectively update and interpret certain software metrics that show the state of the application under test. Machine learning can also quickly gather information from large amounts of data and point developers or testers right to the performance problem. 

AI is also excellent at finding those one-in-a-million anomalies which testers might just not catch, according to Jonathon Wright, chief technology evangelist at testing company Keysight. 

In the blog,  “What is Artificial Intelligence in Software Testing?,” Igor Kirilenko, Parasoft’s VP of Development, explains that these AI capabilities “can review the current state of test status, recent code changes, code coverage, and other metrics, decide which tests to run, and then run them,” while machine learning (ML) “can augment the AI by applying algorithms that allow the tool to improve automatically by collecting the copious amounts of data produced by testing.”

By 2025, 70% of enterprises will have implemented an active use of AI-augmented testing,

up from 5% in 2021, according to Gartner’s “Market Guide for AI-Augmented Software testing Tools.” Also by 2025, organizations that ignore the opportunity to utilize AI-augmented testing will spend twice as much effort on testing and defect remediation compared with their competitors that take advantage of AI. 

AI-augmented software testing tools can provide capabilities for test case and test data generation, test suite optimization and coverage detection, test efficacy and robustness, and much more. 

“AI can change the game here, because even in the decades that we’ve had test automation tools, there’s very little that it offered you regarding any guidance like how do I determine the test cases that I need?” Herschmann said. 

The post Targeting a key to automated testing appeared first on SD Times.

Eggplant’s software test automation platform leverages artificial intelligence and analytics to automate test creation and test execution. Keysight’s solutions optimize networks and help bring products to market faster and at lower costs. It features designed simulation, prototype validation, manufacturing test, and optimization in networks and cloud environments. 

“Joining forces with Keysight gives Eggplant the ability to scale our intelligent automation platform and reach more organizations across the globe,” said Dr. John Bates, CEO of Eggplant. “We share a vision to accelerate innovation and together we will be able to help customers on their digital transformation journey. We’re proud of what we’ve accomplished through our employees and partners’ contributions, and we’re excited about this next chapter.”

As part of the acquisition, the companies will enable bi-directional leverage of measurement technologies and provide expanded offerings. 

The acquisition is valued at $330 million. Eggplant CEO John Bates will join the Keysight leadership team and report to Soon-Chai Gooi, president of its electronic industrial solutions group. 

“As a recognized leader and trusted advisor in layer 1-7 design and test, Keysight is excited to add Eggplant’s test capabilities for the software application layer, aligning with our strategy to grow our first-to-market software-centric solutions,” said Ron Nersesian, Keysight chairman and CEO. “We’re thrilled to welcome the Eggplant team to the Keysight family and look forward to working together in the fast-growing intelligent software test market with differentiated software-as-a-service technologies.”

The post Eggplant gets acquired by Keysight Technologies appeared first on SD Times.

“Optimizing the customer experience is the holy grail for all digital businesses,” said Antony Edwards, COO of Eggplant. “But most teams we speak to tell us that they don’t have the time or skills for performance and usability testing. These new capabilities in Eggplant’s DAI suite mean that every testing team can now quickly and easily do performance and usability testing in addition to the functional testing they are already doing.”

Sensory acquires Vocalize.ai
Speech recognition company Sensory has acquired Vocalize.ai, which is a software tools company that offers benchmarking, accuracy assessments, and bias evaluations for speech technologies. After the acquisition, Vocalize.ai will remain as an independently operated division.

“As more companies add voice to their products, there is growing need for an independent evaluation service and software tools that ensure a quality user experience,” said Joe Murphy, CEO of Vocalize.ai. “It’s exciting to have access to the deep bench of AI and machine learning talent and resources of Sensory. It is also important to recognize that Vocalize.ai will operate as an independent company under the Sensory umbrella. In this model, we will continue to provide quality evaluations and competitive benchmarking services for the entire voice-enabled industry.”

Assembla updates its Project Management tool
Assembla has released the latest version of its project management tool. The tool enables developers to build and deploy software quickly and with precision thanks to Assembla Tickets and sprint workflows, the company explained.

New features include sprint planning, nested tasks and subtasks, and the ability to change cardwall columns and sort by assigned users.

Sencha Ext JS 6.7 now available
Sencha has announced that it is releasing the latest version of Ext JS. Sencha Ext JS 6.7 will feature improvements to the Modern toolkit, providing new ways for developers to quickly design and develop web apps.

New features of Ext Js 6.7 include the addition of grid filtering, locking grid, chip, multiselect combobox, color picker, and virtual scroller.

The post SD Times news digest: Eggplant’s DAI Suite, Sensory acquires Vocalize.ai, and Sencha Ext JS 6.7 appeared first on SD Times.

“Rapid change and accelerating application delivery is a topic that used to really be something only technology and Silicon Valley companies talked about. Over just the past few years, it has become something that almost every organization is experiencing,” said Lubos Parobek, vice president of product for the testing company Sauce Labs. “They all feel this need to deliver apps faster.”

RELATED CONTENT: A guide to automated testing tools

This sense of urgency has businesses looking to leverage test automation even further and go beyond just automating repetitive tasks to automating in dynamic environments where everything is constantly changing. “As teams start releasing even weekly, let alone daily or multiple times a day, test automation needs to change. Today test automation means ‘automation of test execution,’ but the creation and maintenance of tests, impact analysis and the decision of which test to run, the setup of environments, the reviewing of results, and the go/no-go decision are all entirely manual and usually ad-hoc,” said Antony Edwards, CTO of the test automation company Eggplant. “The key is that test automation needs to expand beyond the ‘test execution’ boundary and cover all these activities.”

Pushing the limits
Perhaps the biggest drivers for test automation right now are continuous integration, continuous delivery, continuous deployment and DevOps because they are what is pushing organizations to move faster and get software into the hands of their users more quickly, according Rex Black, president of the Rex Black Consulting Services, a hardware and software testing and quality assurance consultancy.

“But the only way for test automation to provide value and to not be seen as a bottleneck is for it to be ‘continuous,’” said Mark Lambert, vice president of products at the automated software testing company Parasoft.

According to Lambert, this happens in two ways. First, the environment has to be available at all times so tests can be executed at anytime and anywhere. Secondly, the tests need to take change into account. “Your testing strategy has to change resistance built into it. Handling change at the UI level is inherently difficult, which is why an effective testing strategy relies on a multi-layer approach. This starts with a solid foundation of fully automated unit tests, validating the granular functionality of the code, backed up with broad coverage of the business logic using API layer testing,” said Lambert. “By focusing on the code and API layers, tests can be automatically refactored leaving a smaller set of the brittle end-to-end UI level tests to manage.”

Part of that strategy also means having to look at testing from a different angle. According to Eggplant’s Edwards, testing has shifted from testing to see if something is right, to testing to see if something is good. “I am seeing more and more companies say, ‘I don’t really care if my product complies with a [specification] or not,’ ” he said. “No one wants to be the guy saying no one is buying our software anymore, and everyone hates it, but at least it complies with the spec.” Instead, testing is shifting from thinking about the requirements to thinking about the user. Does the software increase customer satisfaction, and is it increasing whatever the business metric is you care about?

“If you care about your user experience, if you care about business outcome, you need to be testing the product form the outside in, the way a user does,” Edwards added.

Looking at it from the user’s side involves monitoring performance and the status of a solution in production. While that may not seem like it has anything to do with testing or automation, it’s about creating automated feedback loops and understanding the technical behavior of a product and the business outcome, Edwards explained. For example, he said if you look at the page load speed of all your pages and feed that back into testing, instead of automating tests that say every page has to respond in 2 seconds, you can get more granular and say certain pages need to load faster while other pages can take up to 10 seconds and won’t have a big impact on experience.

“Testing today is too tied to the underlying implementation of the app or website. This creates dependencies between the test and the code that have nothing to do with verification or validation, they are just there because of how we’ve chosen to implement test automation,” Edwards said.

But just because you aren’t necessarily testing something against a specification anymore, doesn’t mean you shouldn’t be testing for quality, according to Thomas Murphy, senior director analyst at the research firm Gartner. Testing today has gone from a calendar event to more of a continuous quality process, he explained.

“There is a fundamental need to be shipping software every day or very frequently, and there is no way that testing can be manual. You don’t have time for that. It needs to be fast,” he said.

Some ways to speed things up is to capture the requirements and create the tests upfront. Two approaches that really drove the need for automating testing are test-driven development (TDD) and behavior-driven development (BDD). TDD is the idea that you are going to write the test first, then write the code to pass that test, according to Sauce Labs’ Parobek. BDD is where you enable people like the business analyst, product manager or product owners to write tests at the same time developers are developing code.

These approaches have helped teams get software out multiple times a day because they don’t have to wait for days to create the tests and get back results, and it enables them to understand if they make a mistake right away, Parobek explained.

However, if a developer is submitting new code or pull requests to the main branch multiple times a day, it can be hard to keep up with TDD and BDD, making automated testing impossible because there aren’t tests already in place for these changes. In addition, it slows down the process because now you have to go in manually to make sure the code that is being submitted doesn’t break any key existing function, according to Sauce Labs’ Parobek.

But Parobek does explain if you write your test correctly and follow best practices, there are ways around this. “As you change your application and as you add new functionality, you do not just create new tests, but you might have to change some existing tests,” he said.

Parobek recommends page object modeling as a best practice. It enables users to create tests in a way that is very easy to change when the behavior of the app is changed, he explained.  “It enables you to abstract out and keep in one place changes so when the app does change, you are able to change one file that then changes a variety of test cases for you. You don’t have to  go into 100 different test cases and change something 100 times. Rather you just change one file that is abstracted through page objects,” he said.

Another best practice, according to Parobek, is to be smart about locators. Locators enable automated tests to identify different parts of the user interface. A common aspect of locators is IDs. IDs enable tests to identify elements. For example, when an automated test goes in and needs to test a button, if you’ve attached a locator ID to it, the test can recognize the button even if you moved it somewhere else on the page. Other approaches to locators are to use names, CSS selectors, classes, tags links, text and XPath. “Locators are an important part for creating tests that are simpler and easier to maintain,” said Parobek.

In order to successfully use locators, Parobek thinks it is imperative that the development and QA teams collaborate better. “If QA and development are working closely together, it is easy to build apps that make it easier to test versus development not thinking about testability.”

No matter how much you end up being able to automate, Black explained in order to be successful at it, you will still always have to go back to the basics. If you become too aspirational with automation and have too many failed attempts, it can reduce management’s appetitive for wanting to invest. “You need to have a plan. You need to have an architecture,” Black said. “The plan needs to include a business case so you can prove to management it is not just throwing money into a bright shiny object.”

“It’s the boring basics. Attention to the business case. Attention to the architecture. Take it step by step and course correct as you go,” Black added.

The promise of artificial intelligence in automated testing
As artificial intelligence (AI) advances, we are seeing it be implemented in more tools and technologies as a way to improve user experience provide business value. But when it comes to test automation, the promise of AI is more inspirational than operational, RBCS’ Black explained.

“If you go to conferences, you will hear about people wanting to use it, and tool vendors making claims that they are able to deliver on it. But at this point, I have not had a client tell me or show me a successful implementation of test automation that relies on AI in a significant way,” he said. “What is happening now is that tool vendors are sensing that this is going to be the next hot thing and are jumping on that AI train. It is not a realized promise yet.”

When you think about AI, you think about a sentient element figuring things out automatically, according to Gartner’s Murphy, when in reality it tends to be some repeated pattern of learning something to be predictive or learning from past experiences. In order to learn from past experiences, you need a lot of data to feed into your machine learning algorithm. Murphy explained AI is still new and a lot of the test information that companies have today is very fragmented, so when you hear companies talk about AI in regards to test automation it tends to be under-delivering or over-promising.

Vendors that say they are offering an AI-oriented test automation tool are often just performing model-based testing, according to Murphy. Model-based testing is an approach where tests are automatically generated from models. The closest thing we have out there to an AI-based test automation tool are image-based recognition solutions that understand if things are  broken, and can show when it happened and where through visual validation, Murphy explained.

However, Black does see AI having potential within the test automation space in the future; he just warns businesses against investing in any technologies too soon. Areas where Black sees the most potential for AI include false positives, and flaky tests.

False positives happen when a test returns a failed result, but it turns out the software is actually working correctly. A human being is able to recognize this when they look further into correcting the result. Black sees AI being used to apply human reasoning and differentiate the correct versus incorrect behavior.

Flaky tests happen when a test fails once, but passes when the test runs again. This unpredictable result is due to the variation of the system architecture, the test architecture, the tool, or the test automation, according to Black. He sees AI being used to handle validation issues like this by bringing a more sophisticated sense of what fit for use means to the testing efforts.

Kevin Surace, CEO of Appvance.ai, sees AI being applied to test automation, but in different levels. Surace said there are 5 levels of AI that can be applied to test automation:

1. Scripting/coding
2. “Codeless” capture/playback
3. Machine learning: self-healing human-created scripts and money bots
4. Machine learning: Near full automation with auto-generated smart scripts
5. Machine learning full automation: auto-generated smart scripts with validation

When deciding on AI-driven testing, Surace explained the most important qualification is to learn what type of level of AI a vendor is offering. According to Surace, many vendors have offerings at levels one and two, but there are very few vendors that can actually promise levels three and above.

In the future, Parasoft’s Lambert expects humans will just be looking at the results of test automation with the machine actually doing the testing in an autonomous way. But for now, the real value of AI and machine learning will be used to augment human work and spot patterns and relationships in the data in order to guide the creation and execution of tests, he explained.

Still, Black warns to enter AI for test automation with caution. “Organizations that want to try to use AI-based test automation at this point in time should be extremely careful and extremely conservative in how they pilot that and how they roll that out. They need to remember that the tools are going to evolve dramatically over the next decade, and making hard, fast and difficult to change  large investments in automation may not be a wise thing in the long term,” he said.

Manual practices remain
Despite the efforts to automate as much as possible, things for the time being will still require a human touch.

According to Rex Black, president of the Rex Black Consulting Services (RBCS),  a hardware and software testing and quality assurance consultancy, you can break testing down into two overlapping categories: 1. Verification, where a test makes sure the software works as specified; and 2. Validation tests, where you make sure tests are fit for use. For now, Black believes validation will remain manual because it is very hard to do in an automated fashion. For example, he explained, if you developed a video game, you can’t automate for things like: Is it fun? Is it engaged? Is it sticky? Do people want to come back and keep playing it?

“At this point, automation tools are really about verifying that the software works in some specified way. The test says what is suppose to happen and checks to see if it happens. There is always going to be some validation that will need to be done either by people,” he said.

Lubos Parobek, vice president of product for the testing company Sauce Labs explained that even if we get to a point where everything is automated in the long-term future, you will still always want a business stakeholder to take a final look and do a sanity check that everything works as expected to a human.

“Getting a complete view of customer experience isn’t just about validating user scenarios, doing click-counts and sophisticated ‘image analysis’ to make sure the look and feel is consistent — it’s about making sure the user is engaged and enjoying the experience. This inherently requires human intuition and cannot be fully automated,” added  Mark Lambert, vice president of products for automated software testing company Parasoft.

Robotic process automation
Test automation vendors are flocking to this idea of robotic process automation (RPA). RPA is a business process automation approach used to cut costs, reduce errors and speed up processes, so what does this have to do with test automation?

According to Thomas Murphy, senior director analyst at Gartner, RPA and test automation technologies have a high degree of overlap. “Essentially both are designed to replicate a human user performing a sequence of steps.”

Anthony Edwards, CTO of the test automation company Eggplant, explained that on a technical level, test automation is about automating user journeys across an app and verifying that what is supposed to happen, happens. RPA aims to do just that. “So at a technical level they are actually the exact same thing, it’s simply the higher level intent and purpose that is different. But if you look at a script that automates a user journey there is no way to tell if it has been created for ‘testing’ or for ‘RPA’ just by looking at it,” said Edwards. “The difference for some people would be that testing focuses on a single application whereas RPA typically works across several systems integrated together.”

Over the next couple of years, Gartner’s Murphy predicts we will see more test automation vendors entering this space as a new way to capitalize on market opportunity. “By moving into the RPA market, they are expanding their footprint and audience of people they go after to help them,” he said.

This move is especially important as more businesses move toward open-source technologies for their testing solutions.

Rex Black, president of the Rex Black Consulting Services (RBCS), a hardware and software testing and quality assurance consultancy, sees the test automation space moving towards open source because of cost. “It’s easier to get approval for  a test automation project if there isn’t a significant up-front investment in a tool purchase, especially if the test automation project is seen as risky. Related to that aspect of risk is that so many open-source test automation tools have been successful over recent years, so the perceived risk of going with an open-source tool is lower than it used to be,” he said.

The post Pushing automated testing to its limits appeared first on SD Times.

To take these capabilities even further, testing and performance solution provider Eggplant has announced Eggplant RPA, an intelligent RPA tool that leverages the company’s Digital Automation Intelligence (DAI) solution.

“Although the Robotic Process Automation industry is fairly new, Eggplant has a long heritage in automating the un-automatable. Our customers have naturally used Eggplant for all kinds of automation including RPA and so it is was listening to our customers and looking at their use cases, that lead us to create Eggplant RPA. When faced with the prospect of RPA, many organizations believe that it is too technical and that a lot of developers need to be involved. However, with the powerful modeling and fusion automation of Eggplant RPA, we are proving this is simply not the case,” said Antony Edwards, CTO of Eggplant.

Eggplant created DAI for test automation. DAI is designed to automate manual developer processes and intelligently understand and act on screen images and text. With this technology inside the Eggplant RPA solution, companies can automatically convert and migrate files, provide onboarding and offboarding of new employees to companies, provide a modeling interface for non-technical users, and enable attended or unattended RPA and automation for multiple systems. Industries currently using Eggplant RPA include federal organizations for database migration and audit trails, hospitals for security patches, and marketing automation, according to the company.

“Eggplant RPA can work with Eggplant Real Customer Experience Insights (CXI) to record and analyze the usage patterns of real users in production and feed this into the Eggplant Modeler as a robotic process. In addition, Eggplant RPA can be used to provide an API for a legacy application that has no native API. To achieve this, a REST API can be customized that then triggers Eggplant automation actions through an application’s user interface,” the company wrote in its announcement.

Other features include end-to-end automation, a universal fusion engine, data-driven automation, and automated discovery.

The post Eggplant launches Robotic Process Automation solution appeared first on SD Times.

“Hasura makes it easy for developers to close the loop between their serverless backends and their apps. When serverless functions running business logic are triggered, propagating information back to the end-user on say an app used to be a challenge. Hasura’s real-time GraphQL makes it easy for developers to do this without any effort. Developers can use realtime GraphQL to build reactive UX to go along with their async backends,” said Tanmai Gopal, co-founder and CEO of Hasura.

Pusher launches Beams API
Pusher has announced Beams, a new API designed to help developers ensure that transactional information is delivered to users.

Beams provides a hosted service that can manage the device token lifecycle for both iOS and Android applications. It also provides Insights to help track delivery acknowledgement and open events. Finally, it will offer a Debug Console for troubleshooting issues in real-time.

“Notifications are the lifeline of an application, but managing the infrastructure behind the scenes is a headache,” said Jordan Harp, product manager at Pusher. “The well-kept secret of the push notification industry is that no one knows if your transactional notifications actually got delivered. Things like your ride-share driver arriving, a meeting reminder, or a bank transaction might arrive in time or get delivered late – there was no way of checking that. This is the problem we are solving for our customers.”

The Linux Foundation launches new blockchain course
The Linux Foundation has announced that enrollment for its new blockchain course, “LFD271 – Hyperledger Fabric Fundamentals,” is now open. The course will introduce core concepts of blockchain and distributed ledger technologies. It will also introduce the core architecture and components that comprise Hyperledger Fabric applications.

Later this year, it will release Certified Hyperledger Fabric Administrator and Certified Hyperledger Sawtooth Administrator exams.

“Blockchain technology adoption is increasing at a rapid pace – with TechCrunch reporting blockchain jobs as the second-fastest growing in today’s labor market – leading to a shortage of professionals who are qualified to implement and manage it on an enterprise scale,” said Clyde Seepersad, general manager of training & certification at The Linux Foundation. “After seeing more than 100,000 students take our free introductory Hyperledger course, we knew it was time for more advanced training options, and certification exams to demonstrate the extent of professionals’ knowledge.”

Eggplant releases two new products as part of its open-source communities initiative
Eggplant has announced the release of two new products as part of its open-source communities initiative. According to Eggplant, these products will help out the JMeter and Selenium communities.

Eggplant Performance for JMeter is a version of Eggplant’s load testing solution that can be used with JMeter Test Plants. According to the company, this solution adds “strong test composition, environment management, dynamic control, and result analytics to JMeter’s existing scripting capability.”

Eggplantium allows customers to run Selenium WebDriver scripts against mobile devices in a reliable way.

The post SD Times news digest: Hasura GraphQL Engine, Pusher’s Beams API, and The Linux Foundation’s blockchain course appeared first on SD Times.

GitHub hopes that by open-sourcing the solution, others will be able to benefit and “contribute to a common standard software load balancing solution that runs on commodity hardware in physical datacenter environments.”

Sauce Labs launches Continuous Testing Services
Sauce Labs has launched Continuous Testing Services in order to help organizations on their journey to automation and continuous testing. As part of this release, it is also launched a 6-8 week program for teams that are starting to adopt continuous testing called Sauce Start.

According to Sauce Labs, Sauce Start is targeted at companies that have little or no test automation in place; little or no Selenium or Appium experience; have test automation in place, but lack integration with their continuous integration architecture; or are using legacy test automation solutions and are looking to move to modern tools.

Eggplant launches Eggplant Release Insights
Eggplant has launched Eggplant Release Insights to help organizations visualize the impact on user satisfaction and business outcomes if they were to release another version of their product now.

Eggplant Release Insights works together with Eggplant AI to provide an extensive set of comprehensive predictors to determine release quality, such as ones for bug content, development quality, test coverage, and usability quality.

“DevOps teams are drowning in data but, in this world of continuous everything, they lack the ability to easily predict outcomes and the effect on the business,” said Antony Edwards, CTO of Eggplant. “Optimizing products to drive growth is essential for every organization and our “release rating” will provide teams with real time insights to immediately understand the quality and the impact on the user before release. This further cements Eggplant’s leadership in deploying AI, machine learning, and analytics today to optimize the customer experience and accelerate the pace of DevOps.”

Revulytics adds new feature tracking and reporting capabilities
Revulytics has launched new feature tracking and reporting capabilities in its analytics offering Lifetime Feature Usage. The update enables every feature tracked in Usage Intelligence to be filtered based on environmental properties as well as usage by particular audiences.

Other improvements to Usage Intelligence include Sunburst visualizations, ReachOut delivery caps, regular expressions, a date installed filter, and a date last seen filter.

“Our mission is to increase the power of product usage analytics to help our customers build better products by answering more questions about how users interact with their application. The introduction of Lifetime Feature Usage analysis extends our existing set of feature usage reports, giving our customers a deeper understanding of how specific user groups behave throughout their lifetime and reasons for churn,” said Keith Fenech, VP of Software Analytics. “Additionally, customers can better target ReachOut in-application messaging campaigns to influence that behavior and increase feature adoption and retention.”

The post SD Times news digest: GitHub’s open-source load balancer, Sauce Labs’ Continuous Testing Services, and Eggplant Release Insights appeared first on SD Times.

“Our goal has always been around trying to help people make software that delights their users,” said Antony Edwards, CTO of Eggplant.

Eggplant helps a wide variety of companies test all sorts of solutions, from point-of-sale terminals to vending machines to banking websites. “We test the whole user experience and we can do this in a nice SaaS platform so that you can scale up and down easily,” said Edwards.

Edwards believes that if you’re doing true test-driven development, you cannot start with a testing approach that is centered on code rather than on the user. “What you should be validating at the end of your sprint is validating that you have delivered that benefit, have delivered that functionality, that story to the user. And the only way to do that properly is to come at it from the user perspective,” said Edwards.

Most software testing vendors work to analyze code and find a way to ensure that that code complies with the necessary specifications, Edwards explained. What Eggplant does when testing is that it ensures that the software is behaving correctly and doing the things that will make the user happy. “I think testers need to start looking at the customers first and not really care that much if we comply with this little bit of the spec, that little bit of the spec,” said Edwards.

“I definitely think that more testers need to be thinking, how can I drive the business outcome?” Edwards said. “And I have to say, as I talk to companies these days, I’m hearing more and more people thinking that way.” He believes this should be the starting point for testing these days.

The company’s testing product, Eggplant AI, uses AI, deep learning, and analytics to accelerate the process of testing. “We’ve always had that advantage in that we’re always testing from the user’s perspective, but the other thing that other people have issues with in testing is the amount of effort that you have to put in.”

As more and more teams move to DevOps and reduce the length of their project cycles, it is the testing aspect that falls behind and cannot keep up, Edwards explained.

By bringing AI and analytics into the process, Eggplant is able to automatically generate test cases. It allows people to create a lightweight model of their application, which can be used to create millions or billions of different test cases that provide complete, comprehensive coverage of the application.

Since a company cannot run billions of test cases every night, Eggplant AI also uses neural networks and deep learning to identify a couple thousand test cases that are the most likely to help find defects and improve the user experience.

The company also recently acquired NCC Group’s Web Performance division and its solution, which Edwards explained does real user and synthetic monitoring of systems. “The magic thing these guys do it they understand what your users are doing, how they’re behaving, what journeys they’re taking and the demographics of those people converting,” said Edwards.

They also understand the technical behavior of the website and are able to build a model that will show how that behavior influences the business outcome and customer satisfaction.

“And then of course the reason we think it’s a good match is that we then bring that back into the testing process. We’re already focusing on the user experience and now we’ve got a lot more data to understand what really matters,” said Edwards.

According to Edwards there are three steps organizations can take to make their testing process more user-focused.

First, they should use user analytics to focus testing efforts. For example, looking at the parts of the applications that users are using, determining what annoys them, and finding out their demographic are good areas to take a look at. “Currently, this kind of information doesn’t appear in any test strategy,” said Edwards. “It should.”

Second, they should test from the user’s perspective. The user only cares about what appears on the screen or what happens when they press a button. “Test through the eyes of the user,” Edward said.

Finally, test objectives should be redefined as increasing user satisfaction and business outcomes instead of as covering requirements.

“I’m seeing more and more testers saying what we can really offer, we know what good looks like, we can get information about the customers, and we can be making sure that the new releases they’re doing every week or two weeks are going to be a net benefit to customer satisfaction rather than reduction,” said Edwards.

The post Eggplant approaches testing from the user perspective to deliver better business outcomes appeared first on SD Times.