Episode 1:  Is AI coming for your job?

The post The Rage Cage appeared first on SD Times.

This capability integrates into mabl’s SaaS platform so that users can enhance the value of existing functional tests, move performance testing to an earlier phase of the development lifecycle, and cut down on infrastructure and operations costs.

“The primary goal is to help customers test application changes under production load before they release them so that they can detect any new bottlenecks or things that they would have experienced as the changes hit production before release,” said Dan Belcher, co-founder of mabl.

According to the company, these API load testing capabilities allow for the unification of functional and non-functional testing by utilizing functional API tests for performance and importing Postman Collections to cut down on the time it takes to create tests. 

Mabl also stated that this performance testing lowers the barrier to a sustainable and collaborative performance testing practice, even for teams that do not have dedicated performance testers or specific performance testing tools. 

“Anyone within the software team can use it, so it is not limited to just the software developers or just the performance experts,” Belcher said. “Because we’re low-code and already handling the functional testing, it makes it super easy for the teams to be able to define and execute performance tests on their own without required specialized skills.”

Furthermore, these tests can also be configured to run alongside functional tests on demand, on a schedule, or as a part of CI/CD pipelines. 

The post Mabl’s load testing offering provides increased insight into app performance appeared first on SD Times.

Even if an app delivers the best, most innovative functionality, it won’t matter if loading or transactions take too long. Further, as users continue to grow increasingly impatient, the definition of “too long” continues to shrink.

As a result, it is critical to validate application performance, both before new services are launched and before periods of peak activity. For example, for a retailer, it is vital to do extensive performance testing in advance of Black Friday. 

Traditionally, teams have been doing what we’d refer to as “classic” performance testing. Through this approach, teams do end-to-end testing based on high volumes of virtual users or synthetic transactions. Quite often, this type of performance testing is likely to delay new releases by several weeks. Further, this type of testing is extremely costly, invariably consuming a lot of staff time and resources. These challenges are particularly problematic in modern continuous delivery lifecycles, where keeping cycle times to a minimum is critical. 

To combat these challenges, it is imperative that teams gain the ability to speed software releases, while consistently ensuring they’re not introducing performance issues into production environments. 

Introducing Continuous Performance Testing

While a lot has been written about continuous testing, much of the focus has tended to center on continuous functional testing. A subset of continuous testing, continuous performance testing (CPT) is based on the principle of “continuous everything” in DevOps. As opposed to having a single performance testing phase, CPT is employed in situations in which performance testing needs to happen across different phases of the continuous integration/continuous delivery (CI/CD) lifecycle. 

CPT is a key enabler of continuous delivery. With CPT, teams can ensure apps are peak-performance ready at all times, so they can release new code without lengthy performance testing delays. There are three keys to enabling the successful implementation of CPT: 

•     Teams must be able to specify performance requirements at the component level. These requirements must either be tied to functional requirements or product features, or they need to be tied to an application’s specific system components.
•     Teams must be able to test each component in isolation.
•     Teams have to be able to test frequently as application changes occur. 

Continuous Performance Testing: Three Best Practices

As organizations seek to employ CPT in their environments, there are a few key practices that will help ensure implementation success. Each of these practices is detailed in the following sections. 

Test at the Lowest Possible Level of Granularity

With CPT, most testing can be done at the unit, component, or API levels. By establishing testing at the component level, teams can test early and often. This component-level approach offers advantages in speed and operational efficiency.  

Another key advantage is that this approach reduces the number of tests that have to be completed: If component-level tests don’t pass, teams don’t need to run higher-level tests. This means teams can reduce the amount of resource-intensive, end-to-end tests that have to be executed. This also means more testing happens at the CI level, rather than at the CD level, where minimizing lead time for changes is most critical. 

Establish Frequent, Change-Driven Testing 

Once teams begin to do more granular, component-level testing, they can then employ another approach that helps reduce elapsed testing time: change-impact testing. Through this approach, teams can focus testing on specific parts of applications that have changed. At a high level, there are two ways to make this happen:

•     “Inside-out” approach. In this scenario, teams take an inside-out approach by analyzing the impact of changes made in the code of application components. 
•     “Outside-in” approach. In this case, taking an outside-in approach refers to focusing on analyzing the impact of changes made to application requirements or behavior. Through this outside-in approach, every time a requirement is changed, teams flag the set of tests that have been affected. In many organizations, this approach has reduced the amount of ongoing testing that is required by approximately 70%.

Scale Testing of Individual Components 

As mentioned above, doing end-to-end performance testing is expensive and time consuming. Through CPT, teams can effectively scale testing on specific components and reduce their reliance on these resource-intensive tests. To further scale component-level testing, teams can integrate CPT activities with CI/CD orchestration engines. In this way, teams can automate a range of efforts, including provisioning of environments, deployment of app components and test assets, execution of tests, capture and dissemination of test results, and post-testing cleanup. Teams can also leverage continuous service virtualization and continuous test data management, which can further boost scalability and test coverage. 

To learn more, view my earlier article entitled, “Optimize Continuous Delivery of Microservices Applications with Continuous Performance Testing.” This article features step-by-step guidance for implementing CPT across all the stages of the CI/CD pipeline.

The post Why performance testing is so vital and so difficult appeared first on SD Times.

“VHS started as a project that filled a need related to our performance testing and optimization portfolio, namely, accurate load generation,” Abrahams said. “Our mission as a company is to extend the concept of application performance from being a reactive mindset focused on operations teams, to a proactive, automatic and continuous process that includes and empowers the application developers themselves. Part of that mission is ensuring that developers in the community are not only aware that proactive solutions are available to them, but that they’re able to contribute and help build tomorrow’s application performance solutions.”

According to the company, current methods for recording and replaying app traffic did not provide clear enough or precise results. VHS aims to provide load generational aligned with actual live production to better guarantee performance testing and forecasted traffic. 

As part of the community-driven project initiative, StormForge is asking the open-source community to help rename the project in Q1 of 2021. “The name VHS wouldn’t be particularly easy to find in a Google search, anyway, and the acronym is already taken in most places that matter, so the rename will be happening sooner rather than later,” Abrahams wrote. 

The post SD Times Open-Source Project of the Week: VHS appeared first on SD Times.

The biggest mistake organizations make when preparing for holiday sales is decreasing the required amount of security testing of their web and mobile applications in favor of tight release deadlines, said global director of application security strategy at Checkmarx, Matt Rose.

“Proper security testing is a must and should not be overshadowed by the need for enhanced features or functionality that may not even be utilized if an application is hacked or down to a DDoS attack,” he said.

(Related: How DevOps security is lacking)

Organizations might look to cut testing processes because of their shorter release deadlines. Sometimes, security testing is cut because “cool” application features are seen as generating revenue, whereas security testing is not, said Rose. It’s a narrow-minded view, because if the application has security issues, the new revenue-generating feature may never be available to the user, he said.

Different organizations can assign different levels of responsibilities to developers during the holiday season, but all companies should review how developers would support operations during critical times like Black Friday and Cyber Monday, according to Michael Butt, senior product marketing manager at BigPanda. And, just like those in operations, developers need to understand how much stress peak shopping times will have on systems during the holiday season, he said.

Developers can also prepare for the holiday season by properly testing their applications for stability and security, because the “potential for unanticipated load or exposure to hackers is a real threat,” said Rose.

If developers fail to do this, retailers can expect worst-case scenarios like being blacklisted by users, he said, especially if they fear that a platform is unstable and their personal information is at risk.

“The holiday selling season is a very short time period, and any downtime or instability of their web or mobile applications could potentially have very damaging implications to a retailer’s bottom line,” said Rose. “If an application fails to meet the consumer’s expectations, they will simply take their business somewhere else.”

Mobile applications have changed the world of digital business and e-commerce, and now that organizations are going to a mobile-first world, all of that mobile traffic adds to the holiday load, said Butt.

Just the nature of these mobile applications and how they have developed opens a new category for risk, said Rose. Many organizations outsource mobile application development to third parties, and if these third parties do not know if proper security testing was done to applications, it increases the chances of hackers attacking, according to him.

“The third parties are paid to develop these mobile apps based on a set of functionality criteria,” said Rose. “If security requirements are not properly defined by the outsourced development teams, they will probably not be included in the application, which is a huge risk to organization contracting the third party.”

The post Security testing should be on every DevOps team’s Black Friday checklist appeared first on SD Times.

The impact is largely felt in the testing and development community. Consider a few decades ago when the traditional IT environment comprised the standalone PC and the situation was far simpler. Developers only had two browsers to deal with, and there was only one platform to test them on. Developers would simply set the browsers up on one machine, install the application and run through a few functions to make sure everything was working. At that time, manual testing was the viable and effective option.

The proliferation of mobile devices and applications has dramatically altered the development landscape. For instance, today’s testing teams have six or more browsers to contend with. Agile increases the relentless rate at which new application releases and updates are generated, which must be tested against a plethora of new mobile devices continually being introduced to the market. The browser/device configurations are multiplying at an exponential rate, and complexity is reaching alarming levels.

Any organization delivering applications has a number of complex testing and delivery challenges to overcome, which are caused by the mobile multiplier effect. If left unchecked, these challenges will cause product delays, end user frustration and, worst of all, loss of revenue.

Functional testing and performance testing
Being assured that applications meet requirements for end users across devices, and ensuring that the applications perform adequately even as demand scales, is made even more difficult in an agile environment. Agile has actually made software testing a more difficult challenge as it increases the rate at which new releases and updates are generated, which, by default, increases the pace at which testing must take place.

Functional testing has been able to keep up reasonably well with agile as the tools have matured to allow manual testing to develop into automated testing. Factor in the need to perform functional tests across multiple mobile devices, as well as the need to have rigorous functional testing on all devices on which the application is supported becomes even more critical.

Performance testing has become even more important in an agile environment, because if applications don’t perform well and scale under demand, large business impacts can occur. This is often the result of the way distributed server application architectures are implemented. The use of virtual machine technology and cloud resources has, in many cases, changed application resource allocation from static to dynamic. However, allocation isn’t the only concern. What has been found in the APM world is that other root cause issues can lead even the most functionally and transactionally sound applications to degrade if problems are not found and fixed during testing.

Organizations need to work harder than ever to be sure that the application releases they generate with agile methods will perform satisfactorily under all operating conditions, including across a wide range of devices.

Prepare and plan for the mobile multiplier
Many organizations do not have an effective mobile strategy in place, and so they start out by focusing on a couple of their most common use cases. Then, as soon as they attempt to deliver real business functionality to their users, they realize they need to accommodate testing of multiple use cases based on the operating systems and device configurations that their audiences are using. In addition, the cost of testing multiple use cases grows exponentially as the complexity of the testing environment increases.

Finance departments are naturally skeptical about footing the bill for thorough testing of complex mobile environments, which requires different levels of domain knowledge and expertise, as well as increased time and manpower to validate applications (if costs are reaching excessive levels and dramatically impacting return on investment). Companies certainly do not want to spend more time and money on testing an application than they did developing it.

Delivering performance and capacity
In light of the mobile multiplier effect, performance and capacity are key areas for consideration. Application quality can be defined as performance plus capacity, and as with quality and cost, there will usually be a trade-off. If performance levels are poor and an application is ridiculously unresponsive, users will quickly seek an alternative. The competition is fierce; today’s users are fast to dismiss a brand, and their loyalty will quickly evaporate when faced with an app that’s taking an age to respond.

Capacity refers to an application’s range and scope (what is does and how extensive it is), and this has a major impact on whether or not an excellent user experience is delivered. If an application is seriously limited in scope, it will be simple to test but will also be of little use to the end user. The more developers put into an application, the more engaging and valuable it becomes for the person using it, and the more difficult and expensive it becomes to test it. The bottom line is all about prioritizing performance and capacity levels to suit your application.

Mobile and agile
Agile development, with its emphasis on working software and regular testing, needs test automation, which enables levels of code coverage and confidence impossible through other means. Not only does automated testing help catch defects earlier in the cycle, it essentially builds quality into the process.

Today’s application developers and testers need a mobile strategy with an integrated and automated approach. This allows them to effectively and efficiently develop and test apps, while taking into account the plethora of operating systems and devices, and managing the workload generated by the mobile multiplier effect.

Cloud-based performance testing will ensure capacity even in the most extreme performance scenarios. By allowing test teams to instantly deploy existing performance test scripts to cloud-based load generators, the load is created on preconfigured systems provisioned in the cloud. This eliminates the effort and cost related to extending the on-premise test infrastructure, which only the highest-load scenarios would need.

In addition, cloud-based services can provide a diagnosis of any performance related issues when they arise, giving teams the detailed diagnostics they need to pinpoint the nature and location of the problem in order to remediate quickly. Combined with an on-premise performance monitor, it’s straightforward to understand the demands on the server infrastructure in the data center, providing end-to-end transparency.

By harnessing the power of the cloud, developers can execute mobile testing quickly across multiple platforms and devices, ensuring applications work exactly as intended and generate an exceptional user experience for everyone.

The post Guest View: Mobile testing in an agile world appeared first on SD Times.

“Traditional testing takes too much time,” he said. “You have to plan ahead and calculate your steps; it removes spontaneity.”

To help developers save time and quickly run large-scale performance tests of their API, mobile app, mobile website and Web apps, BlazeMeter has introduced FollowMe, a module that provides scriptless and spontaneous performance testing.

(Related: How to use crowdsourcing)

“When you don’t need to have a script, you can choose on the fly what you want to test and just do it,” Girmonsky said. “It isn’t just about saving time, with this new technology, there are things you can do now that weren’t available for you to the past.”

FollowMe provides virtual crowdsourcing for performance testing, and can trigger more than 1 million virtual users and imitate how real users will interact with the app.

“The ability to stress your application in ways you couldn’t do in the past helps developers figure out problems quicker and avoid potential downtime,” Girmonsky said. “Traditionally you had to invest a lot of time, and today you can do it easy and fast and a bit cheaper.”

Developers will also get real-time performance analytics with FollowMe, so they can easily make changes when they need to.

“By simplifying the testing process, the usage of testing will increase and therefore the quality of products and better user experience resulting from testing will also increase,” said Girmonsky.

More information is available here.

The post BlazeMeter introduces scriptless performance testing appeared first on SD Times.