A Browser-Level Load Testing Tool For the Cloud – InApps Technology 2022

Tricentis sponsored this post.

Black Friday is fast approaching and anyone who has prepared for this retail apocalypse in the past knows it comes in a hurry.  As eCommerce teams worldwide begin preparing for this all-important sale, the Flood team is ready to alleviate your pain with new cloud load testing capabilities that can help you make this the best Black Friday ever.

Earlier this year, we announced the official release of Flood Element, a tool that can dramatically reduce scripting time for complex scenarios.  Over the course of this year, we have seen it used to test everything under the sun: from simple websites to the most complex web-based ERP systems, with tests supporting up to 100,000 users and beyond. Additionally, teams are starting to run Element tests alongside their traditional protocol-based load testing tools.

In this article, I explain how Flood Element improves on traditional load testing tools to make your life easier, with simplified scripting and maintenance. I hope this exciting new tool will reduce the number of headaches you experience this holiday season and help you unlock your app’s full potential!

How Existing Load Testing Tools Work

Most existing load testing tools like LoadRunner, JMeter and Gatling work by interacting over various protocols that are specific to each application type.A protocol is a set of rules that allows entities to communicate with each other. For example, consider languages. If you want to communicate with someone in Spanish, you must know the rules that apply to that language. It’s the same story with electronic devices: they need to establish a protocol with associated rules in order to transmit and control data flow.  Thus, these protocol-level load testing tools work more as a simulated interaction two systems would have when communicating, rather than impersonating real user behavior.

Where Do Protocol-Level Testing Tools Excel?

Testing at the protocol level means that you need to define end-user tasks within a tool to simulate the communication to the target server on a network communication level. Most of the tools either compile or interpret the specified routines and use small and efficient engines that are not real browsers. As a result, the cost of simulating a user is low. Tools like JMeter or Gatling that support the most common testing protocols (HTTP, HTTPS, LDAP, JMS and JDBC) can typically achieve upwards of 1,000 virtual users on a common load generator. However, the scripting requires deep knowledge of the application architecture to create and maintain representative scripts.

What Options Exist Besides Protocol-Level Tools?

For those that prefer not to test with protocol-level testing tools, a new class of browser-level load testing tools is emerging. Testing at the browser level allows you to define actions as a user would, by interacting with locatable objects within a web page rendered in a real browser. Examples of these tools include Selenium or Flood Element (based on Google’s Puppeteer), which are capable of managing the most common browsers Chrome or Firefox.

Are you wondering which type of load testing tool is right for you this holiday season?  It all comes down to your requirements, of course. If your requirement includes page rendering time, you will certainly prefer a browser-level tool for its real user simulation. On the other hand, if you need to test with millions of concurrent users, a protocol-level tool will provide a more reasonable cost for this large-scale simulation.

Let’s check out some of the pros and cons of these two families of tools:

At first sight, the browser-level tools seem to be the hands-down the leader for load testing.   Certainly, any entry-level or beginner testers will prefer these tools because the skills needed to get accurate results are minimal. When testing end to end business scenarios, many more advanced testers will gravitate to protocol-level tools.  Why? Because these E2E  tests require you to interact with the different layers of your application including API, web, mobile and more.  However, many teams are now finding that a mix of protocol and browser-level users can achieve the most complete testing outcome.

A few years ago, when I was working in one of the largest banks of the world, we coined a similar browser + protocol-level user option that we called  “Gomez.” Gomez is an automated browser test that was launched alongside a protocol-level load test to measure more user-centric response time. We found that this hybrid approach allowed us to generate very high levels of load at a low cost while measuring a more accurate user response time than protocol-level tools alone can provide.

Why Two Tools Are Better Than One

The response is simple: to get the best performance test with the lowest cost and effort. A protocol-level test alone would be the least expensive to execute — but it would take the longest time to create and it might not provide a realistic result. A browser-level test is faster to build and more accurate but is more expensive to execute. So, the question remains: How do we determine how many users to simulate with browser-level users vs protocol-level users in this hybrid simulation?

First, let’s look at today’s cost for each type of user.  For example, you can test with 10,000 real browsers with Flood Element (Puppeteer) for 1 hour for $700. By contrast, a protocol-level tool like JMeter or Gatling is able to run 200k threads concurrently for the same price. So for a test under 10,000 users, you should probably test with browser-level users alone in order to save the hassle of needing to write and maintain a protocol-level script.  However, for a range of 50,000 to 100,000 or more users, it becomes clear that there is a material saving to writing an additional protocol-level script to run alongside your browser-level test.

Putting It All Together

Browser-level and protocol-level testing tools each bring their own unique capabilities to the table. Ultimately, a combination of both might be your best bet this holiday season. You can experiment with browser-based tools like Element or Selenium and protocol-level tools like JMeter or Gatling to assess your own personal tastes for both.  Once you have decided on the right tools for your hybrid load test, you can head on over to Flood to sign up for a free trial. There, you will be rewarded with five free node hours you can use to execute your hybrid load test and easily assess the performance of your website before the Black Friday masses descend!.

Feature image via Pixabay.

InApps Technology is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Tricentis.