The Challenge of Scaling WebSockets – InApps 2022

When Tim Berners-Lee outlined the first “web” of hypertext documents so he could more easily get, share and communicate whilst working at CERN in the late 1980s, it’s unlikely even he could have imagined the impact this would have on all our lives. Today we take our online experiences for granted, but it has taken a lot of great ideas and development to get to this point.

Tim Berners-Lee’s initial creations led to the establishment of HTML, URI and HTTP, the three most fundamental building blocks of the internet. As we progressed through the ’90s, HTTP evolved and was standardized, and we saw JavaScript born to embed scripting capabilities into browsers.

Towards the end of the 20th century, we saw AJAX enter the fray, suddenly allowing parts of a web page to be updated without the entire page needing to be refreshed. As we moved into the 21st century, AJAX, alongside HTTP, laid the foundation for the real-time internet, and the development of Comet also brought significant advances, allowing a web server to push data to a browser when updates were available. The challenge was that these technologies made new, “almost” real-time experiences possible but brought with it challenges around scalability, latency and resource requirements — not a sustainable model when the growth of the internet was accelerating.

In 2008, developers Michael Carter and Ian Hickson both experienced the challenge and pain of using Comet to implement something that resembled a real-time solution. As a result, they hatched a plan to introduce a new standard for modern, real-time communication on the web based on bidirectional full-duplex communication between client and server over a single connection.

Added initially to Google Chrome in 2010, and all major browsers shortly after, and standardized by RFC 6455, the “WebSocket Protocol’’ was invented. We entered the era of truly real-time internet.

What Do WebSockets Make Possible?

WebSockets can be used for many different purposes, such as streaming data between backend services such as enterprise applications or supporting multidirectional communications. We can broadly group WebSocket use cases into two distinct categories:

• Real-time updates, where the communication is unidirectional, and the server streams low-latency (and often frequent) updates to the client. For example, live sports updates, alerts, real-time dashboards, or location tracking.
• Bidirectional communication, where both the client and the server send and receive messages. Examples include chat, virtual events and virtual classrooms (the last two usually involve features like polls, quizzes and Q&As). WebSockets can also be used to underpin multiuser synchronized collaboration functionality, such as enabling multiple people to edit the same online document simultaneously.

Many of these experiences are the ones we’ve come to expect and rely on during our day-to-day use of the internet. This brings with it new challenges, such as how to deal with the sheer volume of people accessing services and the vast amount of data required to deliver them.

The Challenge of Scale

For WebSockets to work, and for every user to see one another, they must be connected to the same WebSocket server. This means that the number of active users you can support is directly related to how much hardware your server has. Modern JavaScript runtime environments are pretty good at managing concurrency, but once you reach a few hundred to a few thousand users, you’re going to need to scale your hardware vertically to keep all the users in sync.

Scaling can either be vertical — i.e., procuring the most powerful hardware you can — or, once you’ve hit the technical limitations of what hardware is capable of, horizontal, which is better in the long run but costs more. Neither is an ideal solution.

But why are WebSockets hard to scale? The main challenge is that connections to your WebSocket server need to be persistent. And even once you’ve scaled out your server nodes both vertically and horizontally, you also need to provide a solution for sharing data between the nodes. Any state needs to be stored out-of-process to ensure that all the nodes have the same view of state.

In addition to having to share state using additional technology, broadcasting to all subscribed clients becomes difficult because any given WebSocket server node knows only about the clients connected to itself. There are ways around this, such as using some form of direct connection between the cluster nodes that are handling the traffic or by using an external pub/sub mechanism. This is sometimes called “adding a backplane” to your infrastructure, but this ends up being another moving part that makes scaling WebSockets difficult.

Moving to the Cloud

One way that the whole server capacity problem can be solved is to move the traffic to the cloud. Serverless WebSockets can help organizations build, extend and deliver real-time capabilities. Businesses can have an elastic WebSocket capability without having to manage or scale WebSocket servers because they can spin up capacity as needed on their cloud provider of choice.

However, many developers choose a server-based approach because it’s the one they know. In addition, we have only recently seen the likes of Amazon Web Services introduce ways to do serverless WebSockets so there is still education needed across the industry. As more and more services rely on streaming data and the appetite for real time grows, we will likely see an increasing number of cloud providers introduce serverless WebSocket capabilities.

Making Your Implementation a Success

There is no denying that WebSockets are a critical component of the real-time web, but organizations must carefully consider their implementations. How many users will you be serving? Do you expect sudden spikes in traffic? Ask yourself these questions and think carefully about what you need; it will help you to choose the right path and ensure the digital experiences you deliver are up to scratch.

Whether to host these services on your own servers or in the cloud is a critical decision that can affect your ability to serve a growing user base effectively. But even serverless implementations require significant development time, so it is prudent to choose an effective way to deliver your services.

Ultimately, delivering real-time capabilities using WebSockets requires additional architectural and engineering complexity, especially when done at scale. Working with an expert third-party can help you build and maintain a low-latency, scalable, fault-tolerant real-time infrastructure. This approach is becoming increasingly attractive, allowing developer teams to focus on frontend functionality rather than what is going on in the background. WebSockets are a great technology, it is now up to organizations to maximize their potential in a scalable way.

InApps is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Ably, Real.

Feature image via Pixabay.