How MongoDB Thrives in Chaos – InApps 2022

MongoDB sponsored this post.

If you see MongoDB on the sponsor list of Chaos Conf, Oct. 6-8, your first thought might be, “Why is MongoDB sponsoring a conference about chaos engineering?”

The answer is simple. Having built the world’s most popular distributed database, we know a thing or two about building systems that operate well in highly-contested, networked environments.

If you’re building highly scalable, distributed systems, then you need a database that works well in these environments — and we’ve got one.

Chaos engineering is a high-effort, high-cost undertaking. That doesn’t mean an environment with a single web server and a single database. It’s only worthwhile when you’re working with large clusters of servers running hordes of services and replicated databases, ideally across multiple hosting providers or availability regions.

In an environment like this, where services are spinning up and down to handle your requirements, redundancy is fundamental. Because of either your intentional efforts to induce chaos in the network, or just simply because, at this scale, chaos happens, you really need a database that’s distributed by default and self-healing.

MongoDB is designed to run with multiple copies of your data stored across multiple servers. In the case of server or network failure — provided enough servers are available — the cluster will elect a new primary and continue running. In cases when too few servers are available, the cluster will go into a read-only state until the network problem is resolved and the servers can again achieve quorum.

In the event of a network partition, MongoDB keeps your data safe until the cluster is automatically re-combined

MongoDB clusters can be run across multiple availability zones, or even hosting platforms, for extra redundancy. If you’ve ever had nightmares of losing a whole availability zone and having your service stay available, MongoDB will be able to help with that. In fact, if you want all of this to be taken care of for you, MongoDB’s hosted database solution, MongoDB Atlas, supports multiple availability zones without any effort on your part — just select the option.

Proven in Difficult Network Conditions

We’re all-in on running complex network tests as part of our release processes — so much so that we built our own tool, Evergreen, to do just that.

Evergreen’s public dashboard shows the results of MongoDB integration tests

In the case of network partitions or node failure, MongoDB prioritizes consistency over availability. The service supports automatically retryable writes, increasing the likelihood of availability in situations where the cluster is temporarily unavailable — such as during an election within the cluster — without having to manually add this feature to your application code.

As part of every release, we run Jepsen tests that induce hard-to-reproduce errors and evaluate data correctness and safety in the face of extreme failure scenarios — including simultaneous network partitions, drifting systems clocks, and repeated node crashes. The design is such that in every circumstance, when a network partition is over the cluster comes back together without any loss of data.

MongoDB is a broad product. To ensure the correct behavior of our database against requirements, we’ve built a JavaScript fuzzing framework that takes our existing suite of tests and heuristically mutates them, creating chaotic inputs to reveal bugs in untested code paths.

MongoDB Atlas even includes a feature to force a node failover, so that you can ensure your services do their part in the case of availability issues.

MongoDB Atlas allows you to explicitly test your software against primary failover

Despite the fact that MongoDB has always been distributed by design, the latest release of MongoDB 4.4 introduced new features that ensure that this core feature now works better than ever. Mirrored reads mean that, in the event of a failover, secondary servers are ready to go with pre-warmed caches. Resumable initial sync ensures that new servers added to the cluster can robustly handle intermittent network errors while getting up to speed with the rest of the cluster. Streaming topology changes ensure that when a new primary is elected after a node failure or maintenance event, cluster topology changes are now streamed back to the drivers in real-time. As a result, clients can react immediately to cluster state changes, switching open connections as needed.

Among many new features, MongoDB 4.4 includes several resiliency features

Designed to be Distributed

Pervasive monitoring is essential in chaotic environments. In the case of MongoDB, instances, clusters, and even MongoDB Atlas, can be easily monitored at a low level. The database gives you change streams — direct, streaming access to the operations being conducted against your data. MongoDB Atlas even provides serverless functions built on this platform, to allow developers to build and deploy JavaScript triggers based on this very feature.

MongoDB’s schema-less design suits agile, microservice architectures, so that your services’ data models can be changed to suit changing requirements — without the risk, difficulty or cost of large database schema migrations. Of course, it also supports schema validation if that’s something you require.

And finally, with MongoDB Realm database’s sync feature, you can build mobile or other edge applications that can deal with unreliable networks, continuing to stay available and store data locally until the next time they’re online, when data is again synchronized with the server.

So if you need a distributed database that can handle parts of the cluster being made unavailable, can host your data in nodes around the World, and can be monitored at the finest level of granularity, you should definitely consider MongoDB for your next project.

In short, MongoDB thrives in chaos — and that’s why we’ve chosen to be a part of Chaos Conf.

Feature image via Pixabay.

At this time, InApps does not allow comments directly on this website. We invite all readers who wish to discuss a story to visit us on Twitter or Facebook. We also welcome your news tips and feedback via email: [email protected].