Implementing Stateful Messaging – InApps Technology 2022

Lightbend sponsored this post.

The definition of “cloud native” we’re using here means a friendly and optimized distributed computing environment. Key features of this type of environment are elasticity and resilience, as well as synchronous (point-to-point), asynchronous, and parallel communication. A Reactive messaging infrastructure provides loosely-coupled design, which is paramount to distributed computing. In this post, we’ll look at why messages are so important, and we’ll reveal how messaging types, Event Sourcing, and Command Query Responsibility Segregation (CQRS) can be applied to cloud native applications.

What Are Messages?

First, let’s define what a message is. Wikipedia has a great definition: “A message is a discrete unit of communication intended by the source for consumption by some recipient or group of recipients.” We will also add an additional key piece of information to this definition: messages in a distributed system must be immutable.

Now that we have a working definition, let’s look at the different types of messages we will explore:

• Command
• Event (declarative/historical)
• Query (question)

The type of message or message pattern you choose will have a direct effect on how your application behaves.

Looking at Commands (Imperative)

Commands are imperative in nature and represent a request from “somewhere” to do “something.” That “something” often results in a request to change state on “somewhere.” As a result, a command can be denied. In addition, a command, if valid, is often translated into one or more events. For example:

Commands In Scala

Commands In Java

Typically, a command follows the VerbNoun format, such as PickupGroceriesas we see above.

Looking at Events (Declarative / Historical)

Events, on the other hand, are declarative. They represent that “something” has occurred. They are historical in nature. In our example above, we see that in processing a valid PickupGroceries command, four events are generated: CarDrivenToMarket, GroceriesPickedup, CarDrivenHome and GroceriesPutAway. You will also notice that the structure of an event is the inverse of a command: NounVerb. Another thing to note about an event is that because of their declarative nature, they cannot be denied.

Events should be atomic in nature, and represent a form of state that has transpired against a domain aggregate.

Account Register Example

One of the best ways to understand Event Sourcing is to look at the canonical example of a bank account register. In a mature business model, the notion of tracking behavior is quite common. Consider, for example, a bank accounting system. A customer can make deposits, write checks, make ATM withdrawals and transfer money to another account.

In the image above, we see a typical bank account register. The account holder starts out by depositing $10,000.00 into the account. Next, they write a check for $4,000.00, make an ATM withdrawal, write another check and finally make a deposit. We persist each transaction as an independent event. To calculate the balance, the delta of the current transaction is applied to the last known value. As a result, we have a verifiable audit log that can be reconciled to ensure validity. The current balance at any point can be derived by replaying all the transactions up to that point. Additionally, we have captured the real intent of how the account holder manages their finances.

Looking at Queries (Questions)

Queries are similar to commands, in that they are a request for information. Queries do not change state, so they usually do not result in an event(s). However, they can result in events if auditing of query projections is required, or a log of all queries that are executed. Queries will often rely on synchronous or point-to-point communication (not a requirement), whereas Command/Events typically use asynchronous fire-and-forget communication.

Message-Based Abstractions — Event Sourcing

As we discussed, Event Sourcing provides a means by which we can capture the real intent of our users. In an event-sourced system, all data operations are viewed as a sequence of events that are recorded to an append-only store. This pattern can simplify tasks in complex domains by:

• Avoiding the requirement to synchronize the data model and the business domain.
• Improving performance, scalability and responsiveness.
• Providing consistency for transactional data.
• Maintaining full audit trails and history that may enable compensating actions.

For additional information, check out this great article for using Event Sourcing to overcome the complexities of distributed systems and CAP (Consistency, Availability, Partition) theorem.

Message-Based Abstractions — Command Query Responsibility Segregation (CQRS)

Command Query Responsibility Segregation (CQRS) is a pattern by which we can segregate operations that read data from operations that write data, by using separate interfaces. This pattern can:

• Maximize performance, scalability and security
• Support the evolution of the system over time through higher flexibility
• Prevent writes (update commands) from causing merge conflicts at the domain level.

This pattern implements two distinct paths: a Command-side(Write) and Query-side(Read). In most cases, these will be separate microservices running on their own JVMs.

Why Consistency Is Explicit in Message-Based Systems

Consistency is often taken for granted when designing traditional monolithic systems, as you have tightly-coupled services connected to a centralized database. These types of systems default to strong consistency since there is only one path to the data store for a given service — and that path is synchronous in nature.

However, in distributed computing this is not the case. By design, distributed systems are asynchronous, loosely-coupled, and rely on patterns such as atomic shared memory systems and distributed data stores that achieve availability and partition tolerance. Therefore, strongly-consistent systems are not distributable as a whole contiguous system, as identified by the CAP theorem.

Systems with strong consistency can be distributed. They just need to use something to coordinate their efforts, such as a distributed atomic commit or distributed consensus. Obviously, this emphasizes the consistency from CAP, whereas the availability and partitioning suffers a lot.

Consistency Models

In distributed computing, a system supports a given consistency model if operations follow specific rules as identified by the model. The model specifies a contractual agreement between the programmer and the system, wherein the system guarantees that if the rules are followed, the memory will be consistent and the results will be predictable.

Eventual Consistency

Eventual consistency is a consistency model used in distributed computing that informally guarantees that if no new updates are made to a given data item, eventually all accesses to that item will return the last updated value. Eventual consistency is a pillar in distributed systems — often under the moniker of optimistic replication — and has origins in early mobile computing projects.

A system that has achieved eventual consistency is often said to have converged, or achieved replica convergence. While stronger models like linearizability (strong consistency) are trivially eventually consistent, the converse does not hold. Eventually, consistent services are often classified as BASE (Basically, Available, Soft state, Eventual consistency) semantics, as opposed to more traditional ACID (Atomicity, Consistency, Isolation, Durability) guarantees.

Causal Consistency

Causal consistency is a stronger consistency model that ensures that the operations process in the order expected. More precisely, partial order over operations is enforced through metadata. For example, if operation A occurs before operation B, then any data center that sees operation B must-see operation A first. There are three rules that define potential causality:

1. Thread of Execution: If A and B are two operations in a single thread of execution, then A -> B if operation A happens before B.
2. Reads-From: If A is a write operation and B is a read operation that returns the value written by A, then A -> B.
3. Transitivity: For operations A, B, and C, if A -> B and B -> C, then A -> C. Therefore, the causal relationship between operations is the transitive closure of the first two rules.

Causal consistency is stronger than eventual consistency, since it ensures that these operations appear in order.

Conclusion

Message-driven architectures are well suited for stateful, cloud-native requirements, and should utilize different types of messages or message patterns (Commands, Events, Queries), as well as message-based abstractions such as Event Sourcing and CQRS.

If you would like to explore stateful applications in greater detail, download a copy of “Build Stateful Cloud Native Applications” by Jonas Bonér, creator of Akka and CTO at Lightbend — and get started on the path to running stateful services in a simple and efficient way.

Feature image via Pixabay.

At this time, InApps Technology does not allow comments directly on this website. We invite all readers who wish to discuss a story to visit us on Twitter or Facebook. We also welcome your news tips and feedback via email: [email protected].