The Chunnel Problem – InApps 2022

The Channel Tunnel, or Chunnel, is an engineering marvel that connects England with France. Trains, and cars that ride on the trains, run/drive on the left-hand side in England and on the right-hand side in France. Somewhere in the middle of the Strait of Dover, the handedness changes, seamlessly. This bridging of gaps between two different perspectives needs to occur often in IT systems and is sometimes done well, like the Chunnel, and sometimes not so well. In this post, we’ll explore the bridging of the gaps between frontend and backend systems.

Frontends — your mobile app, your website, etc. — want to deal with data and concepts that make sense to their users. For example, they may want to provide the delivery status of a customer’s order or a customer’s updated shopping cart on an eCommerce app. Backend systems, on the other hand, typically expose APIs that reflect the shape of the data on the backend. For example, they store and process data that makes sense in the context of the business and the technology — normalized tables, REST interfaces, and so on.

The two are like the two sides of the Chunnel. And increasingly, the frontends want to deal with data in the form of GraphQL. So queries that the apps make using GraphQL have to get connected with data and actions that require SQL and REST interfaces. So the more specific Chunnel problem is translating GraphQL into its subparts from SQL and REST. For the purpose of this blog, we will focus on REST.

Which Direction Do You Travel in?

For a minute, assume that the English end of the Chunnel represents the frontend, and the backend systems are the French end.

If you start with the English side, you might have:

But a weather API at the backend might take lat, lon as opposed to city, might return a weather forecast for the next 10 days, might produce fields that say temp as opposed to temperature, might encode description in a two-letter code and so on. So the call to the backend and the response from the backend must be made conformant to the frontend (English) view.

However, if you start on the French side, you might take the same JSON response and create a GraphQL API that looks like:

In this way, you get very high fidelity to the backend, but you also get all the details the backend produces. Any autogeneration is forced to generate names of types and queries.

As you can see, neither the frontend-first (English) view nor the backend-first (French) view is perfect. The former eliminates a lot of useful information and requires manual or semi-manual mapping. The latter floods the frontend developer with a lot of interfaces, oftentimes with complicated, auto-generated names. What you want is a meet-in-the-middle approach.

Other Core Problems in the Chunnel

While we will not go into details, there are other issues to be addressed:

• There may be no way to meet in the middle. Perhaps the two sides are fundamentally irreconcilable. For example, a frontend API that returns all orders for a product to facilitate recalls might require a backend API that is the reverse of “all products of an order,” and such an API might not exist.
• The tracks might be really slow. While this could happen for a number of reasons, a common reason is that GraphQL allows recursive paths through enterprise data. So, often, the backends that support it end up being inundated with requests. (A GraphQL request might generate 500 calls for the different backends!)
• The gauges on the two sides might be different. (Am I carrying the analogy of Chunnel too far?) This is a fundamental gap. GraphQL is strongly typed, but often the underlying REST is weakly typed. I like the analogy of Golang/Java as strongly typed, but JavaScript is weakly typed. Neither is good nor bad; they are just different.
• Road signs in and out could be missing. In the REST world, a machine-readable description of what exists is available via OpenAPI specs. However, in our analysis, only one in 100 APIs had completely correct OpenAPI specs. This leads to guessing on the part of someone building the connectivity between frontend and backends. GraphQL, on the other hand, is completely introspectable, unless it has been deliberately turned off.
• The pagination model is common in GraphQL, but is left more as an exercise to the developer in REST.
• The filter and selection model is ad hoc in GraphQL; it is equally ad hoc in REST, but the two are almost always different.
• Versioning is central to REST constructs, while GraphQL takes a strong opinion on avoiding versioning.
• AuthN/AuthZ is sufficient in REST, but it is insufficient in GraphQL.

Summary

Fundamentally, there are no frontends without backends. However, the language, perspective, and implementations of the two sides are completely different. The Chunnel can and must be built. But it requires a good understanding of the two sides and intelligent “meet-in-the-middle” approaches. We at StepZen are building a set of capabilities to help build that Chunnel.

Photo by Carrie Johnson from Pexels.