“Microservices,” along with “serverless” and “DevOps,” are terms that are hard to define because they describe patterns and architectures as much as specific technologies. Yet, survey data points to a strong correlation between many of these subjects. Last month, we reported that 81 percent of people using microservices in production adhere to DevOps or continuous delivery practices.
A survey conducted by Lightbend in mid-2017 focusing on “fast data” showed another strong link between microservices adoption and use of streaming technologies. Note that fast data is a term meant to indicate that speed, not size (i.e., big data) is the frame of reference. While only 34 percent of all respondents used microservices in production, the figure rose to 50 percent among those that use advanced fast data use cases in production. Advanced use cases include real-time personalization, machine learning, and IoT pipelines, and excludes activities like Extract, Transform and Load (ETL) and traditional statistical analysis.
A survey of Kafka users conducted by Confluent supports the aforementioned finding as 50 percent said that microservices is one of their Kafka use cases.
Over the last few years, Kafka Streams, Apache Spark Streaming and Akka Streams (supported by Lightbend) have gained traction because they allow developers to work with data in-flight. In many ways, they have become the glue that connects data pipelines. While much of the data is handled in a modular approach, not all is dealt with in real time. Fifty-six percent of respondents said that at least half of their workloads included a real-time component.
When asked how fast they actually need their data, developers said most use cases do not need to be accessed within a second or a minute. Thirty-one percent required that capability to integrated data streams and 32 percent needed that speed for artificial intelligence and/or machine learning. For many of the other use cases, batch processing, which can happen hourly or even daily fits their needs.
Feature image by Pine Watt via Unsplash.
