Data is everywhere, being used not only by individuals but by companies that want to harness data’s power to increase sales and retain customers long term.
Built as an open source project to manage streaming data in real time, Apache Kafka powers not only LinkedIn, where it originated, but also companies such as Uber and many others that need to have real-time access to information about their products and services. Some of the creators of Kafka started a company that offers users an enterprise-level version of Kafka, called Confluent.
“There is a change happening in the way companies use data, we wanted to make Confluent practical for [enterprise] companies to use,” said Confluent CEO and Co-Founder Jay Kreps. “Confluent helps organizations to cooperate around their data.”
On Tuesday, Confluent released Confluent 2.0, which boasts the newly-updated Apache Kafka 0.9 core.
Bridging the Gap Between Past and Future
Described by Kreps as, “What you would get if you crossed a distributed file system with a messaging system,” Kafka 0.9 offers not only persistent data, replication, and fault-tolerance, but has introduced a new Java client and a C client to better serve those working in these environments.
Kafka offers real-time data streams which are able to be viewed and acted upon as they happen, with scalability at the core. As Kafka offers a language-independent protocol, developers can use any language to tap into Kafka on their own stack, along with adding community-built connectors and frameworks.
“We’ve improved our Java client and C clients while also updating our connector area to better integrate with a client’s ecosystem,” said Kreps. The updated connector platform, known as Kafka Connect, is now fault-tolerant, which is critical for enterprise users operating at scale. Allowing for central management of connectors and frameworks, Kafka Connect reduces common pain points often encountered when setting up a data streaming service.
Security is a key feature of Confluent 2.0, with Kafka 0.9 adding security features which will help teams run Confluent at scale. These features include authenticating users requesting access to data, setting permission levels on data streams, and the ability to maintain security levels across a data cluster, said Kreps. An additional feature offered in Confluent 2.0 is the ability to throttle reads to ensure that services are not interrupted by processes using an excess of resources.
Delivering Data to the Enterprise
As more organizations begin to realize the potential impact real-time data streaming can have in their business, Confluent 2.0 has seen a number of unique use cases.
Far from the traditional clients of web development teams, Confluent has seen a rise in adoption by financial organizations and IoT platforms alike. As more machines and devices are connected to the internet, the data generated and collected from these devices must not only be stored securely but acted upon to generate continued sales.
With support for Hadoop, Elastic Search, and file system data set to arrive in a future update to Kafka, Confluent will continue to provide businesses the ability to work with Kafka in whatever language they choose, while being able to use the database tools they prefer when working at scale.
With the addition of a Java Client, a C client, and Kafka Connect, Confluent 2.0 offers businesses not only the ability to see their data streams in real time, but to utilize the information provided. Making data easier to use at scale is something which can not only benefit developers and enterprises, but the customers using products to enrich their lives by connecting to the Internet.
The company also announced the inaugural Kafka Summit, which will take place in San Francisco on April 26th, 2016.
TNS managing editor Joab Jackson contributed to this story.
Feature Image: Confluent.
