CoreOS, along with a cadre of external volunteers, is updating the open source etcd distributed key service so it can serve tens of thousands of requests per second, a major bump from the throughput offered by the version now available.
This improvement makes etcd version 3, released last week in beta, more suitable to help production-scale deployments of Kubernetes, which relies on etcd for coordinating communications across large numbers of containers, according to Brandon Philips, CoreOS chief technology officer who created etcd.
Etcd v. 3 features a redesigned storage engine, 10which uses memory about ten times more efficiently than the current version, and an updated API to accommodate more synchronization primitives, Philips said at the kickoff keynote of the CoreOS Fest, being held in Berlin this week.
The current version of etcd, Brandon said, was designed more for storing and updating configuration information for servers, and so required less throughput. Version 3 of etcd, rivals the performance, in terms of throughput, with the likes of Apache Zookeeper, a similar value store that provides data for coordinating Hadoop services.
The software is able to accommodate greater traffic thanks to its use of gRPC (Google Remote Call Procedure). It now provides incremental snapshots, which should further boost its reliability in high availability deployments. Garbage collections has been improved, eliminating the latency spikes that could occur with previous versions. Making use of off-heap storage, etcd v3 can now store millions of keys without interference from garbage collections.
Philips originally designed etcd as a simple database for reading, writing and comparing keys. As a distributed system, if one node goes down, the other nodes can reorganize themselves to continue to offer the service. Its use in Kubernetes, however, demanded a greater scalability, Philips said, in a follow-up technical session.
Etcd provides more consistent performance than Apache ZooKeeper, which incurs spikes in latency for garbage collection, CoreOS CTO Brandon Philips said.
The beta release of etcd v3 was one of a number of technical announcements the company made at the event.
The company also launched JWT Proxy, a service-to-service authentication designed especially for allowing microservices to communicate with each other. One service can authenticate directly with another service, even if the two are written in different languages. The company has also previewed software, called quayctl, for pulling appc and Docker container images using BitTorrent peer-to-peer file sharing protocol.
CoreOS is a sponsor of InApps Technology.
InApps Technology is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker.
