The etcd key-value store, an essential component for the Google Kubernetes container orchestrator, may be the next project that the Cloud Native Computing Foundation (CNCF) takes under its wing.
Interim CNCF executive director Chris Aniszczyk mentioned the etcd consideration during a talk at the Linux Foundation’s Linux Collaboration Summit, held this week near Lake Tahoe, California.
“Etcd is next line,” Aniszczyk promised. The technical committee is currently mulling over accepting the technology and CoreOS, the open source technology’s current steward, seems amiable to the idea. CNCF’s technical oversight committee was formed earlier this month, and Kubernetes was quickly ushered in as the first project.
“Technologies such as ZooKeeper, etcd and Consul are getting traction in this space. It is a common pattern, and that is why we believe there should be a distributed key-value store inside of CNCF,” said Alexis Richardson, who is head of the CNCF technical oversight committee, during a TNS podcast recording.
“And let’s not beat around the bush. Kubernetes depends on etcd,” Richardson added. “If Kubernetes is in the Foundation, etcd should be in the Foundation too.”
The etcd technology is a distributed key-value store. Operational data stored on etcd is propagated across multiple servers, so if one server goes off-line, one of the others in the etcd cluster can quickly answer queries. “The whole idea is to have a key value store that is resilient to machine failure,” Philips said.
Etcd was designed to store “really important cluster configuration information,” Philips said. Resilient key value stores are necessary for running cloud-native apps. Information for scheduling, DNS lookups, load balancing, or service discovery is so important for a cluster “that you don’t want one individual to fail and for that information not to be readable or writable.”
Philips based etcd on a Google white paper describing the Google’s own, never-publicly-released, key-value store called Chubby. etcd was subsequently integrated into Kubernetes, as well as into a number of other container-based technologies developed by Docker, CoreOS, and others. The etcd project now has over 400 contributors and several maintainers.
“Etcd is a project of quality. It not a brand new thing that is poorly written. It has gone through a number of iterations and has proven itself in battle,” Richardson said.
CNCF was formed last year to help reduce the technical confusion in the cloud container space, by helping to harmonize the different approaches being developed. Members include container-oriented startups such as CoreOS and Docker as well as larger IT firms such as IBM, Red Hat, and Cisco.
Gifting etcd to a third-party such as CNCF would be advantageous in a number of ways, Philips noted. An organization would be a neutral shepherd, which would help relieve any qualms that the software is controlled by a single biased party. “It is a little bit less off-putting for competitors to participate,” Philips said, noting that the Kubernetes copyright was transferred from Google to CNCF when that project was accepted.
An organization could also handle all the routine management duties such as enforcing copyright and registering domain names.
Another benefit CNCF would bring etcd is the 10,000 node cluster it has on-hand, donated by Intel, that could be used to test new versions of the software. Having this resource on-hand “will lower the cost of testing considerably,” Richardson said.
The etcd development team could certainly use this cluster to help finish new API they are working on to help etcd, already pretty speedy, to work even more quickly, by using CPU and memory more efficiently. A new API is required so that etcd can use the Google-developed gRPC (Google Remote Procedure Call), a new protocol based on HTTP/2 and Protocol Buffers, Philips said. Kubernetes uses the rather verbose JSON for exchanging key information, so this new protocol will hasten deliveries considerably.
Cisco, CoreOS, Docker, IBM, Intel, and Red Hat are sponsors of InApps.
InApps is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker.
