Update GitHub Dumps REST Calls for Facebook’s GraphQL

`

Hoping to streamline the remote querying services of its site, GitHub is moving its API (application programming interface) from REST calls to Facebook’s GraphQL API.

In the company’s recent user conference GitHub Universe, Kyle Daigle, GitHub platform engineering manager, explained that the company has experienced issues as the company scaled its REST-based services, causing a lot of pain both for developers and managers watching budgets.

The company has expanded exponentially over the past nine years since the first user committed code to GitHub. Now over 10,000 new users are committing their first code every day, according to Chris Wanstrath, CEO and cofounder of GitHub.

REST API

The REST API is responsible for over 60 percent of the requests made to the GitHub database tier, according to a blog announcing the switchover. This is partly because, by its nature, hypermedia navigation requires a client to repeatedly communicate with a server so that it can get all the information it needs.

Retrieving requested data with REST can require multiple API calls, especially on mobile applications. That’s expensive. Therefore, the design team was looking for a way to get a more granular data return.

The second problem became apparent as the company’s service scaled to meet its crush of demand, from over 6 million users having and 250,000 OAuth apps. The REST API was not supplying the functionality they needed, including assurance of type-safety for user-supplied parameters, generating client patches, and generating documentation for their code.

In addition, Daigle explained that when it became necessary to make changes to the REST API, they found it very hard to know who would be impacted, creating unwanted consequences.

The Test

A bit of investigation found a lot of tools available. They decided that a good test project would be implementing emoji reactions on comments. Daigle said, “one or two people worked one or two weeks” to create a proof of concept to be used across the tools for evaluation.

GraphQL uses object-oriented programming to target data requests and reduce calls to the server.

One of the tools they found was GraphQL, was developed in 2012 at Facebook to solve its own mobile app issues. Dan Schafer, software engineer at Facebook and co-creator of GraphQL, said the team threw out old sequential ways of retrieving data and created a new map of how they wanted to use the data at a granular level.

By defining objects in your code, you can make one request to fetch the names of your first three repositories, their total number of stars, number of forks, watchers and open issues.

The GraphQL query code:

Returned the results:

This ability to request data at a granular level frees workflows from the limitations of parsing massive JSON blobs, enabling all kinds of new possibilities.

Daigle’s team found the GraphQL documentation succinct and implementing the server to be straightforward. From the blog post:

Drawing off our experiences in supporting the REST API, we worked quickly to implement our existing services to work with GraphQL. This included setting up logging requests and reporting exceptions, OAuth and AuthZ access, rate limiting, and providing helpful error responses. We tested our schema to ensure that every part of was documented and we wrote linters to ensure that our naming structure was standardized.

Then they ran the tests. Daigle had just one word to describe their reaction: Woah.

Implementing GraphQL

As GitHub is the first external commercial user of GraphQL, Facebook worked closely with GitHub engineers to launch the product. On a go-forward basis, every GitHub new project will be built on GraphQL and the team is working on retrofitting current functionality to work with GraphQL.

Part of what is exciting about GraphQL, said Schafer, is that it’s bringing a lot of changes to GitHub. In the past, once a feature was launched, a REST API would have to be added to it, said Schafer. Now, GitHub engineers are working to move old features onto the new GraphQL API platforms.

The new Project feature (also launched at GitHub Universe), was built using the REST API to get it launched, explained Daigle, but GitHub engineers built out GraphQL objects in order to make the future conversion easier. The GraphQL took several hours to build, he said, but the REST API took “significantly longer.” They are excited to be moving to this new protocol, he stated.

“I don’t want this to come off as a condemnation of REST,” said Daigle, “but we have different needs now that GraphQL can solve for us.”

Want To Get Involved?

GitHub Engineers are looking for community members who want to get involved. Early access program is now available, and they are looking for feedback from integrators. This is a beta version, Daigle cautioned, and there are likely to be bugs.

The new GitHub Community (also launched at GitHub Universe) will be used to track feedback.

There are also two new open source projects created by the GitHub engineering team working to implement GraphQL:

• github/graphql-client, a client that can be integrated into Rails for rendering GraphQL-backed views.

• github/github-graphql-rails-example, a small app built with Rails that demonstrates how you might interact with our GraphQL schema.

Woah, indeed.

Feature image: Kyle Daigle, Platform Engineering Manager at GitHub, explaining the team’s reaction to their first GraphQL test. Taken by T.C. Currie.