Scarf Takes Aim at Package Manager Lock-In with Scarf Gateway – InApps Technology 2022

Scarf, a startup emerging from stealth this week, wants to help open source developers measure their software’s usage and connect with their commercial end users, and they’re doing so with the launch of a public beta of Scarf Gateway.

For the majority of open source maintainers, the extent of project usage statistics can be limited to download numbers and public popularity metrics like GitHub stars. While this can serve as bragging rights, it does little to inform maintainers about how or where their project is being used, and when questions and issues come rolling in, it can be a surprise to find out that your software is being relied upon by multinational corporations and not just individual developers like yourself.

Scarf Gateway actually takes aim at this central idea, while also attempting to solve another problem faced by open source projects, that of lock-in when dealing with package managers. Scarf Gateway provides a secure, central access point for software packages independent of a host registry, so that switching from Docker Hub to GitHub Container Registry is a non-breaking change that doesn’t involve any downtime or effort from end users.

“If another provider comes down to the market, it’s not necessarily trivial to just switch over and move all your users over. It’s much more complex than that. That’s a problem that we’re really addressing. It’s essentially just a really thin redirect layer that sits in front of your registry, and allows you to use your own domain to host your containers,” explained Scarf co-founder and CEO Avi Press in an interview.

“With Scarf, the project gets to supply that URL, and effectively decouples you from the registry. You can switch registries as many times as you want. Users never have to be impacted by that change ever again. And then because Scarf is the first thing that gets hit and redirects or proxies to the ultimate registry, we can supply all that data about what’s coming through to the maintainers.”

While the idea of collecting this sort of data may sound like a potentially risky business, Press assures that personally identifying information (PII) is not collected, and that Scarf Gateway makes sure to not step afoul of regulations such as General Data Protection Regulation (GDPR), which could quickly complicate matters. Rather, Scarf Gateway collects information such as unique installs, client container runtime information, package version, course-grained location, device and operating system information, and generally any other metadata that might be useful, while making sure to discard PII.

“We’re trying to really normalize this for maintainers. It should not be a taboo thing to want to understand how your software is being used, because we’re in a world where banks and hospitals and nuclear power plants are all running on open source software, up and down their stacks,” said Press. “When critical infrastructure is sitting on top of software, where the people who write that software don’t have any idea what’s going on, that is an increasingly problematic situation. Ideally, the maintainers can find issues ahead of time, and solve them proactively, rather than have to just wait for people to come in on GitHub. It’s really crazy that the most technologically sophisticated people on the planet are using surveys to get at this right now.”

At launch, Scarf Gateway will be available to work with Docker containers, but Press says that the company is looking forward to adding other package types, such as for Java and Python, in the near future.

While the timing of the release is fortuitous, with the recent issues around Docker Hub and image pull rate limits, Press says that the feature is one that has been in the works for a little over six months now, coming as a direct result of a customer request.

For users concerned that moving their package registry information over to Scarf is simply a lock-in of a different form, Press notes that the company plans on releasing Scarf Gateway as open source once it is generally available, though he also points to these additional features around gathering information as reason to host with Scarf.

Using Scarf Gateway under its current feature set is free and will remain free, with additional paid features, such as service level agreements and integrations with business intelligence tools like Segment, available in the future.

InApps Technology is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: scarf.sh, Docker.