How We Automated Dev Releases to Maven Central – InApps 2022

Bringing DevSecOps best practices and tools into open source communities can be challenging. It can be difficult to understand what tools to implement, where to publicize them and how to get your community involved. By leveraging GitHub secrets using Terraform and Vault, an open source community can not only improve their open source developer experience but lessen the burden on its infrastructure team as well. 

Through the use of tools such as Terraform Cloud from HashiCorp, Aqua Security’s Trivy, and GitHub Actions, the infrastructure and developer experience teams at Camunda combined forces to build a CI/CD release tool that allows developers to automate their releases to Maven Central, while also ensuring that projects with critical security vulnerabilities were informed of failing tests in real time. In this article, we’ll share tips, lessons learned, and dive into how to use these DevSecOps best practices to empower and secure your open source community projects.

Focusing on Automation and Security

Automating a CI/CD release workflow is something that not only benefits open source community maintainers but allows for new contributors to an open source project to contribute in a meaningful way.

When working with automation in the CI/CD ecosystem, particularly in open source, security is at the heart of any project. At times, organizations and community maintainers may be hesitant to adopt a new workflow if it doesn’t have security features or policies built into it. There are a variety of security tools and improvements that can be added to existing CI/CD workflows in GitHub Actions, GitLab CI, and Jenkins. 

The team at Camunda chose Aqua Security’s Trivy to implement in its GitHub Action; though at the time of implementation, they faced a challenge where GitHub Actions couldn’t run concurrent actions. This led to the team pair programming together on a solution that allowed them to still utilize Trivy, which involved implementing the tool via a Bash script and returning the results of the scan via a Sarif file. Any project utilizing the automated release GitHub Action whose project had a security vulnerability would not be able to release their project to Maven Central automatically if it was found to have a high or critical vulnerability.

What Terraform Brings to Open Source DevSecOps

CI/CD workflows need secrets in order to deploy artifacts to destinations like the Sonatype-managed Apache Maven Central or Docker Hub.

The need to efficiently, transparently and yet securely manage these secrets, especially in larger organizations, usually leads to the following questions:

• What if a credential needs to be rotated?
• How to efficiently distribute secrets?
• How to ensure a new project is onboarded quickly and efficiently?

The distribution aspect can be partially addressed by GitHub, which allows assigning secrets to an organization and allows inheriting them to either all or select projects. This leaves us with the task of efficiently maintaining this list of repositories, as well as updating the actual secret values whenever there’s a change.

HashiCorp Terraform is a configuration management tool that is well known in the operations, DevOps and site reliability engineering (SRE) community. The team at Camunda chose Terraform to mirror secrets from Vault, a “single source of truth” secret store, into the respective destinations.

The configuration management code for this is maintained in a single Git repository.

Thanks to the automation provided by Terraform Cloud, onboarding new projects or adding new secrets became a simple task:

The project name is simply added to an existing list in a Terraform code file, and the changes are reviewed and merged. Terraform Cloud picks up the change and, using the official GitHub provider from HashiCorp and the github_actions_secret resource, the change is immediately reflected in the GitHub organization’s configuration. This process ensures that secrets cannot be tampered with and changes have to pass peer review before being applied.

The same techniques can be employed to manage app installations for repositories, as well as create and manage many aspects of GitHub Repositories.

Best Practices to Empower Your Community

Security looks different to everyone. What works for one large open source project may not work for a smaller community focusing on open source extensions to a product or platform. Encourage your community to come together to collaborate on and improve their existing automation tooling by offering to pair program with them, or by having GitHub issue templates available that allow them to quickly open a pull request to fix a bug or request a new feature. Another approach could be encouraging them to build a feature they suggest to improve your project’s existing CI/CD workflow and working with them to see that through. Encouraging security awareness is also key. Tell your community why you chose the tools you did, what they do, and why they matter.

Additional resources:

InApps is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker, Camunda.

Feature image via Pixabay.