Amazon Web Services (AWS) sponsored this post.
Though some people cast cloud as open source’s kryptonite, the opposite is almost certainly true. If we look back at some of the most celebrated open source projects of the past decade, it’s hard to escape the conclusion that cloud has nurtured and accelerated open source adoption. From MySQL to PhoneGap to [name your favorite project], the cloud has helped to increase open source’s utility and adoption.
Not that all the impact flows one way. As Isaac Schlueter, the founder of npm Inc. (the company that created the world’s largest software registry, npm) put it in an interview, “I suspect that open source had a bigger effect on cloud than the other way around.”
A perfect symbiosis, perhaps? Let’s take a look.
Taking the ‘War’ Out of Software
From our vantage point in 2020, when cloud services are largely a given, it’s easy to forget the pre-cloud “duct tape and Band-Aids” era of software, as Brian Leroux described it. Leroux should know. While today he’s the co-founder of Begin, previously Leroux helped to launch the PhoneGap open source project in 2009, as an early employee at Nitobi Software. For those like Leroux who lived through this pre-cloud era, it’s perhaps easier to appreciate just how big an impact the cloud has had on open source.
Why? Because back then software infrastructure was painful to scale and operate.
Though PhoneGap had been available to download and use, a year after the project’s launch (2010) Nitobi was still earning much of its revenue by training people on “How to PhoneGap,” as Leroux explained in an email interview. “We had a classroom full of people from the same company running Windows laptops, expecting to build iPhone apps, which is and was impossible. Needless to say, it was a nightmare and people got mad.”
After that training, Leroux went to a café and built the first version of PhoneGap Build, a cloud service that does the heavy lifting of compiling software for mobile devices, removing the need for developers to constantly maintain native SDKs. “We ironically could not build a cross-platform mobile solution without a plethora of compilers and host operating systems,” he says. “By moving the proprietary bits into the cloud, we could seamlessly build for all platforms from any development environment that had a web browser.” He says the cross-platform experience was just too hard without the cloud. A month later at JSConf 2010, Nitobi announced PhoneGap Build and “had something ridiculous like 80,000 signups in the first month. The rest is history.”
So that’s one way cloud has helped: hiding some of the complexity of software. But the benefits don’t end there.
Easy Scaling
For enterprises hoping to embrace the latest and greatest open source software, there was also the difficulty associated with running the underlying infrastructure. Again, we may not remember the “duct tape and Band-Aids” era, but it was a real crimp on open source adoption. In a 2008 blog post, Drupal founder Dries Buytaert called out the difficulties inherent in scaling application infrastructure:
“It is not always easy to scale Drupal — not because Drupal sucks, but simply because scaling the LAMP stack (including Drupal) takes no small amount of skill. You need to buy the right hardware, install load balancers, setup MySQL servers in master-slave mode, setup static file servers, setup web servers, get PHP working with an opcode cacher, tie in a distributed memory object caching system like memcached, integrate with a content delivery network, watch security advisories for every component in your system and configure and tune the hell out of everything.”
“Either you can do all of the above yourself,” Buytaert said, “or you outsource it to a company that knows how to do this for you.” One of the hardest challenges at that time was to scale relational databases like MySQL, something that he hoped the still-nascent AWS (or another vendor) might remedy, even as he lamented, “I don’t see strong signals about it becoming fundamentally easier for mere mortals to scale MySQL.”
And yet it did. Roughly a year-and-a-half after Buytaert penned his post, AWS launched Amazon Relational Database Service (Amazon RDS) for MySQL. Suddenly the arcane magic required to scale the “M” in the LAMP stack was covered. At the same time, a variety of companies that depended heavily on MySQL were contributing back, hardening the database and extending its functionality.
More Simple, More Adoption
Nor was MySQL alone in this. According to Craig Kerstiens, an active PostgreSQL contributor and postgresweekly.com curator, “PostgreSQL definitely saw some adoption growth as it became available on the cloud.” While the data to prove this is limited, DB-Engines database trends show a spike in PostgreSQL popularity in 2014, perhaps coinciding with the launch of Amazon RDS for PostgreSQL in November 2013.
How did cloud help? “In general [the] Postgre[SQL] community has always been short on DBAs and [it’s] less simple to just get up and running. A button click of managed cloud made adoption much greater,” Kerstiens said. By facilitating adoption of PostgreSQL, Kerstiens concludes, more users piled onto PostgreSQL. Importantly, they didn’t need to be a DBA or consult with one in order to embrace the power of PostgreSQL.
Now play this out across hundreds of thousands of other open source projects, and you get a sense for just how much open source has benefited from the cloud. This was not coincidental. Amazon, earliest to the cloud in 2006, had years of experience running a variety of open source projects at scale. Small wonder, then, that open source worked so well in the cloud. As AWS vice president Matt Wilson says, “[T]he cloud was built for running it!”
Taking Open Source Mainstream
There’s another, less technical reason that cloud computing helped enterprises embrace open source: it integrated support for open source software into a variety of cloud services.
According to a 2019 Red Hat survey of enterprise IT leaders, one of the biggest barriers to using open source today is a lack of enterprise-grade support. I personally worked for a variety of open source companies, starting in 2000, and a lack of support for open source software was an oft-cited reason companies couldn’t switch from supported, proprietary products, a fact called out in a variety of enterprise surveys, government research and academic studies.
As cloud companies like AWS began offering hosted and/or managed open source software services, they de-risked open source by “baking” support for the software into the service. More than just a support contract, however, cloud services like Amazon Relational Database Service (Amazon RDS) helped enterprises keep up with the rapid pace of open source innovation. Instead of having to download an open source project, fiddle with the knobs to make it work, and assume any associated legal liability, enterprises could just run the cloud service without any bother. As good as open source was before, cloud made it better.
But what about Schlueter’s suggestion that as much as the cloud shaped open source, open source shaped the cloud? I’ve dug into that in a follow-up post: Why Cloud Is Impossible without Open Source.
Feature image via Pixabay.
