Databricks is an analytics, data engineering and AI platform that is available on all three major US-based hyperscaler clouds. But it’s also a little different on each one. On Microsoft Azure, Databricks is available as a first-party service called Azure Databricks and is therefore just as integrated into, and accessible from, the Azure portal as Microsoft’s own services, like Synapse Analytics and HDInsight. Databricks is also available as a third-party but integrated offering on Google Cloud. In both cases, Databricks is available on a pay-as-you-go (“PAYGO”) basis.
But if you have wanted to run Databricks on Amazon Web Services (AWS), which happens to be the cloud on which Databricks was first launched, you’ve probably had to provision and manage it through Databricks’ own console. That’s because up until now, Databricks has only been available on the AWS Marketplace on a multi-year contract basis. If you wanted to work on a PAYGO basis on AWS, you had to use Databricks’ own platform. That, in turn, meant separate accounts, separate budgets and a completely separate experience in working with Databricks versus other first- and third-party data and analytics tools on AWS. Suffice it to say, this created friction for customers with AWS S3-based data lakes who wanted to use Databricks’ “lakehouse” platform to query, analyze and build machine learning models on that data.
Payday for PAYGO
Today, the two companies are announcing that, from here on, Databricks will be available on AWS’ Marketplace on a PAYGO basis, enabling customers who prefer that billing model to procure, provision, pay for and manage Databricks clusters along with their other AWS assets. This will remove friction, and allow customers to use Databricks more easily alongside AWS tools like Elastic MapReduce (EMR) and Athena, on the very same data. It also opens up a variety of scenarios for using Databricks more casually, experimentally, or for limited-term — but still important — projects. Databricks believes strongly that the AWS Marketplace PAYGO option should hasten lakehouse adoption.
Databricks’ VP of Marketing, Joel Minnick, and AWS’ Director of Category Management, Mona Chadha, briefed InApps to give us a detailed understanding of the announcement. Minnick commented that the two companies have “both become big believers in the data lakehouse.” Minnick further stated that customers are being driven by “what they go and achieve with their data and, increasingly, what they want to go and achieve with AI.” Chadha agreed, saying “this is really streamlining that customer experience.” Chadha also pointed out that services companies can embed PAYGO Databricks on AWS in their own solutions, explaining that “MSPs [managed service providers], GSIs [global systems integrators] and resellers… now have another option to offer more of a holistic solution to customers…”
Come on in, the Water’s Fine
Beyond the partnership, business and billing model aspects of this announcement, what does it really mean for the analytics world? Essentially, it tells us that the data lake model of using multiple engines and technologies against the same data is a mainstream concept now, and customers don’t want any impediments to working that way. If an organization is to be data-driven, then its people should be able to use the technology of their choice to analyze their data on a discretionary or even a capricious basis. That’s how user buy-in comes about.
Such buy-in is also what makes projects successful, and is the fuel that accelerates them. In fact, Minnick ended the briefing by explaining that “the problem [Databricks and AWS] are really trying to solve for is time-to-value.” He said the goal is “just making it much, much easier and faster to find those lakehouse use cases and get them up and running as quickly as possible.”
Certainly, if an organization wants digital transformation to be more than rhetoric, it will need to have that ease of implementation, and vendors who want to be successful will need to provide it.
AWS is a sponsor of InApps.
