Filebase Migrates Data from S3 to Decentralized Storage – InApps Technology 2022

One of the challenges that exist for all development environments is data availability — whether it’s on a storage array in your data center or stored in cloud object storage like Amazon S3. If connectivity between that storage and the user is severed in some way, it’s no longer accessible. Building out the infrastructure and application layer to avoid data inaccessibility becomes expensive, especially when you factor in copying it all to geographically separate zones and structuring in fail-safes (in case one of those zones experiences an outage).

In cloud deployments, we’re seeing more multicloud solutions emerging to address this, but this may be an area where Web3 can shine if the complexity of configuration can be eliminated. Filebase, a layer 2 decentralized storage company, is showing some early promise at becoming one of the key solutions for onboarding data to Web3 without the complexity. InApps Technology interviewed the Filebase founders back in May 2021 and they’ve made some interesting progress in the intervening months.

What Is Filebase?

That original interview does an excellent job of explaining what Filebase does, so I won’t spend a lot of time redefining their role as a storage provider. In a nutshell, Filebase is a centralized entity for decentralized storage — meaning that you upload your data to Filebase and they do the heavy lifting of distributing it to one or more decentralized storage solutions. They further simplify this by providing an S3-compatible API, so that you can take data from S3 and easily migrate to decentralized storage.

Rather than relying on a single decentralized storage provider, they provide options. It’s early days in decentralized storage and as Filebase COO Zac Cohen put it in an interview, “from an interoperability standpoint, it’s a little bit early to pick a winner in this space and every single blockchain is very different.”

One additional point to note here is that Filebase doesn’t require any cryptocurrency staking in order to leverage decentralized storage. They have obfuscated this away by holding currency as part of their business model and abstracting this component of decentralization away from the Filebase user.

Objects May Be Smaller Than They Appear

One of the complications of using decentralized storage directly is the minimum object size. Cloud object storage like S3 or Azure Blob Storage support minimum block sizes of 64k all the way down to zero-byte files for S3. For decentralized storage providers, the minimums are substantially larger. Filebase currently interoperates with Sia, Skynet (which leverages the Sia network), and Storj. Some example minimum block sizes for popular decentralized storage networks are:

To deal with these minimums, you effectively need to make a smaller file look like it meets the minimum by file packing.

Filebase created a proprietary file packing engine that is isolated on a per customer, per bucket, and per network basis.

Cohen explains, “File packing is a problem with Web3 that no one is aware of and it is one of the reasons no one is using decentralized storage. Today, Filecoin has 30 petabytes of data usage on 15 exabytes of capacity, which is larger than Wasabi, Backblaze and Digital Ocean combined. File packing is also the reason why OpenSea is still using Amazon.”

Here again, Filebase is looking at making decentralized storage adoption easier by abstracting away complexity. Filebase created a proprietary file packing engine that is isolated on a per customer, per bucket, and per-network basis, which Cohen says is, “a similar abstraction layer to the way Coinbase abstracts away the purchase of a cryptocurrency by just clicking a buy button. It’s very difficult to solve these issues at the layer 1 level, or they would have done it already.”

Increased Redundancy at Reduced Cost

When you upload a file to a bucket created via Filebase, you get the equivalent of three geographically regionally distributed buckets on Amazon, without paying the transfer fees to copy your data to three separate locations.

According to Cohen, 1TB of data uploaded to S3 runs about $115 per month as of this writing, with the price running closer to $200 by the time you’ve copied that same data to two additional geographically discrete locations. Filebase provides the same level of redundancy for about $6 per month leveraging decentralized storage.

When I asked Cohen why the cloud providers aren’t offering a competing solution, he replied, “They make lots of money off of inter-bucket regional transfer. I’m trying to convince people not to have seven copies of their data. Just put one copy on decentralized storage and you truncate all of that work.

For now, Filebase also does not charge retention fees or API fees, which further trims the storage budget.

Migrating Data into Filebase

Filebase has been compatible with S3 via an API since our original article about them in May 2021. They just announced a partnership with Flexify.IO, which is already known for their storage virtualization and migration solutions. This further simplifies the process of migrating data from Amazon S3, Azure Blob Storage, Backblaze B2, Wasabi, and Google Cloud Platform to decentralized storage via Filebase. This has the advantage of allowing dApp developers to start building from a place of familiarity within their favorite cloud ecosystem, and migrating to decentralized storage when it makes sense.

For its part in the process, Flexify.IO charges $0.03 per gigabyte to migrate to Flexify from most cloud storage options.

Is the Future of Storage Decentralized?

It remains to be seen what the future holds for storage. Most enterprise technology departments are still beholden to the laws and governance of the countries they operate in, which is unlikely to change as long as there are countries with laws.

However, I do see a ton of potential for decentralized storage to eliminate some of the outages we see when individual regions of various cloud providers experience downtime. And while cost savings alone isn’t likely to move anyone away from S3 and cloud competitors, Filebase is definitely on to something here that may speed Web3 adoption and facilitate some very practical applications of decentralized data.