WebAssembly Aims to Eliminate the File System – InApps 2022

WebAssembly continues to disrupt the IT ecosystem. The latest plans for the WebAssembly System Interface (WASI) aims to, over time, do away with the file system, and possibly even the operating system as well, according to a Qcon talk held in May by Lin Clark, a senior principal engineer at the edge services platform Fastly, as well as a co-founder of the WASI Bytecode Alliance.

Originally designed for the browser, WebAssembly (WASM) is a secure build-once-run-anywhere runtime that can ingest code from your favorite programming language (assuming your favorite language is C/C++, C# or Rust) and emit superfast bytecode targeted for the browser. “WebAssembly is a compilation target, and not a programming language,” explained Liam Randall, co-founder of the WasmCloud framework, in a session at KubeCon+CloudNativeCon EU, a virtual conference held last month. “As an embedded engine, WebAssembly’s key value proposition is around speed, efficient size and security make it an attractive choice for executing code from third parties.”

WASI itself was conceived as a way to move WebAssembly out of the browser. To do this, and in order for code to run fast, it needs to be as closely tailored to the specific processor architecture as much as possible. This required creating a low-level abstraction over various ISAs architectures. A very modular interface, WASI provides a small sets of “narrowly-scoped” APIs tailored to the major operating system. And thanks to its modular architecture, more platforms, and higher-level cross-platform functionalities, can be added in later.

Ultimately, WASI developers want to move away from a core concept most operating systems are reliant upon, the file system, with the argument that it would improve performance.

The idea of “file systems” in CompSci is a metaphor really, in that “files” are used to store sets of data, so that the data can be shared across programs.

“The file achieved its central position in system interfaces during a very different time in software development,” Clark noted. These days, however, as we plug together more components to build applications, the file format brings along various deployments and security issues: “The way that these file systems are used is basically like having one big pile of global shared mutable state,” she said.

For developers who wish to use WASI on specific hardware can set a software flag, which will tell the WASI runtime to run the bytecode for a particular platform, allowing the program to get access to a file system, or to a virtual file system.

But WASI is fostering other, faster and more portable options, Clark explained in her talk.

One is the WASI File System interface, in which WASI offers its own virtual file system to the developer. In this approach, the developer would open and read files using the same calls, but the files themselves would reside entirely within WASM module.

“This means that we don’t have that global shared mutable state problem that the file system introduces. Even though these look like files in the source code, under the hood, they would use WASI I/O types, things like streams and arrays, that would give them that full portability,” Clark explained.

The downside to this approach is that size of the modules would balloon, given the virtual file system each WebAssembly module would be carrying.

For full portability and efficiency, a third API, known as WASI I/O, could be used. In this case, the program would not call files at all, but more specifically the pointers to the I/O types themselves, such as data arrays or strings. “And with this, the developer no longer even thinks in terms of files. It’s all just these pure I/O types,” she explained.

“We’re hoping to see the rest of the ecosystem gradually migrate to only using WASI I/O,” Clark said.

There will be lots of optimizations that can be enjoyed with this architecture, once WASI is aware of the I/O types that it is handling, she explained. For instance,  it could vastly streamline the communications between two separate containers on the same pod working on a common task. Typically data that gets passed among containers must be serialized, moved into memory, then moved again into kernel memory so it can be passed to the sidecar, which duplicates this process yet once again to move the data out onto the network (and where, presumably at the receiving end, all these processes must be done again in reverse).

In a WASI setup, this copying process could be reduced to mere nanoseconds, Clark promised. The sidebar and the main program can be consolidated into a single process, where calls to the network socket are replaced by much speedier synchronous function calls within the program itself. “There are no intermediate serialization and deserialization steps here, and no heavyweight calls to the kernel or inter-process communication,” Clark said.

A spokesperson for QCon told us that the videos from this latest QCon will be available online by the end of the year, both on the InfoQ site and on YouTube. So definitely check out this talk when it is posted. The next QCON event, to be held both virtually and live in New York, will run from Nov. 1-5.

WebAssembly Secures Kubernetes

WebAssembly is already making its presence known in the Kubernetes community, and not just with Krustlet, a Kubernetes Kubelet to run WASM workloads instead of containers.

At KubeCon+CloudNativeCon EU, also held back in May, Flavio Castelli, engineering manager at open source enterprise software company SUSE, and SUSE Software Architect Rafael Fernández López, discussed how to use WASM to script secure container workloads, through KubeWarden, a Kubernetes policy engine and admission controller they created that can interpret policies as web assembly modules.

Kubernetes best practices policy is to write user permission policies-as-code, as declarative syntax that can be ingested by machines. The trouble with this policy-as-code approach is that it requires a lot of time from the developer to do correctly, either by populating extensive YAML files, or even by learning a new language to express a complicated set of rules.

KubeWarden, currently in alpha release, provides a way to write the policies in Rust or Golang, which then can be compiled into modules that are executed by Kubernetes in an isolated WASM sandbox by the KubeWarden Policy Server. The modules, which can be reusable, can be stored in a container registry or a policy hub and pulled by the server.

KubeWarden uses Kubernetes Custom Resource Definitions (CRDs) and a controller to manage the policies, relieving the dev of burden of learning the low-level webhook validation technique and other bits of K8s administrivia.

Dr. CSS

Also featured at QCon was a fascinating talk on how the British Broadcasting System (BBC) is testing WASM as a way to standardize the handling of its content across platforms.

Tim Pearce, a BBC research and development software engineer, noted in his presentation how the BBC is seeking to make the full range of its audio, video and print content “more personalized, interactive and immersive,” a tall task given that BBC shows are already run on a variety of mobile, desktop and Smart TV platforms. The media company has settled on a common format “Object-Based Media” which can be run from the company’s “Single Source Player,”

“Object-based media allows the content programs to change according to the requirements of each individual audience member. The objects refer to different assets they used to make a piece of content that could be large objects, such as the audio or video that’s used to make a drama, or small objects, like the individual frame or video, a caption, or other on-screen graphics,” he explained. “By breaking a piece of media down into separate objects that attach meaning to them, and describing how they can be rearranged, a program can change to reflect the context of an individual viewer.

Right now this is done by generating multiple artifacts for different platforms, such as for mobile and web browsers. Ideally though, WASM bytecode could be used across all these services.

The media company is testing Emscripten, a WebAssembly toolchain, to compile interactive media assets written in C or C++, into WebAssembly bytecode. This provides a way to offer an identical user experience on the Web that the company now offers through its desktop and mobile apps. Plus the media experience would identical across all different browsers, something difficult to achieve natively because of how the different browsers implement CSS — while bringing near-native performance to the Web.

“This approach has delivered a smooth, consistent user experience across different web browsers,” Pearce explained.