Microsoft Refines the Windows Subsystem for Linux – InApps 2022

Microsoft’s newest update to Windows 10, called The Creators Update, will contain Windows Subsystem for Linux, a tool that could make Windows 10 much more appealing to the increasing number of developers considering a move from Mac OS because they find the MacBook Pro underpowered for their needs.

WSL is often called ‘Bash on Windows’ because Bash is the entry point, and “a Bash-like experience” was one of the original goals, Microsoft’s Rich Turner told InApps. He’s responsible for the Windows console as well as WSL, but WSL goes far beyond having Bash as an alternative shell on Windows. Bash is just a starting point to unlock all the tools of the Linux command line.

WSL, he explained, is “a Linux-compatible environment that looks and behaves just like Linux, and allows you to run all your Linux code — your Linux build system, your GNU tools and everything else you need to run, build and test your application without having to fire up VMs.” WSL still uses the Windows kernel; it just uses it to run the system calls ELF64 Linux binaries depend on, via a pico driver with Microsoft’s clean room implementation of the Linux syscall interface.

Microsoft placed WSL in Windows largely for developers, because of the way so many open source tools and languages and libraries assume developers are using Linux. “Many of these have hard dependencies on Linux behaviors, the Linux file system layer, Linux networking socket interaction mechanisms and so on, which made some of those things struggle to work well on Windows, because Windows has a slightly different way of doing a lot of those things.”

Core languages and runtimes like node and Python and Ruby work well enough on Windows. But if you want access to the same gems, packages, libraries and modules you’d use on Linux, so you can use the exact same toolchain, you need more fundamental compatibility than just porting some of X Windows or the GNU libraries to Windows the way Cygwin and MSYS do, because that doesn’t help you when it comes to binary packages.

“A lot of Ruby gems are compiled and people take a dependency on those compiled gems and then they run into a problem on Windows. A lot of gems expect files to be in a particular location, and on Windows that looks completely different. We need the ability to load and run binaries from Linux without modification,” Turner said.

The idea isn’t to turn Windows into Linux, but to put the Linux tools that developers depend on alongside Windows tools like Visual Studio (and productivity applications like Office).

It’s also not about giving up on the idea of the Linux desktop, Dustin Kirkland, Canonical’s technical lead for Microsoft Ubuntu development told InApps. “I see it as a beautiful way to introduce the UNIX and Linux way of communicating with a computer through a command line, as a gateway to tens of thousands of open source tools. The opportunity to deliver the Linux way and open source way to literally billions of Windows users is too good to pass up.”

His own lightbulb moment came when using Visual Studio to build the Ubuntu image for WSL when he had to change one specific term across some 17 different files. Instead of hunting for an unfamiliar GUI command, he realized he could use recursive grep and sed against the project in his Documents folder in Windows, the way he would on Ubuntu.

“The two really worked beautifully together. I’m far more comfortable in vi than in any graphical editor, so being able to pop down to a vi window and create and edit files and do it natively, do ssh natively on the system, is super powerful,” he said.

Daily Driver

If you tried WSL early on and were disappointed, it’s time for another look. The version of WSL in the Windows 10 Anniversary Update was an early release to get developer feedback about what tools they needed WSL to run. “It was a snapshot into where we’re going,” Turner emphasized, noting it had obvious gaps.

“You couldn’t ping, you couldn’t look at your ifconfig to see how your network was configured. We couldn’t run Java, we couldn’t run npm because it wasn’t able to enumerate the network configuration. Those all work now, and we can run MySQL and Postgres and Apache and Nginx and node and Ruby and Java and Python, and even Core CLR works now for ASP.NET.”

In Creator’s Update, key tools like SSH and the GDB GNU debugger work more reliably. And you can even configure WSL as the target for Visual C++ for Linux in Visual Studio, so you can edit and debug visually and then compile and build in WSL.

Making those tools work has meant “adding huge numbers of newer capabilities either in new syscalls or by expanding the breadth and depth of our syscall implementations, allowing more tools and libraries to run,” he said. Adding a new syscall or a new capability to an existing syscall often fixes 20 or 30 issues, making for fast improvements.

Kirkland calls the progress outstanding. “We’ve seen the Windows kernel team filling out even more of WSL, capturing more system calls, and ensuring that anything and everything you would expect to work in Linux continues to work. We see things like Screen and Tmux and Byobu now working very well. We’re starting to see bits and pieces that provide the initialization procedures so we can start doing things in the not too distant future, like perhaps containers.”

Canonical maintains the user mode images that Windows 10 systems downloads when you first run Bash. So far those have been Ubuntu 14.04, with a new image about every three months (that’s the frequency Microsoft suggested Windows users would be comfortable with, Kirkland noted). With Creators Update, that switches to Ubuntu 16.04 (Xenial, the version released last April.)

That’s a big improvement, because it means much newer versions of commonly-used libraries, compilers and utilities, all natively packaged and one packaged app install away.

Ubuntu 16.04 will be installed by default if the Creators Update is the first time you’re using WSL on a PC. If you already have 14.04, Windows won’t update your distro. Turner says that’s because of the very strong feedback from the community that they didn’t want the update to be automatic). You can do an in-place upgrade using sudo apt dist-upgrade, if you want a clean 16.04 instance, use lxrun /uninstall /full to remove your Ubuntu instance and then reinstall it with lxrun install.

Creators Update also fixes some simple annoyances, like mouse support in the console (as well as adding 24-bit color). It also integrates the Windows and WSL environments more closely. “You can run a command on the Ubuntu system that affects the Windows system,” Kirkland explained; “so you can edit files in real time and have the file updated in both Notepad and vi, or you can launch an app from Linux that triggers an event in Windows — or vice versa.”

That integration also means you can see Linux processes in the Windows task manager. “Supporting Tmux allows you to have multiple panes and each pane is running its own Bash, so if you look in task manager you’ll see multiple instances of Bash, the main one that’s a child of init, and instances for each of your tiles,” Turner explained. “If you run MySQL, you’ll see MySQL in task manager.”

That gives you a handy way of dealing with runaway Linux processes or error-prone Bash scripts. You can just right-click on them in task manager and kill the process. Network and system monitoring tools can also see WSL processes, because they’re exposed to the Windows Management Interface and they use the Windows networking stack and the Windows firewall. Enterprises may want this so that they can use their existing security monitoring tools with WSL processes as well as Windows ones.

That also fixes a big management problem for businesses whose developers might be using dozens of VMs that can’t be monitored the way Windows tools are. “They can get developers off Hyper-V and VMware VMs which bypass most of the Windows network stack and talk directly to the network card,” Turner explained.

Sticking with the Command Line

Now that the WSL platform covers the majority of mainstream developer scenarios, the emphasis is shifting to what Turner called the more esoteric and edge cases, as well as more developer requests, and there will continue to be more updates and improvements.

One thing you shouldn’t expect WSL to do — officially — is support the Linux desktop. Users have been experimenting this, running everything from the Ubuntu desktop to Firefox. The fact that they work is purely a by-product of making WSL compatible with the developer tools it’s designed for, Turner said.

“We’ve been very clear that the reason we are building WSL is to provide an environment for developers to get their work done. There are some Linux GUI tools that used by developers but by and large, the majority of tools they want to be able to run are compilers and debuggers and build engines and so on. We are only focusing our efforts on command line tools and scenarios.”

Microsoft isn’t discouraging those experimenters, though. “It’s been so much fun for us to watch,” he said, “and we’re not doing anything to prevent it, but it’s not something we’re focusing our efforts on.”

Feature image: The Windows kernel handles Linux system calls for the Windows Subsystem for Linux using a pico process (Microsoft).