Phenquestions
Microsoft recently announced that they will soon be shipping a Linux Kernel that's integrated right into Windows 10. This will allow developers to leverage the Windows 10 platform when developing applications for Linux. In fact, this is the next step in the evolution of Windows Subsystem for Linux (WSL). Let's review WSL version 1 before getting into the knitty-gritty of Linux kernel intergation and what it means.
If you want to get started with WSL (v1) here is a guide for that.
1. WSL version1
The Windows Subsystem for Linux should really be called a Linux subsystem for Windows. It offers a driver (a subsystem) for the Windows OS, that translates Linux system calls into native Windows 10 system calls that the NT kernel understands.
This creates a somewhat believable illusion for Linux binaries that they are, in fact, running on top of a Linux kernel! This works well enough that not only can you run simple 64-bit binaries compiled for Linux, but you can run an entire Userland (a.k.a a Linux distribution like Debian or Ubuntu) on top of WSL v1. So when you install Ubuntu from Microsoft store, it just fetches the Ubuntu userland binaries that Canonical ships for the Microsoft's WSL v1 enviroment.
However, WSL v1 is still far from perfect. Certain semantics that a Linux system would expect are totally unacceptable on Windows. For example, you can't change the file name of an open file on Windows, but you can do that on Linux.
Other obvious examples include WSL v1's poor filesystem performance and its inability to run Docker.
2. WSL v2 the Linux Kernel
This new version of WSL solves all of these problems by including the Linux kernel running in the Hyper-V hypervisor. Throughout their announcement and demos, Craig Loewen and his colleague from Microsoft stressed on the point that, moving forward, Microsoft is going to invest hugely in virtualization technology.
Using a Linux Kernel solves all the semantic issues that can't be solved using just a system call translation layer. This Linux kernel is going to be very light-weight and will be maintained by Microsoft with all the changes to the Linux Kernel made open source. In their demo, they showed how you can just migrate the existing WSL v1 apps to run unmodified into a WSL v2 environment. The file system performance has improved by 3-4 times and the whole system feels a lot more responsive.
3. How does it work?
So basically, we have a Linux VM with a lot of userland apps, e.g, OpenSUSE or Debian or Ubuntu userland. You get more than one distro on your Windows machine, with a single Linux VM, so that's neat. But it also implies that every time you open your WSL Ubuntu terminal, you are booting a complete VM! That's going to take a while, right?
Actually, no. Linux kernel is actually small enough and lightweight enough that Microsoft was able to get insane boot times with it (~ 1second). They achieved this by removing all the bootloader code and since it is a VM, they directly load the Linux kernel into its address space in the memory and set up a few states of the VM that the kernel expects. This is the same mechanism that powers another new Windows 10 feature that is Windows Sandbox.
It has a very small memory footprint, it loads up in a second and the native ext4 filesystem format allows it to deliver a very smooth user experience. The VM only runs when you need it.
Moreover, it is not completely isolated like a traditional VM. You can very easily interact with the rest of the Windows system, including files in your Windows drives using WSL v2. It is not an isolated VM but an integrated part of Windows 10. How does it achieve that?
4. Long live the Plan 9
With WSL v1, accessing files and directories on your Windows guests was trivial. Your Linux userland is just an app on Windows, so it can read and write files originally belonging to the native OS pretty easily.
With WSL v2, you have a VM running with its virtual harddisk (formatted with ext4, of course) and if we want a similar experience like that of WSL v1, we need some extra mechanisms. Enter Plan 9.
Plan 9, also written as, 9P is an OS originally developed at Bell Labs. While it is unlikely that you will ever find it running in production, it still lives on as various other operating systems adopt interesting ideas that are incubated in it, including the 9P Protocol for a file server.
5. File systems
WSL v2 will have a 9P protocol server running on your Windows 10 host, and a 9P client running inside your WSL apps. This allows you to access Windows 10 files natively inside your WSL environment. The C: drive will be mounted at /mnt/c, just like WSL v1, and every file inside the Windows 10 host can be reached by the WSL environment.
The converse will also be true. There will be a 9P protocol server running in your Linux environment with its corresponding client on Windows 10 host. This will allow users to access their Linux (ext4) file system from the Windows 10 environment. Allowing you to edit your source code or config files using your favorite IDE installed on Windows, listing all the directories using Windows 10 file explorer, and a lot more. Essentially, you will be running your favorite Linux distros with the Windows 10 UI.
6. Docker
In their announcement, Microsoft also hinted that this new environment will also be used by Docker to ship their future Docker for Windows apps. Since there's a Linux kernel, running Docker on top of it, is going to be quite easy. In the demo, they ran Docker on top of Ubuntu running WSL v2 and it worked as if it is running on a native Ubuntu installation.
To give a very superficial example, if you have worked with Dockerfiles on a Windows system, you must have noticed the security issues that are caused because of the lack of Unix like file permissions. That will no longer be an issue. Docker on Windows already uses a custom VM to provide Linux containers, presumably, it will now use WSL v2 to use the Linux Kernel that microsoft ships.
Conclusion
Overall, I am very impressed with what Microsoft is doing to provide a haven for Linux developers who also want to use Windows. Hopefully, over the long run, it will encourage a lot of cross-pollination between the two different ecosystems.
If you are running Windows 10 Home, Pro or enterprise edition you can get a taste of WSL v2 by opting in for the Preview builds of Windows 10. Here's a guide on how to do that.
