When it comes to Docker containers, according to the usual thinking, that is Linux containers (LXC), not much to do with Windows, at best, is to run Docker containers in the Windows Linux virtual machine.

However, since Windows Server 2016, there are Windows-native Docker containers, which are no longer just for Linux, and Docker containers can now run Windows systems, with each Windows container sharing the host Windows kernel (–isolation= process,) or use a Windows kernel in a highly optimized virtual machine (–isolation=hyperv).

We say that since Windows Server 2016, including now Windows Server 2019, Windows Server 2022, and Windows 10 and 11 for desktop systems, with the help of Docker Desktop, you can also run Windows containers.

The original installation of Docker Desktop on Windows desktop can be used to run Linux containers, so it can be seen that on Windows desktop (e.g. Windows 7, 10, 11) two types of containers can be run

  • Linux containers: Each container runs a Linux instance, with resources isolated by the cgcroups namespace. By default, Docker Desktop’s LinuxEngine is used.
  • Windows containers: The containers run Windows instances in process isolation mode, where the containers share the host’s Windows kernel, and Hyper-V isolation mode, where the containers use the kernel of a highly optimized virtual machine. You need to enable the Hyper-V feature of Windows and switch Docker Desktop to use WindowsEngine.

By adding Windows containers to the traditional LXC concept, Docker’s architecture looks like this

Docker’s architecture

Knowledge about Windows Docker containers can be found in Microsoft’s official documentation Containers on Windows Documentation.

The most basic images for Windows are the following four, listed in order of weight from heaviest to lightest.

  1. Windows: contains the full set of Windows APIs and system services (but not Server-related), such as Windows 10 image 20H2.
  2. Windows Server: Contains the full set of Windows APIs and system services, allowing the use of most service features, such as GPU acceleration if you need it.
  3. Windows Server Core: Includes only a subset of Windows Server APIs that are primarily used to support the .NET framework. Most services are also included (e.g. Fax service is not).
  4. Nano Server: The lightest Windows Server image, containing only some services that support the .NET Core API.

We can build our own image by choosing one of the above base images, or a quicker way is to choose an image where someone else has added the packages we need. For example, to run Python on Windows you can choose 3.10.2-windowsservercore-ltsc2022; https://mcr.microsoft.com/dotnet/framework/sdk:4.8 for .net sdk4.8.

For a rough comparison of the image file sizes (which vary greatly by version), here are the image sizes listed with docker images

  1. mcr.microsoft.com/windows:20H2 size 16.2G
  2. mcr.microsoft.com/windows/server:ltsc2022 size 11.4 G
  3. mcr.microsoft.com/windows/servercore:ltsc2022 size 4.96 G, mcr.microsoft.com/windows/servercore:ltsc2016 but 12 G
  4. mcr.microsoft.com/windows/nanoserver:ltsc2022 is 295 M in size, like an embedded system

It is only possible to pull/run/build Windows images on Windows platforms and requires that the current Windows platform be compatible with the image version, unlike Linux containers which do not have any requirements for the current platform. We will learn more about this later.

To install Docker on Windows Server (including the current Windows 2016, 2019, 2022), execute the following command in PowerShell.

1

Install-Module -Name DockerMsftProvider -Repository PSGallery -Force

If prompted to install NuGet, select Y. If the above command requires TLS, execute the following command first.

1

[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::Tls12;

Finally, install Docker.

1

Install-Package -Name docker -ProviderName DockerMsftProvider

To install Docker Desktop under Windows 10 or 11.

The next example is Windows 10 (also for Windows 11), to see the difference between the two container types (Linux/Windows), for which a clean system was installed, and the following is the hardware and software environment of the test machine.

  1. CPU: Intel i7-7700 @3.60GHz
  2. Memory: 48 GB
  3. Windows 10 Pro, 21H1, OS Version: 10.0.19043
  4. Docker Desktop 4.5.1 (74721), and its requirements for WSL 2
  5. Additional Windows features such as Containers, Hyper-V, Windows Hypervisor Platform are not yet enabled, but Virtual Machine Platform is found to be enabled by default

Linux containers on Windows

After just installing Windows 10 Pro + Docker Desktop, by default, LinuxEngine is used, so only Linux containers are supported, run docker version to see the server/client version.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

C:\Users\yanbin>docker version
Client:
 Cloud integration: v1.0.22
 Version:           20.10.12
 API version:       1.41
 Go version:        go1.16.12
 Git commit:        e91ed57
 Built:             Mon Dec 13 11:44:07 2021
 OS/Arch:           windows/amd64
 Context:           default
 Experimental:      true
 
Server: Docker Desktop 4.5.1 (74721)
 Engine:
  Version:          20.10.12
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.12
  Git commit:       459d0df
  Built:            Mon Dec 13 11:43:56 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.12
  GitCommit:        7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

We see Server Engine OS/Arch: linux/amd64 above. The linux/amd64 of Server means that Docker can only run Linux containers.

At this point we can run the following command.

1
2

docker pull busybox
docker run busybox echo hello world!

It is also possible to build Linux images, such as Dockerfile content.

1
2

FROM ubuntu:20.04
CMD echo hello world!

Then execute the build and test.

1
2

docker build -t test .
docker run test

It is illegal to try to pull or run a Windows container at this point.

1
2
3

C:\Users\yanbin> docker pull mcr.microsoft.com/windows/nanoserver:20H2
20H2: Pulling from windows/nanoserver
no matching manifest for linux/amd64 in the manifest list entries

Of course, run/build Windows images don’t work either, because you have to pull the Windows image before run/build. The reason is that it doesn’t match the current linux/amd64.

Exploring Docker Desktop LinuxEngine for Windows

1
2
3
4

c:\Users\yabqi>docker context ls
NAME                TYPE                DESCRIPTION                               DOCKER ENDPOINT                             KUBERNETES ENDPOINT   ORCHESTRATOR
default *           moby                Current DOCKER_HOST based configuration   npipe:////./pipe/docker_engine                                    swarm
desktop-linux       moby                                                          npipe:////./pipe/dockerDesktopLinuxEngine

Go to the host of Docker.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

c:\Users\yanbin>docker run --net=host --ipc=host --uts=host --pid=host -it --security-opt=seccomp=unconfined --privileged --rm -v /:/host alpine chroot /host
root@docker-desktop:/# uname -a
Linux docker-desktop 5.10.16.3-microsoft-standard-WSL2 #1 SMP Fri Apr 2 22:23:49 UTC 2021 x86_64 GNU/Linux
root@docker-desktop:/# free
               total        used        free      shared  buff/cache   available
Mem:        39336428      586016    37845116      377180      905296    37951412
Swap:       10485760           0    10485760
root@docker-desktop:/# cat /proc/cpuinfo | grep processor
processor       : 0
processor       : 1
processor       : 2
processor       : 3
processor       : 4
processor       : 5
processor       : 6
processor       : 7

With 48G of physical memory, the Docker host can use up to 39G of memory and all 8 cores of the CPU. This is different from the default allocation of 2G RAM and half the total number of CPU cores in Docker Desktop under Mac OS X.

Note that Hyper-V is not enabled on Windows 10 so far, and the command to view the physical memory is 48G, the Docker host can use up to 39G of memory and all 8 cores of the CPU. This is different from the default allocation of 2G of memory and half the total number of CPU cores for Docker Desktop on Mac OS X.

Note that Hyper-V is not enabled on Windows 10 so far, check with the command.

1
2
3
4
5
6
7
8
9

PS C:\Windows\system32> Get-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V
 
 
FeatureName      : Microsoft-Hyper-V
DisplayName      : Hyper-V Platform
Description      : Provides the services that you can use to create and manage virtual machines and their resources.
RestartRequired  : Possible
State            : Disabled
CustomProperties :

Switching to use Windows containers

To use Windows containers in Windows 10, first switch to Windows containers for Docker Desktop by tapping Docker Desktop on the system bar and going to its context menu.

context menu

Or use the DockerCli command to switch

1

"c:\Program Files\Docker\Docker\DockerCli.exe" -SwitchWindowsEngine

If you want to switch back to LinuxEngine from WindowsEngine, the parameter is -SwitchLinuxEngine.

Or use the -SwitchDaemon parameter to switch back and forth between LinuxEngine and WindowsEngine.

1

c:\Program Files\Docker\Docker\DockerCli.exe" -SwitchDaemon

However, trying to switch to WindowsEngine, either through the UI or by command, now brings up an error container.

error

The reason for this is that Hyper-V is not turned on, which can be done in the Turn Windows features on or off interface.

Turn Windows features on or off interface

Or use the PowerShell command prompted earlier

1

Enable-WindowsOptionalFeature -Online -FeatureName $("Microsoft-Hyper-V", "Containers") -All

Windows will restart automatically after Hyper-V is turned on. After that, you can successfully switch Docker Desktop to Window containers mode by performing the previous step, and then check the docker version.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


C:\Users\yanbin>docker version
Client:
 Cloud integration: v1.0.22
 Version:           20.10.12
 API version:       1.41
 Go version:        go1.16.12
 Git commit:        e91ed57
 Built:             Mon Dec 13 11:44:07 2021
 OS/Arch:           windows/amd64
 Context:           default
 Experimental:      true
 
Server: Docker Desktop 4.5.1 (74721)
 Engine:
  Version:          20.10.12
  API version:      1.41 (minimum version 1.24)
  Go version:       go1.16.12
  Git commit:       459d0df
  Built:            Mon Dec 13 11:42:13 2021
  OS/Arch:          windows/amd64
  Experimental:     false

The version information is displayed more simply, and we see that Server / OS/Arch has become windows/amd64.

If you try to run the Linux container at this point it won’t work either.

1
2
3
4
5

C:\Users\yanbin>docker run busybox
Unable to find image 'busybox:latest' locally
latest: Pulling from library/busybox
docker: no matching manifest for windows/amd64 10.0.19043 in the manifest list entries.
See 'docker run --help'.

Build and run Windows images/containers

1
2

C:\Users\yanbin>docker run mcr.microsoft.com/windows:20H2 cmd /c "echo hello world!"
hello world!

Build the Windows image with the following Dockerfile contents.

1
2

FROM mcr.microsoft.com/windows/servercore:ltsc2016
CMD echo hello world!

1
2
3

C:\Users\yanbin>docker build -t test .
C:\Users\yanbin>docker run test
hello world!

Docker Desktop under WindowsEngine

Switch to WindowsEngine and look at the docker context.

1
2
3
4

C:\Users\yanbin>docker context ls
NAME                TYPE                DESCRIPTION                               DOCKER ENDPOINT                               KUBERNETES ENDPOINT   ORCHESTRATOR
default *           moby                Current DOCKER_HOST based configuration   npipe:////./pipe/docker_engine                                      swarm
desktop-windows     moby                                                          npipe:////./pipe/dockerDesktopWindowsEngine

A desktop-windows appears instead of the previous desktop-linux.

As for the host of the Windows container, it is related to the isolation mode, there are two types, process isolation and Hyper-V isolation.

process isolation

It is similar to the cgroups naming isolation of Linux containers, all containers share the current system kernel, the container is actually a process under the current system. The current system is the host of the Windows container, which requires the running container to be of the same version as the current operating system, otherwise the kernel cannot be shared, and the following error will occur.

1
2

C:\Users\yanbin>docker run -it --isolation=process mcr.microsoft.com/windows/servercore:ltsc2016 ping localhost -t
docker: Error response from daemon: hcsshim::CreateComputeSystem 2cc0e4e8ce8c0f7e8c82bef76df4a9dbf0672d6ffe29776814891ce27c6bb3fe: The container operating system does not match the host operating system.

Similar to the Linux container, processes started with –isolation=process can be listed with the Get-Process command.

Hyper-V isolation

The default value of -isolation in the docker runtime is hyperv, where the container runs in a highly optimized virtual machine, and the container processes do not appear on the current system, but are wrapped in individual vmwp virtual machine processes. That is, that virtual machine is the host of the Windows container. But where can I see the supposedly highly optimized virtual machine? Not in Hyper-V Manager, not with the Get-VM command, too highly optimized.

Learned about the two isolation modes for Windows containers, which might explain why docker run -p 80:8080 ... The default hyperv isolation is 80 to the highly optimized virtual machine, not the current operating system, you should try with -isolation=process, just make sure the current system version and the container system version to be highly consistent. —– verified that even with –isolation=process -p 80:8080, the ports are still not mapped out.

And the speed of starting the container may be related to the choice of isolation mode, and then highly optimized virtual machine should also be slower than the process isolation way to start the container, because process isolation is essentially a local process. —– actual test seems to be not much different, anyway, are much slower than the Linux container, at least an order of magnitude difference.

On a particular Windows operating system, not all Windows images support process isolation, Hyper-V isolation is supported, refer to the Windows container version compatibility list Windows container version compatibility.

Windows system and container version compatibility

Knowing this can guide us in choosing what version of Windows to build images on and what version of Windows to run containers on. This is not a problem at all for Linux containers, because basically a Linux system that can run docker commands is free to build/run images of any Linux distribution. When you first encounter Windows containers, it is easy to get frustrated when you choose a Windows machine that can run docker commands as WindowsEngine, and you want to pull, run or build an image, but you still have the Linux container mindset in your head.

For example, pull the image of mcr.microsoft.com/windows/servercore:ltsc2022 on Windows 2016.

1
2
3
4
5
6

PS C:\docker> docker pull mcr.microsoft.com/windows/servercore:ltsc2022
ltsc2022: Pulling from windows/servercore
8f616e6e9eec: Extracting [==================================================>] 1.252 GB/1.252 GB
898469748ff6: Download complete
failed to register layer: re-exec error: exit status 1: output: ProcessUtilityVMImage C:\ProgramData\docker\windowsfilter\ce66a282a87a379aca443a594025eee342160907305a3f4323f2baaa38d89937\Util
ityVM: The system cannot find the path specified.

But pull mcr.microsoft.com/windows/servercore:ltsc2016 is fine. docker run/build based on incompatible image versions is also the same problem, because rub/build needs to be pulled first.

1
2

PS C:\> docker run mcr.microsoft.com/windows/servercore:ltsc2016 cmd /c "echo hello"
docker: Error response from daemon: hcsshim::CreateComputeSystem a9c5b39283e525ca8cbf688f95d7aaabfdd1eb7dd21914d9e587144a455125ca: The container operating system does not match the host operating system.

Therefore, a clear understanding of Windows system and container version compatibility is not likely to make us frustrated and even a bit frantic when we come into contact with Windows containers.

For Windows systems that correspond to Windows image versions, and whether Hyper-V or process isolation is supported, please refer to this list Windows container version compatibility. Basically, newer versions are compatible with older versions, e.g. Windows Server 2019 can run Windows Server 2019 and Windows Server 2016 under Windows Server 2019, and Windows Server 2022 can run all versions from Windows 2016 to Windows 2022. However, if process isolation is supported, the platform must be the same as the container version, e.g. Windows Server 2019 can only support Windows Server 2019 if the Windows container is run in process isolation.

Likewise, follow this compatibility table when pulling or building with docker. Build with a higher OS platform for broader compatibility, but use –isolation=process for the benefit of running containers with the exact same version.

This strict matching of Windows image versions to Windows hosts doesn’t defeat the purpose of running Linux containers, we can run other Linux containers in a Linux host without even caring what kernel version or distribution it is. It’s a bit like when Windows dabbled with Java and came up with Visual J++, which directly undermined Java’s claim to write once and run everywhere.

AWS support for Windows containers

I’ve been using Linux container services in ECS before, because I had to specify the image when defining the Task, and at that time I was not in the dark about Windows containers, so I always thought ECS didn’t support Windows containers at all. Only recently I noticed that AWS has provided a lot of optimized Windows Server ECS AMI to run docker Windows containers, for example.

1
2
3
4
5
6
7
8

Windows_Server-2016-English-Core-Containers-*
Windows_Server-2016-English-Full-ECS_Optimized-*
Windows_Server-2019-English-Core-Containers
Windows_Server-2019-English-Full-ECS_Optimized-*
Windows_Server-2022-English-Core-Containers-*
Windows_Server-2022-English-Full-ECS_Optimized-*
Windows_Server-20H2-English-Core-Containers
Windows_Server-20H2-Core-ECS_Optimized-*

etc.

The version of Windows can be obtained with the ver , systeminfo commands.

Attempting to use docker run --isolation=hyperv on the selected Windows_Server-2016-English-Core-Containers-2016 prompts that there is no hypervisor.

1
2
3

PS C:\docker> docker run --isolation=hyperv mcr.microsoft.com/windows/servercore:ltsc2016 cmd /c "echo hello"
C:\Program Files\Docker\docker.exe: Error response from daemon: container fba5ff713c0e9a4fc439747c6792251855c06dfcb1649c3e58d72280b9d62a23 encountered an error during CreateContainer: failure
in a Windows system call: No hypervisor is present on this system. (0xc0351000) extra info.......................

Although ECS supports Windows containers, Windows images are very large, starting at 10G, which is a beast compared to Linux images of about 100-200M, which can seriously affect the speed of building, pushing, and pulling images. And it’s also slower to start a Windows container after the image is downloaded. So for Windows applications you might want to run past the ECS and let the ELB connect directly to the EC2 Target Group.

Boot speed test

It is not yet possible to test the startup speed in different isolation modes for the same image, because the --isolation=hyperv isolation level cannot be used on Windows Server.

Windows containers in different isolation modes

Test the speed of booting Windows containers on Windows Server 2016 with process isolation and Hyper-V isolation respectively.

1
2

PS C:\docker> Measure-Command {docker run --isolation=process mcr.microsoft.com/windows/servercore:ltsc2016 cmd /c "echo hello" | Out-Host} | findStr TotalSeconds
TotalSeconds      : 6.1539651

The average time taken is 6.15 seconds.

This is not comparable to starting a Linux container under Linux.

1
2
3
4
5

$ time docker run ubuntu:20.04 echo hello
 
real    0m0.361s
user    0m0.019s
sys 0m0.005s

The time to boot a ubuntu:20.04 under Windows 10 is.

1
2

PS C:\Windows\system32> Measure-Command {docker run ubuntu:20.04 echo hello | Out-Host} | findStr TotalSeconds
TotalSeconds      : 1.7179686

Summary

The Windows container is not only big, but the Windows container is also slow to start.

here is still a problem with -p port mapping that does not start the corresponding port on the machine where the docker command is executed.

So we still support Linux where we can use it.

Port mapping problem (solved)

Running Windows Server Docker container

1

docker run --name aspnet_sample --rm -it -p 8080:80 mcr.microsoft.com/dotnet/framework/samples:aspnetapp

As usual with Linux containers, first use docker ps to verify that the port mapping is set (only Name and Ports are shown below)

1
2

PS C:\Windows\system32> docker ps --format "{{.Names}}: {{.Ports}}"
aspnet_sample: 0.0.0.0:8080->80/tcp

The port mapping is fine, from 8080 on the host to 80 on the container

First check if port 8080 is open on the host with a Linux-like netstat -na|grep 8080

1
2

PS C:\Windows\system32> netstat -na|findstr 8080
PS C:\Windows\system32>

Found nothing, then added telnet double authentication

1
2

PS C:\Windows\system32> telnet localhost 8080
Connecting To localhost...Could not open connection to the host, on port 8080: Connect failed

At this point, check the IP and port number of the container

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

PS C:\Windows\system32> docker exec aspnet_sample ipconfig
 
 
Ethernet adapter vEthernet (Container NIC 91439e0f):
 
    Connection-specific DNS Suffix  . : ec2.internal
    Link-local IPv6 Address . . . . . : fe80::e878:9772:931:e2cd%19
    IPv4 Address. . . . . . . . . . . : 172.25.8.94
    Subnet Mask . . . . . . . . . . . : 255.255.240.0
    Default Gateway . . . . . . . . . : 172.25.0.1
 
PS C:\Windows\system32> docker exec  aspnet_sample netstat -na|findstr 80
TCP 0.0.0.0:80 0.0.0.0:0 LISTENING
TCP [::]:80 [::]:0 LISTENING

The port 80 started in the container is fine, so accessing http://172.25.8.94:80 through the container IP is fine, but some places introduce access to http://172.25.8.94:8080 to access (this path does not work). And one reason is to use container IP in earlier versions, which earlier is not clear.

1

After the application starts, navigate to http://localhost:8080 in your web browser. You need to navigate to the application via IP address instead of localhost for earlier Windows versions

see https://hub.docker.com/_/microsoft-dotnet-framework-samples/?tab=description

The other day I’ve been troubled by this illusion, seeing port 8080 from netstat -na and telnet localhost 8080, and instinctively thinking that the Windows Server container port mapping is not working, and that further port mapping with it for ECS will not work. In fact, this is just a display bug, see Open issue: [Windows] Port binding is not visible with ’netstat’ but works correctly. #30300

In this case, the port mapping is actually successful, but you can’t access it with localhost:8080, use ipconfig to find the host IP, and then access the host IP:8080 is able to pass.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

PS C:\Windows\system32> ipconfig
 
........
 
Ethernet adapter Ethernet:
 
   Connection-specific DNS Suffix  . : ec2.internal
   Link-local IPv6 Address . . . . . : fe80::64ea:22d5:4620:79a8%4
   IPv4 Address. . . . . . . . . . . : 10.255.60.241
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . : 10.255.60.1

Then

1

telnet 10.255.60.241 8080

There is no problem, that is, accessing http://10.255.60.241:8080 from the remote is possible, so naturally it is not a problem as a port mapping for ECS.

ECS

netstat -na is an illusion, localhost:8080 does not work either, nor does 127.0.0.1:8080, which means that docker -p 8080:80 when starting a Windows container only listens on port 8080 of the NIC, and is only visible to netstat -na, and even PowerShell command Get-NetTCPConnection are not visible. But one more -p 8080:80 will expose the problem.

1
2

PS C:\Windows\system32> docker run -p 8080:80 mcr.microsoft.com/dotnet/framework/samples:aspnetapp
C:\Program Files\Docker\docker.exe: Error response from daemon: failed to create endpoint dazzling_clarke on network nat: HNS failed with error : The object already exists.

Port 8080 is occupied, proving that 8080 is bound somewhere we can’t see.

Since then, using Windows containers in ECS is worth practicing further, and the following task is about how to control the size of the Windows Docker image and put it in the Docker Registry with a fast network.