Docker usb device windows

Hardware developers have missed out on the benefits of Docker and similar localized container solutions for one big reason: USB. Today we’re seeing how devices can start to reliably connect from the host system to the container.

I have a career-long passion (borderline obsession) with developer tools. I want them to be great and search for ways to make them better for every developer. All steps in the program / test / debug loop require access to devices via USB. For some time, Docker has supported USB device access via --device on Linux, but left Windows and macOS users in a lurch. As of Docker Desktop 4.35.0, it is now possible to access USB devices on Windows and macOS as well! It’s early days and caveats abound, but we can genuinely say Docker is starting to finally become a useful tool for hardware developers.

Why USB in Docker Matters

Working with USB devices inside containers adds a bit of complexity compared to native tools. They add a variable in the process and are yet another tool to manage. But there’s value in using containers for development: namely reproducibility and environment isolation. It’s also worth noting that USB device access isn’t only relevant to hardware devs; anyone using a USB-based peripheral like a midi controller also know the struggle.

On Linux, Docker can take advantage of native OS features to allow containerized processes to directly interact with USB devices. However, on Windows and macOS, Docker runs within a virtual machine (VM), meaning it doesn’t have direct access to the host’s hardware in the same way Linux does.

USB Limitations for macOS and Windows Users

For years, Windows and macOS users have faced a clunky workaround:

• Running Docker inside a Linux VM
• Passing USB devices through to that VM
• Passing them into Docker containers.

While this approach works, it is resource-intensive, slow, and far from seamless. Many developers voiced frustration over this in GitHub issues, such as this long-running thread that discusses the challenges and demands for native USB passthrough support.

In reality, most hardware developers don’t end up leveraging Docker for local development because of the lack of USB access, only relegating containers to CI/CD and test automation.

A Solution: USB/IP

USB/IP is an open-source protocol that is part of the Linux kernel that allows USB devices to be shared over a network. By using a USB/IP server on the host system, USB devices can be exposed to clients (such as remote servers or containers) over the network. The client, running inside a Docker container, can access and interact with the device as though it were directly connected to the container.

In theory, enabling USB/IP for Docker would allow USB devices to be accessed from within containers, bypassing the need for complex VM setups. This would simplify the experience, offering access to USB devices without the overhead of virtualization.

Making It Happen: Collaborating with Docker

While USB/IP provided the perfect solution for Docker’s USB problem, Docker hadn’t yet implemented USB/IP as part of its official features.

In February of this year, I reached out to the Docker team to see if it would be possible to integrate USB/IP into Docker Desktop. I got connected to Piotr Stankiewicz, a senior software engineer at Docker, to discuss the possibility of adding support for USB/IP. Piotr was immediately interested, and after a few discussions, we began collaborating on requirements.

Finally, after several months of work by Piotr and testing by me, Docker Desktop 4.35.0 was released, featuring support for USB/IP.

Key Concepts Before We Begin

Before diving into the steps for enabling USB device access, it’s important to understand the basic configuration. The key idea is that USB devices are passed through from the host system into a shared context within Docker. This means that every container on your machine could theoretically access the USB device. While this is great for development, be cautious on untrusted machines, as it could expose your USB devices to potentially malicious containers.

Setting Up a Persistent Device Manager Container

One of the best ways to manage this is by setting up a lightweight “device manager” container, which I’ll refer to as devmgr. This container will act as a persistent context for accessing USB devices, so you don’t have to repeat the setup every time you want to develop code in a new container. By keeping devmgr running, you can ensure that the USB device is always available to other containers as needed.

I’ve created a simple image (source) and pushed it to Docker Hub for convenience, though you can use your own image as a device manager.

Serving up USB

Next we need to figure out how to expose USB devices from the host machine, aka “the server” in USB/IP terms. Windows already has a mature USB/IP server you may have not known about; it’s called usbipd-win and is what enables USB access on WSL (which I previously wrote about.) Unfortunately, macOS doesn’t have a complete server but I was able to use this Python library to program a development board (I also tried this Rust library but it’s missing support for USB to serial devices). Developing a USB/IP server for macOS is certainly one area the community can contribute!

With the pre-reqs out of the way, we can get going.

Getting Started with Windows

1. Install prerequisites

Make sure you install Docker, usbipd-win and any device drivers your USB devices might need.

2. Select a USB device to share

Open an admin Powershell and list out the available USB devices. Note the Bus ID as we’ll use that to identify the device next.

usbipd list

3. Share a USB device via USB/IP

Binding a device to the USB/IP server makes it available to Docker and other clients. You can confirm the device is shared by calling list again.

usbipd bind -b <BUSID>

Here’s an example of the output from my machine:

PS C:\Users\jberi> usbipd list
Connected:
BUSID   VID:PID   DEVICE                                                      STATE
2-1     10c4:ee60 CP2102 USB to UART Bridge Controller                        Not shared
2-3     06cb:00bd Synpatics UWP WBDI                                          Not shared
2-4     13d3:5406 Integrated Camera, Integrated IR Camera, Camera DFU Device  Not shared
2-10    8087:0032 Intel(R) Wireless Bluetooth(R)                              Not shared

Persisted:
GUID                                DEVICE

PS C:\Users\jberi> usbipd bind -b 2-1
PS C:\Users\jberi> usbipd list
Connected:
BUSID   VID:PID   DEVICE                                                      STATE
2-1     10c4:ee60 CP2102 USB to UART Bridge Controller                        Shared
2-3     06cb:00bd Synpatics UWP WBDI                                          Not shared
2-4     13d3:5406 Integrated Camera, Integrated IR Camera, Camera DFU Device  Not shared
2-10    8087:0032 Intel(R) Wireless Bluetooth(R)                              Not shared

Persisted:
GUID                                DEVICE

4. Create a container to centrally manage devices

Start a devmgr container with the appropriate flags (--privileged --pid=host are key.)

docker run --rm -it --privileged --pid=host jonathanberi/devmgr

5. See which devices are available to Docker

In the devmgr container, we can list all the devices that the USB/IP client can grab (USB/IP calls it attaching.) One important note – we need to use host.docker.internal as the remote address to make Docker magic happen.

usbip list -r host.docker.internal

6. Attach a device to Docker

Now we want to attach the device shared from the host. Make sure to use the <BUSID> from the previous step.

usbip attach -r host.docker.internal -b <BUSID>

7. Verify that the device was attached

USB/IP can confirm the operation but since you now have a real-but-virtual device, ls /dev/tty* or lsusb should also work. You’ll need the /dev name for the next step anyway.

usbip port

Here’s another example of the output from my machine:

b6b86f127561:/# usbip list -r host.docker.internal
usbip: error: failed to open /usr/share/hwdata//usb.ids
Exportable USB devices
======================
 - host.docker.internal
        2-1: unknown vendor : unknown prodcut (239a:0029)
           : USB\VID_239A&PID_))29\4445FEEF71F2274A
           : unknown class / unknown subclass / unknown protocol (ef/02/01)
           :  0 - unknown class / unknown subclass / unknown protocol (02/02/00)
           :  1 - unknown class / unknown subclass / unknown protocol (0a/00/00)
           :  2 - unknown class / unknown subclass / unknown protocol (08/06/50)

b6b86f127561:/# usbip attach -r host.docker.internal -b 2-1
b6b86f127561:/# usbip port
usbip: error: failed to open /usr/share/hwdata//usb.ids
Imported USB devices
======================
Port 00: <Port in Use> at Full Speed(12Mbps)
       unknown vendor : unknown prodcut (239a:0029)
       1-1 -> usbip://host.docker.internal:3240/2-1
           -> remote bus/dev 002/001
b6b86f127561:/# lsusb
Bus 001 Device 001: ID 1d6b:0002
Bus 001 Device 009: ID 239a:0029
Bus 002 Device 001: ID 1d6b:0003
b6b86f127561:/# ls /dev/ttyA*
/dev/ttyACM0

8. Use your newly-attached USB device

The moment of truth! We can finally use our USB device in a development container. From the Docker perspective it is a real USB device so you can use it to flash, debug, and twiddle bits. You can, of course, use it for non-development things like hooking up a MIDI* controller – but hardware is more fun, right?

There’s unlimited configurations here so I’ll just give an example of how I’d flash a Zephyr application using an image from github.com/embeddedcontainers. Note we’re passing the /dev with --device like we would on a Linux host.

docker run --rm -it -v ${PWD}:/workdir --device "/dev/ttyACM0" ghcr.io/embeddedcontainers/zephyr:arm-0.17.0SDK

Watching west flash work for the first time felt like ✨magic✨.

* One caveat with USB/IP is that Docker may need to enable device drivers for your particular hardware. Many popular USB to Serial chips are already enabled. File an issue on github.com/docker/for-win to request additional drivers or for any other related issues.

Getting Started with macOS

The steps for macOS are nearly identical to Windows except we need to use a different USB/IP server. I’ll use one that I know works with a CP2102. It’s still flaky and only partially implemented, so we need the community to pitch in!

git clone https://github.com/tumayt/pyusbip
cd pyusbip
python3 -m venv .venv
source .venv/bin/activate
pip install libusb1
python pyusbip

Watch as we flash and monitor an ESP32 running Zephyr in Docker!

Картинка с сайта: blog.golioth.io

As before, Docker may need to enable device drivers for your hardware, so file any issue at github.com/docker/for-mac.

Limitations to the USB/IP approach

Obviously the glaring issue here is the UX. The manual setup is clunky and each time a device reboots, it may need to be re-attached–auto attach works pretty well in usbipd-win, though! Also, we need a proper USB/IP server implementation for macOS. Lastly, device support may be limited based on driver availability.

However, I’m optimistic that these challenges can be overcome and that we will see solutions in short time (Open Source, ftw 💪.)

Conclusion

The ability to access USB devices within Docker on both Windows and macOS is a big leap forward for developers, hardware enthusiasts, and anyone working with USB peripherals. While there are still some setup hurdles and device limitations to overcome, this feature has the potential to streamline development and testing processes.

Give it a try and see how it works for you. Just keep in mind that it’s still early days for this feature, so you may run into some bumps along the way. Share your feedback on our forum, and let’s help improve the developer experience of hardware developers everywhere!

Jonathan Beri is the founder and CEO of Golioth, an IoT SaaS platform making it easier for embedded devices to connect to the internet and do all the things that large scale computing enables. He can be found most nights and weekends tinkering with containers and dev boards.

I needed to configure Docker container that can recognize a USB device. I used --device option for the first release. It mounts a specific path device to the container. However, it will never recognize it again when the USB device is disconnected/connected again while the container is running.

I needed to fix this problem. Namely, I have to set up Docker container to let the container recognize the USB that is connected while the container is running.

It took me a whole day to find out the solution. I hope this post helps you out. By the way, the Docker container runs on a Linux system. If you need to try a USB device in a container on Linux system but your host system is Windows, you need to use WSL or VM.

This might help if you use WSL but some devices are not available. Use VirtualBox if your device is not available in WSL.

See this post to set up USB2.0/3.0 in VirtualBox.

Combination of mount and device-cgroup-rule

Let’s look at the solution first. Add the following two options at the same time.

--mount=type=bind,source=/dev/bus/usb,target=/dev/bus/usb
--device-cgroup-rule=a *:* mrw

Setting --device-cgroup-rule=a *:* mrw means that it allows all resources to Docker. It’s not a good idea. This will be explained later.

Device option can’t be used for USB hotplug

The device that I used was assigned under /dev/bus/usb. It’s a scanner. When it’s connected, it’s assigned to /dev/bus/usb/001/002 for example. However, it’s assigned to /dev/bus/usb/001/003 when it’s disconnected and connected again.

When configuring --device=/dev/bus/usb:/dev/bus/usb, it can use USB devices that are already connected before the container starts. When a USB is newly connected, it can’t be opened due to a permission error. It shows an error like below. Privileged is added to for the existing devices but not for the newly added devices. That’s why permission error occurs.

open /dev/bus/usb/001/001: operation not permitted

Actually, --device option is not recommended to use for ephemeral devices.

The –device option cannot be safely used with ephemeral devices. You shouldn’t add block devices that may be removed to untrusted containers with –device.

https://docs.docker.com/engine/reference/commandline/run/#device

Automatic binding by mount option when USB is connected

To bind a new path to the container, the target path /dev/bus/usb needs to be bound.

I first used –volume but it’s better to use –mount because it doesn’t write anything there, doesn’t have to be shared with another container, and doesn’t need a backup. More detail for the good use case of volume/mount is described in the official page.

I tried all Linux capabilities by adding --cap-add option but it didn’t work.

Allow the target device by device-cgroup-rule

To make it work, the device needs to be an allowed device. It can be added by using --device-cgroup-rule.

--device-cgroup-rule=a *:* mrw

The first character is type. The possible values are as follows.

• a: all
• c: char
• b: block

The next *:* is major:minor of a device number. Details including type are here.

The last one is rights.

• m: mknod
• r: read
• w: write

It’s not a good idea to allow all types and devices because it leads to security risks. It’s necessary to find out which type the desired device is.

I executed ls -l /dev. Then, I disconnected the device followed by ls -l /dev again. Comparing the two results, I found the target device.

# when the device is connected
crw-rw-rw-  1 root tty       5,   2 Oct 25 08:43 ptmx
# after it's disconnected
crw-rw-rw-  1 root tty       5,   2 Oct 25  2023 ptmx

As you can see the numbers 5, 2 in the result above, they are the major and minor numbers for the device.

Since the device that I needed was a scanner, it should be enough to set read right with char --device-cgroup-rule=c 5:2 r but it actually didn’t work for some reason. It worked only when the type was set to all --device-cgroup-rule=a 5:2 r.

Overview

To bind the newly created path under /dev/bus/usb, the path needs to be bound. Then, the device needs to be added to the cgroup.

--mount=type=bind,source=/dev/bus/usb,target=/dev/bus/usb
--device-cgroup-rule=c 5:2 r

If it’s necessary to use the options in devcontainer in VSCode, the following configurations need to be added to devcontainer.json.

"runArgs": [
    "--device-cgroup-rule=a 5:* r",
],
"mounts": [
    "type=bind,source=/dev/bus/usb,target=/dev/bus/usb"
],

If it still doesn’t work as expected, it might be related to a user that is used in Docker container. Check which user is used in Docker container and which user owns the target path in the host machine.

If you also need a scanner in Golang, check the following post too.