Skip to content

Commit 3d58e9d

Browse files
author
Shivani Kumari
committed
fix: improve Arch Linux nftables note formatting
1 parent d702e21 commit 3d58e9d

1 file changed

Lines changed: 7 additions & 380 deletions

File tree

Lines changed: 7 additions & 380 deletions
Original file line numberDiff line numberDiff line change
@@ -1,385 +1,12 @@
1-
Arch Linux---
2-
description: Troubleshooting the Rootless mode
3-
keywords: security, namespaces, rootless, troubleshooting
4-
title: Troubleshooting
5-
weight: 30
6-
---
7-
8-
### Distribution-specific hint
9-
10-
{{< tabs >
11-
}}
12-
{{< tab name="Ubuntu" >}}
13-
- Ubuntu 24.04 and later enables restricted unprivileged user namespaces by
14-
default, which prevents unprivileged processes in creating user namespaces
15-
unless an AppArmor profile is configured to allow programs to use
16-
unprivileged user namespaces.
17-
18-
If you install `docker-ce-rootless-extras` using the deb package (`apt-get
19-
install docker-ce-rootless-extras`), then the AppArmor profile for
20-
`rootlesskit` is already bundled with the `apparmor` deb package. With this
21-
installation method, you don't need to add any manual the AppArmor
22-
configuration. If you install the rootless extras using the [installation
23-
script](https://get.docker.com/rootless), however, you must add an AppArmor
24-
profile for `rootlesskit` manually:
25-
26-
1. Create and install the currently logged-in user's AppArmor profile:
27-
28-
```console
29-
$ filename=$(echo $HOME/bin/rootlesskit | sed -e 's@^/@@' -e 's@/@.@g')
30-
$ [ ! -z "${filename}" ] && sudo cat <<EOF > /etc/apparmor.d/${filename}
31-
abi <abi/4.0>,
32-
include <tunables/global>
33-
34-
"$HOME/bin/rootlesskit" flags=(unconfined) {
35-
userns,
36-
37-
include if exists <local/${filename}>
38-
}
39-
EOF
40-
```
41-
2. Restart AppArmor.
42-
43-
```console
44-
$ systemctl restart apparmor.service
45-
```
46-
47-
{{< /tab >}}
481
{{< tab name="Arch Linux" >}}
49-
- Add `kernel.unprivileged_userns_clone=1` to `/etc/sysctl.conf` (or
50-
`/etc/sysctl.d`) and run `sudo sysctl --system`
51-
{{< /tab >}}
52-
{{< tab name="openSUSE and SLES" >}}
53-
- `sudo modprobe ip_tables iptable_mangle iptable_nat iptable_filter` is required.
54-
This might be required on other distributions as well depending on the configuration.
55-
56-
- Known to work on openSUSE 15 and SLES 15.
57-
{{< /tab >}}
58-
{{< tab name="CentOS, RHEL, and Fedora" >}}
59-
- For RHEL 8 and similar distributions, installing `fuse-overlayfs` is recommended. Run `sudo dnf install -y fuse-overlayfs`.
60-
This step is not required on RHEL 9 and similar distributions.
61-
62-
- You might need `sudo dnf install -y iptables`.
63-
{{< /tab >}}
64-
{{< /tabs >}}
65-
66-
## Known limitations
67-
68-
- Only the following storage drivers are supported:
69-
- `overlay2` (only if running with kernel 5.11 or later)
70-
- `fuse-overlayfs` (only if running with kernel 4.18 or later, and `fuse-overlayfs` is installed)
71-
- `btrfs` (only if running with kernel 4.18 or later, or `~/.local/share/docker` is mounted with `user_subvol_rm_allowed` mount option)
72-
- `vfs`
73-
- cgroup is supported only when running with cgroup v2 and systemd. See [Limiting resources](./tips.md#limiting-resources).
74-
- Following features are not supported:
75-
- AppArmor
76-
- Checkpoint
77-
- Overlay network
78-
- Exposing SCTP ports
79-
- To use the `ping` command, see [Routing ping packets](./tips.md#routing-ping-packets).
80-
- To expose privileged TCP/UDP ports (< 1024), see [Exposing privileged ports](./tips.md#exposing-privileged-ports).
81-
- `IPAddress` shown in `docker inspect` is namespaced inside RootlessKit's network namespace.
82-
This means the IP address is not reachable from the host without `nsenter`-ing into the network namespace.
83-
- Host network (`docker run --net=host`) is also namespaced inside RootlessKit.
84-
- NFS mounts as the docker "data-root" is not supported. This limitation is not specific to rootless mode.
85-
86-
## Troubleshooting
87-
88-
### Unable to install with systemd when systemd is present on the system
89-
90-
``` console
91-
$ dockerd-rootless-setuptool.sh install
92-
[INFO] systemd not detected, dockerd-rootless.sh needs to be started manually:
93-
...
94-
```
95-
`rootlesskit` cannot detect systemd properly if you switch to your user via `sudo su`. For users which cannot be logged-in, you must use the `machinectl` command which is part of the `systemd-container` package. After installing `systemd-container` switch to `myuser` with the following command:
96-
``` console
97-
$ sudo machinectl shell myuser@
98-
```
99-
Where `myuser@` is your desired username and @ signifies this machine.
100-
101-
### Errors when starting the Docker daemon
102-
103-
**\[rootlesskit:parent\] error: failed to start the child: fork/exec /proc/self/exe: operation not permitted**
104-
105-
This error occurs mostly when the value of `/proc/sys/kernel/unprivileged_userns_clone` is set to 0:
106-
107-
```console
108-
$ cat /proc/sys/kernel/unprivileged_userns_clone
109-
0
110-
```
111-
112-
To fix this issue, add `kernel.unprivileged_userns_clone=1` to
113-
`/etc/sysctl.conf` (or `/etc/sysctl.d`) and run `sudo sysctl --system`.
114-
115-
**\[rootlesskit:parent\] error: failed to start the child: fork/exec /proc/self/exe: no space left on device**
116-
117-
This error occurs mostly when the value of `/proc/sys/user/max_user_namespaces` is too small:
118-
119-
```console
120-
$ cat /proc/sys/user/max_user_namespaces
121-
0
122-
```
123-
124-
To fix this issue, add `user.max_user_namespaces=28633` to
125-
`/etc/sysctl.conf` (or `/etc/sysctl.d`) and run `sudo sysctl --system`.
126-
127-
**\[rootlesskit:parent\] error: failed to setup UID/GID map: failed to compute uid/gid map: No subuid ranges found for user 1001 ("testuser")**
128-
129-
This error occurs when `/etc/subuid` and `/etc/subgid` are not configured. See [Prerequisites](./_index.md#prerequisites).
130-
131-
**could not get XDG_RUNTIME_DIR**
132-
133-
This error occurs when `$XDG_RUNTIME_DIR` is not set.
134-
135-
On a non-systemd host, you need to create a directory and then set the path:
136-
137-
```console
138-
$ export XDG_RUNTIME_DIR=$HOME/.docker/xrd
139-
$ rm -rf $XDG_RUNTIME_DIR
140-
$ mkdir -p $XDG_RUNTIME_DIR
141-
$ dockerd-rootless.sh
142-
```
143-
144-
> [!NOTE]
145-
>
146-
> You must remove the directory every time you log out.
147-
148-
On a systemd host, log into the host using `pam_systemd` (see below).
149-
The value is automatically set to `/run/user/$UID` and cleaned up on every logout.
150-
151-
**`systemctl --user` fails with "Failed to connect to bus: No such file or directory"**
152-
153-
This error occurs mostly when you switch from the root user to a non-root user with `sudo`:
154-
155-
```console
156-
# sudo -iu testuser
157-
$ systemctl --user start docker
158-
Failed to connect to bus: No such file or directory
159-
```
160-
161-
Instead of `sudo -iu <USERNAME>`, you need to log in using `pam_systemd`. For example:
162-
163-
- Log in through the graphic console
164-
- `ssh <USERNAME>@localhost`
165-
- `machinectl shell <USERNAME>@`
166-
167-
**The daemon does not start up automatically**
168-
169-
You need `sudo loginctl enable-linger $(whoami)` to enable the daemon to start
170-
up automatically. See [Advanced Usage](./tips.md/#advanced-usage).
171-
172-
### `docker pull` errors
173-
174-
**docker: failed to register layer: Error processing tar file(exit status 1): lchown &lt;FILE&gt;: invalid argument**
175-
176-
This error occurs when the number of available entries in `/etc/subuid` or
177-
`/etc/subgid` is not sufficient. The number of entries required vary across
178-
images. However, 65,536 entries are sufficient for most images. See
179-
[Prerequisites](./_index.md#prerequisites).
180-
181-
**docker: failed to register layer: ApplyLayer exit status 1 stdout: stderr: lchown &lt;FILE&gt;: operation not permitted**
182-
183-
This error occurs mostly when `~/.local/share/docker` is located on NFS.
184-
185-
A workaround is to specify non-NFS `data-root` directory in `~/.config/docker/daemon.json` as follows:
186-
```json
187-
{"data-root":"/somewhere-out-of-nfs"}
188-
```
189-
190-
### `docker run` errors
191-
192-
**docker: Error response from daemon: OCI runtime create failed: ...: read unix @-&gt;/run/systemd/private: read: connection reset by peer: unknown.**
193-
194-
This error occurs on cgroup v2 hosts mostly when the dbus daemon is not running for the user.
1952

196-
```console
197-
$ systemctl --user is-active dbus
198-
inactive
3+
Arch Linux defaults to the `iptables-nft` backend. This may cause networking
4+
issues when running Docker in rootless mode.
1995

200-
$ docker run hello-world
201-
docker: Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:385: applying cgroup configuration for process caused: error while starting unit "docker
202-
-931c15729b5a968ce803784d04c7421f791d87e5ca1891f34387bb9f694c488e.scope" with properties [{Name:Description Value:"libcontainer container 931c15729b5a968ce803784d04c7421f791d87e5ca1891f34387bb9f694c488e"} {Name:Slice Value:"use
203-
r.slice"} {Name:PIDs Value:@au [4529]} {Name:Delegate Value:true} {Name:MemoryAccounting Value:true} {Name:CPUAccounting Value:true} {Name:IOAccounting Value:true} {Name:TasksAccounting Value:true} {Name:DefaultDependencies Val
204-
ue:false}]: read unix @->/run/systemd/private: read: connection reset by peer: unknown.
205-
```
6+
If you encounter issues, consider switching to the legacy iptables backend or
7+
ensure nftables compatibility.
2068

207-
To fix the issue, run `sudo apt-get install -y dbus-user-session` or `sudo dnf install -y dbus-daemon`, and then relogin.
9+
See the Arch Linux announcement:
10+
https://archlinux.org/news/iptables-now-defaults-to-the-nft-backend/
20811

209-
If the error still occurs, try running `systemctl --user enable --now dbus` (without sudo).
210-
211-
**`--cpus`, `--memory`, and `--pids-limit` are ignored**
212-
213-
This is an expected behavior on cgroup v1 mode.
214-
To use these flags, the host needs to be configured for enabling cgroup v2.
215-
For more information, see [Limiting resources](./tips.md#limiting-resources).
216-
217-
### Networking errors
218-
219-
This section provides troubleshooting tips for networking in rootless mode.
220-
221-
Networking in rootless mode is supported via network and port drivers in
222-
RootlessKit. Network performance and characteristics depend on the combination
223-
of network and port driver you use. If you're experiencing unexpected behavior
224-
or performance related to networking, review the following table which shows
225-
the configurations supported by RootlessKit, and how they compare:
226-
227-
| Network driver | Port driver | Net throughput | Port throughput | Source IP propagation | No SUID | Note |
228-
| -------------- | -------------- | -------------- | --------------- | --------------------- | ------- | ---------------------------------------------------------------------------- |
229-
| `slirp4netns` | `builtin` | Slow | Fast ✅ ||| Default in a typical setup |
230-
| `vpnkit` | `builtin` | Slow | Fast ✅ ||| Default when `slirp4netns` isn't installed |
231-
| `slirp4netns` | `slirp4netns` | Slow | Slow ||| |
232-
| `pasta` | `implicit` | Slow | Fast ✅ ||| Experimental; Needs pasta version 2023_12_04 or later |
233-
| `lxc-user-nic` | `builtin` | Fast ✅ | Fast ✅ ||| Experimental |
234-
| `bypass4netns` | `bypass4netns` | Fast ✅ | Fast ✅ ||| **Note:** Not integrated to RootlessKit as it needs a custom seccomp profile |
235-
236-
For information about troubleshooting specific networking issues, see:
237-
238-
- [`docker run -p` fails with `cannot expose privileged port`](#docker-run--p-fails-with-cannot-expose-privileged-port)
239-
- [Ping doesn't work](#ping-doesnt-work)
240-
- [`IPAddress` shown in `docker inspect` is unreachable](#ipaddress-shown-in-docker-inspect-is-unreachable)
241-
- [`--net=host` doesn't listen ports on the host network namespace](#--nethost-doesnt-listen-ports-on-the-host-network-namespace)
242-
- [Network is slow](#network-is-slow)
243-
- [`docker run -p` does not propagate source IP addresses](#docker-run--p-does-not-propagate-source-ip-addresses)
244-
245-
#### `docker run -p` fails with `cannot expose privileged port`
246-
247-
`docker run -p` fails with this error when a privileged port (< 1024) is specified as the host port.
248-
249-
```console
250-
$ docker run -p 80:80 nginx:alpine
251-
docker: Error response from daemon: driver failed programming external connectivity on endpoint focused_swanson (9e2e139a9d8fc92b37c36edfa6214a6e986fa2028c0cc359812f685173fa6df7): Error starting userland proxy: error while calling PortManager.AddPort(): cannot expose privileged port 80, you might need to add "net.ipv4.ip_unprivileged_port_start=0" (currently 1024) to /etc/sysctl.conf, or set CAP_NET_BIND_SERVICE on rootlesskit binary, or choose a larger port number (>= 1024): listen tcp 0.0.0.0:80: bind: permission denied.
252-
```
253-
254-
When you experience this error, consider using an unprivileged port instead. For example, 8080 instead of 80.
255-
256-
```console
257-
$ docker run -p 8080:80 nginx:alpine
258-
```
259-
260-
To allow exposing privileged ports, see [Exposing privileged ports](./tips.md#exposing-privileged-ports).
261-
262-
#### Ping doesn't work
263-
264-
Ping does not work when `/proc/sys/net/ipv4/ping_group_range` is set to `1 0`:
265-
266-
```console
267-
$ cat /proc/sys/net/ipv4/ping_group_range
268-
1 0
269-
```
270-
271-
For details, see [Routing ping packets](./tips.md#routing-ping-packets).
272-
273-
#### `IPAddress` shown in `docker inspect` is unreachable
274-
275-
This is an expected behavior, as the daemon is namespaced inside RootlessKit's
276-
network namespace. Use `docker run -p` instead.
277-
278-
#### `--net=host` doesn't listen ports on the host network namespace
279-
280-
This is an expected behavior, as the daemon is namespaced inside RootlessKit's
281-
network namespace. Use `docker run -p` instead.
282-
283-
#### Network is slow
284-
285-
Docker with rootless mode uses [slirp4netns](https://github.com/rootless-containers/slirp4netns) as the default network stack if slirp4netns v0.4.0 or later is installed.
286-
If slirp4netns is not installed, Docker falls back to [VPNKit](https://github.com/moby/vpnkit).
287-
Installing slirp4netns may improve the network throughput.
288-
289-
For more information about network drivers for RootlessKit, see
290-
[RootlessKit documentation](https://github.com/rootless-containers/rootlesskit/blob/v2.0.0/docs/network.md).
291-
292-
Also, changing MTU value may improve the throughput.
293-
The MTU value can be specified by creating `~/.config/systemd/user/docker.service.d/override.conf` with the following content:
294-
295-
```systemd
296-
[Service]
297-
Environment="DOCKERD_ROOTLESS_ROOTLESSKIT_MTU=<INTEGER>"
298-
```
299-
300-
And then restart the daemon:
301-
```console
302-
$ systemctl --user daemon-reload
303-
$ systemctl --user restart docker
304-
```
305-
306-
#### `docker run -p` does not propagate source IP addresses
307-
308-
This is because Docker in rootless mode uses RootlessKit's `builtin` port
309-
driver by default, which doesn't support source IP propagation. To enable
310-
source IP propagation, you can:
311-
312-
- Use the `slirp4netns` RootlessKit port driver
313-
- Use the `pasta` RootlessKit network driver, with the `implicit` port driver
314-
315-
The `pasta` network driver is experimental, but provides improved throughput
316-
performance compared to the `slirp4netns` port driver. The `pasta` driver
317-
requires Docker Engine version 25.0 or later.
318-
319-
To change the RootlessKit networking configuration:
320-
321-
1. Create a file at `~/.config/systemd/user/docker.service.d/override.conf`.
322-
2. Add the following contents, depending on which configuration you would like to use:
323-
324-
- `slirp4netns`
325-
326-
```systemd
327-
[Service]
328-
Environment="DOCKERD_ROOTLESS_ROOTLESSKIT_NET=slirp4netns"
329-
Environment="DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=slirp4netns"
330-
```
331-
332-
- `pasta` network driver with `implicit` port driver
333-
334-
```systemd
335-
[Service]
336-
Environment="DOCKERD_ROOTLESS_ROOTLESSKIT_NET=pasta"
337-
Environment="DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=implicit"
338-
```
339-
340-
3. Restart the daemon:
341-
342-
```console
343-
$ systemctl --user daemon-reload
344-
$ systemctl --user restart docker
345-
```
346-
347-
For more information about networking options for RootlessKit, see:
348-
349-
- [Network drivers](https://github.com/rootless-containers/rootlesskit/blob/v2.0.0/docs/network.md)
350-
- [Port drivers](https://github.com/rootless-containers/rootlesskit/blob/v2.0.0/docs/port.md)
351-
352-
### Tips for debugging
353-
354-
**Entering into `dockerd` namespaces**
355-
356-
The `dockerd-rootless.sh` script executes `dockerd` in its own user, mount, and network namespaces.
357-
358-
For debugging, you can enter the namespaces by running
359-
`nsenter -U --preserve-credentials -n -m -t $(cat $XDG_RUNTIME_DIR/docker.pid)`.
360-
361-
## Uninstall
362-
363-
To remove the systemd service of the Docker daemon, run `dockerd-rootless-setuptool.sh uninstall`:
364-
365-
```console
366-
$ dockerd-rootless-setuptool.sh uninstall
367-
+ systemctl --user stop docker.service
368-
+ systemctl --user disable docker.service
369-
Removed /home/testuser/.config/systemd/user/default.target.wants/docker.service.
370-
[INFO] Uninstalled docker.service
371-
[INFO] This uninstallation tool does NOT remove Docker binaries and data.
372-
[INFO] To remove data, run: `/usr/bin/rootlesskit rm -rf /home/testuser/.local/share/docker`
373-
```
374-
375-
Unset environment variables PATH and DOCKER_HOST if you have added them to `~/.bashrc`.
376-
377-
To remove the data directory, run `rootlesskit rm -rf ~/.local/share/docker`.
378-
379-
To remove the binaries, remove `docker-ce-rootless-extras` package if you installed Docker with package managers.
380-
If you installed Docker with https://get.docker.com/rootless ([Install without packages](./_index.md#install)),
381-
remove the binary files under `~/bin`:
382-
```console
383-
$ cd ~/bin
384-
$ rm -f containerd containerd-shim containerd-shim-runc-v2 ctr docker docker-init docker-proxy dockerd dockerd-rootless-setuptool.sh dockerd-rootless.sh rootlesskit rootlesskit-docker-proxy runc vpnkit
385-
```
12+
{{< /tab >}}

0 commit comments

Comments
 (0)