Repository navigation
Cannot Use testcontainers with Isolated Containers #712
Description
I do have a docker setup, providing container isolation with a user namespace. Following configuration:
/etc/docker/daemon.json
{
"userns-remap": "default"
}/etc/subuid
dockremap:1476256:65536
/etc/subgid
dockremap:1476256:65536
When I use mvn test which tries to start containers, I get following error (cannot connect to Ryuk):
[main] INFO org.testcontainers.DockerClientFactory - Docker host IP address is localhost
[main] INFO org.testcontainers.DockerClientFactory - Connected to docker:
Server Version: 18.05.0-ce
API Version: 1.37
Operating System: Arch Linux
Total Memory: 3948 MB
[testcontainers-ryuk] WARN org.testcontainers.utility.ResourceReaper - Can not connect to Ryuk at localhost:32770
java.net.ConnectException: Verbindungsaufbau abgelehnt (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at java.net.Socket.<init>(Socket.java:434)
at java.net.Socket.<init>(Socket.java:211)
at org.testcontainers.utility.ResourceReaper.lambda$start$2(ResourceReaper.java:119)
at java.lang.Thread.run(Thread.java:748)
[testcontainers-ryuk] WARN org.testcontainers.utility.ResourceReaper - Can not connect to Ryuk at localhost:32770
java.net.ConnectException: Verbindungsaufbau abgelehnt (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at java.net.Socket.<init>(Socket.java:434)
at java.net.Socket.<init>(Socket.java:211)
at org.testcontainers.utility.ResourceReaper.lambda$start$2(ResourceReaper.java:119)
at java.lang.Thread.run(Thread.java:748)
I assume that the ryuk container must be started with the command line option --userns=host (docker ignores user namespaces).
Is it possible to integrate this commandline option?
Activity
kiview commentedon Jun 12, 2018
AFAIK docker-java doesn't suport user namespaces ATM.
nathanburrell commentedon Sep 28, 2018
Rather than disabling userns remapping on the ryuk container, we can investigate why the ryuk container cant run with that feature enabled as that would be the better fix.
As some environments may never ever allow a user to turn of the remapping feature for a container. As some cloud CI/CD providers (such as bitbucket pipelines and others) run in a multi tenanted environment and have user namespace remapping always enabled for security purposes.
bsideup commentedon Sep 28, 2018
@nathanburrell is it possible to turn it on locally on Mac? Then I can experiment with it locally and maybe figure out why it doesn't work :)
nathanburrell commentedon Sep 28, 2018
It sure is, you can specify daemon args following the steps here
https://docs.docker.com/v17.12/docker-for-mac/#daemon-tab
The Arg you want to provide is
--userns-remap=default
More information about user namespace remapping is here https://docs.docker.com/engine/security/userns-remap/
nathanburrell commentedon Oct 9, 2018
Any updates on this?
Over the coming months we will be removing the exclusion list as it leaves potential security holes within our product, and I want to ensure test containers still works with pipelines when that time comes.
kiview commentedon Oct 9, 2018
I'll try to see if I can reproduce it locally on Linux when setting the flag on the daemon.
The actual ryuk implementation is pretty slim, so I wonder where the problem comes from:
https://github.com/testcontainers/moby-ryuk/blob/master/main.go
schrieveslaach commentedon Oct 10, 2018
@kiview, if you have a new SNAPSHOT version, I can test it on my machine.
rnorth commentedon Dec 13, 2018
I was trying to reproduce the other day so that we can look into why - however, I had trouble with the
userns-remapsetting on Docker for Mac. It seems that setting this in the daemon JSON configuration causes the daemon to fail to start up.I haven't yet had time to try this in another context (e.g. with docker-machine).
Sorry for there being no progress made on this ticket so far. If anybody has ability to diagnose why this setting is incompatible with Ryuk, we'd be keen to get your input.
jrehwaldt commentedon Dec 14, 2018
Suppose the issue with bitbucket pipeline is indeed the same you can test it against their dockerized pipeline runner as follows:
You should see the same log as posted in the issue description.
rnorth commentedon Dec 15, 2018
Ah, that's a good suggestion, thanks @jrehwaldt. I'll give this a try and see what I can find.
rnorth commentedon Dec 15, 2018
Hmm, either the userns-remapping or the docker-in-docker nature of this is meaning that none of the exposed ports on any containers are accessible from Mac (not just Ryuk). I imagine this might be a limitation of Docker for Mac's TCP proxy.
I think we'll need to do this on Linux, though, so I'll pursue that avenue.
rnorth commentedon Dec 15, 2018
We seem to have two issues for the same thing (this and #700), and it will get confusing having both. As such, I'll close this as the newer of the two. Please continue discussion as needed on #700.
marcinar commentedon Aug 12, 2026
@rnorth I don't think this was ever resolved. The problem is that Ryuk can't access the mapped domain socket unless userns remapping is turned off, and this can't be worked around without broadening the socket permissions.
The other issue was resolved by disabling Ryuk entirely, which doesn't apply here since this isn't about CI. It would be nice if there was a simple configuration option to pass --userns=host similar the option for disabling the privileged mode.
nathanburrell commentedon Aug 17, 2026
Hey @marcinar you can disable userns-remap (amongst other advanced features such as choosing your own docker version) by using runtime v3.
See https://support.atlassian.com/bitbucket-cloud/docs/enable-and-use-runtime-v3/