A debug session. I ran into a problem in CI where accessing /var/run/docker.sock from within a container failed with a permission error. I had this specific part working before. So I did a bit of bisecting and added debug output and found the critical difference. In this case I get a permission error (EACCES):
uid=2000(buildkite-agent) gid=0(root) groups=0(root)
In this case not:
uid=2000(buildkite-agent) gid=1001 groups=1001
The difference is in the unix group membership specifics of unix user uid=2000(buildkite-agent) within the specific container. If the user is member of gid=1001 then access is allowed, if the user is member of gid=0 then access is denied.
I found that in the erroneous case I was passing this command line argument to docker run ...:
-u 2000:
Nothing after the colon. This is where the group ID (gid) belongs. docker run ... did not error out. That is no good. This input was treated the same as -u 2000 or -u 2000:0.
But why was there no gid after the colon when I have this in my Makefile?
-u $(shell id -u):${DOCKER_GID_HOST}Because when this line was run DOCKER_GID_HOST (a Make variable, not an environment variable) was actually not set.
A shell program would catch this case (variable used, but not set) when being used with the -o nounset option, and error out. Make does not have this kind of protection. There is no general protection against using variables that are not set. As far as I know!
Okay, but why was the variable DOCKER_GID_HOST not set when I have
DOCKER_GID_HOST := $(shell getent group docker | awk -F: '{print $$3}')right before executing docker run? Well, because this is a Makefile. Where you cannot set a variable in one line of a recipe, and use it in the next one (a pattern that we use in almost every other programming environment).
The lines of a recipe are executed in independent shells. The next line’s state is very much decoupled from the previous line’s state, that’s how Makefiles work.
This is probably the most important thing to know about Make, and one of the most common mistakes, and I certainly knew this before, and I certainly made this same mistake before. And I made it again, like probably every single time that I had to do things with Make.
Makefiles are flexible and great, and sometimes they make you waste a great deal of time compared to other development environments, man.
Stack Overflow threads on the matter, with goodies like workarounds, best practices, and generally helpful discussion:
Leave a Reply