config.json: permission denied

cmardonespino 2020-05-13 15:14:29 UTC #1

IMPORTANT, please fill the questions

We assume you are using Bitnami to deploy your application

Which version of the application are you using?: 2.222.3
Please choose how you got the application: Installer (Windows, Linux, macOS), cloud image (AWS, GCE, Azure, ...) or VM (VMDK, VBOX): From AWS MarketPlace https://aws.amazon.com/marketplace/server/procurement?productId=447af561-b7e2-4d03-8a67-4842db5439cb
Have you installed any plugin or modified any configuration file?: Yes,
Authorize Project
Docker plugin
GitHub Authentication plugin
GitHub Organization Folder Plugin
Matrix Authorization Strategy
OWASP Markup Formatter Plugin
Pipeline
Describe here your question/suggestion/issue (expected and actual results): It is a question. I have installed docker following the next documentation https://docs.bitnami.com/aws/apps/jenkins/configuration/install-docker-debian/. Once docker installed, I have added at the docker group the user bitnami & jenkins and then, I have restarted the machine were is running Jenkins. After it, I created a pipeline where Jenkins execute the comand "docker ps", but it fail returning the following error:

20:46:16  + docker ps
20:46:16  WARNING: Error loading config file: /root/.docker/config.json: stat /root/.docker/config.json: permission denied
20:46:16  Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.40/containers/json: dial unix /var/run/docker.sock: connect: permission denied

I dont know whats going on if when I execute through SSH with the user Jenkins I dont have problems but when I build the pipeline doenst works... it is worth mentioning that in the pipeline the command sh "whoami" it return jenkins

Steps to reproduce the issue (if relevant):
install docker through SSH
create pipeline and put a SH command to execute "docker ps"
build pipeline

Copy the apache log (if relevant):

20:46:16  + docker ps
20:46:16  WARNING: Error loading config file: /root/.docker/config.json: stat /root/.docker/config.json: permission denied
20:46:16  Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.40/containers/json: dial unix /var/run/docker.sock: connect: permission denied

Jenkins user has different group as the Jenkins runner user

amoreno 2020-05-14 08:17:24 UTC #2

Hi, @cmardonespino.

We have a Support Tool that will gather relevant information for us to analyze your configuration and logs. Could you please execute it on the machine where the stack is running by following the steps described in the guide below?

How to Run the Bitnami Support Tool in a cloud image or virtual machine

Please note that you need to paste the code ID that is shown at the end.

cmardonespino 2020-05-14 16:00:53 UTC #3

Yeah! Sure! Thank so much for your reply !

The output code is: 92e1d23c-6e6b-0b2d-7666-54d3c5a0a1eb

I did a couple of things after this post.
1) I saw that the HOME environment variable of the jenkins was defined in the /root/ path, weird ...

BITNAMI_SCRIPTS_DIR	/opt/bitnami/scripts
HOME	/root
INVOCATION_ID	X
JENKINS_HOME	/opt/bitnami/jenkins/jenkins_home
JOURNAL_STREAM	X
LANG	en_US.UTF-8
PATH	/opt/bitnami/nami/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bitnami/apache/bin:/opt/bitnami/apache2/bin:/opt/bitnami/common/bin:/opt/bitnami/git/bin:/opt/bitnami/gonit/bin:/opt/bitnami/java/bin
PWD	/bitnami/jenkins/jenkins_home
SHLVL	0
TERM	vt220

So I added an environment variable called HOME (in the Jenkins configuration) and I replace it to $JENKINS_HOME. After it I got the same permission denied but now in the path /var/run/docker.sock ... So as a fast solution until you have answered me I executed in the machine by ssh sudo chmod o+rw /var/run/docker.sock and is working ... But obviously it is not secure so I hope exists a better solution

amoreno 2020-05-15 08:35:28 UTC #4

Hi, @cmardonespino.

What's the output of these commands?

ls -l /root/.docker
ls -l /var/run

The user running Jenkins should have access to both the config.json file and the Docker daemon socket. If the user running Jenkins is a member of the docker group, you can just make sure that the docker group has permissions to those files, no need to give permissions to others.

Let me know how it goes.

Regards,
Alejandro

cmardonespino 2020-05-15 13:22:39 UTC #5

Hi @amoreno!

Yeah, sure, I just added the user jenkins in the docker group and it is working only when I execute "docker" through SSH ... but through a pipeline, jenkins dont have permissions... its weird !

ls -l /root/.docker -> it doesnt exists root@ip-X:/home/bitnami# ls -las /var/run 0 lrwxrwxrwx 1 root root 4 Nov 13 2019 /var/run -> /run bitnami@ip-X:~$ sudo su jenkins $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES $

As I said in the last post when I installed docker in the Jenkins machine through SSH, I can do whatever command with docker, but when I want to execute docker through a Jenkins pipeline I got the permission denied. So I did the things that I said above...

Help me pleae

cmardonespino 2020-05-15 18:11:39 UTC #6

I have created a new machine with the same Jenkins AMI and I got the same issue

14:05:58  + docker inspect -f . infrastructureascode/aws-cli:1.16.287
14:05:58  WARNING: Error loading config file: /root/.docker/config.json: stat /root/.docker/config.json: permission denied
14:05:58  
14:05:58  Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.40/containers/infrastructureascode/aws-cli:1.16.287/json: dial unix /var/run/docker.sock: connect: permission denied
[Pipeline] isUnix
[Pipeline] sh
14:05:59  + docker pull infrastructureascode/aws-cli:1.16.287
14:05:59  WARNING: Error loading config file: /root/.docker/config.json: stat /root/.docker/config.json: permission denied
14:05:59  Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.40/images/create?fromImage=infrastructureascode%2Faws-cli&tag=1.16.287: dial unix /var/run/docker.sock: connect: permission denied

I has executed the command sudo chmod o+rw /var/run/docker.sock. This is the only way that I found so Jenkins can execute docker commands through an pipeline

cmardonespino 2020-05-15 20:08:57 UTC #8

I understand less and less

root@ip-X:/etc/systemd/system/sockets.target.wants# cat docker.socket
[Unit]
Description=Docker Socket for the API
PartOf=docker.service

[Socket]
ListenStream=/var/run/docker.sock
SocketMode=0660
SocketUser=root
SocketGroup=docker

[Install]
WantedBy=sockets.target
root@ip-X:/etc/systemd/system/sockets.target.wants# grep 'docker' /etc/group
docker:x:999:bitnami,jenkins

root@ip-X:/etc/systemd/system/sockets.target.wants# grep 'docker' /etc/group
docker:x:999:bitnami,jenkins

I still understand why a Jenkins pipeline cant execute docker !!

amoreno 2020-05-18 08:31:11 UTC #9

Hi, @cmardonespino.

Is it possible that your pipeline is running as a different user? What happens if you execute whoami in your pipeline?

cmardonespino 2020-05-18 13:35:37 UTC #10

As I said in my first post, yes, my pipeline is using the Jenkins user...

09:34:56  + whoami
09:34:56  jenkins

carey 2020-05-19 10:45:16 UTC #11

I am having exactly the same problem with the same bitnami/jenkins AMI, from which I created an instance about the same time as @cmardonespino, getting the same error starting yesterday when we started running our first pipeline.

carey 2020-05-19 11:01:54 UTC #13

My support tool code is 6c8fdec1-5c62-01fb-865b-ad352df3d5a9

Thanks,

Carey

amoreno 2020-05-19 11:30:14 UTC #14

Sorry, @cmardonespino. I meant id instead of whoami to check the groups.

I've been able to reproduce this issue: running id as user jenkins from the command line returns:

uid=1001(jenkins) gid=1003(jenkins) groups=1003(jenkins),999(docker)

However, running it in the pipeline, it returns this instead:

uid=1001(jenkins) gid=0(root) groups=0(root)

This explains why it's not able to run docker commands. I'll forward this to another team so they can continue investigating this.

In the meantime, I can provide you with a different workaround, which is changing the group of /var/run/docker.sock to root:

sudo chgrp root /var/run/docker.sock

@carey, please confirm that you are able to replicate this post. If that's not the case, please open a new ticket for your case.

Regards,
Alejandro

carey 2020-05-19 11:54:15 UTC #15

I can confirm that I can replicate the post. I get:

Running as SYSTEM
Building in workspace /opt/bitnami/jenkins/jenkins_home/workspace/test
[test] $ /bin/sh -xe /tmp/jenkins1592150378067783436.sh
+ id
uid=1001(jenkins) gid=0(root) groups=0(root)
Finished: SUCCESS

Thank you, @amoreno. I am now able to progress to the next error in the pipeline

Carey

rafaelrios 2020-05-20 07:24:28 UTC #16

Hi @carey,
It seems that jenkins is running with gid=0:

$ ps -ef | grep jenkins
jenkins   1602     1  5 06:40 ?        00:02:08 /opt/bitnami/java/bin/java -jar /opt/bitnami/jenkins/jenkins.war

$ ps -o uid,gid -p 1602
UID   GID
1001     0

I am creating an internal task to check the reasons behind this, and how to fix it.

We will come back as soon as we have news to share.

Best regards

cmardonespino 2020-05-21 03:10:07 UTC #17

When I execute id it returns the same as to you

23:03:28  + id
23:03:28  uid=1001(jenkins) gid=0(root) groups=0(root)

After the change of the group is working well. I hope you can fix soon! Thank you for you help! @amoreno

cmardonespino 2020-05-21 03:11:11 UTC #18

Thank you !! I will stay attentive to whatever new news!

system 2020-06-04 03:11:18 UTC #19

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Bitnami provides free all-in-one installers, virtual machines and cloud images for popular open source applications. Get them now!