Updating ssl certificate with bncertbot fails because of httpd.bin on port 80 and 443

Fadalken 2020-12-02 19:33:00 UTC #1

Keywords: Apache Airflow - AWS - Technical issue - Secure Connections (SSL/HTTPS)
Description:
Hello,

I am trying to renew the ssl certificate for my website with the bncertbot. It fails with this message:

Error: There has been an error.
Cannot bind to port 80 and/or 443. These ports are used for Let's Encrypt to
verify the domain DNS configuration. Please stop any services using those ports,
and ensure your system user has permissions to bind to them.

I tried to see what is listening on that port and this came back

sudo netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:9418 0.0.0.0:* LISTEN 28621/git-daemon
tcp 0 0 127.0.0.1:587 0.0.0.0:* LISTEN 16187/sendmail: MTA
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 5454/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 16187/sendmail: MTA
tcp6 0 0 :::443 :::* LISTEN 5693/httpd.bin
tcp6 0 0 :::9418 :::* LISTEN 28621/git-daemon
tcp6 0 0 :::80 :::* LISTEN 5693/httpd.bin
tcp6 0 0 :::22 :::* LISTEN 5454/sshd
udp 0 0 0.0.0.0:68 0.0.0.0:* 945/dhclient

So I tried to kill it with:
sudo kill -9 5693
but it is immediately back with another PID.
I also cannot restart apache because of the same problem (I think), if I run:
sudo /opt/bitnami/ctlscript.sh start
I get:
/opt/bitnami/mysql/scripts/ctl.sh : mysql started at port 3306
/opt/bitnami/php/scripts/ctl.sh : php-fpm started
Syntax OK
(98)Address already in use: AH00072: make_sock: could not bind to address [::]:80
(98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
AH00015: Unable to open logs
/opt/bitnami/apache2/scripts/ctl.sh : httpd could not be started

I also tried to run the bitnami support tool, but it also exits with an error:
sudo /opt/bitnami/bnsupport-tool

Error: There has been an error.
The application will exit now.
Press [Enter] to continue:

I know enough things to get a server like this running from scratch but if its about this debugging thing, I am in over my head and still have lots to learn.
Any pointers or help would be greatly appreciated.

gongomgra 2020-12-03 09:34:40 UTC #2

Hi @falk.zimmermann93,

Thanks for using Bitnami. The bncert tool should take care of stopping the services before creating the SSL certificate. Can you run the next command to stop Apache and then run the bncert tool again?

sudo /opt/bitnami/ctlscript.sh stop apache

After that, run the bncert too. You can also start Apache later with

sudo /opt/bitnami/ctlscript.sh start apache

Can you also run some commands to check if your server is running out of memory or disk space?

free -h
df -h
uptime

Fadalken 2020-12-03 09:46:51 UTC #3

Hi @gongomgra,
thank you for coming back to me.
I ran the stop apache command and this came back:
sudo /opt/bitnami/ctlscript.sh stop apache

Failed to unmonitor apache: write /var/lib/gonit/state: no space left on device
Syntax OK
/opt/bitnami/apache2/scripts/ctl.sh : apache not running

then I tried the bncert command and this came back:
sudo /opt/bitnami/bncert-tool

Unable to initialize installer.
Is /tmp writable ?
Is SELinux enabled? You may need to disable it temporarily
Please see below for details
http://docs.fedoraproject.org/en-US/Fedora/13/html/SELinux_FAQ/index.html#id3037154

Its a different problem than I had before, now I know even less, what is going on.

And about the disk space here the commands you mentioned:
free -h

total used free shared buff/cache available
Mem: 990M 115M 391M 4.4M 483M 633M
Swap: 634M 73M 560M

df -h

Filesystem Size Used Avail Use% Mounted on
udev 488M 0 488M 0% /dev
tmpfs 100M 11M 89M 11% /run
/dev/xvda1 9.7G 9.7G 0 100% /
tmpfs 496M 0 496M 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 496M 0 496M 0% /sys/fs/cgroup
tmpfs 100M 0 100M 0% /run/user/1000
/dev/loop1 29M 29M 0 100% /snap/amazon-ssm-agent/2012
/dev/loop4 98M 98M 0 100% /snap/core/9993
/dev/loop2 98M 98M 0 100% /snap/core/10444
/dev/loop3 56M 56M 0 100% /snap/core18/1932
/dev/loop5 33M 33M 0 100% /snap/amazon-ssm-agent/2996

I realise this one might be hard to read in this format. So in short, everything is free execpt for the /dev/loops those are packed and the /dev/xvda1.
Should I free something up here?

uptime

09:40:33 up 677 days, 8:28, 1 user, load average: 0.00, 0.00, 0.00

I am guessing 600+ days is not an optimal uptime? I should restart somehow?

Thanks for having a look into my post!

gongomgra 2020-12-04 10:36:49 UTC #4

Hi @Fadalken,

Thanks for sending the output. According to it, your server is running out of disk space, so you will need to increase the EBS volume first. Please follow the next guide to do so

https://docs.bitnami.com/aws/how-to/modify-ebs-volume/

You may also need to run some additional commands in your instance. Please check the official docs at https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/recognize-expanded-volume-linux.html

I am guessing 600+ days is not an optimal uptime? I should restart somehow?

That's fine. Restarting is not needed usually on servers, their principal mission is to be up as much time as possible, so your website is always available

Fadalken 2020-12-07 18:25:33 UTC #5

Hi @gongomgra,

I have increased the disk space. What I do not understand is, why I had to increase disk size, as the size of the website had not changed. Isn't all used space that came after initiating the website then some cache data of running the website that should be possible to be deleted? I had googled for that but the results didnt point towards an answer I would understand. Anyways, here the df -h output:

Filesystem Size Used Avail Use% Mounted on
udev 488M 0 488M 0% /dev
tmpfs 100M 3.3M 96M 4% /run
/dev/xvda1 15G 9.6G 5.0G 66% /
tmpfs 496M 0 496M 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 496M 0 496M 0% /sys/fs/cgroup
/dev/loop0 98M 98M 0 100% /snap/core/10444
/dev/loop1 98M 98M 0 100% /snap/core/9993
/dev/loop2 56M 56M 0 100% /snap/core18/1932
/dev/loop3 33M 33M 0 100% /snap/amazon-ssm-agent/2996
/dev/loop4 29M 29M 0 100% /snap/amazon-ssm-agent/2012
tmpfs 100M 0 100M 0% /run/user/1000

After doing so, the bn support tool worked again! Thanks for hinting to space issues! So here is the support code:
5dbcea87-c84a-2aed-1837-9cbe77dfec65

Still, ssl certification does not work. Still because of the ports being occupied, as I understand it. I can also not restart apache because of the same issue.

Here the error output of the bn restart command:
sudo /opt/bitnami/ctlscript.sh restart

Syntax OK
/opt/bitnami/apache2/scripts/ctl.sh : apache not running
/opt/bitnami/php/scripts/ctl.sh : php-fpm not running
/opt/bitnami/mysql/scripts/ctl.sh : mysql not running
/opt/bitnami/mysql/scripts/ctl.sh : mysql (pid 2073) already running
/opt/bitnami/php/scripts/ctl.sh : php-fpm started
Syntax OK
(98)Address already in use: AH00073: make_sock: unable to listen for connections on address [::]:80
(98)Address already in use: AH00073: make_sock: unable to listen for connections on address 0.0.0.0:80
no listening sockets available, shutting down
AH00015: Unable to open logs
/opt/bitnami/apache2/scripts/ctl.sh : httpd could not be started

And the error of the bn cert tool:

Thanks again for your continuous replies.

Ibone 2020-12-08 12:02:59 UTC #6

Hi @Fadalken,

Could you run the following commands and copy the output in the thread?

netstat -lnp | grep :80
netstat -lnp | grep :443

Regards,
Ibone.

Fadalken 2020-12-08 12:11:20 UTC #7

Hi @Ibone,

Sure thing!
Here the output of the first command:

sudo netstat -lnp | grep :80
tcp6 0 0 :::80 :::* LISTEN 1554/apache2 -

The second command has no output with sudo and as the normal user:

(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)

gongomgra 2020-12-09 10:01:02 UTC #8

Hi @Fadalken,

Thank you for providing us with the bnsupport bundle. I think I know what is your issue now. You are running the Apache service installed using the system package manager (apt-get), and it is preventing the Bitnami service to be started (and it is also being started every time on its own)

-----------------------------------
Check Apache path:
-----------------------------------
Running: ps aux | grep 'apache\|mysql\|nginx\|/opt/bitnami'
In: /opt/bitnami

Output:

root      1554  0.0  0.4  71596  4504 ?        Ss   18:09   0:00 /usr/sbin/apache2 -k start
www-data  1556  0.0  0.3 360744  3800 ?        Sl   18:09   0:00 /usr/sbin/apache2 -k start
www-data  1557  0.0  0.3 360744  3800 ?        Sl   18:09   0:00 /usr/sbin/apache2 -k start

Can you run the next commands to disable and stop it?

sudo systemctl disable apache2
sudo systemctl stop apache2

After that, it is also a good practice to uninstall this Apache service, but this step is optional though

sudo apt-get remove --purge apache2

You should be able to request your SSL certificate once you have disabled and stopped the apache2 service

system 2020-12-23 10:01:10 UTC #9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Bitnami provides free all-in-one installers, virtual machines and cloud images for popular open source applications. Get them now!