Apache stuck - website return 504

Keywords: LAMP/MAMP/WAMP - AWS - Technical issue - Application configuration

bnsupport ID: 7a4166e3-fc41-af65-a76a-900eafa1ba57

bndiagnostic output:

? Apache: Found possible issues
? Resources: Found possible issues
https://docs.bitnami.com/general/apps/wordpress/troubleshooting/debug-errors-apache/
https://docs.bitnami.com/bch/apps/moodle/troubleshooting/deny-connections-bots-apache/
https://docs.bitnami.com/installer/faq/linux-faq/administration/increase-memory-linux/

Description:
Dear Support ;

your diagnostic tool return “Your instance has little available RAM memory.”
that is not correct as I checked the free command output and the available free RAM space is half of the real memory size.

when the website return 504 , I detach the server then that server will not accept any request , I tried the following command { sudo telnet localhost 80 , sudo telnet localhost 443 } but no connection then I restarted the services and run the commands again and it was connected.

in short, the main issue is the Apache server is unable to serve any request >> 504 error >> restart Apache >> Apache return to serve the requests.

Hi @baker.khamiseh,

There are a lot of requests coming from the same IP’s:

 -----------------------------------
 Check performance issues: Count number of requests for the 10 most active IP addresses in the last 100.000 requests
 -----------------------------------
 Running: tail -n 100000 access_log | awk '{print $1}' | sort | uniq -c | sort -nr | head -n 10 | awk '{print $1}'
 In: /opt/bitnami/apache2/logs/

 Output:

 20319
 20123
 94
 48
 45
 32
 23
 20
 16
12

This could indicate a bot attack. There are also suspicious looking messages in the apache log file trying to read the /etc/passwd file:

[Sun Nov 07 04:02:36.434885 2021] [authz_core:error] [pid 29941:tid 139960451405568] [client xxx.xx.xx.xxx:35342] AH01630: client denied by server configuration: /etc/passwd

Can you check the following guide to block those IP’s?

https://docs.bitnami.com/bch/apps/moodle/troubleshooting/deny-connections-bots-apache/

Regards,
Michiel

bro; look ;

the output for the following command return only for 9-11-2021 the issue was yesterday on 8-11-2021 ;

at that time when you try to telnet the localhost on ports 80 and 443 and nothing happened, this means the Apache server is not serving any requests until you restart the server, that is the issue!

note: when the issue was raised, no logs for Apache were available.

BR
Baker

Hi @baker.khamiseh,

You can check the logs in /opt/bitnami/apache2/logs/error_log.

To check the number of requests per IP you can run this command:

sudo tail -n 100000 /opt/bitnami/apache2/logs/access_log | awk '{print $1}' | sort | uniq -c | sort -nr | head -n 10 

This will list the IP addresses which tried to access your site. The high number of unique requests does indicate a bot attack, which can cause the 504 error.

Regards,
Michiel

Dear michiel;

the website return the same 504 today, I checked the server specs again its normal, the was solved when the services was restarted before that the command: telnet localhost 80 was unable to connect, this means the apache server was stuck, note the following error_log : on line 173PHP message: PHP Warning: chmod(): Operation not permitted in /opt/bitnami/apache/htdocs/wp-admin/includes/class-wp-filesystem-direct.php.

BR
Baker

Hi @baker.khamiseh,

Did you check the unique IP addresses that should be blocked? I recommend launching a new instance and migrate your application there.

Best regards,
Michiel

Dear Michiel ;

yes, i already checked the IP, it is back to ELB so now the issue is here.

the website again returns

504 error, and the issue is on the Apache server, why? because when I run { sudo telnet localhost 80 } its returns an error and this means apache is not serving any request.

BR

Hi @baker.khamiseh,

When I check the IP with curl Apache seems to be ok. If the issue continues can you run the bndiagnostic tool again?

Regards,
Michiel