Bitnami Magento stack showing high CPU usage for EC2 t3.xlarge instance

Keywords: Magento - AWS - Technical issue - Other

bnsupport ID: 4531282e-87a6-9e76-bfb2-bea54f5516b4

bndiagnostic output:

? Apache: Found possible issues
? Connectivity: Found possible issues
https://docs.bitnami.com/general/apps/wordpress/troubleshooting/debug-errors-apache/
https://docs.bitnami.com/bch/apps/moodle/troubleshooting/deny-connections-bots-apache/
https://docs.bitnami.com/general/faq/administration/use-firewall/

bndiagnostic failure reason: The suggested guides are not related with my issue

Description:
Hi there,
I am using t3.xlarge instance for my Bitnami Magento stack. All of sudden, my CPU usage went high while I was uploading docs to the server.

I restarted my instance as well but it has not got fixed. It’s continuously more than 70%. Following is the error log shown when I did run the diagnostic tool.

[Thu Aug 12 07:37:07.225305 2021] [proxy_fcgi:error] [pid 1098:tid
140197891057408] (104)Connection reset by peer: [client **ip_address**:57048]
AH01075: Error dispatching request to : , referer: https://**.***.79.**/
 [Thu Aug 12 07:37:07.240376 2021] [proxy_fcgi:error] [pid 1011:tid
140197949806336] [client **ip_address**:56982] AH01067: Failed to read FastCGI
header, referer: https://**.***.79.**/
 [Thu Aug 12 07:37:07.240416 2021] [proxy_fcgi:error] [pid 1011:tid
Press [Enter] to continue:
140197949806336] (104)Connection reset by peer: [client **ip_address**:56982]
AH01075: Error dispatching request to : , referer: https://**.***.79.**/

Any kind of help will be really appreciated.

Hi @zobi.gondal.89,

It seems Apache can’t connect to PHP-FPM. You can get more info from the PHP-FPM’s log files. Those files were rotated but you can find them inside the /opt/bitnami/php/logs folder. Could you please let us know the errors you find in those files?

Thanks

Hi Jota,
Thanks for replying back. The following is from the PHP-FPM log file.

[13-Aug-2021 07:31:24] WARNING: [pool www] server reached pm.max_children setting (200), consider raising it
[13-Aug-2021 09:44:23] WARNING: [pool www] child 4166 exited on signal 11 (SIGSEGV) after 3269.406087 seconds from start
[13-Aug-2021 09:44:23] WARNING: [pool www] child 4206 exited on signal 11 (SIGSEGV) after 3260.507827 seconds from start
[13-Aug-2021 09:45:04] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 129 idle, and 189 total children
[13-Aug-2021 09:45:07] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 16 children, there are 129 idle, and 190 total children
[13-Aug-2021 09:45:10] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 129 idle, and 191 total children
[13-Aug-2021 09:45:14] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 129 idle, and 192 total children
[13-Aug-2021 09:45:27] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 129 idle, and 193 total children
[13-Aug-2021 10:12:24] WARNING: [pool www] child 4083 exited on signal 11 (SIGSEGV) after 4952.132342 seconds from start
[13-Aug-2021 10:14:17] WARNING: [pool www] child 4091 exited on signal 11 (SIGSEGV) after 5064.848052 seconds from start
[13-Aug-2021 10:15:58] WARNING: [pool www] server reached pm.max_children setting (200), consider raising it
[13-Aug-2021 10:41:17] WARNING: [pool www] server reached pm.max_children setting (200), consider raising it
[13-Aug-2021 10:42:25] WARNING: [pool www] child 4077 exited on signal 11 (SIGSEGV) after 6752.545980 seconds from start
[13-Aug-2021 10:57:01] WARNING: [pool www] child 4172 exited on signal 11 (SIGSEGV) after 7629.015599 seconds from start
[13-Aug-2021 10:57:17] WARNING: [pool www] child 5353 exited on signal 11 (SIGSEGV) after 4374.071708 seconds from start
[13-Aug-2021 10:57:17] WARNING: [pool www] child 5375 exited on signal 11 (SIGSEGV) after 4303.331883 seconds from start
[13-Aug-2021 10:59:17] WARNING: [pool www] child 6339 exited on signal 11 (SIGSEGV) after 1194.637662 seconds from start
[13-Aug-2021 11:00:02] WARNING: [pool www] child 6340 exited on signal 11 (SIGSEGV) after 1238.608304 seconds from start
[13-Aug-2021 11:00:32] WARNING: [pool www] child 6345 exited on signal 11 (SIGSEGV) after 1156.010151 seconds from start
[13-Aug-2021 11:00:32] WARNING: [pool www] child 4074 exited on signal 11 (SIGSEGV) after 7839.704519 seconds from start
[13-Aug-2021 11:03:50] WARNING: [pool www] child 4165 exited on signal 11 (SIGSEGV) after 8037.038999 seconds from start
[13-Aug-2021 11:05:27] WARNING: [pool www] child 6343 exited on signal 11 (SIGSEGV) after 1452.431757 seconds from start

Hi @jota
I am wondering if you got a chance to take a look at the logs I shared earlier

Hello @zobi.gondal.89,

Did you try increasing the values of php-fpm parameters? Maybe this could help to sort the issue out. In additon to this, using swap could also be helpful. In order to add swap memory to your instance, please follow these steps:

  • Allocate space to a file, for instance 1GB:
fallocate -l 1G swapfile 
  • Format this newly created file to swap
mkswap swapfile
  • Mount this file as swap memory
sudo swapon swapfile
  • Using the following command you can check that this new swap is enabled:
free -h

Hope it helps.

Hi @davidg
I tried to increase the following php-fpm variables in opt/bitnami/php/etc/php-form.d/www.conf file.
php_value_error_reporting: ‘E_ALL’
pm_max_children: 500
pm_max_requests: 2000
pm_min_spare_servers: 1
pm_max_spare_servers: 25
pm_process_idle_timeout: 150
php_value_disable_functions: 0

And then I restarted the apache and php-fpm services too but the CPU is still hitting 100%.

I used the “free -h” command and the following are the results.
total used free shared buff/cache available
Mem: 15Gi 6.5Gi 6.4Gi 1.2Gi 2.6Gi 7.6Gi
Swap: 634Mi 0B 634Mi

Following is the PHP-FPM.log file’s output.

[18-Aug-2021 07:27:04] WARNING: [pool www] child 760 exited on signal 15 (SIGTERM) after 368583.470654 seconds from start
[18-Aug-2021 07:30:13] ERROR: Another FPM instance seems to already listen on /opt/bitnami/php/var/run/www.sock
[18-Aug-2021 07:30:13] ERROR: FPM initialization failed

I am not sure where to go next?

Hi @zobi.gondal.89,

I understand there is a typo in the path you wrote, right? This guide explains where to place those values

https://docs.bitnami.com/general/apps/wordpress/configuration/configure-phpfpm-processes/

Please ensure the syntax in the conf file is the correct one and remember to restart the service once you apply the changes.

Thanks

Hi,
Yeah, that was the typo in the path, sorry about that. ok, I followed the helping link you shared and update
/opt/bitnami/php/etc/php-fpm.d/www.conf file with following option.

pm=ondemand

And restarted all services including php-fpm. But it did not bring the CPU usage down.

So as for second attempt, I tried to locate “/opt/bitnami/php/etc/common-ondemand.conf” but this file is not there. So, I manually created “common-ondemand.conf” file and added following as suggested in the link.

pm=ondemand
pm.max_children=5
pm.start_servers=2
pm.min_spare_servers=1
pm.max_spare_servers=3

I restarted all services but did not work.
www.conf file

[www]
include=/opt/bitnami/php/etc/environment.conf
include=/opt/bitnami/php/etc/common.conf
pm=dynamic
listen=/opt/bitnami/php/var/run/www.sock
listen.allowed_clients=127.0.0.1

; Memory settings adapted to the machine
; The file below will be overwritten after restarts
include=/opt/bitnami/php/etc/memory.conf

pm=ondemand

Hi @jota

I think I am doing something wrong because the CPU usage is still hitting 99%.

Hi @zobi.gondal.89

We will review the documentation. Creating the file will not modify the parameters in the PHP-FPM configuration. You will need to perform these changes:

  • Modify the www.conf file and remove the line you included (pm=ondemand). This is just if you want to use PHP-FPM in “on demand” mode.
  • You just need to configure the parameters in the /opt/bitnami/php/etc/memory.conf file. You can reduce the number of PHP-FPM processes to use to reduce the usage of resources in the machine.

Happy to help!


Was my answer helpful? Click on :heart:

Hi @jota Thanks for your reply.

  • ok, I removed the on-demand from “www.conf” file.
  • I just updated the memory.conf file with the following changes.
    pm.max_children=80
    pm.start_servers=20
    pm.min_spare_servers=15
    pm.max_spare_servers=20
    pm.max_requests=200

And restarted PHP-FPM and then all services but it did not resolve the issue. Just some extra information, the instance type I am using is t3.xlarge.

I am wondering if it’s possible can you take a look at the server via SSH?

Hi @zobi.gondal.89,

If modifying the number of PHP-FPM processes in the instance doesn’t fix the performance issues, I suggest you change the instance type you are using to deploy the application. Please note that you are using burstable instances which performance may change. You can try using the new generation of burstable instances (t3a) or change to one of the fixed performance instances

https://aws.amazon.com/ec2/instance-types/

Let us know if that solves the problem.

Hi @jota
I changed my instance type from t3.xlarge to t3a.xlarge. And as soon as I started the server, it went high in the skies around 94%.

Is it possible if you can share the recommend PHP-FPM settings for Magento 2?

Thanks

Hi @zobi.gondal.89,

that’s a question for the Magento’s developers. Please note that the application requires different configuration depending on the usage (number of connections, users, …). Can you ask in the community forum of Magento to know how you can troubleshoot this performance issue with the application and configure the parameters correctly?

Let us know if you have any questions.

It’s an underdevelopment website and not available publically. Anyway, I’ll see in Magento resources and forums.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.