Bitnami wordpress site performance issue

shiv.c 2021-01-25 06:43:06 UTC #1

Keywords: WordPress - AWS - Technical issue - Other
Description:
We are having performance issues on our Bitnami WordPress instance with woocommerce having a little over 500k products, where we notice extremely long server response times (~60 seconds or sometimes even a gateway timeout).

Upon running show processlist; command in mysql console, we found a task named "event_scheduler" which is always running in the "Waiting on empty queue" state.

As recommended, we also have the W3 total cache plugin installed on WordPress with the pagecache enabled with the Disk:Enhanced setting. Other cache options are also enabled with the Disk setting.

We have run the bnsupport-tool on both sites but the tool was unable to upload the results, I have therefore uploaded them to google drive and sent the link as a reply on the welcome private message with beltran.

davidg 2021-01-25 10:49:15 UTC #2

Hello @shiv.c,

Have you checked our optimization guide for WP:
https://docs.bitnami.com/aws/apps/wordpress/troubleshooting/optimize-bitnami-wordpress/

Apart from that, could you share your support output with us again in the private message I sent to you.

Regards

shiv.c 2021-01-25 11:59:32 UTC #3

Hello @davidg

Thank you for your reply.

I have sent the link to you in the private message. Regarding the optimization guide, we are using the W3 Total Cache plugin with the page speed cache enabled and set to disk:enhanced. Are the other mentioned plugins such as WP Super Cache, Quick Cache and Minify to be installed in addition to W3 Total Cache?

Upon using the top command, I see the iowait (wa) parameter is extremely high, >90 consistently.

davidg 2021-01-26 10:28:03 UTC #4

Hello @shiv.c,

I can see that the Apache log is full of lines related to PageSpeed. Could you try disabling PageSpeed as shown here and seeing if the situation improves?:

https://docs.bitnami.com/aws/apps/wordpress/administration/use-pagespeed/#disable-pagespeed

Don't forget to restart your Apache service after disabling it.

I hope it helps.

shiv.c 2021-01-27 13:12:14 UTC #5

@davidg thank you for your reply.

I have disabled PageSpeed as shared in the link above. I will observe to see if this causes any improvements.

Meanwhile, there is another issue I noticed on the mysql terminal. On running show processlist; command, there are two queries which execute for an extended period of time.
I am unsure of the nature of their sensitivity, I have, therefore, shared a screenshot with you on private message. Could these be affecting performance?

gongomgra 2021-01-28 10:48:10 UTC #6

Hi @shiv.c,

Can you check it with the WordPress developers in their official forums? They may be able to tell you what are those requests and why they are running them.

https://wordpress.org/support/forums/

Additionally, can you check our guide for troubleshooting server performance and share the results with us?

https://docs.bitnami.com/aws/faq/troubleshooting/troubleshoot-server-performance/

Apart from the above, we have a Support Tool that will gather relevant information for us to analyze your configuration and logs. Could you please execute it on the machine where the stack is running by following the steps described in the guide below?

How to Run the Bitnami Support Tool in a cloud image or virtual machine

Please note that you need to paste the code ID that is shown at the end.

shiv.c 2021-01-29 13:55:45 UTC #7

@gongomgra Thank you for your reply.

I have uploaded the results of these commands to google drive and shared with you through private message.

I already ran the bn support tool and shared the output with @davidg through google drive since the tool was unable to upload the results itself. I have shared the same google drive link with you in a private message.

shiv.c 2021-02-06 06:25:39 UTC #8

I am facing this issue again where the website is not opening. Tried the df -i command and the output is as shown below.

It shows 100% usage of Inodes. How can I find which process is responsible for this and why this is happening. How can I resolve this issue?

shiv.c 2021-02-11 11:43:19 UTC #9

Hi,
Waiting to hear back on this issue. Do we need to increase server disk space? That too, however, I don't see as a permanent solution since I need to find the reason for these inodes getting used up.

gongomgra 2021-02-11 12:48:38 UTC #10

Hi @shiv.c,

Sorry for not getting back to you earlier, but we had some issues in our system and we lost track of your ticket.

The inodes store metadata for every file on your system. If your filesystem is full of small files, it will consume all the available inodes on your server rapidly, even if you have free room on your disk. Can you check if this is your case?

shiv.c 2021-02-12 10:40:51 UTC #11

Hi @gongomgra,

Yes, this is the case in our system. Our disk has plenty of free space but inodes are full indicating a lot of small files. This is shown in the image below:

The Inode usage is 100% but disk usage is only 39%. I have tried commands to locate small files however due to 100% inode usage, I get an error saying there is no space on the disk to create files in /tmp for the commands to run.

Please advise on where I can look for small files in order to delete them and how I can resolve this issue.

jsalmeron 2021-02-12 11:48:39 UTC #12

Hi,

You could go the /opt/bitnami folder and execute this snippet (there are better ways to do it but just came up with this fast dirty solution)

for i in $(ls); do if [[ -d $i ]]; then echo "$i : $(find $i -type f | wc -l)"; fi; done

This should list the folders and the number of files. This could give us a hint on where the biggest amount is.

shiv.c 2021-02-12 12:48:28 UTC #13

Hi @jsalmeron, Thank you for your reply. I believe the command is missing a "done" at the end. I ran the following updated command, please let me know if the correction I made to the command is fine.

for i in $(ls); do if [[ -d $i ]]; then echo "$i : $(find $i -type f | wc -l)"; fi; done

It is running right now and I will update the result here as soon as it is complete.

jsalmeron 2021-02-12 14:46:04 UTC #14

Hi,

Yes sorry, I missed that when copy-pasting

shiv.c 2021-02-12 14:51:09 UTC #15

Just got the results back as shown in the image below

/opt/bitnami

I ran the same command inside the apps directory since it seems that has an abnormally high number of files, will post the results when ready

Low performance and frequent periodic gateway timeouts while loading website

shiv.c 2021-02-15 08:02:10 UTC #16

I have been able to pinpoint the location of the directory with a large number of files using that command shared above. Following is the result:

Ran the command inside /opt/bitnami/apps/wordpress/htdocs/wp-content/cache:

I use W3ToalCache plugin on the website and have enabled all the caches which are available with the free plan and set all to disk:enhanced. We have around 500K products on the website.

miruiz 2021-02-16 10:32:29 UTC #17

Hi @shiv.c,

Looks like the inode issue is caused by the W3 Total Cache configuration and we may not be able to give you the best assistance in this case.

Maybe by reaching the W3 Total Cache team using their Support Forum (https://wordpress.org/support/plugin/w3-total-cache/) they may be able to help you resolve this issue.

I have searched for similar issues and this seems to be a frequent problem for deployments exceeding a certain size, but looks like there is not a unique solution and it may vary depending on your requirements. E.g. Purging cache or using a Memory cache service such as Memcached.

I hope it helps.

shiv.c 2021-02-18 12:10:55 UTC #18

@miruiz thank you for your reply. I will check out the links shared by you and contact the W3 Total Cache team using their Support Forum.

miruiz 2021-02-23 16:46:18 UTC #19

I'm happy to help.

If you need any further assistance don't hesitate to let us know.

Bitnami provides free all-in-one installers, virtual machines and cloud images for popular open source applications. Get them now!