So here is the forum I found... https://community.bitnami.com/t/sudden-503-service-unavailable-error-for-wordpress-multisite-3-8-1-1-on-m3-medium-ec2/23468
The reason why I knew the APC was the culprit was because I kept restarting the servers, happened on both VM's, and it would resolve for about a half an hour or so. Then it would resort to spinning out the servers again. They just slowly hang.
So I did the Xcache thing and it added the caching service and cleared up the issue for the most part.
the settings for this don't seem too clear at all though of how and where to make any adjustments.
What I notice now though is that the servers work fine but if a change is made to a group of files, usually those being php because we have a bunch of wordpress sites, the servers can hang for a bit before the "calm / settle" in and then everything seems to be fine.
A couple weeks though this turned into a disaster as I couldn't calm them down, didn't think to stop the xcache, and it took about a day for the servers to resolve themesleves.
I turned the VM's up to 16 cores and they still wouldn't budge. nothing in htop or logs to state any issues or problems.
Now, my servers are on whisper mode and instead of the 16 core 25% - 35% average cpu they are around 3.5 - 6.5% cpu usage.
So again, what do I do about the xcache or caching in general to know and fix any problems in the future. I would like to go back to 8 cores now as this seems feasible.