@michiel Updating pagespeed to the latest version was unsuccessful, I also updated my virtualhost settings to return a 404 for outside hosts which was also unsuccessful. It did what it was supposed to, but pagespeed was still caching ourside urls.
In the end, I was able to modify the pagespeed conf file to include the following two lines:
This seems to be doing the trick for now. The documentation for this can be found here:
I'm not even sure that these cache folders would have ever caused a problem, but it wasn't a risk I wanted to take. Thanks for your help in researching this issue.