Varnish not working on https pages

Keywords: PrestaShop - Google Cloud Platform - Technical issue - Other
bnsupport ID: 2048631f-baaf-64b7-2190-86aa105d8fd0
Description:
I’ve followed the tutorial in https://docs.bitnami.com/aws/apps/wordpress/administration/configure-use-varnish/#use-varnish-tm-with-Sal and I’ve done exactly what’s instructed but to no effect at all. Varnish only works on http pages, on https it just goes directly to apache.

I’m using varnish on port 80 so I’ve tried to set ProxyPass and ProxyPassReverse to 127.0.0.1:80 instead of 127.0.0.1:81.

I’ve been hours apon hours around this issue, and haven’t found any solution.

Sorry I got the tutorial link wrong, the correct link is: https://docs.bitnami.com/aws/apps/wordpress/administration/configure-use-varnish/#use-varnish-tm-with-ssl

I inserted in the /opt/bitnami/apache2/conf/bitnami/bitnami-ssl.conf (Approach A) file:

...
ProxyPreserveHost On
RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
ProxyPass "/"  "http://127.0.0.1:80/"
ProxyPassReverse "/"  "http://127.0.0.1:80/"
...

``
Varnish is in port 80, the port the website is being served on, but it’s not working from what I see using the commands “varnishlog” and “varnishstat”.

Hi @jlago3577,

I just accessed your site and it seems it’s working properly

I also took a look at the configuration files and everything seems to be properly updated too.

Let me know if you need further help.

Thanks

Varnish isn’t working.
When I use varnishlog, only on http pages does it output.
And varnishstat is like shown in the image below:

Hi @jlago3577,

Please note that you are using Cloudflare and it caches the requests the different users make so it doesn’t need to connect to your server every time there is a new request.

If you want to connect to your instance directly and test your website, you can modify you host file and include your domain and the IP of your instance and access it using your browser.

https://docs.rackspace.com/support/how-to/modify-your-hosts-file/

I’ve added to my /etc/hosts file the website’s IP followed by it’s url, but connecting to the website still does not output anything from varnishlog, and actually it never outputs anything for as long as I leave it and browse around the website.
Using varnishstat I can see the Hitrate is 0 on MAIN.backend_reuse, and varnish has been running non stop for more than a day, with the website receiving more than a thousand visits every day.

Running sudo netstat -nplt outputs, along with other info, the follwing:

tcp6 0 0 :::80 :::* LISTEN varnishd
tcp6 0 0 :::81 :::* LISTEN httpd
tcp6 0 0 :::443 :::* LISTEN httpd

So apache is serving on ports 81 and 443, and varnish on port 80 which is where the website is being served on.

Thanks for the reply. :slight_smile:

Hi @jlago3577,

You can verify if Varnish is serving your requests by running these commands in the command line

curl -LI http://yourdomainhere.com
curl -LI https://yourdomainhere.com

When I run curl -LI http://www.codistec.com/pt, it outputs HTTP/1.1 302 Found and redirects to the https version. On the server with varnishlog running, it outputs

-   Begin          bereq 32793 fetch
-   Timestamp      Start: 1610976347.890111 0.000000 0.000000
-   BereqMethod    HEAD
-   BereqURL       /pt/
-   BereqProtocol  HTTP/1.1
-   BereqHeader    Host: www.codistec.com
-   BereqHeader    User-Agent: curl/7.68.0
-   BereqHeader    Accept: */*
-   BereqHeader    X-Forwarded-For: 188.83.84.214
-   BereqMethod    GET
-   BereqHeader    Accept-Encoding: gzip
-   BereqHeader    X-Varnish: 32794
-   VCL_call       BACKEND_FETCH
-   VCL_return     fetch
-   BackendOpen    24 boot.default 127.0.0.1 81 127.0.0.1 53882
-   BackendStart   127.0.0.1 81
-   Timestamp      Bereq: 1610976347.890424 0.000313 0.000313
-   Timestamp      Beresp: 1610976347.890747 0.000636 0.000323
-   BerespProtocol HTTP/1.1
-   BerespStatus   302
-   BerespReason   Found
-   BerespHeader   Date: Mon, 18 Jan 2021 13:25:47 GMT
-   BerespHeader   Server: Apache/2.4.46 (Unix) OpenSSL/1.1.1d
-   BerespHeader   Location: https://www.codistec.com/pt/
-   BerespHeader   Content-Length: 212
-   BerespHeader   Content-Type: text/html; charset=iso-8859-1
-   TTL            RFC -1 10 -1 1610976348 1610976348 1610976347 0 0
-   VCL_call       BACKEND_RESPONSE
-   BerespUnset    Location: https://www.codistec.com/pt/
-   BerespHeader   Location: https://www.codistec.com/pt/
-   TTL            VCL -1 3600 0 1610976348
-   VCL_return     deliver
-   Storage        malloc s0
-   ObjProtocol    HTTP/1.1
-   ObjStatus      302
-   ObjReason      Found
-   ObjHeader      Date: Mon, 18 Jan 2021 13:25:47 GMT
-   ObjHeader      Server: Apache/2.4.46 (Unix) OpenSSL/1.1.1d
-   ObjHeader      Content-Length: 212
-   ObjHeader      Content-Type: text/html; charset=iso-8859-1
-   ObjHeader      Location: https://www.codistec.com/pt/
-   Fetch_Body     3 length stream
-   BackendReuse   24 boot.default
-   Timestamp      BerespBody: 1610976347.890848 0.000737 0.000101
-   Length         212
-   BereqAcct      156 0 156 210 212 422
-   End
*   << Request  >> 32793
-   Begin          req 32792 rxreq
-   Timestamp      Start: 1610976347.889994 0.000000 0.000000
-   Timestamp      Req: 1610976347.889994 0.000000 0.000000
-   ReqStart       188.83.84.214 52070
-   ReqMethod      HEAD
-   ReqURL         /pt/
-   ReqProtocol    HTTP/1.1
-   ReqHeader      Host: www.codistec.com
-   ReqHeader      User-Agent: curl/7.68.0
-   ReqHeader      Accept: */*
-   ReqHeader      X-Forwarded-For: 188.83.84.214
-   VCL_call       RECV
-   ReqHeader      Cookie:
-   ReqUnset       Cookie:
-   ReqHeader      Cookie:
-   ReqUnset       Cookie:
-   ReqHeader      Cookie:
-   ReqUnset       Cookie:
-   ReqHeader      Cookie:
-   ReqUnset       Cookie:
-   ReqHeader      Cookie:
-   ReqUnset       Cookie:
-   ReqHeader      Cookie:
-   ReqUnset       Cookie:
-   ReqHeader      Cookie:
-   ReqUnset       Cookie:
-   ReqHeader      Cookie:
-   ReqUnset       Cookie:
-   ReqHeader      Cookie:
-   ReqUnset       Cookie:
-   ReqHeader      Cookie:
-   ReqUnset       Cookie:
-   ReqHeader      Cookie:
-   ReqUnset       Cookie:
-   VCL_return     hash
-   VCL_call       HASH
-   VCL_return     lookup
-   VCL_call       MISS
-   VCL_return     fetch
-   Link           bereq 32794 fetch
-   Timestamp      Fetch: 1610976347.890844 0.000850 0.000850
-   RespProtocol   HTTP/1.1
-   RespStatus     302
-   RespReason     Found
-   RespHeader     Date: Mon, 18 Jan 2021 13:25:47 GMT
-   RespHeader     Server: Apache/2.4.46 (Unix) OpenSSL/1.1.1d
-   RespHeader     Content-Length: 212
-   RespHeader     Content-Type: text/html; charset=iso-8859-1
-   RespHeader     Location: https://www.codistec.com/pt/
-   RespHeader     X-Varnish: 32793
-   RespHeader     Age: 0
-   RespHeader     Via: 1.1 varnish-v4
-   VCL_call       DELIVER
-   RespHeader     X-Cache: MISS
-   RespUnset      Server: Apache/2.4.46 (Unix) OpenSSL/1.1.1d
-   RespUnset      Via: 1.1 varnish-v4
-   VCL_return     deliver
-   Timestamp      Process: 1610976347.890864 0.000870 0.000021
-   Debug          "RES_MODE 0"
-   RespHeader     Connection: keep-alive
-   Timestamp      Resp: 1610976347.890896 0.000902 0.000032
-   ReqAcct        84 0 84 230 0 230
-   End

*   << Session  >> 32792
-   Begin          sess 0 HTTP/1
-   SessOpen       188.83.84.214 52070 :80 10.132.0.3 80 1610976347.889942 21
-   Link           req 32793 rxreq
-   SessClose      REM_CLOSE 0.143
-   End

When I do curl -LI https://www.codistec.com/pt, I get the warning;

curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

Because I’m following this https://docs.rackspace.com/support/how-to/modify-your-hosts-file/.
If I remove the server IP and url from /etc/hosts, I get served the cloudflare cache:

HTTP/2 200 
date: Mon, 18 Jan 2021 13:35:04 GMT
content-type: text/html; charset=utf-8
...
server: cloudflare
...

The varnishstat Main.backend_reuse is 1, and Main.cache_hit is 3 after 10 hours of the server running and receiving visitors.

And varnishlog doesn’t output anything from this command.

That’s an error related with the certificate. Can you add the -k flag to the command?

curl -LIk https://www.codistec.com/pt

Running curl -LIk https://www.codistec.com/pt I got:

HTTP/2 200 
date: Tue, 19 Jan 2021 13:51:52 GMT
server: Apache/2.4.46 (Unix) OpenSSL/1.1.1d
x-powered-by: PHP/7.3.26
expires: Thu, 19 Nov 1981 08:52:00 GMT
cache-control: no-store, no-cache, must-revalidate
pragma: no-cache
set-cookie: ...; PHPSESSID=ge1m5lo2ca5tfvjarm2e7t8asa; path=/
set-cookie: ...; expires=Mon, 08-Feb-2021 13:51:52 GMT; Max-Age=1728000; path=/; domain=www.codistec.com; secure; HttpOnly
set-cookie: ; expires=Mon, 08-Feb-2021 13:51:52 GMT; Max-Age=1728000; path=/; domain=www.codistec.com; secure; HttpOnly
content-type: text/html; charset=utf-8

Varnishlog didn’t output.

6th day of trying to solve this… man

I’ve tried adding

ProxyPreserveHost       On
ProxyPass               / http://127.0.0.1:80/
ProxyPassReverse        / http://127.0.0.1:80/

on apache/conf/vhosts/prestashop-https-vhost.conf instead of /apache/conf/bitnami/bitnami-ssl.conf.
Now, running curl -LIk https://MYWEBSITE.com I’m getting HTTP/1.1 302 Found in a loop :

X-Cache: MISS
Vary: X-Forwarded-Proto
HTTP/1.1 302 Found
Date: Tue, 19 Jan 2021 19:15:59 GMT
Server: Apache/2.4.46 (Unix) OpenSSL/1.1.1d
Content-Length: 208
Content-Type: text/html; charset=iso-8859-1
Location: https://www.codistec.com/en
X-Varnish: 23
Age: 0
X-Cache: MISS
Vary: X-Forwarded-Proto

HTTP/1.1 302 Found
Date: Tue, 19 Jan 2021 19:15:59 GMT
Server: Apache/2.4.46 (Unix) OpenSSL/1.1.1d
Content-Length: 208
Content-Type: text/html; charset=iso-8859-1
Location: https://www.codistec.com/en
X-Varnish: 25
Age: 0
X-Cache: MISS
Vary: X-Forwarded-Proto

HTTP/1.1 302 Found
Date: Tue, 19 Jan 2021 19:15:59 GMT
Server: Apache/2.4.46 (Unix) OpenSSL/1.1.1d
Content-Length: 208
Content-Type: text/html; charset=iso-8859-1
Location: https://www.codistec.com/en
X-Varnish: 27
Age: 0
X-Cache: MISS
Vary: X-Forwarded-Proto

HTTP/1.1 302 Found
Date: Tue, 19 Jan 2021 19:15:59 GMT
Server: Apache/2.4.46 (Unix) OpenSSL/1.1.1d
Content-Length: 208
Content-Type: text/html; charset=iso-8859-1
Location: https://www.codistec.com/en
X-Varnish: 29
Age: 0
X-Cache: MISS

Every solution online to this problem involves setting the command in the external address (433) conf:

RequestHeader set X-Forwarded-Proto expr=%{REQUEST_SCHEME}

and in the .htaccess:

SetEnvIf X-Forwarded-Proto https HTTPS=on

But this doesn’t solve the issue.

FINALLY!
Its working…

I had to insert in /opt/bitnami/apache2/conf/vhosts/prestashop-https-vhost.conf:

ProxyPreserveHost On
RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
ProxyPass "/"  "http://127.0.0.1:80/"
ProxyPassReverse "/"  "http://127.0.0.1:80/"

It doesn’t work if the code goes in /opt/bitnami/apache2/conf/bitnami/bitnami-ssl.conf.
AND
To stop it from looping 302s, I had to add RewriteCond %{HTTP:X-Forwarded-Proto} !https in /opt/bitnami/apache2/conf/vhosts/prestashop-vhost.conf

RewriteEngine On
>>RewriteCond %{HTTP:X-Forwarded-Proto} !https<<
RewriteCond %{HTTPS} !=on
RewriteCond %{HTTP_HOST} !^(localhost|127.0.0.1)
RewriteRule ^/(.*) https://%{SERVER_NAME}/$1 [R,L]

Could you verify if these changes are alright for a production environment please? @jota
I’m not very familiar with this.

Every HTTPS request now outputs from varnishlog, however I’m finding it strange how all of them show RespHeader X-Cache: MISS.

*   << Request  >> 294917
    -   Begin          req 294916 rxreq
    -   Timestamp      Start: 1611105254.360055 0.000000 0.000000
    -   Timestamp      Req: 1611105254.360055 0.000000 0.000000
    -   ReqStart       127.0.0.1 53578
    -   ReqMethod      GET
    -   ReqURL         /en/module/productcomments/ListComments?id_product=790&page=1
    -   ReqProtocol    HTTP/1.1
    -   ReqHeader      Host: www.jlago.xyz
    -   ReqHeader      Sec-Ch-Ua: "Google Chrome";v="87", " Not;A Brand";v="99", "Chromium";v="87"
    -   ReqHeader      Accept: */*
    -   ReqHeader      Dnt: 1
    -   ReqHeader      X-Requested-With: XMLHttpRequest
    -   ReqHeader      Sec-Ch-Ua-Mobile: ?0
    -   ReqHeader      User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36
    -   ReqHeader      Sec-Fetch-Site: same-origin
    -   ReqHeader      Sec-Fetch-Mode: cors
    -   ReqHeader      Sec-Fetch-Dest: empty
    -   ReqHeader      Referer: https://www.jlago.xyz/en/thermometers/790-capillary-thermometer-0-to-120-c-200cm.html
    -   ReqHeader      Accept-Encoding: gzip, deflate, br
    -   ReqHeader      Accept-Language: pt-PT,pt;q=0.9,en-US;q=0.8,en;q=0.7,es;q=0.6
    -   ReqHeader      Cookie: __cfduid=da5e531f646710b934392f4ed4da625bd1610978880; PrestaShop-ee8fc2dd028879d4e3b66e9dcfd618bf=def50200e0eb39836087acaf66cf3b18c49578b186e6e91124003284a60a8206a1b3308cf80c1b9344dcbeb361125ec25dae4ab8bbed7e6575178ffdc8a5ca1d0daa47f39785fc2aa67e
    -   ReqHeader      X-Forwarded-Proto: https
    -   ReqHeader      X-Forwarded-For: 188.83.84.214
    -   ReqHeader      X-Forwarded-Host: www.jlago.xyz
    -   ReqHeader      X-Forwarded-Server: www.jlago.xyz
    -   ReqHeader      Connection: Keep-Alive
    -   ReqUnset       X-Forwarded-For: 188.83.84.214
    -   ReqHeader      X-Forwarded-For: 188.83.84.214, 127.0.0.1
    -   VCL_call       RECV
    -   ReqUnset       Accept-Encoding: gzip, deflate, br
    -   ReqHeader      Accept-Encoding: gzip
    -   VCL_return     pass
    -   VCL_call       HASH
    -   VCL_return     lookup
    -   VCL_call       PASS
    -   VCL_return     fetch
    -   Link           bereq 294918 pass
    -   Timestamp      Fetch: 1611105254.463457 0.103401 0.103401
    -   RespProtocol   HTTP/1.1
    -   RespStatus     200
    -   RespReason     OK
    -   RespHeader     Date: Wed, 20 Jan 2021 01:14:14 GMT
    -   RespHeader     Server: Apache/2.4.46 (Unix) OpenSSL/1.1.1d
    -   RespHeader     X-Powered-By: PHP/7.3.26
    -   RespHeader     Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
    -   RespHeader     Content-Type: application/json
    -   RespHeader     X-Varnish: 294917
    -   RespHeader     Age: 0
    -   RespHeader     Via: 1.1 varnish-v4
    -   VCL_call       DELIVER
    -   **RespHeader     X-Cache: MISS**
    -   RespUnset      X-Powered-By: PHP/7.3.26
    -   RespUnset      Server: Apache/2.4.46 (Unix) OpenSSL/1.1.1d
    -   RespUnset      Via: 1.1 varnish-v4
    -   VCL_return     deliver
    -   Timestamp      Process: 1611105254.463506 0.103451 0.000050
    -   RespHeader     Accept-Ranges: bytes
    -   RespHeader     Content-Length: 55
    -   Debug          "RES_MODE 2"
    -   RespHeader     Connection: keep-alive
    -   Timestamp      Resp: 1611105254.463541 0.103485 0.000034
    -   ReqAcct        3191 0 3191 275 55 330
    -   End

Hi @jlago3577,

Thanks for the feedback. I’ll notify the documentation team to update the guide accordingly. I didn’t notice you needed to add the configuration to the prestashop-https-vhost.conf file.

What happens if you retry the query several times? Do you always get a cache miss?

It’s most likelly normal for it to get MISSs in regular pages.

In https://dh42.com/blog/fastest-prestashop/ is says:

What this means is that Varnish is not caching your page files. You do not want Varnish to cache them, the way that Prestashop and other e-commerce programs work it will cause problems if the pages are cached. What you are wanting is for the static files to be cached. Try entering one of your image file addresses in the box on isvarnishworking.com and it should look like the image below.

If I do curl -LIk https://www.codistec.com/595-large_default/suportes-para-ar-condicionado-490x370.jpg for example I get a x-cache: HITx-cache: HIT

Well… I turned off varnish again. It’s giving me Error 500 from time to time, the apache logs don’t report the error.
I’ll look into it later.

The 500 errors weren’t being reported on the apache file, so I thought it was because of varnish.
I think it was due to cookie size, I added -p workspace_client=128k to the varnish args in run.sh and it’s working now.

However I’m sometimes getting error 403 if I load many pages, even if I turn off mod_evasive. Are there any settings for DDOS protection in varnish or apache other than the evasive module? @jota

It’s sometimes failing to load images because I get those error 403… varnish is really problematic.

Its not related to htaccess rules. I can reload the page and the images show up.