Health Check Fails regardless of http/s

danmansan 2020-09-07 03:41:09 UTC #1

Keywords: MODX - AWS - Technical issue - Other
Description:
Hello Bitnamers!

I have a few ec2 instances running the bitnami Modx for AWS stack - they've been working fine for years now.

We are using application load balancers to redirect all http traffic to https and then point that traffic to our instance which is listening on 443. No matter what i fiddle within the apache config I can never seem to get a heatlh check to pass (tried both 80 and 443 healthcheck with various paths).

Obviously our http/s traffic is reaching our servers as the web pages are loading correctly- it just seems that the health checks are failing.

There seems to be an old thread related to this that was never fully solved - https://community.bitnami.com/t/aws-lightsail-load-balancer-health-check-fails/71084/9

I would be highly in your debt for whoever knows how to get the health checks to work.We want to add fault tolerant support for our application and without a working health check I seem to be stuck!

Thanks

Ibone 2020-09-07 10:46:20 UTC #2

Hi @danmansan,

Thanks for using Bitnami! Did you follow this guide https://docs.bitnami.com/aws/how-to/configure-elb-ssl-aws/?
If the answer it's 'yes' could you tell us what step are you having trouble with?

Regards,
Ibone.

danmansan 2020-09-10 16:11:55 UTC #3

@Ibone Thanks for helping me out with this. Yes I have followed the guide to setup the elastic load balancer with ssl.

The load balancer is serving the https traffic correctly- the website load with no issue.

However inside the AWS console - for the target group that the load balancer is forwarding traffic to- it shows the status as unhealthy- see the sceenshot below:

regardless if I use a target group on 443 or 80 the result is the same.

Ibone 2020-09-11 09:03:45 UTC #4

Hi @danmansan,

Could you check this link https://aws.amazon.com/premiumsupport/knowledge-center/troubleshoot-classic-health-checks/?

On this page appears this:

Redirection is not configured on the backend. Redirection configured on the backend can result in a 301 or 302 response code, resulting in failed health checks. For example, if you have a redirection from HTTP:80 to HTTPS:443 on the backend, then HTTP health checks on port 80 will fail, unless you change the health check to HTTPS and health check port to 443. Note: You can simulate the health check request sent by the load balancer using the curl -Ivk http command from an instance in the subnet that is associated with the load balancer.

Could be your case?

Regards,
Ibone,

danmansan 2020-09-15 07:37:05 UTC #5

@Ibone

I have checked the link - here is some more information:

-The health check is using https protocol on 443.
-When I change the health check success code to 301 - it starts to show as healthy- this does not feel like a good way to use the health check so I continue with how to setup the bitnami.conf to correctly return code 200 for the health check.

-When I run the curl -lvk command with the following results:

Trying X.X.X.X...

Connected to X.X.X.X (X.X.X.X) port 443 (#0)

ALPN, offering http/1.1

Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH

error setting certificate verify locations, continuing anyway:

CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none

TLSv1.2 (OUT), TLS header, Certificate Status (22):

TLSv1.2 (OUT), TLS handshake, Client hello (1):

TLSv1.2 (IN), TLS handshake, Server hello (2):

TLSv1.2 (IN), TLS handshake, Certificate (11):

TLSv1.2 (IN), TLS handshake, Server key exchange (12):

TLSv1.2 (IN), TLS handshake, Server finished (14):

TLSv1.2 (OUT), TLS handshake, Client key exchange (16):

TLSv1.2 (OUT), TLS change cipher, Client hello (1):

TLSv1.2 (OUT), TLS handshake, Finished (20):

TLSv1.2 (IN), TLS change cipher, Client hello (1):

TLSv1.2 (IN), TLS handshake, Finished (20):

SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384

ALPN, server accepted to use http/1.1x

Server certificate:

subject: OU=Domain Control Validated; OU=PositiveSSL; CN=www.devblockdrop.tech

start date: Sep 11 00:00:00 2019 GMT

expire date: Dec 9 23:59:59 2020 GMT

issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA

SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.

HEAD /health-check HTTP/1.1
Host: X.X.X.X
User-Agent: curl/7.45.0
Accept: /

< HTTP/1.1 301 Moved Permanently
HTTP/1.1 301 Moved Permanently
< Date: Tue, 15 Sep 2020 07:04:19 GMT
Date: Tue, 15 Sep 2020 07:04:19 GMT
< Server: Apache
Server: Apache
< X-Frame-Options: SAMEORIGIN
X-Frame-Options: SAMEORIGIN
< Location: 'https://www.X.X.X.X/health-check
Location: 'https://www.X.X.X.X/health-check
< Content-Type: text/html; charset=iso-8859-1
Content-Type: text/html; charset=iso-8859-1

-AS you can see above, the issue is the HTTP/1.1 301 Moved Permanently which is causing the heath check to fail when the success code is set to 200.

-so i found this article to setup a bypass for the url:

using this:

RewriteCond %{HTTP_HOST} !^www.example.com$ [NC]
RewriteCond %{REQUEST_URI} !^/health-check$
RewriteRule ^ 'http://www.example.com/$1 [R=301,L]

and setting the health check path to /health-check

unfortunately, this does not seem to work for me- i cannot get the code 200 for my healthcheck at path /health-check using both http and https types of health check- the curl command still returns the 301 result

perhaps there is something missing in my config to get the bypass to work

Ibone 2020-09-15 09:59:21 UTC #6

Hi @danmansan,

The load balancer and the instance will be connected by IP otherwise the load balancer uses this rule:

And check the force of https you can check this guide https://docs.bitnami.com/aws/how-to/configure-elb-ssl-aws/#step-5-force-https

Regards,
Ibone.

danmansan 2020-09-16 04:49:05 UTC #7

thanks - actually I finally realized the issue when you pointed out the last mention -

RewriteRule ^ 'http://www.example.com/$1 [R=301,L]

the ending bit > [R=301,L] is where that error was coming from -

I had to go through the bitnami.conf and remove any rewrite rule that contained a 301 condition in it - there werea few im susupecting from previous redirects rules left over from a time before the load balancer which solved the problem- our load balancer already redirects all traffic fro 80 to 443 so it is not needed and caused failing health checks on 443.

thanks!

Ibone 2020-09-16 07:12:14 UTC #8

Hi @danmansan,

I'm glad the problem is solved. I close the topic. Do not hesitate to write us back if you have any other questions.

Regards,
Ibone.

Bitnami provides free all-in-one installers, virtual machines and cloud images for popular open source applications. Get them now!