After Reboot, Apache will not restart

aarosson48 2019-11-14 21:58:56 UTC #1

Keywords: Re:dash - AWS - Technical issue - Other
bnsupport ID: 384eb7e8-1bcf-92d1-0839-4f0989f7e62c
Description:
Bitnami support tool code:
384eb7e8-1bcf-92d1-0839-4f0989f7e62c

After restarting server, I wasn't able to get Re:Dash to come back up.
Getting error :
/home/bitnami# sudo /opt/bitnami/ctlscript.sh restart
Syntax OK
/opt/bitnami/apache2/scripts/ctl.sh : apache not running
/opt/bitnami/apps/redash/scripts/ctl_redash_gunicorn.sh : redash_gunicorn not running
/opt/bitnami/apps/redash/scripts/ctl_redash_celery_worker.sh : redash_celery_worker not running
/opt/bitnami/apps/redash/scripts/ctl_redash_celery_scheduled.sh : redash_celery_scheduled not running
/opt/bitnami/redis/scripts/ctl.sh : redis stopped
waiting for server to shut down.... done
server stopped
/opt/bitnami/postgresql/scripts/ctl.sh : postgresql stopped
waiting for server to start.... done
server started
/opt/bitnami/postgresql/scripts/ctl.sh : postgresql started at port 5432
/opt/bitnami/redis/scripts/ctl.sh : redis started at port 6379
/opt/bitnami/apps/redash/scripts/ctl_redash_gunicorn.sh : redash_gunicorn could not be started
/opt/bitnami/apps/redash/scripts/ctl_redash_celery_worker.sh : redash_celery_worker could not be started
/opt/bitnami/apps/redash/scripts/ctl_redash_celery_scheduled.sh : redash_celery_scheduled could not be started
Syntax OK
/opt/bitnami/apache2/scripts/ctl.sh : httpd could not be started

jota 2019-11-15 08:12:25 UTC #2

Hi @aarosson48,

It seems you changed the SSL keys in your installation in the past and that's the reason why Apache can't be started (please check the /opt/bitnami/apache2/logs/error_log file)

[Thu Nov 14 21:51:14.744195 2019] [ssl:emerg] [pid 5267] AH02565: Certificate and private key localhost:443:0 from /opt/bitnami/apache2/conf/server.crt and /opt/bitnami/apache2/conf/server.key do not match
AH00016: Configuration Failed

Can you take a look at these guides to ensure you copied the correct files?

https://docs.bitnami.com/aws/apps/redash/administration/check-ssl-certificate/
https://docs.bitnami.com/aws/apps/redash/administration/enable-https-ssl-apache/

Happy to help!

Was my answer helpful? Click on

aarosson48 2019-11-15 23:59:27 UTC #3

Thanks, I have added a lets encrypt cert and that has the apache2 running but I still have some errors
/opt/bitnami/apps/redash/scripts/ctl_redash_celery_worker.sh : redash_celery_worker could not be started
/opt/bitnami/apps/redash/scripts/ctl_redash_celery_scheduled.sh : redash_celery_scheduled could not be started
Syntax OK

jota 2019-11-18 10:07:07 UTC #4

Hi @aarosson48,

Could you please share the log files inside the /opt/bitnami/apps/redash/logs folder?

Thanks

aarosson48 2019-11-18 15:44:47 UTC #5

I hope this helps. The site said I can't upload attachments. So I just copy and pasted here. the folder in the error has 755 permissions... also nothings in it. Thinking I should probably just reinstall the software and then just reload the database base and theme.

This is what shows in the log after everything stopped working.
redash_gunicorn.log:
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: Permission denied
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: Permission denied
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: Permission denied

redash_celery_worker.log:
This typically means that you attempted to use functionality that needed
to interface with the current application object in a way. To solve
this set up an application context with app.app_context(). See the
documentation for more information.
[2019-11-14 16:37:48,389][PID:2773][ERROR][ForkPoolWorker-11] Task redash.tasks.refresh_queries[4b311ceb-e279-44e6-885a-aad737533c97] raised unexpected: RuntimeError('Working outside of application context.\n\nThis typically means that you attempted to use functionality that needed\nto interface with the current application object in a way. To solve\nthis set up an application context with app.app_context(). See the\ndocumentation for more information.',)
Traceback (most recent call last):
File "/opt/bitnami/apps/redash/htdocs/venv/lib/python2.7/site-packages/celery/app/trace.py", line 382, in trace_task
R = retval = fun(*args, **kwargs)
File "/opt/bitnami/apps/redash/htdocs/redash/worker.py", line 63, in call
File "/opt/bitnami/apps/redash/htdocs/venv/lib/python2.7/site-packages/werkzeug/local.py", line 343, in getattr
return getattr(self.getcurrent_object(), name)
File "/opt/bitnami/apps/redash/htdocs/venv/lib/python2.7/site-packages/werkzeug/local.py", line 302, in getcurrent_object
return self.__local()
File "/opt/bitnami/apps/redash/htdocs/venv/lib/python2.7/site-packages/flask/globals.py", line 51, in findapp
raise RuntimeError(appctx_err_msg)
RuntimeError: Working outside of application context.

This typically means that you attempted to use functionality that needed
to interface with the current application object in a way. To solve
this set up an application context with app.app_context(). See the
documentation for more information.
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: Permission denied
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: Permission denied
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: Permission denied

redash_celery_scheduled.log:

Restarting celery worker (/opt/bitnami/apps/redash/htdocs/venv/bin/celery worker --pidfile /opt/bitnami/apps/redash/logs/redash_celery_scheduled.pid --app=redash.worker -c2 -Qscheduled_queries --maxtasksperchild=10 -Ofair)

worker: Warm shutdown (MainProcess)

-------------- celery@ip-172-31-7-22 v4.2.1 (windowlicker)
---- **** -----
--- * *** * -- Linux-4.4.0-1087-aws-x86_64-with-debian-stretch-sid 2019-09-10 00:27:54
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app: redash:0x7fac75214850
- ** ---------- .> transport: redis://127.0.0.1:6379/0
- ** ---------- .> results: redis://127.0.0.1:6379/0
- *** --- * --- .> concurrency: 2 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> scheduled_queries exchange=scheduled_queries(direct) key=scheduled_queries

[2019-11-14 15:58:42,794][PID:1721][INFO][root] Latest version: 8.0.0 (newer: True)
[2019-11-14 15:59:01,646][PID:2113][INFO][root] Latest version: 8.0.0 (newer: True)
Stale pidfile exists - Removing it.
[2019-11-14 16:02:17,742][PID:2593][INFO][root] Latest version: 8.0.0 (newer: True)
Stale pidfile exists - Removing it.
[2019-11-14 16:26:19,893][PID:2924][ERROR][ForkPoolWorker-3] Signal handler raised: ImportError('cannot import name authentication',)
Traceback (most recent call last):
File "/opt/bitnami/apps/redash/htdocs/venv/lib/python2.7/site-packages/celery/utils/dispatch/signal.py", line 288, in send
response = receiver(signal=self, sender=sender, **named)
File "/opt/bitnami/apps/redash/htdocs/redash/worker.py", line 73, in init_celery_flask_app
File "/opt/bitnami/apps/redash/htdocs/redash/__init__.py", line 101, in create_app
ImportError: cannot import name authentication
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: not found
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: Permission denied
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: Permission denied
sh: 1: /opt/bitnami/apps/redash/htdocs/bin/run: Permission denied

jota 2019-11-19 10:46:37 UTC #6

Hi @aarosson48,

thanks for the information.

This path doesn't exist in your installation. After taking a second look to the files that the Bitnami support tool provided, it seems you created a htdocs.bak folder inside the /opt/bitnami/apps/redash one. That one contains the bin/run file, however the current htdocs folder you have doesn't and its permissions are incorrect. It seems you deployed redash manually, is this the case? If so, I don't know the changes you apply so you probably broke the installation.

Bitnami provides free all-in-one installers, virtual machines and cloud images for popular open source applications. Get them now!