#1 By: DCollins, June 7th, 2012 13:54
I am using the BitNami Redmine stack 2.0.1 on Windows Server 2008 and around every two to three hours, Redmine will return a "503 service temporarily unavailable error" when trying to access it. The only way to restore Redmine is to restart the redmineThin1 and redmineThin2 services.
If I try to access Redmine while it's down, these errors appear in \BitNami Redmine Stack\apache2\logs\error.log
[Thu Jun 07 12:10:35 2012] [error] (OS 10061)No connection could be made because the target machine actively refused it. : proxy: HTTP: attempt to connect to 127.0.0.1:3001 (127.0.0.1) failed
[Thu Jun 07 12:10:35 2012] [error] ap_proxy_connect_backend disabling worker for (127.0.0.1)
It appears that ports 3001 and 3002 correspond to the redmineThin1 and redmineThin2 services. If one service isn't restarted, then it will continue to throw this error even if Redmine is accessible. The Redmine production.log doesn't show anything about this and reconnect is set to true in the database.yml file.
Does anyone know how I can stop these services from failing or locking up? Also, does anyone know what the log files for the redmineThin services are located? Any help with this would be greatly appreciated.
#2 By: Beltran Rueda Borrego, June 8th, 2012 12:09
It is strange. The Thin log files are into the apps/redmine/htdocs/log folder. Could you try to start it manually instead of using the services?
Open the "Use Redmine" console from the shorcuts
> cd C:\\Program Files\\BitNami Redmine Stack\\apps\\redmine\\htdocs
> bundle exec thin start -e production -p 3001 --prefix /redmine
Check if you see any error in the output. Did you install any plugin?
#3 By: DCollins, June 8th, 2012 15:06
Thank you for responding. Whenever I try to run the commands you give me, I get this error
ruby.exe: illegal switch in RUBYOPT: -F (RuntimeError)
But, I have run the services manually by using (with either ports 3001 or 3002):
C:\\Program Files\\BitNami Redmine Stack\\ruby\\bin>thin -V -D start -p 3002 -e prouction -c "C:\\Program Files\\BitNami Redmine Stack/apps/redmine/htdocs" --prefix/redmine
I turn on tracing and debugging with -V and -D. There were no error messages displayed in the console when the service was running. There are no thin log files in my apps/redmine/htdocs/log folder and I don't have any plugins installed. Is writing log output to a file disabled somewhere for the thin services?
Also since I've manually started the redmineThin1 service today around 10:16 it hasn't locked up yet, but today isn't as busy in the office so maybe that has something to do with it. If manually starting them DOES prevent locking, then I would never be able to see the errors. What I need are the files that the thin services write to or at least a way to enable logging to a file. The only files in my htdocs/log are production.log and delete.me. Is there something that I can do to logging.rb? Thank you.
#4 By: Beltran Rueda Borrego, June 9th, 2012 05:20
It does not prevent the locking it is just to see what kind of error do you have. If the error is related to the server load, you can try to install two more thin servers. Edit the apps/redmine/script/serviceinstall.bat file and include two lines more changing the port. Then run the script from a command prompt "serviceinstall.bat REMOVE" and "serviceinstall.bat INSTALL". It is also necessary to include the new servers in the apps/redmine/conf/redmine.conf file and restart the Apache server. I hope it helps.
#5 By: Luigi_Sorangelo, June 11th, 2012 04:22
Hi, I have the same problem on a PC running Windows Vista, when Redmine is unused for some time to the service Thin fails to start, I tried to reinstall the service but there was no change.
#6 By: Pedro_Calvo, June 11th, 2012 07:58
Same issue Redmine stack 2.0.0 on Windows Server 2008 R2 Standard
Does it have to do with Apache bugs #39499 and #37770?
We are trying this configuration:
SetEnv force-proxy-request-1.0 1
SetEnv proxy-nokeepalive 1
#7 By: kaysa, June 11th, 2012 09:39
#8 By: Beltran Rueda Borrego, June 11th, 2012 15:42
These are very old issues, the Stack uses Apache 2.2.22 version.
#9 By: mharen, June 11th, 2012 22:10
I work with @DCollins and just discovered the site had locked up. I restarted just the Apache service. This didn't seem to restart the Thins...but it worked. I was surprised. Does that help to possibly point us back at Apache?
#10 By: houstek, June 12th, 2012 08:36
I think I have the same problem as @DCollins but witch fresh install of BitNami Redmine stack 2.0.2 on w2k3. Since yesterday, the server has not responded three times
in Apache error.log
[Tue Jun 12 09:59:47 2012] [error] (OS 10061)No connection could be made because the target machine actively refused it. : proxy: HTTP: attempt to connect to 127.0.0.1:3001 (127.0.0.1) failed
[Tue Jun 12 09:59:47 2012] [error] ap_proxy_connect_backend disabling worker for (127.0.0.1)
In production.log, i found
DEPRECATION WARNING: The InstanceMethods module inside ActiveSupport::Concern will be no longer included automatically. Please define instance methods directly in CollectiveIdea::Acts::NestedSet::Model instead. (called from include at C:/BitNami_Redmine_Stack/apps/redmine/htdocs/lib/plugins/awesome_nested_set/lib/awesome_nested_set/awesome_nested_set.rb:58)
#11 By: davidelucca, June 14th, 2012 07:12
Same problem here: same errors, same logs, everything. I have noticed that the problem occurs not only after some inactivity, but also during activity: all of a sudden I get the login form, after login comes the 503 error and the services must be restarted
#12 By: Beltran Rueda Borrego, June 14th, 2012 18:26
Thanks for posting it, we will try to reproduce it on our side.
#13 By: Pedro_Calvo, June 15th, 2012 06:48
Yes. Same here too. Errors, logs... login form and HTTP 503...
#14 By: davidelucca, June 15th, 2012 08:36
I have just reinstalled 2.0.2, always the same behavior... just after a few pages served, login form appears, a few more pages and then http 503 is back
#15 By: davidelucca, June 18th, 2012 09:08
yet no solutions or workarounds? I have a production environment almost unusable here... I'm in trouble :(
#16 By: llayman, June 18th, 2012 11:19
UPDATE: This does not really fix the problem...
I have encountered a similar problem. I don't know if it's related, but maybe it will help someone. BTw, I'm running 2.0.2 on Windows Server 2008 RC2 64bit.
I recently installed the 'redmine_backlogs' plugin. After doing so, I restarted all services from the Manager Tool. Both Thin clients would not start and my Redmine site was showing 503.
I manually started the thin client using the command previously posted:
thin -V -D start -p 3001 -e production -c "C:\\Redmine/apps/redmine/htdocs" --prefix /redmine
The server started with no problem. THEN I tried to start the second thin client and received an error:
C:\\Redmine\\apps\\redmine\\htdocs>thin -V -D start -p 3002 -e production -c "C:\\Redmine/apps/redmine/htdocs" --prefix /redmine
>> Using rack adapter
←[31mCould not find Ascii85-1.0.1 in any of the sources←[0m
←[33mRun `bundle install` to install missing gems.←[0m
I don't know why, but the second Redmine instance seemed to have forgotten all of its gems. I should say that I did run 'bundle install' durng the course of installing the plugin.
Anyway, I then ran the following:
bundle install --without development test
This installed a bunch of missing gems. I was then able to start the second thin client. I killed both clients and was able to start Redmine using the Manager tool. All seems to be running smoothly. Anyway, I'm not sure if this will be of use to anyone else, but try:
- Starting a thin client from the command line
- Trying to start the second thin client from the command line
- If one fails, run 'bundle install --without development test' in your REDMINE_ROOT/apps/redmine/htdocs folder.
#17 By: llayman, June 18th, 2012 16:59
After some more experimenting, the problem is definitely with the thin clients. One of them will crash after light use, thus resulting in seemingly semi-random 503 errors as the load balancer diverts traffic between the two clients. Eventually, both thin clients will crash.
#18 By: Beltran Rueda Borrego, June 18th, 2012 17:10
If you run the thin servers from the command line, could you see any error?
#19 By: llayman, June 19th, 2012 09:46
After running from the command line, one of my users encountered a 500 error, after which 503 showed up. Here is the screenshot of the console:
Redmine has become unresponsive since after reboot following the above error as well. No error was displayed in hte console -- the only thing I can find in the production.log is below.
Started GET "/redmine/queries/12/edit" for 127.0.0.1 at Tue Jun 19 10:07:26 -0400 2012
Processing by QueriesController#edit as HTML
Completed 500 Internal Server Error in 0ms
ActiveRecord::StatementInvalid (Mysql::Error: Lost connection to MySQL server during query: SELECT `queries`.* FROM `queries` WHERE `queries`.`id` = ? LIMIT 1):
#20 By: Beltran Rueda Borrego, June 19th, 2012 11:15
The second error seems that the database was not started. Could you try the following?
#platforms :mri_19, :mingw_19 do
# group :mysql do
# gem "mysql2", "~> 0.2.7"
Run the following command in the redmine/htdocs folder:
bundle install --local --without development test postgresql sqlite
You can try with the following command
thin start -e production -p 3001 --prefix /redmine
or you can start the services again.
next page →