Linux Shell
username: bitnami
password: bitnami
Changes to Machine
/opt/bitnami/apps/gitlab/htdocs/config/gitlab.yml
/opt/bitnami/apps/gitlab/htdocs/config/environments/production.rb
/opt/bitnami/apps/gitlab/htdocs/config/initializers/smtp_settings.rb
References
http://wiki.bitnami.com/Applications/BitNami_GitLab#How_to_configure_the_email_settings_of_GitLab.3f
Converting MySQL DB on the old Bitnami Gitlab to Postgres
The new version of GitLab uses Postgres DB instead of mysql, so we need to convert our mysql DB to postgres:
cd /opt/bitnami/mysql/bin
git clone https://github.com/gitlabhq/mysql-postgresql-converter.git -b gitlab
cd mysql-postgresql-converter
mysqldump --compatible=postgresql --default-character-set=utf8 -r bitnami_gitlab.mysql -u bitnami bitnami_gitlab –p
Note that the username and password for the mysql database is stored in database.yml. Look into there to see what the username and password is.
python db_converter.py bitnami_gitlab.mysql bitnami_gitlab.psql
ed -s bitnami_gitlab.psql < move_drop_indexes.ed
Note, if you don’t have ed package, do sudo apt-get update and sudo apt-get install to get it:
bitnami@linux:/opt$ sudo apt-get update
bitnami@linux:/opt$ sudo apt-get install ed
See section Info about Gitlab CI DB below for why I chose not to also migrate the bitnami_gitlabci database over as well. You might want to at least create the bitnami_gitlabci.psql file anyways just in case (if so, use the same steps as above just rename bitnami_gitlab to bitnami_gitlabci)
Links where I got the steps above from:
Use the databases in this link for migrating mysql to postgres, the actual steps are in the second link:
https://docs.bitnami.com/installer/apps/gitlab-legacy/#migration-process-for-gitlab-with-mysql-gitlab--7114
Use these steps to convert from mysql to postgres, do this before getting the new Gitlab version, but don’t use gitlabhq_production, use the databases listed in the link above (bitnami_gitlab and bitnami_gitlabci), but with the steps in the link below:
Download and Install GitLab v 8.7.2
Start the new Gitlab version, install latest Gitlab from this URL:
Version of GitLab that is compatible with old PostgresSQL and works with Ubuntu 16.04:
https://packages.gitlab.com/gitlab/gitlab-ce/packages/ubuntu/precise/gitlab-ce_8.7.2-ce.0_amd64.deb
After installation, you should see a folder under /opt called gitlab
Change config on new Gitlab version to match what was in the old:
New config can be found here:
sudo vi /etc/gitlab/gitlab.rb
Old config can be found here:
sudo vi ./opt/bitnami/apps/gitlab/htdocs/config/gitlab.yml
Also make sure to enable postgres db in the gitlab.rb file:
postgresql['enable'] = true
Link that explains the step above:
https://docs.gitlab.com/ee/update/mysql_to_postgresql.html#converting-a-gitlab-backup-file-from-mysql-to-postgres
After changes to gitlab.rb are done, reconfigure gitlab using:
sudo gitlab-ctl reconfigure
run the below rake test to find out if there are any problems, at this point there shouldn’t be because it is fresh gitlab instance:
sudo gitlab-rake gitlab:check
sudo gitlab-rake gitlab:check SANITIZE=true –trace
(optional, might not even work for some reason) create a backup of latest gitlab before we do DB migration:
sudo gitlab-rake gitlab:backup:create
Use this checklist to determine if the old postgresSQL database on the existing GitLab is compatible with the new version 8.4.2:
https://docs.gitlab.com/omnibus/settings/database.html#upgrade-packaged-postgresql-server
Import .psql file into new Postgres DB
Make sure you can access postgres DB:
bitnami@linux:/opt/gitlab/bin$ sudo gitlab-rails dbconsole
Or, to run psql without the gitlab-rails dbconsole command:
(note you may need to switch to superuser gitlab-psql first)
/opt/gitlab/embedded/bin/psql --port 5432 -h /var/opt/gitlab/postgresql –d gitlabhq_production
I got the line above from:
WinSCP the bitnami_gitlab.psql file created above over to the new machine (if you are installing new GitLab on a new machine).
Import .psql DB into the new Gitlab postgres DB:
sudo su gitlab-psql switch to superuser role gitlab-psql first
gitlab-psql@linux:~$ /opt/gitlab/embedded/bin/psql --port 5432 -h /var/opt/gitlab/postgresql -f /opt/gitlab/bin/bitnami_gitlab.psql -d gitlabhq_production
After importing the bitnami_gitlab.psql you need to do db:migrate, this will fail because of a lot of inconsistencies between old and new postgres DB.
sudo gitlab-rake db:migrate RAILS_ENV=production
Now this is where the majority of work will be. Need to modify the database so that the rails migration schemas can apply their changes on top of our existing tables.
After doing the database migration, first got the following error:
Couldn't create database for {"adapter"=>"postgresql", "encoding"=>"unicode", "collation"=>nilpt/gitlab/postgresql", "port"=>5432, "socket"=>nil, "sslmode"=>nil, "sslrootcert"=>nil, "sslca
rake aborted!
ActiveRecord::StatementInvalid: PG::InsufficientPrivilege: ERROR: permission denied for relat
: SELECT "schema_migrations".* FROM "schema_migrations"
To fix this issue, I did:
Gave role gitlab superuser:
first switch to gitlab-psql user:
sudo –s gitlab-psql
then go into postgres DB and give gitlab superuser role:
ALTER USER gitlab WITH SUPERUSER;
Useful postgres commands:
\dt : show tables
\du : show users
select * from table; : shows if there is stuff inside a table or not
Fixing the database inconsistency errors:
Now this is where the majority of work will be. Need to modify the database so that the rails migration schemas can apply their changes on top of our existing tables.
Link to a forum where they are also trying to migrate Bitnami Gitlab DB over to latest Gitlab:
A lot of the steps below are from this link above
Same error we get when we do migration of DB:
column projects.pending_delete does not exist
Links to people with same issue:
I did the following:
comment out default_scope from app/models/project.rb.
Then get this error:
relation "project_group_links" already exists.
Solution described here:
Fixed it by running these commands in dbconsole:
sudo gitlab-rails dbconsole
gitlabhq_production=# INSERT INTO schema_migrations (version) values ('20130711063759');
INSERT 0 1
gitlabhq_production-# INSERT INTO schema_migrations (version) values ('20130820102832');
INSERT 0 1
gitlabhq_production=#
Then after trying migration again:
sudo gitlab-rake db:migrate RAILS_ENV=production
I get this error:
ERROR: relation "git_hooks" already exists
Fixed with:
INSERT INTO schema_migrations (version) values ('20140319135450');
Then got this error:
"users_star_projects" already exists
Fixed with:
DROP TABLE users_star_projects;
See here for fixes to further errors related to DB migration:
Then here for rest of DB issues:
Some of the commands I had to do in DB console (see second link above for the rest)
Some of the more complicated commands I had to in the DB console
alter table application_settings drop column default_group_visibility;
alter table application_settings add column twitter_sharing_enabled boolean default false;
gitlabhq_production=# alter table application_settings add column help_text character varying(50) NOT NULL DEFAULT '';
Once your db:migrate command succeeds, do reconfigure (sudo gitlab-ctl reconfigure )
and restart (sudo gitlab-ctl restart)
Now, visit the IP of your site and it should be running, if not check the GitLab log file for more errors or see 500 something went wrong section below if you get that.
Otherwise, if everything works and the migrate, reconfigure, check and restart commands succeed, proceed to Transfer Git repository to new GitLab
Log files in New GitLab:
bitnami@linux:/$ sudo vi /var/log/gitlab/reconfigure/15100
bitnami@linux:/$ sudo vi /var/log/gitlab/postgresql/current
/var/log/gitlab/sidekiq/current
root@linux:/opt/gitlab/bin# vi /var/log/gitlab/gitlab-rails/production.log
Transfer Git repository to new GitLab
Notice that
sudo gitlab-rake gitlab:check
Shows that the repositories are empty.
On old bitnami gitlab, repositories are located here:
root@linux:/# sudo du -s ./opt/bitnami/apps/gitlab/repositories
315148 ./opt/bitnami/apps/gitlab/repositories
root@linux:/#On new Gitlab, gitlab.rb says where repositories are with this field in the gitlab.rb:
git_data_dirs
by default it is
# git_data_dir "/var/opt/gitlab/git-data"
WinSCP the old repositories folder to the new machine, then,
Copy over all the folders within the old repositories folder into the new repositories folder
Steps above are from this Link:
Then, tell gitlab to import these new repos
sudo gitlab-rake gitlab:import:repos
then when you do
sudo gitlab-rake gitlab:check
you will get these errors:
(Ovid TODO: paste missing hooks error here)
To fix, run this command:
sudo -u git -H /opt/gitlab/embedded/service/gitlab-shell/bin/create-hooks
Now after reconfigure (sudo gitlab-ctl reconfigure )
and restart (sudo gitlab-ctl restart)
the Git repositories should show up on the site! Your done!
500 something went wrong issue (after no more errors in db:migrate)
Find issue in /var/log/gitlab/sidekiq/current
Started GET "/api/v3/internal/check" for 127.0.0.1 at 2017-11-07 14:22:19 -0800
Started GET "/api/v3/internal/check" for 127.0.0.1 at 2017-11-07 14:23:36 -0800
Started GET "/api/v3/internal/check" for 127.0.0.1 at 2017-11-07 14:23:45 -0800
Started GET "/users/sign_in" for 127.0.0.1 at 2017-11-07 14:30:38 -0800
Processing by SessionsController#new as HTML
Completed 500 Internal Server Error in 84ms (ActiveRecord: 3.3ms)
In /var/log/gitlab/gitlab-rails/production.log you will see:
ActionView::Template::Error (undefined method `help_text' for #<ApplicationSetting:0x000000136ffac8>):
26:
27: - if extra_sign_in_text.present?
28: = markdown(extra_sign_in_text)
29: - if help_text.present?
30: %h3 Need help?
31: %hr
32: %p.slead
app/helpers/application_settings_helper.rb:19:in `help_text'
app/views/layouts/devise.html.haml:29:in `_app_views_layouts_devise_html_haml___2036178018103411599_74014680'
app/controllers/sessions_controller.rb:22:in `new'
lib/gitlab/middleware/go.rb:16:in `call'
lib/gitlab/middleware/readonly_geo.rb:29:in `call'
To fix, do this command in the dbconsole:
bitnami@linux:/opt/gitlab/bin$ sudo gitlab-rails dbconsole
gitlabhq_production=# alter table application_settings add column help_text character varying(50) NOT NULL DEFAULT '';
Other problems
note that many of the problems below are due to the fact that bitnami stack was installed as root, so many commands required sudo in order to succeed.
note: when doing bundle install commands, try prepending sudo if the gem install fails, or try manually installing the gems.
had an issue with charlock_holmes gem not installing had to do:
sudo bundle install
note: version of ruby might need to be upgraded in order for bundle install commands to work. Upgraded to ruby 2.1.2 using "rbenv install 2.1.2"
note: when fixing the dashboard I had issue with this command:
ruby bin/rake migrate_iids RAILS_ENV=production
fixed by adding sudo in front
mysql –u bitnami –p (password is 7c5343e323 )
Biggest issue was with the mysql commands to drop database, create database and copy tables from the bk.sql files to the mysql DB.
For these steps, whenever I got a Access denied for user 'root' ...
I just removed the "-p" option from the mysql commands.
If that doesn’t work, look into your config/databse.yml file for the username and password field.
Also, when doing the "grant all" commands, if you get a Access denied message, do
flush privileges;
command first.
After that, the previous command should work.
Within the database.yml file, I changed the production password and the development password to 'bitnami'.
Do NOT change the password for the test environment, comment just above says that it is auto-generated.
useful commands:
sudo ./ctlscript.sh [start | stop | restart ]
To fix sharlock holmes issue during bundle install:
sudo apt-get install libicu-dev
check gitlab application status (note this needs mysql to be running):
sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production
change /etc/sudoers to point to the new bitnami by issuing the following command:
sudo visudo
when changing gitlab.yml, make sure to also update git: bin_path to the correct path (see original)
note: if getting an error logging into mysql, look into database.yml for username and password
If then, you get the following error when doing a sql command via command line:
ERROR 1290 (HY000): The MySQL server is running with the --skip-grant-tables option so it cannot execute this statement
run this command first: FLUSH PRIVILEGES;
remember to remove the --skip-grant-tables from the my.cnf file afterwards.
root@linux:/opt/bitnami# mysql -u root
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)
root@linux:/opt/bitnami#
To fix Gollum-lib issue during bundle install, downgrade bundle from current version 1.16 to 1.12.3:
gem uninstall bundler
gem install bundler --version '1.12.3'
Summary
This text will be hidden
if you get permissions denied error when doing bundle install, change the permissions on the directory in which the bundle is (i.e. htdocs/bin)
to list all local users:
cut -d: -f1 /etc/passwd
Submitted Ticket for issue with repositories (FIXED)
Info about Gitlab CI DB:
Enable disable CI from gitlab.rb:
What does GitLab CI do:
Do we need Gitlab CI ? Database in old bitnami gitlab has stuff inside the gitlab CI DB, but importing into gitlabhq_production is not trivial.
Big note: According to this link, existing projects cannot have CI disabled, only for new projects, what are the implications of this if gitblab CI DB is empty?:
Based on the information in the links above, I decided that the bitnami_gitlabci database in mysql doesn’t need to be imported into the gitlabhq_production database in the new GitLab. GitLab CI doesn’t seem necessary and is not worth the effort of renaming and modifying the mysql bitnami_gitlabci tables so that they can be imported into the postgres DB.
Links
https://docs.bitnami.com/installer/apps/gitlab-legacy/#migration-process-for-gitlab-with-mysql-gitlab--7114
https://docs.bitnami.com/installer/apps/gitlab-legacy/#how-to-upgrade-gitlab
https://docs.bitnami.com/installer/apps/gitlab-legacy/#step-8-check-results
https://docs.gitlab.com/omnibus/settings/configuration.html#configuring-the-external-url-for-gitlab
https://docs.gitlab.com/ee/update/README.html#installation-from-source
https://docs.gitlab.com/omnibus/settings/smtp.html
link to migrate from 7.0 to 7.14:
