Thanks @jota for those details, it's definitely been helpful. I've spent some time over the past couple of weeks working through a solution, and I figure it's worth sharing some notes for anyone else who wants to do something similar.
I've ended up building my own AMI on top of the Bitnami one, and it does a few extra things:
- Install the
jq apt package to allow easy manipulation of JSON data.
- Added a script to copy the database contents (via mysqldump) to RDS if matomo_* tables don't already exist there.
- Added a script to update database credentials in the
config.ini.php file using sed, as suggested.
- Link both files into numbered scripts within
/opt/bitnami/var/init/pre-start/ so they're run when the instance is booted up.
- The second script is also added to root's crontab, because our database credentials have a 1 month TTL, so we refresh them at the start of each month.
The first of those scripts (with things specific to my setup removed):
#!/usr/bin/env bash
# For copying the initial local database contents over
# to RDS. This script will do nothing if matomo tables already
# exist in RDS.
file="/opt/bitnami/apps/matomo/htdocs/config/config.ini.php"
vars=$(curl request that gets initial secrets)
db=$(curl request that gets time-limited username and password)
# Read the old database values from the config file
old_host=$(sed -n -e 's/host = "\([a-zA-Z0-9_\.\-]\+\)"/\1/p' $file)
old_user=$(sed -n -e 's/username = "\([a-zA-Z0-9_\.\-]\+\)"/\1/p' $file)
old_pass=$(sed -n -e 's/password = "\([a-zA-Z0-9_\.\-]\+\)"/\1/p' $file)
old_name=$(sed -n -e 's/dbname = "\([a-zA-Z0-9_\.\-]\+\)"/\1/p' $file)
# Get the new database values from the JSON requests
new_host=$(echo $vars | jq '.data.data.host' | tr -d '"')
new_user=$(echo $db | jq '.data.username' | tr -d '"')
new_pass=$(echo $db | jq '.data.password' | tr -d '"')
new_name=$(echo $vars | jq '.data.data.database' | tr -d '"')
# start database services
# This was taken from existing bitnami scripts, as we can't be sure if
# MySQL is already running
for SERVICE in mysql mariadb postgresql mongodb ; do
/opt/bitnami/ctlscript.sh start $SERVICE 2>&1
done
# If the user table doesn't already existâŚ
if ! /opt/bitnami/mysql/bin/mysql --user $new_user --host $new_host -p$new_pass -e "SHOW TABLES" $new_name | grep matomo_user
then
# Pull down the database dump from the old db
/opt/bitnami/mysql/bin/mysqldump --user $old_user --host $old_host -p$old_pass $old_name > dump.sql
# And upload it into the new (RDS) one
/opt/bitnami/mysql/bin/mysql --user $new_user --host $new_host -p$new_pass $new_name < dump.sql
fi
#!/usr/bin/env bash
# For updating the config file with appropriate RDS credentials.
# Every run will update the file.
file="/opt/bitnami/apps/matomo/htdocs/config/config.ini.php"
vars=$(curl request that gets initial secrets)
db=$(curl request that gets time-limited username and password)
# Get the new database values from the JSON requests
new_host=$(echo $vars | jq '.data.data.host' | tr -d '"')
new_user=$(echo $db | jq '.data.username' | tr -d '"')
new_pass=$(echo $db | jq '.data.password' | tr -d '"')
new_name=$(echo $vars | jq '.data.data.database' | tr -d '"')
# Update the config file
sed -i \
-e 's/host = "\([a-zA-Z0-9_\.\-]\+\)"/host = "'$new_host'"/' \\
-e 's/username = "\([a-zA-Z0-9_\.\-]\+\)"/username = "'$new_user'"/' \
-e 's/password = "\([a-zA-Z0-9_\.\-]\+\)"/password = "'$new_pass'"/' \
-e 's/dbname = "\([a-zA-Z0-9_\.\-]\+\)"/dbname = "'$new_name'"/' \
$file
if [ -f /opt/bitnami/apache2/logs/httpd.pid ]
then
# If apache is running, make sure we restart it so it's got the changes
sudo apachectl restart
fi
We're using Vault as our central secrets storage, as well as having it manage database credentials. I've deliberately left out the calls to Vault from these scripts, though if someone's using that I'm happy to discuss that side of things in more detail.
