AWS Ubuntu Bitnami Ubuntu 16.04.6 LTS MySQL WordPress site could not be started

Keywords: WordPress - AWS - Technical issue - Other
bnsupport ID: bc98013c-1dc8-f250-c504-27da15125da2
Description:
Hi, Bitnami community,

I have recently cleared my AWS Ubuntu disk space and its now 53% used out of 9.7G.

I tried running the script /opt/bitnami/ctlscript.sh restart that resulted in the following output:

Syntax OK
/opt/bitnami/apache2/scripts/ctl.sh : httpd stopped
/opt/bitnami/php/scripts/ctl.sh : php-fpm stopped
/opt/bitnami/mysql/scripts/ctl.sh : mysql not running
/opt/bitnami/mysql/scripts/ctl.sh : mysql could not be started
/opt/bitnami/php/scripts/ctl.sh : php-fpm started.

So I ran the command to read the error log message of mySQL tail -f /opt/bitnami/mysql/data/mysqld.log and the output as of below:

2020-10-21T22:56:18.649992Z 0 [ERROR] [MY-010119] [Server] Aborting
2020-10-21T22:56:20.077706Z 0 [System] [MY-010910] [Server] /opt/bitnami/mysql/bin/mysqld.bin: Shutdown complete (mysqld 8.0.16) MySQL Community Server - GPL.
2020-10-21T22:58:17.886459Z 0 [Warning] [MY-011068] [Server] The syntax ‘expire-logs-days’ is deprecated and will be removed in a future release. Please use binlog_expire_logs_seconds instead.
2020-10-21T22:58:17.887162Z 0 [System] [MY-010116] [Server] /opt/bitnami/mysql/bin/mysqld.bin (mysqld 8.0.16) starting as process 4563
2020-10-21T22:58:17.893234Z 0 [Warning] [MY-013242] [Server] --character-set-server: ‘utf8’ is currently an alias for the character set UTF8MB3, but will be an alias for UTF8MB4 in a future release. Please consider using UTF8MB4 in order to be unambiguous.
2020-10-21T22:58:17.893247Z 0 [Warning] [MY-013244] [Server] --collation-server: ‘utf8_general_ci’ is a collation of the deprecated character set UTF8MB3. Please consider using UTF8MB4 with an appropriate collation instead.
2020-10-21T22:58:18.607408Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2020-10-21T22:58:18.617950Z 0 [ERROR] [MY-000068] [Server] unknown option ‘–add skip-log-bin’.
2020-10-21T22:58:18.618668Z 0 [ERROR] [MY-010119] [Server] Aborting
2020-10-21T22:58:20.151913Z 0 [System] [MY-010910] [Server] /opt/bitnami/mysql/bin/mysqld.bin: Shutdown complete (mysqld 8.0.16) MySQL Community Server - GPL.

My current problem is I am unable to locate the text ‘–add skip-log-bin’ from my.cnf files that are located in “/opt/bitnami/mysql/bitnami/my.cnf” and “/opt/bitnami/mysql/my.cnf”.

What action should I take so I am able to locate the current configuration file which I am able to remove the “–add skip-log-bin”?

Thank you.

Hi @joetan83,

You can add the --skip-log-bin or --disable-log-bin options in the /opt/bitnami/mysql/scripts/ctl.sh file at the end of the MYSQL_START line.

Regards,
Michiel

Hi michiel,

I ran the nano editor with the command:
sudo nano /opt/bitnami/mysql/scripts/ctl.sh .

I have found the --disable-log-bin already in MYSQL_START line as you had suggested.
I removed the ‘–disable-log-bin’ and saving the ctl.sh file and restarted the apache2 server with the command script:

sudo /opt/bitnami/ctlscript.sh restart

Unfortunately, the MySQL server is not running after I typed in the script command:

sudo /opt/bitnami/ctlscript.sh status
php-fpm already running
apache already running
mysql not running

Do you have any other ideas?

Thank you.

Hi @joetan83,

Could you add the error in the logs of MySQL?

tail -f /opt/bitnami/mysql/data/mysqld.log

Regards,
Ibone.

Hi Ibone,

When I ran that command currently there is nothing on the log file. It is empty.

Regards,

joetan83

Hi @joetan83,

could you rerun the bnsupport?

Regards,
Ibone.

Hi Ibone,

I ran the bitnami support tool again with the following code:

faa4ec74-9458-f6eb-8c34-1f9ddcd3a76a

Hi @joetan83,

Could you add in my.cnf the next line:

[mysqld]
disable_log_bin

and restart MySQL:

sudo /opt/bitnami/ctlscript.sh start mysql

I hope that helps you,
Ibone

Hi @Ibone,

I have added the disable_log_bin under /opt/bitnami/mysql/bitnami/my.cnf and /opt/bitnami/mysql/my.cnf.

I ran the script of
sudo /opt/bitnami/ctlscript.sh start mysql

MySQL is still unable to start
/opt/bitnami/mysql/scripts/ctl.sh : mysql could not be started

MySQL logs are still empty.

Not entirely sure what is going on with the MySQL server.

Regards,
joetan83

Hi @joetan83,

At the beginning of the case, you modified the /opt/bitnami/mysql/scripts/ctl.sh, could you copy the file in the thread?

Regards,
Ibone.

Hi @Ibone,

Here’s the ctl.sh file.

 #!/bin/sh

MYSQL_PIDFILE=/opt/bitnami/mysql/data/mysqld.pid

MYSQL_START="/opt/bitnami/mysql/bin/mysqld_safe --defaults-file=/opt/bitnami/mysql/my.cnf --mysqld=mysqld.bin --socket=/opt/bitnami/mysql/tmp/mysql.sock  --datadir=/opt/bitnami/mysql/data --log-error=/opt/bitnami/mysql/data/mysqld.log  --pid-file=$MYSQL_PIDFILE --lower-case-table-names=1 --disable-log-bin"

MYSQL_STATUS=""
MYSQL_PID=""
PID=""
ERROR=0

get_pid() {
    PID=""
    PIDFILE=$1
    # check for pidfile
    if [ -f "$PIDFILE" ] ; then
        PID=`cat $PIDFILE`
    fi
}

get_mysql_pid() {
    get_pid $MYSQL_PIDFILE
    if [ ! "$PID" ]; then
        return
    fi
    if [ "$PID" -gt 0 ]; then
        MYSQL_PID=$PID
    fi
}

is_service_running() {
    PID=$1
    if [ "x$PID" != "x" ] && kill -0 $PID 2>/dev/null ; then
        RUNNING=1
    else
        RUNNING=0
    fi
    return $RUNNING
}

is_mysql_running() {
    get_mysql_pid
    is_service_running $MYSQL_PID
    RUNNING=$?
    if [ $RUNNING -eq 0 ]; then
        MYSQL_STATUS="mysql not running"
    else
        MYSQL_STATUS="mysql already running"
    fi
    return $RUNNING
}

start_mysql() {
    is_mysql_running
    RUNNING=$?
    if [ $RUNNING -eq 1 ]; then
        echo "$0 $ARG: mysql  (pid $MYSQL_PID) already running"
  exit
    fi
    $MYSQL_START >/dev/null 2>/dev/null &
    COUNTER=40
    while [ $RUNNING -eq 0 ] && [ $COUNTER -ne 0 ]; do
        COUNTER=`expr $COUNTER - 1`
        sleep 3
        is_mysql_running
        RUNNING=$?
    done
    if [ $RUNNING -eq 0 ]; then
        ERROR=1
    fi
    if [ $ERROR -eq 0 ]; then
  echo "$0 $ARG: mysql  started at port 3306"
  sleep 2
    else
  echo "$0 $ARG: mysql  could not be started"
  ERROR=3
    fi
}

stop_mysql() {
    NO_EXIT_ON_ERROR=$1
    is_mysql_running
    RUNNING=$?
    if [ $RUNNING -eq 0 ]; then
        echo "$0 $ARG: $MYSQL_STATUS"
        if [ "x$NO_EXIT_ON_ERROR" != "xno_exit" ]; then
            exit
        else
            return
        fi
    fi

  kill $MYSQL_PID

    COUNTER=40
    while [ $RUNNING -eq 1 ] && [ $COUNTER -ne 0 ]; do
        COUNTER=`expr $COUNTER - 1`
        sleep 3
        is_mysql_running
        RUNNING=$?
    done

    is_mysql_running
    RUNNING=$?
    if [ $RUNNING -eq 0 ]; then
            echo "$0 $ARG: mysql stopped"
        else
            echo "$0 $ARG: mysql could not be stopped"
            ERROR=4
    fi
}

cleanpid() {
    rm -f $MYSQL_PIDFILE
}

if [ "x$1" = "xstart" ]; then
    start_mysql
elif [ "x$1" = "xstop" ]; then
    stop_mysql
elif [ "x$1" = "xstatus" ]; then
    is_mysql_running
    echo "$MYSQL_STATUS"
elif [ "x$1" = "xcleanpid" ]; then
    cleanpid
fi

exit $ERROR

Hi @joetan83,

Could you run the next command?

MYSQL_PIDFILE=/opt/bitnami/mysql/data/mysqld.pid
sudo  /opt/bitnami/mysql/bin/mysqld_safe --defaults-file=/opt/bitnami/mysql/my.cnf --mysqld=mysqld.bin --socket=/opt/bitnami/mysql/tmp/mysql.sock  --datadir=/opt/bitnami/mysql/data --log-error=/opt/bitnami/mysql/data/mysqld.log  --pid-file=$MYSQL_PIDFILE --lower-case-table-names=1 --disable-log-bin

If the console doesn’t respond, MySQL started, but if you receive an error, please add it to this thread.

Regards,
Ibone.

Hi @Ibone

I ran the command below:

sudo /opt/bitnami/mysql/bin/mysqld_safe --defaults-file=/opt/bitnami/mysql/my.cnf --mysqld=mysqld.bin --socket=/opt/bitnami/mysql/tmp/mysql.sock --datadir=/opt/bitnami/mysql/data --log-error=/opt/bitnami/mysql/data/mysqld.log --pid-file=$MYSQL_PIDFILE --lower-case-table-names=1 --disable-log-bin

Output generated on the terminal:

2020-11-01T22:32:19.726188Z mysqld_safe Logging to '/opt/bitnami/mysql/data/mysqld.log'.
2020-11-01T22:32:19.746863Z mysqld_safe Starting mysqld.bin daemon with databases from /opt/bitnami/mysql/data
2020-11-01T22:32:20.122947Z mysqld_safe mysqld from pid file /opt/bitnami/mysql/data/ip-172-31-2-29.pid ended  

MySQL still would not start. :confused:

Hi @joetan83,

On the moment of running the bnsupport tool you only had 58 MB of available memory, so the issue is probably due to a lack of available memory. I suggest upgrading your instance type to make it work:

https://docs.bitnami.com/aws/faq/administration/change-server-type/

Regards,
Michiel

Hi @michiel,

I have upgraded the ec2 instance from nano to micro. However, the MySQL server still will not start.

I have also increased the SWAP memory to 4GB to no avail.

Is there anything else I can do?

Thank you and regards,
Joetan83

Hi @joetan83,

When you run the command the console responds?
Appears something in the logs?

Could you run and copy the result of the following command?

ls -la /opt/bitnami/mysql

Another thing you can do is launch a new instance and migrate your data from the current instance to the new one.

https://docs.bitnami.com/aws/how-to/migrate-wordpress/

If you already configured SSL certificates or modified the server’s configuration in the old instance, you will also need to migrate that configuration to the new server.

Regards,
Ibone.

Hi @Ibone,

Yes I ran the command as you have suggested and posted the output on your last reply.

sudo /opt/bitnami/mysql/bin/mysqld_safe --defaults-file=/opt/bitnami/mysql/my.cnf --mysqld=mysqld.bin --socket=/opt/bitnami/mysql/tmp/mysql.sock --datadir=/opt/bitnami/mysql/data --log-error=/opt/bitnami/mysql/data/mysqld.log --pid-file=$MYSQL_PIDFILE --lower-case-table-names=1 --disable-log-bin

This was the console’s response.

2020-11-05T22:17:03.421786Z mysqld_safe Logging to '/opt/bitnami/mysql/data/mysqld.log'.
2020-11-05T22:17:03.442163Z mysqld_safe Starting mysqld.bin daemon with databases from /opt/bitnami/mysql/data
2020-11-05T22:17:04.095218Z mysqld_safe mysqld from pid file /opt/bitnami/mysql/data/ip-172-31-2-29.pid ended

I ran the bitnami script to start mysql server again:

sudo /opt/bitnami/ctlscript.sh start mysql

Output generated:

/opt/bitnami/mysql/scripts/ctl.sh : mysql  could not be started
Monitored mysql

and when I typed the command:

sudo tail -f /opt/bitnami/mysql/data/mysqld.log

The terminal freezes…

Hi @joetan83,

Can you paste the output of the following command:

ls -la /opt/bitnami/mysql

Regards,
Michiel

Hi @michiel,

The following output from the command:

drwxr-xr-x 13 root    root    4096 Oct 28 23:40 .
drwxr-xr-x 18 root    root    4096 Oct 21 22:15 ..
drwxr-xr-x  2 root    root    4096 Oct 23 12:47 bin
drwxr-xr-x  2 bitnami root    4096 Oct 28 23:40 bitnami
-r-xr--r--  1 root    root 8490016 May 13  2019 bnconfig
-rw-r--r--  1 root    root    1024 Oct 22 00:11 .bnconfig.swp
drwxr-x---  7 bitnami root    4096 Nov  8 06:25 data
drwxr-xr-x  3 root    root    4096 May 15  2019 include
drwxr-xr-x  6 root    root    4096 May 15  2019 lib
-rw-r--r--  1 root    root  335809 Apr 13  2019 LICENSE
-rw-r--r--  1 root    root  101807 Apr 13  2019 LICENSE.router
-rw-r--r--  1 bitnami root     663 Oct 28 21:52 my.cnf
-rw-r--r--  1 root    root    1024 Oct 28 23:40 .my.cnf.swp
-rw-r--r--  1 root    root     687 Apr 13  2019 README
-rw-r--r--  1 root    root     700 Apr 13  2019 README.router
drwxrwxr-x  2 root    root    4096 May 15  2019 run
drwxr-xr-x  2 root    root    4096 Oct 29 21:57 scripts
drwxr-xr-x 28 root    root    4096 May 15  2019 share
drwxr-xr-x  2 root    root    4096 May 15  2019 support-files
drwxrwxrwx  2 root    root    4096 Oct 21 22:58 tmp
drwxr-xr-x  3 root    root    4096 May 15  2019 var

Hi @joetan83,

It seems the default-character-set is set to UTF8 in the /opt/bitnami/mysql/my.ini file. Could you try setting it back to utf8mb4?And restart mysql:

sudo /opt/bitnami/ctlscript.sh restart mysql

Regards,
Michiel