[SOLVED] MariaDB won't start. "mysqld got signal 4;"

[autofyrsto]

Hi. My MariaDB seems to have broken after a recent @world update on a hobby web server. The server is running Gentoo (stable) x86. The update included a gcc upgrade to 10.2.0 and a MariaDB upgrade to 10.5.9. A few things were initially broke on upon this update, including portage and dhcpcd. I believe I've attributed these initial breakages to SELinux, which I've set to "permissive" until I've solved all other problems. In the meantime, I rebuilt the system with "emerge -qae @world".

After rebuilding the system, MariaDB will not start. I've tried many remedies suggested online, including:
- downgrading MariaDB and running mysql_upgrade,
- ensuring permissions of /var/lib/mysql are 750 with user:group set to mysql:mysql,
- renaming ibdata1 and ib_logfile* and restarting MariaDB,
- renaming /var/lib/mysql and running "emerge --config dev-db/mariadb",
- unmerging dev-db/mariadb and emerging dev-db/mysql, and
- unmerging and emerging dev-db/mariadb with CONFIG_PROTECT="-*"

None of the above was able to get MariaDB up and running. The output when starting MariaDB is:

[alamahant]

Hi
p;ease move
/etc/mysql/mariadb.d/50-distro-server.cnf ###and
/etc/init.d/mysql

to another location and reemerge mariadb.

Also
emerge --info mariadb
chown mysql. /var/run/mysqld
chmod 755 /var/run/mysqld
grep mysql /etc/passwd
_________________

[trilithium]

Signal 4 is SIGILL; "Illegal instruction". You will get this error if GCC has been configured to generate code not supported by your CPU. Make sure optimization flags in /etc/portage/make.conf matches what the CPU is capable of.

[autofyrsto]

Thanks for the suggestions. I re-named /etc/mysql/mariadb.d/50-distro-server.cnf and /etc/init.d/mysql and then re-emerged dev-db/mariadb. The latter file, /etc/init.d/mysql, was apparently not replaced when dev-db/mariadb was re-emerged. I had to restore the original file to attempt restarting MariaDB, but the output was the same as above, with "ERROR: mysql failed to start". After renaming, /var/log/mysql/mysqld.err looks like this:

[Hu]

It might be useful to know the specific illegal instruction, and the file in which it is found. If mysqld generates a core file, you can use gdb to examine the core to extract this. You won't even need debug symbols in this case, since we only need binary file granularity, and gdb can do that without symbols.

[trilithium]

I can't see anything wrong with those optimization flags. There is a more specific target for the Pentium M architecture (-march=pentium-m) but since that CPU is a Pentium III variant -march=i686 should be entirely safe and not generate unsupported instructions. While it could still be a code generation regression in GCC 10.2.0 (e.g. a jump into the middle of an otherwise valid instruction could also cause SIGILL) such bugs are rare (generating correct code is always first priority for compiler developers) and nothing in the GCC 10 changes looks relevant.

[alamahant]

What is the output of

getenforce
?
The only possible culprit i see in my ignorance is your
hardened/selinux
profile.
Maybe this profile makes gcc generate weird instructions also
_________________

[autofyrsto]

The output of getenforce is:

[alamahant]

@fedeliallalinea
suggested to run "quickpkg" the old gcc before --depcleaning it so as to install it presto in cases like this.
I just use
emerge --depclean -a --exclude=gcc
_________________

[Hu]

OP's emerge --info shows that gcc-9.3 is still installed. It could be made active without rebuilding gcc, if OP chooses to go that route. I don't expect using an older gcc will help here, but if OP wants to try it, it is easier than he seems to think it will be.

[autofyrsto]

Yes! I did actually attempt earlier to downgrade to gcc-9.3.0, and then rebuild MariaDB. It didn't seem to work at the time because when I tried it I neither noticed nor understood the difference between these two errors:

[autofyrsto]

Okay. It looks like I'm back online. The answer, simply enough, was to find the correct version of the ibdata1 file from my several attempts at re-naming and re-starting MariaDB, copy that back to /var/lib/mysql, and make sure the ownership was mysql:mysql on it. One last question before can I mark this particular thread as "solved":

How might I fill out a bug report on this, if I ought to? Is this as a GCC bug, or is it a MariaDB bug? (Or is it user error? )

Best regards and many thanks to you all for your helpful suggestions that got my server back up.