Recently, I was upgrading one of my computers with gentoo on board. As I haven’t updated the system for a long time, I decided to sit down and do it. Those unfamiliar with gentoo probably don’t know how and how long this update takes place. Blood, sweat and tears. Something wrong every time. But that’s gentoo’s charms. After completing such a two-day update, there is always satisfaction with a well-functioning hardware. I myself did’t know that I had so many applications during these six months and there were some five hundred of them to be upgraded without a graphical environment. Maybe it’s my mistake because I forget to add the –oneshot option when installing, but I’m sure that most of the applications will be updated to the latest versions.
The surprise that happened to me this time was the inability to log into the terminal either via ssh or locally. The first thought that came to my mind is that someone broke into the server in some way, but after a quiet reflection I remembered that the server is under @world updating. I went through the logs and found entries about missing pam_cracklib.so libraries for password audit. As I have a system configured with PAM, the loss of this library during the update caused these crashes.
The only one file where the pam_cracklib.so library was used was /etc/pam.d/system-auth. I found alternative on some forum as the libpwquality library. After the installation we can use pam_pwquality.so instead of pam_cracklib.so.
- password requisite pam_cracklib.so try_first_pass retry=3 minlen=8 lcredit=-1 dcredit=1 difok=4 maxrepeat=2 ocredit=1 ucredit=1 + password requisite pam_pwquality.so try_first_pass local_users_only retry=3 authtok_type=
System has been restored after installing libpwquality from chroot and after editing system-auth file.
The above solution did not bring the expected effect and after restarting the system I couldn’t log in again. I was forced to uninstall PAM completely. To do this, you need to access the chroot environment from the live system:
mount /dev/xdcx /mnt mount -t proc proc /mnt/proc mount -t sysfs sys /mnt/sys mount -o bind /dev /mnt/dev mount -t devpts pts /mnt/dev/pts chroot /mnt emerge --unmerge pam emerge shadow nano /etc/portage/make.conf ... USE="... -pam" ... emerge --ask --update --deep --with-bdeps=y --newuse @world
Another surprise was the damaged mysql table. To regain the smooth operation of the system, it was necessary to perform the following operations:
ps -aef | grep mysql kill -9 PID rm -rf /var/lib/mysql/* mysqld_safe --initialize mysql -u root -p < backup_databases.sql systemctl mysqld start
To fix the problem with re-login to the mysql server:
mysqld --skip-grant-tables FLUSH PRIVILEGES; ALTER USER root@localhost IDENTIFIED BY 'pa$$w0rd';
If you get ‘Unable to lock /var/lib/mysql/ibdata1 error: 11’ error, first make sure there is no mysqld process running in the background:
ps -aef | grep mysql kill -9 PID systemctl start mysqld nano /etc/mysql/mysql.d/50-distro-server.cnf ... innodb_data_file_path = /var/lib/mysql/ibdata1:10M:autoextend ...
After a three days of updates and a few beers drunk, I have a fully functioning gentoo ecosystem again.