These days we are doing some maintenance tasks on hundreds of RHEL servers and we update them using a shell script. We failed to update part of them and after I fixed the failure I would like to share the solution here.
Usually after the update script was run, we would reboot the servers to use the latest kernel and we found the failure was caused by the terminated yum update command abnormally. In other words, we rebooted the servers too early.
Run the command 'package-cleanup --dupes' and we would get the rpm packages with multiple versions:
[root@xxxxxx ~]# package-cleanup --dupes Loaded plugins: fastestmirror firewalld-filesystem-0.5.3-5.el7.noarch firewalld-filesystem-0.6.3-2.el7_7.1.noarch grub2-common-2.02-0.76.el7.1.noarch grub2-common-2.02-0.80.el7.noarch libipa_hbac-1.16.2-13.el7_6.8.x86_64 libipa_hbac-1.16.4-21.el7.x86_64 audit-libs-2.8.5-4.el7.x86_64 audit-libs-2.8.4-4.el7.x86_64 libsss_autofs-1.16.2-13.el7_6.8.x86_64 libsss_autofs-1.16.4-21.el7.x86_64 python-firewall-0.6.3-2.el7_7.1.noarch python-firewall-0.5.3-5.el7.noarch grub2-tools-2.02-0.76.el7.1.x86_64 grub2-tools-2.02-0.80.el7.x86_64 bind-license-9.11.4-9.P2.el7.noarch bind-license-9.9.4-74.el7_6.1.noarch samba-common-4.8.3-4.el7.noarch samba-common-4.9.1-6.el7.noarch libsss_sudo-1.16.4-21.el7.x86_64 libsss_sudo-1.16.2-13.el7_6.8.x86_64 nss-3.36.0-7.1.el7_6.x86_64 nss-3.44.0-4.el7.x86_64 python-sssdconfig-1.16.2-13.el7_6.8.noarch python-sssdconfig-1.16.4-21.el7.noarch libsss_idmap-1.16.4-21.el7.x86_64 libsss_idmap-1.16.2-13.el7_6.8.x86_64 NetworkManager-libnm-1.12.0-10.el7_6.x86_64 NetworkManager-libnm-1.18.0-5.el7_7.1.x86_64 grub2-pc-modules-2.02-0.80.el7.noarch grub2-pc-modules-2.02-0.76.el7.1.noarch dracut-033-554.el7.x86_64 dracut-033-564.el7.x86_64 sssd-client-1.16.2-13.el7_6.8.x86_64 sssd-client-1.16.4-21.el7.x86_64 plymouth-core-libs-0.8.9-0.31.20140113.el7.x86_64 plymouth-core-libs-0.8.9-0.32.20140113.el7.x86_64 libcom_err-1.42.9-13.el7.x86_64 libcom_err-1.42.9-16.el7.x86_64 dhcp-libs-4.2.5-77.el7.x86_64 dhcp-libs-4.2.5-68.el7_5.1.x86_64 openssl-libs-1.0.2k-16.el7_6.1.x86_64 openssl-libs-1.0.2k-19.el7.x86_64 selinux-policy-3.13.1-229.el7_6.12.noarch selinux-policy-3.13.1-252.el7.1.noarch libss-1.42.9-16.el7.x86_64 libss-1.42.9-13.el7.x86_64 openssh-7.4p1-21.el7.x86_64 openssh-7.4p1-16.el7.x86_64 dhcp-common-4.2.5-77.el7.x86_64 dhcp-common-4.2.5-68.el7_5.1.x86_64 grub2-tools-minimal-2.02-0.80.el7.x86_64 grub2-tools-minimal-2.02-0.76.el7.1.x86_64 e2fsprogs-libs-1.42.9-13.el7.x86_64 e2fsprogs-libs-1.42.9-16.el7.x86_64
We could remove some of them using 'package-cleanup --cleandupes' command and would get errors for others:
[root@xxxxxx ~]# package-cleanup --cleandupes Loaded plugins: fastestmirror Not removing bind-license-9.9.4-74.el7_6.1.noarch because it is required by 2 installed packages Not removing libsss_idmap-1.16.2-13.el7_6.8.x86_64 because it is required by 1 installed package Not removing dracut-033-554.el7.x86_64 because it is required by 2 installed packages Not removing plymouth-core-libs-0.8.9-0.31.20140113.el7.x86_64 because it is required by 1 installed package Not removing libsss_autofs-1.16.2-13.el7_6.8.x86_64 because it is required by 1 installed package Not removing libss-1.42.9-13.el7.x86_64 because it is required by 1 installed package Not removing grub2-pc-modules-2.02-0.76.el7.1.noarch because it is required by 1 installed package Not removing libipa_hbac-1.16.2-13.el7_6.8.x86_64 because it is required by 1 installed package Not removing sssd-client-1.16.2-13.el7_6.8.x86_64 because it is required by 1 installed package Not removing selinux-policy-3.13.1-229.el7_6.12.noarch because it is required by 2 installed packages Not removing grub2-common-2.02-0.76.el7.1.noarch because it is required by 2 installed packages Not removing grub2-tools-2.02-0.76.el7.1.x86_64 because it is required by 2 installed packages Not removing python-firewall-0.5.3-5.el7.noarch because it is required by 1 installed package Not removing dhcp-common-4.2.5-68.el7_5.1.x86_64 because it is required by 1 installed package Not removing openssl-libs-1.0.2k-16.el7_6.1.x86_64 because it is required by 1 installed package Not removing firewalld-filesystem-0.5.3-5.el7.noarch because it is required by 1 installed package Not removing libsss_sudo-1.16.2-13.el7_6.8.x86_64 because it is required by 1 installed package Not removing nss-3.36.0-7.1.el7_6.x86_64 because it is required by 1 installed package Not removing e2fsprogs-libs-1.42.9-13.el7.x86_64 because it is required by 1 installed package Not removing openssh-7.4p1-16.el7.x86_64 because it is required by 2 installed packages Not removing dhcp-libs-4.2.5-68.el7_5.1.x86_64 because it is required by 1 installed package Not removing NetworkManager-libnm-1.12.0-10.el7_6.x86_64 because it is required by 2 installed packages Not removing grub2-tools-minimal-2.02-0.76.el7.1.x86_64 because it is required by 2 installed packages Not removing samba-common-4.8.3-4.el7.noarch because it is required by 6 installed packages Not removing libcom_err-1.42.9-13.el7.x86_64 because it is required by 1 installed package Not removing python-sssdconfig-1.16.2-13.el7_6.8.noarch because it is required by 1 installed package ......................
At the first time I tried to resolve these conflict one by one. For instance, the package dhcp-common-4.2.5-68.el7_5.1.x86_64 could not be removed and I tried to know what package blocked this action. I run 'yum remove dhcp-common-4.2.5-68.el7_5.1.x86_64' but input 'n' to interrupt such removal and found package dhclient was the reason.
In fact, if the update script run well, there should be just one version, the newer one, packages on the system. But it was terminated, so not all the old packages were updated and removed. For this reason, I updated the dependent package dhclient to latest version, then run 'package-cleanup --cleandupes' command again and it let me to remove the old dhcp-common-4.2.5-68.el7_5.1.x86_64.
Using the above way most conflicts could be resolved except the glibc-common package.
I run below commands to confirm the newer version was installed correctly, then removed the old one and updated the dependent glibc.i686.
[root@xxxxxx yum.repos.d]# rpm -V glibc-common-2.17-260.el7_6.6.x86_64 S.5...... /usr/bin/gencat S.5...... /usr/bin/getconf S.5...... /usr/bin/getent S.5...... /usr/bin/iconv S.5...... /usr/bin/locale S.5...... /usr/bin/localedef S.5...... /usr/bin/makedb S.5...... /usr/bin/pldd S.5...... /usr/bin/rpcgen S.5...... /usr/bin/sprof S.5...... /usr/sbin/build-locale-archive S.5...... /usr/sbin/zdump S.5...... /usr/sbin/zic S.5...... d /usr/share/doc/glibc-common-2.17/ChangeLog.bz2 S.5...... /usr/share/i18n/charmaps/IBM273.gz [root@xxxxxx yum.repos.d]# rpm -V glibc-common-2.17-292.el7.x86_64 [root@xxxxxx yum.repos.d]# rpm -e --justdb glibc-common-2.17-260.el7_6.6.x86_64 error: Failed dependencies: glibc-common = 2.17-260.el7_6.6 is needed by (installed) glibc-2.17-260.el7_6.6.i686 [root@xxxxxx yum.repos.d]# rpm -e --justdb --nodeps glibc-common-2.17-260.el7_6.6.x86_64 [root@xxxxxx yum.repos.d]# yum update glibc.i686
Finally I could run our update script successfully.
