CloudLinux OS Blog - Issues with CloudLinux OS 7.7 update on systems with kernel v3.10.0-862 or lower
CloudLinux OS Blog

Featured 

Issues with CloudLinux OS 7.7 update on systems with kernel v3.10.0-862 or lower

CL_OS_kernel_warning

Issue description

The recent CloudLinux OS 7.7 update includes the iproute-4.11.0-25.el7[.cloudlinux] package that is not working correctly on systems with a 3.10.0-862 or lower kernel version.
If you apply the update and reboot the server, the network will probably go down.

Issue solution

The solution is to upgrade to the iproute-4.11.0-25.el7.cloudlinux.1 package by running the command:

yum upgrade iproute*

Additional notes

Kernels newer than 3.20.0-862 do not face the issue with the iproute-4.11.0-25.el7[.cloudlinux] package.

Explanations and apologies

We are very sorry about this issue.

We knew that the new iproute-4.11.0-25.el7 package is not compatible with kernels lower than 3.10.0-862 and mentioned it in our release announcement and took the appropriate steps: we added conflicts to the new packages and tested the update on the typical configurations.
But, unfortunately, we did not take into account all cases and some of our clients faced issues after the update.
Now, we are investigating the issue and releasing the hot-fix which is actually a downgrade to the previous stable package.

Once again, please accept our sincerest apologies for this problem and be assured that we have a plan for preventing this from happening again.

Beta: kernel module for CloudLinux 7 hybrid update...
Beta: Alt-PHP updated
 

Comments 19

Guest - Andrew Brown on Wednesday, 04 September 2019 11:39

So how do we run a patch on a virtual machine that now has no network connection??

Is there any file level amendement we can make as I can mount the drive on another machine to access, but I cant run it as the OS> as it has no network

So how do we run a patch on a virtual machine that now has no network connection?? Is there any file level amendement we can make as I can mount the drive on another machine to access, but I cant run it as the OS> as it has no network
Inessa Atmachian on Wednesday, 04 September 2019 12:18

Hi Andrew!
Here is a quick instruction:
1. Boot on another machine (recovery, LiveCD, etc).
2. Mount affected system into the directory including /proc, /dev, etc.
3. Download https://repo.cloudlinux.com/cloudlinux/7/os/x86_64/Packages/iproute-4.11.0-25.el7.cloudlinux.1.x86_64.rpm package into that directory.
4. Chroot into the directory from step 2.
5. Install the package and reboot

Hi Andrew! Here is a quick instruction: 1. Boot on another machine (recovery, LiveCD, etc). 2. Mount affected system into the directory including /proc, /dev, etc. 3. Download https://repo.cloudlinux.com/cloudlinux/7/os/x86_64/Packages/iproute-4.11.0-25.el7.cloudlinux.1.x86_64.rpm package into that directory. 4. Chroot into the directory from step 2. 5. Install the package and reboot
Inessa Atmachian on Wednesday, 04 September 2019 12:30

If you have any doubts, please contact our support team at https://cloudlinux.zendesk.com/hc/requests/new

If you have any doubts, please contact our support team at https://cloudlinux.zendesk.com/hc/requests/new
Guest - Andrew Brown on Wednesday, 04 September 2019 12:36

OK, tried this.

1. Booted another machine and attached the bad OS drive and mounted it to /odin
2. Can see the content of the back OS disk with /proc, /dev/ boot, /root etc
3. Got the file and CHMOD 755 ok
4. CHroot /odin. working ok
5. RPM -i Throwing "error: Failed to initialize NSS library"

Advice please, still have multiple servers offline due to this mistake.

OK, tried this. 1. Booted another machine and attached the bad OS drive and mounted it to /odin 2. Can see the content of the back OS disk with /proc, /dev/ boot, /root etc 3. Got the file and CHMOD 755 ok 4. CHroot /odin. working ok 5. RPM -i Throwing "error: Failed to initialize NSS library" Advice please, still have multiple servers offline due to this mistake.
Guest - Georgi Vasilev on Wednesday, 04 September 2019 13:00

Hi, you should mount bind the dev directory
mount -o bind /dev /odin/dev/

Hi, you should mount bind the dev directory mount -o bind /dev /odin/dev/
Guest - Andrew Brown on Wednesday, 04 September 2019 13:27

Thank you...

These steps got our servers back online....

Now for the debrief and find out how to disable ALL auto updates!

We all make mistakes, but this was a big one and CloudLinux def dropped the ball!

Thank you... These steps got our servers back online.... Now for the debrief and find out how to disable ALL auto updates! We all make mistakes, but this was a big one and CloudLinux def dropped the ball!
Diogo on Wednesday, 04 September 2019 14:47

Hi all,

The solution we found was download the RPM of the new kernel version ( https://www.repo.cloudlinux.com/cloudlinux/7/os/x86_64/Packages/kernel-3.10.0-962.3.2.lve1.5.26.2.el7.x86_64.rpm ) and move it to a pen drive.

Then we started the server, mounted the pen drive and we've installed the kernel using the rpm -ivh command.

After the reboot the network devices will start, and you can run the yum update.

In case you have VPS servers and you're able to manage it, you can create an ISO from the pen drive you used to copy the rpm, upload that ISO to your datastore/virtualization environment and then you can mount it on the VPS server to install the kernel manually.

Hi all, The solution we found was download the RPM of the new kernel version ( https://www.repo.cloudlinux.com/cloudlinux/7/os/x86_64/Packages/kernel-3.10.0-962.3.2.lve1.5.26.2.el7.x86_64.rpm ) and move it to a pen drive. Then we started the server, mounted the pen drive and we've installed the kernel using the rpm -ivh command. After the reboot the network devices will start, and you can run the yum update. In case you have VPS servers and you're able to manage it, you can create an ISO from the pen drive you used to copy the rpm, upload that ISO to your datastore/virtualization environment and then you can mount it on the VPS server to install the kernel manually.
Guest - Ryan Smith on Wednesday, 04 September 2019 15:25

What's the long term plan for the iproute package for users of older kernels? Will you make it so older kernels will always be compatible with this package going forward? We have iproute excluded in /etc/yum.conf now and don't want to encounter any expected issues down the road.

What's the long term plan for the iproute package for users of older kernels? Will you make it so older kernels will always be compatible with this package going forward? We have iproute excluded in /etc/yum.conf now and don't want to encounter any expected issues down the road.
Guest - Ryan Smith on Wednesday, 04 September 2019 23:59

Are there any known issues with Imunify360 under CloudLinux 7.7? I'm seeing a huge increase in incidents after upgrading to CL 7.7 and I'm starting to wonder if Imunify360 is actually blocking the traffic now or not.

Are there any known issues with Imunify360 under CloudLinux 7.7? I'm seeing a huge increase in incidents after upgrading to CL 7.7 and I'm starting to wonder if Imunify360 is actually blocking the traffic now or not.
Guest - Elliott Willink on Thursday, 05 September 2019 01:36

Hope this helps someone, but before you try offline copying RPM's, booting a much older kernel (possibly even rescue/CL 7.2 can't remember exactly which one worked) brought the network back up. Then a "yum downgrade iproute" (Did this twice to get back to the previous version, looked like it had updated twice?) and reboot.

Once it booted back into 7.7 I did a yum update kernel, (reboot just to be sure), yum update iproute, reboot and now everything is up to date and happy.

Hope this helps someone, but before you try offline copying RPM's, booting a much older kernel (possibly even rescue/CL 7.2 can't remember exactly which one worked) brought the network back up. Then a "yum downgrade iproute" (Did this twice to get back to the previous version, looked like it had updated twice?) and reboot. Once it booted back into 7.7 I did a yum update kernel, (reboot just to be sure), yum update iproute, reboot and now everything is up to date and happy.
Already Registered? Login Here
Guest
Tuesday, 19 November 2019

Captcha Image