CloudLinux OS Blog - Issues caused by CloudLinux OS 7.7 update and what we will do to ensure it never happens again
CloudLinux OS Blog

Featured 

Issues caused by CloudLinux OS 7.7 update and what we will do to ensure it never happens again

issue

We want to apologize for the issues that CloudLinux OS 7.7 updated caused. Earning your trust through providing stability, density, and security to your servers is our highest priority. We regret this issue and would like to say “Sorry” to all our customers who rely on us and have been affected by the CloudLinux OS 7.7 update.

Issue investigation

Our test system, which uses a number of synthetic tests, does not include procedures for old kernels v3.10.0-862 or lower.

The other part of the problem is attributed to our deployment process. We release a lot of packages at any given moment to ALL customer's servers and this routine normally goes fine. Yet, we are only human and errors are possible.

Our resolutions

  • First of all, we will include old kernels v3.10.0-862 or lower in our test procedures.
  • Secondly, we will use a gradual roll-out approach for CloudLinux OS updates, the same as we are doing for Imunify360 and ImunifyAV. Gradual roll-out allows us to catch issues quickly without affecting a large number of customers. We will make updates available to 1%, then 2% then 4%, 8%, 16% ... etc. of servers and wait for feedback, instead of rolling it out to all servers at the same time. It might take a week or even a month to roll-out changes to everyone, as we might have to make fixes and re-start the roll-out, but it will cause less issues.
  • We will be implementing gradual roll-out in Q4 of 2019.

Once again, please accept our sincerest apologies and be assured that we have a plan for preventing this from happening ever again.

Igor Seletskiy,
CloudLinux CEO

Beta: alt-mod-passenger updated
Beta: Alt-PHP updated
 

Comments 2

LabShop Wrk on Friday, 04 October 2019 11:02

Have enabled my servers to update on some weekdays using Cron. Example:
Server1
10 22 * * * root /usr/sbin/whmapi1 update_updateconf UPDATES=manual RPMUP=manual SARULESUP=daily
20 22 * * 1 root /usr/sbin/whmapi1 update_updateconf UPDATES=manual RPMUP=manual SARULESUP=daily
Server2
10 22 * * * root /usr/sbin/whmapi1 update_updateconf UPDATES=manual RPMUP=manual SARULESUP=daily
20 22 * * 2 root /usr/sbin/whmapi1 update_updateconf UPDATES=manual RPMUP=manual SARULESUP=daily

Have enabled my servers to update on some weekdays using Cron. Example: Server1 10 22 * * * root /usr/sbin/whmapi1 update_updateconf UPDATES=manual RPMUP=manual SARULESUP=daily 20 22 * * 1 root /usr/sbin/whmapi1 update_updateconf UPDATES=manual RPMUP=manual SARULESUP=daily Server2 10 22 * * * root /usr/sbin/whmapi1 update_updateconf UPDATES=manual RPMUP=manual SARULESUP=daily 20 22 * * 2 root /usr/sbin/whmapi1 update_updateconf UPDATES=manual RPMUP=manual SARULESUP=daily
LabShop Wrk on Friday, 04 October 2019 11:04

Sorry, 20 22 * * 1 root /usr/sbin/whmapi1 update_updateconf UPDATES=daily RPMUP=daily SARULESUP=daily

Sorry, 20 22 * * 1 root /usr/sbin/whmapi1 update_updateconf UPDATES=daily RPMUP=daily SARULESUP=daily
Already Registered? Login Here
Guest
Saturday, 14 December 2019

Captcha Image