Toggle Submenu
  1. Forums
  2. Recent
  3. Tags
Add a reply View Replies (13)
  1. Forums
  2. KernelCare
  3. KernelCare General Discussion

Resolved 042stab127.2 problem

  1. Guest
    Fred Flint
  2. 04.03.2018
  3.  Subscribe via email
Applied kernelcare to two OpenVZ nodes yesterday.

042stab127.2

One of them seems ok but the other one had problems overnight. Really high swap and load even though there was still plenty of unused memory. Individual containers had all their vswap used. Not all containers are using vswap. I rebooted the containers that have vswap and that seemed to correct the swap/load. It's too much of a coincidence that this happened the evening I updated the kernel. Nothing else was done and this server has been running continuously for months

The other node is still ok. Hopefully restarting the containers corrected it and I won't have to roll back the kernel.

Both nodes are Supermicro with Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz and 16GB RAM

Both running kcare 042stab127.2

Base kernel is different though

The one still working is 2.6.32-042stab113.21
The one that had a problem is 2.6.32-042stab108.8
Rate this post:
0
Responses (13)
  1. 04.03.2018 15:03:15
  2. # 1
Fred Flint Accepted Answer
Guest
Posts: 0
Joined: 11.07.2020
0
Votes
Undo
BTW, they were both running the same kcare kernel before the update. The last one released before this meltdown/Spectre one.
  1. 05.03.2018 17:03:40
  2. # 2
Fred Flint Accepted Answer
So am I the only one having this problem? It is definitely something related to Kernelcare. It happen right after I had a heavy process run that clears out all the cache and causes high load for a few minutes. Right after that, instead of the load subsiding, it skyrocketed. Load average was 200 when it is normally below 4. CPU wait state was hopping around between 50% and 95% which means it's waiting for I/O.

I disabled the heavy cron job for now but I suspect the problem is still there and could be triggered again when the heavy cron job runs.
  1. 05.03.2018 18:03:36
  2. # 3
Fred Flint Accepted Answer
I should point out that I have updated other servers with the native 042stab127.2 kernel, not kernelcare, and haven't seen this problem.
  1. 05.03.2018 18:03:55
  2. # 4
Alexandre Parubochyi Accepted Answer
Alexandre Parubochyi
Posts: 187
Joined: 31.01.2017
0
Votes
Undo
Fred, please create a ticket at https://cloudlinux.zendesk.com (KernelCare department) so our support team can help you with the issue.
  1. 09.03.2018 23:03:12
  2. # 5
Peter Laws Accepted Answer
Peter Laws
Posts: 3
Joined: 22.01.2018
0
Votes
Undo
Was there any outcome to this? Was it KC? I've been scared to yum update kernelcare for a while now...
  1. 10.03.2018 13:03:25
  2. # 6
Alexandre Parubochyi Accepted Answer
Alexandre Parubochyi
Posts: 187
Joined: 31.01.2017
0
Votes
Undo
A ticket #26416 has been created to investigate the issue.
BTW, yum update kernelcare shouldn't bring in any problems with binary patches that KC applies.
  1. 10.03.2018 19:03:54
  2. # 7
Peter Laws Accepted Answer
Peter Laws
Posts: 3
Joined: 22.01.2018
0
Votes
Undo
I'm stupid, I thought you needed the latest KC to upgrade to stab127.2...... Seems I've had this kernel version for ages.... :o

as (_8(|) says..... d'oh
  1. 14.03.2018 15:03:11
  2. # 8
Fred Flint Accepted Answer
Guest
Posts: 0
Joined: 11.07.2020
0
Votes
Undo
So far, the high load problem has not re-occurred since the first time where I had restart containers that use vswap.
  1. 19.03.2018 18:03:13
  2. # 9
Vladimir Accepted Answer
Vladimir
Posts: 108
Joined: 04.07.2017
0
Votes
Undo
Glad to hear that.

Feel free to let me know if you have any further questions or concerns.
  1. 17.04.2018 18:04:06
  2. # 10
Fred Flint Accepted Answer
Guest
Posts: 0
Joined: 11.07.2020
0
Votes
Undo
Well it just happened again. I rebooted the 2 containers that seemed to be causing the high load and it went away again. I am going to have to upgrade this node to the real kernel and see if that takes care of it.
  1. 17.04.2018 19:04:11
  2. # 11
Vladimir Accepted Answer
Vladimir
Posts: 108
Joined: 04.07.2017
0
Votes
Undo
Hello.

We would like to check this problem.
Please submit a ticket to https://cloudlinux.zendesk.com, our techs will check the issue in place.
  1. 17.04.2018 20:04:23
  2. # 12
Fred Flint Accepted Answer
Guest
Posts: 0
Joined: 11.07.2020
0
Votes
Undo
Hello.

We would like to check this problem.
Please submit a ticket to https://cloudlinux.zendesk.com, our techs will check the issue in place.


You guys already did and said you couldn't find anything. It seems to be hard to reproduce. I see there was one race condition that was fixed in the latest OVZ kernel so maybe that took care of it. It started happening as soon as up updated Kernelcare and I haven's seen this on other nodes not running KernelCare but running the exact same real kernel. So I think the best thing is to run the real kernel in this case.
  1. 19.04.2018 15:04:40
  2. # 13
Vladimir Accepted Answer
Vladimir
Posts: 108
Joined: 04.07.2017
0
Votes
Undo
I am sorry, but it's hard to help you without a clear understanding what's going on on that server.
We really would like to check this problem in place. Please submit a ticket to https://cloudlinux.zendesk.com and our techs will investigate it further.
  • Page :
  • 1


There are no replies made for this post yet.
Be one of the first to reply to this post!
Guest
Submit Your Response
Upload files or images for this discussion by clicking on the upload button below. Supports gif,jpg,png,zip,rar,pdf
• Insert • Remove Upload Files (Maximum File Size: 2 MB)
Cite your Sources with a URL link to help readers verify facts or find more details.
Add Another Link
Captcha
To protect the site from bots and unauthorized scripts, we require that you enter the captcha codes below before posting your question.