oom killer lsphp
Forum
  1. Forums
  2. CloudLinux and Control Panels
  3. CloudLinux and cPanel
  1. Havri
  2. 22.05.2018
  3.  Subscribe via email
Hello,

I'm trying to debug some of the messages in /var/log/messages that are part of the LVE oom killer. I've already written on the Litespeed forums (as the processes that are getting killed by oom killer are lsphp processes). The link to the thread is the one below:

https://www.litespeedtech.com/support/forum/threads/oom-killer-on-litespeed-processes.16352/

Basically, what I want to know is if the number in these messages "Memory cgroup stats for /lve1421:" and "Memory cgroup stats for /1168" are actually the UID number of the users that are getting their lsphp processes killed. I'll put the messages here also:


May 18 11:05:17 entry01 kernel: oom_kill_process: 1 callbacks suppressed
May 18 11:05:17 entry01 kernel: lsphp invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
May 18 11:05:17 entry01 kernel: lsphp cpuset=lve1421 mems_allowed=0-1
May 18 11:05:17 entry01 kernel: Task in /lve1421 killed as a result of limit of /lve1421
May 18 11:05:17 entry01 kernel: memory: usage 524288kB, limit 524288kB, failcnt 90827
May 18 11:05:17 entry01 kernel: memory+swap: usage 524288kB, limit 9007199254740988kB, failcnt 0
May 18 11:05:17 entry01 kernel: kmem: usage 0kB, limit 9007199254740988kB, failcnt 0
May 18 11:05:17 entry01 kernel: Memory cgroup stats for /lve1421: cache:160KB rss:524128KB rss_huge:442368KB mapped_file:4KB shmem:4KB slab_reclaimable:0KB slab_unreclaimable:0KB swap:0KB inactive_anon:263884KB active_anon:260248KB inactive_file:16KB active_file:8KB unevictable:0KB
May 18 11:05:17 entry01 kernel: Memory cgroup out of memory: Kill process 32246 (lsphp) score 141 or sacrifice child
May 18 11:05:17 entry01 kernel: Killed process 32246 (lsphp) in VE "0" total-vm:415656kB, anon-rss:69556kB, file-rss:3944kB, shmem-rss:4kB
May 18 11:05:17 entry01 kernel: lsphp invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
May 18 11:05:17 entry01 kernel: lsphp cpuset=lve1421 mems_allowed=0-1
May 18 11:05:17 entry01 kernel: Task in /lve1421 killed as a result of limit of /lve1421
May 18 11:05:17 entry01 kernel: memory: usage 524288kB, limit 524288kB, failcnt 92291
May 18 11:05:17 entry01 kernel: memory+swap: usage 524288kB, limit 9007199254740988kB, failcnt 0
May 18 11:05:17 entry01 kernel: kmem: usage 0kB, limit 9007199254740988kB, failcnt 0
May 18 11:05:17 entry01 kernel: Memory cgroup stats for /lve1421: cache:28KB rss:524260KB rss_huge:430080KB mapped_file:4KB shmem:4KB slab_reclaimable:0KB slab_unreclaimable:0KB swap:0KB inactive_anon:263632KB active_anon:260632KB inactive_file:8KB active_file:16KB unevictable:0KB
May 18 11:05:17 entry01 kernel: Memory cgroup out of memory: Kill process 32250 (lsphp) score 145 or sacrifice child
May 18 11:05:17 entry01 kernel: Killed process 32250 (lsphp) in VE "0" total-vm:417720kB, anon-rss:71608kB, file-rss:3940kB, shmem-rss:4kB
May 18 11:05:18 entry01 kernel: lsphp invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
May 18 11:05:18 entry01 kernel: lsphp cpuset=lve1421 mems_allowed=0-1
May 18 11:05:18 entry01 kernel: Task in /lve1421 killed as a result of limit of /lve1421
May 18 11:05:18 entry01 kernel: memory: usage 524288kB, limit 524288kB, failcnt 93999
May 18 11:05:18 entry01 kernel: memory+swap: usage 524288kB, limit 9007199254740988kB, failcnt 0
May 18 11:05:18 entry01 kernel: kmem: usage 0kB, limit 9007199254740988kB, failcnt 0
May 18 11:05:18 entry01 kernel: Memory cgroup stats for /lve1421: cache:32KB rss:524256KB rss_huge:407552KB mapped_file:4KB shmem:4KB slab_reclaimable:0KB slab_unreclaimable:0KB swap:0KB inactive_anon:264108KB active_anon:260152KB inactive_file:8KB active_file:8KB unevictable:0KB
May 18 11:05:18 entry01 kernel: Memory cgroup out of memory: Kill process 32257 (lsphp) score 165 or sacrifice child
May 18 11:05:18 entry01 kernel: Killed process 32257 (lsphp) in VE "0" total-vm:430440kB, anon-rss:82260kB, file-rss:3960kB, shmem-rss:4kB
May 18 11:05:18 entry01 kernel: lsphp invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
May 18 11:05:18 entry01 kernel: lsphp cpuset=lve1421 mems_allowed=0-1
May 18 11:05:18 entry01 kernel: Task in /lve1421 killed as a result of limit of /lve1421
May 18 11:05:18 entry01 kernel: memory: usage 524288kB, limit 524288kB, failcnt 95211
May 18 11:05:18 entry01 kernel: memory+swap: usage 524288kB, limit 9007199254740988kB, failcnt 0
May 18 11:05:18 entry01 kernel: kmem: usage 0kB, limit 9007199254740988kB, failcnt 0
May 18 11:05:18 entry01 kernel: Memory cgroup stats for /lve1421: cache:36KB rss:524252KB rss_huge:403456KB mapped_file:4KB shmem:4KB slab_reclaimable:0KB slab_unreclaimable:0KB swap:0KB inactive_anon:268592KB active_anon:255664KB inactive_file:24KB active_file:8KB unevictable:0KB
May 18 11:05:18 entry01 kernel: Memory cgroup out of memory: Kill process 32258 (lsphp) score 121 or sacrifice child
May 18 11:05:18 entry01 kernel: Killed process 32258 (lsphp) in VE "0" total-vm:405352kB, anon-rss:59204kB, file-rss:3828kB, shmem-rss:4kB
May 18 11:05:19 entry01 kernel: lsphp invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
May 18 11:05:19 entry01 kernel: lsphp cpuset=lve1421 mems_allowed=0-1
May 18 11:05:19 entry01 kernel: Task in /lve1421 killed as a result of limit of /lve1421
May 18 11:05:19 entry01 kernel: memory: usage 524288kB, limit 524288kB, failcnt 97351
May 18 11:05:19 entry01 kernel: memory+swap: usage 524288kB, limit 9007199254740988kB, failcnt 0
May 18 11:05:19 entry01 kernel: kmem: usage 0kB, limit 9007199254740988kB, failcnt 0
May 18 11:05:19 entry01 kernel: Memory cgroup stats for /lve1421: cache:28KB rss:524260KB rss_huge:397312KB mapped_file:4KB shmem:4KB slab_reclaimable:0KB slab_unreclaimable:0KB swap:0KB inactive_anon:273228KB active_anon:251036KB inactive_file:8KB active_file:4KB unevictable:0KB
May 18 11:05:19 entry01 kernel: Memory cgroup out of memory: Kill process 32292 (lsphp) score 157 or sacrifice child
May 18 11:05:19 entry01 kernel: Killed process 32292 (lsphp) in VE "0" total-vm:426240kB, anon-rss:77972kB, file-rss:3924kB, shmem-rss:4kB
May 18 11:05:20 entry01 kernel: lsphp invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
May 18 11:05:20 entry01 kernel: lsphp cpuset=lve1421 mems_allowed=0-1
May 18 11:05:20 entry01 kernel: Task in /lve1421 killed as a result of limit of /lve1421
May 18 11:05:20 entry01 kernel: memory: usage 524288kB, limit 524288kB, failcnt 98512
May 18 11:05:20 entry01 kernel: memory+swap: usage 524288kB, limit 9007199254740988kB, failcnt 0
May 18 11:05:20 entry01 kernel: kmem: usage 0kB, limit 9007199254740988kB, failcnt 0
May 18 11:05:20 entry01 kernel: Memory cgroup stats for /lve1421: cache:28KB rss:524260KB rss_huge:397312KB mapped_file:4KB shmem:4KB slab_reclaimable:0KB slab_unreclaimable:0KB swap:0KB inactive_anon:292304KB active_anon:231960KB inactive_file:12KB active_file:12KB unevictable:0KB
May 18 11:05:20 entry01 kernel: Memory cgroup out of memory: Kill process 32251 (lsphp) score 136 or sacrifice child
May 18 11:05:20 entry01 kernel: Killed process 32251 (lsphp) in VE "0" total-vm:413620kB, anon-rss:66716kB, file-rss:3940kB, shmem-rss:4kB
May 18 11:05:20 entry01 kernel: lsphp invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
May 18 11:05:20 entry01 kernel: lsphp cpuset=lve1421 mems_allowed=0-1
May 18 11:05:20 entry01 kernel: Task in /lve1421 killed as a result of limit of /lve1421
May 18 11:05:20 entry01 kernel: memory: usage 524288kB, limit 524288kB, failcnt 100039
May 18 11:05:20 entry01 kernel: memory+swap: usage 524288kB, limit 9007199254740988kB, failcnt 0
May 18 11:05:20 entry01 kernel: kmem: usage 0kB, limit 9007199254740988kB, failcnt 0
May 18 11:05:20 entry01 kernel: Memory cgroup stats for /lve1421: cache:32KB rss:524256KB rss_huge:399360KB mapped_file:4KB shmem:4KB slab_reclaimable:0KB slab_unreclaimable:0KB swap:0KB inactive_anon:278656KB active_anon:245604KB inactive_file:12KB active_file:12KB unevictable:0KB
May 18 11:05:20 entry01 kernel: Memory cgroup out of memory: Kill process 32252 (lsphp) score 159 or sacrifice child
May 18 11:05:20 entry01 kernel: Killed process 32252 (lsphp) in VE "0" total-vm:426208kB, anon-rss:78764kB, file-rss:3880kB, shmem-rss:4kB




May 20 23:18:51 my_server01 kernel: lsphp invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
May 20 23:18:51 my_server01 kernel: lsphp cpuset=1168 mems_allowed=0-1
May 20 23:18:51 my_server01 kernel: Task in /1168 killed as a result of limit of /1168
May 20 23:18:51 my_server01 kernel: memory: usage 1048576kB, limit 1048576kB, failcnt 224006
May 20 23:18:51 my_server01 kernel: memory+swap: usage 1048576kB, limit 9007199254740988kB, failcnt 0
May 20 23:18:51 my_server01 kernel: kmem: usage 0kB, limit 9007199254740988kB, failcnt 0
May 20 23:18:51 my_server01 kernel: Memory cgroup stats for /1168: cache:16KB rss:1048560KB rss_huge:0KB mapped_file:4KB shmem:4KB slab_reclaimable:0KB slab_unreclaimable:0KB swap:0KB inactive_anon:279160KB active_anon:769404KB inactive_file:8KB active_file:4KB unevictable:0KB
May 20 23:18:51 my_server01 kernel: Memory cgroup out of memory: Kill process 9597 (lsphp) score 152 or sacrifice child
May 20 23:18:51 my_server01 kernel: Killed process 9597 (lsphp) in VE "0" total-vm:572956kB, anon-rss:153572kB, file-rss:5820kB, shmem-rss:4kB
May 20 23:15:02 my_server01 kernel: lsphp invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
May 20 23:15:02 my_server01 kernel: lsphp cpuset=1168 mems_allowed=0-1
May 20 23:15:02 my_server01 kernel: Task in /1168 killed as a result of limit of /1168
May 20 23:15:02 my_server01 kernel: memory: usage 1048576kB, limit 1048576kB, failcnt 222937
May 20 23:15:02 my_server01 kernel: memory+swap: usage 1048576kB, limit 9007199254740988kB, failcnt 0
May 20 23:15:02 my_server01 kernel: kmem: usage 0kB, limit 9007199254740988kB, failcnt 0
May 20 23:15:02 my_server01 kernel: Memory cgroup stats for /1168: cache:8KB rss:1048568KB rss_huge:0KB mapped_file:4KB shmem:4KB slab_reclaimable:0KB slab_unreclaimable:0KB swap:0KB inactive_anon:525060KB active_anon:523512KB inactive_file:0KB active_file:4KB unevictable:0KB
May 20 23:15:02 my_server01 kernel: Memory cgroup out of memory: Kill process 5554 (lsphp) score 122 or sacrifice child
May 20 23:15:02 my_server01 kernel: Killed process 5554 (lsphp) in VE "0" total-vm:467536kB, anon-rss:123824kB, file-rss:3776kB, shmem-rss:4kB
May 20 23:15:02 my_server01 kernel: lsphp invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
May 20 23:15:02 my_server01 kernel: lsphp cpuset=1168 mems_allowed=0-1
May 20 23:15:02 my_server01 kernel: Task in /1168 killed as a result of limit of /1168
May 20 23:15:02 my_server01 kernel: memory: usage 1048576kB, limit 1048576kB, failcnt 223039
May 20 23:15:02 my_server01 kernel: memory+swap: usage 1048576kB, limit 9007199254740988kB, failcnt 0
May 20 23:15:02 my_server01 kernel: kmem: usage 0kB, limit 9007199254740988kB, failcnt 0
May 20 23:15:02 my_server01 kernel: Memory cgroup stats for /1168: cache:8KB rss:1048568KB rss_huge:0KB mapped_file:4KB shmem:4KB slab_reclaimable:0KB slab_unreclaimable:0KB swap:0KB inactive_anon:497324KB active_anon:551124KB inactive_file:4KB active_file:0KB unevictable:0KB
May 20 23:15:02 my_server01 kernel: Memory cgroup out of memory: Kill process 5577 (lsphp) score 120 or sacrifice child
May 20 23:15:02 my_server01 kernel: Killed process 5577 (lsphp) in VE "0" total-vm:465244kB, anon-rss:121892kB, file-rss:3656kB, shmem-rss:4kB
May 20 23:15:03 my_server01 kernel: lsphp invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
May 20 23:15:03 my_server01 kernel: lsphp cpuset=1168 mems_allowed=0-1
May 20 23:15:03 my_server01 kernel: Task in /1168 killed as a result of limit of /1168
May 20 23:15:03 my_server01 kernel: memory: usage 1048576kB, limit 1048576kB, failcnt 223135
May 20 23:15:03 my_server01 kernel: memory+swap: usage 1048576kB, limit 9007199254740988kB, failcnt 0
May 20 23:15:03 my_server01 kernel: kmem: usage 0kB, limit 9007199254740988kB, failcnt 0
May 20 23:15:03 my_server01 kernel: Memory cgroup stats for /1168: cache:8KB rss:1048568KB rss_huge:0KB mapped_file:4KB shmem:4KB slab_reclaimable:0KB slab_unreclaimable:0KB swap:0KB inactive_anon:524512KB active_anon:524060KB inactive_file:0KB active_file:4KB unevictable:0KB
May 20 23:15:03 my_server01 kernel: Memory cgroup out of memory: Kill process 5580 (lsphp) score 121 or sacrifice child
May 20 23:15:03 my_server01 kernel: Killed process 5580 (lsphp) in VE "0" total-vm:465980kB, anon-rss:122476kB, file-rss:3840kB, shmem-rss:4kB
May 20 23:15:04 my_server01 kernel: lsphp invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
May 20 23:15:04 my_server01 kernel: lsphp cpuset=1168 mems_allowed=0-1
May 20 23:15:04 my_server01 kernel: Task in /1168 killed as a result of limit of /1168
May 20 23:15:04 my_server01 kernel: memory: usage 1048576kB, limit 1048576kB, failcnt 223310
May 20 23:15:04 my_server01 kernel: memory+swap: usage 1048576kB, limit 9007199254740988kB, failcnt 0
May 20 23:15:04 my_server01 kernel: kmem: usage 0kB, limit 9007199254740988kB, failcnt 0
May 20 23:15:04 my_server01 kernel: Memory cgroup stats for /1168: cache:8KB rss:1048568KB rss_huge:0KB mapped_file:4KB shmem:4KB slab_reclaimable:0KB slab_unreclaimable:0KB swap:0KB inactive_anon:524464KB active_anon:523980KB inactive_file:0KB active_file:4KB unevictable:0KB
May 20 23:15:04 my_server01 kernel: Memory cgroup out of memory: Kill process 5611 (lsphp) score 123 or sacrifice child
May 20 23:15:04 my_server01 kernel: Killed process 5611 (lsphp) in VE "0" total-vm:469340kB, anon-rss:125292kB, file-rss:3764kB, shmem-rss:4kB




Also, I want to know if the solution to these problems is to increase the PMEM for that user's LVE.

Let me know if you need any other info.

Thank you.
Rate this post:
  1. 30.05.2018 11:05:32
  2. # 1
Igor Ghertesco Accepted Answer
Posts: 154
Joined: 07.08.2015
0
Votes
Undo
Hello,

Correct, usually LVE_ID = UID, and those messages mean that user is hitting PMEM limit.
  • Page :
  • 1


There are no replies made for this post yet.
Be one of the first to reply to this post!
Guest
Submit Your Response
Upload files or images for this discussion by clicking on the upload button below. Supports gif,jpg,png,zip,rar,pdf
• Insert • Remove Upload Files (Maximum File Size: 2 MB)
Captcha
To protect the site from bots and unauthorized scripts, we require that you enter the captcha codes below before posting your question.