Working with LVE
About LVE
LVE is a kernel level technology developed by the CloudLinux team. The technology has common roots with container based virtualization. Yet, it is lightweight, and transparent. The goal of LVE is to make sure that no single web site can bring down your web server.
Today, a single site can consume all CPU resources, IO resources or apache processes -- and bring the server to the halt.
LVE prevents that.
It is done via collaboration of apache modules and kernel.
mod_hostinglimits is apache module that:
- Detects VirtualHost from which the request came.
- Detects if it was meant for cgi or PHP script.
- Switches apache process used to serve that request into LVE for the user determined via SuexecUserGroup for that virtual host.
- Lets apache to serve the request.
- Removes apache process from user's LVE.
The kernel makes sure that all LVEs get fair share of the server's resources. This means, for example, that 20 apache processes serving a heavy site will use the same amount of CPU as one apache process serving a smaller site.
We plan to expand this technology to include IO (coming soon)
Each LVE limits amount of entry processes (Apache processes entering into LVE) to prevent single site exhausting all apache processes. If the limit is reached -- mod_hostinglimits will not be able to place apache process into LVE, and will return error code 503 (server busy).
This way very heavy site would slow down and start returning 503 errors, without affecting other users.
Checking if LVE is installed
To use LVE you have to have CloudLinux kernel installed, and LVE module loaded
you can check the kernel by running the following command:
uname -r
You should see something like 2.6.18-294.8.1.el5.lve0.7.30 The kernel should have lve in its name. To see if lve kernel module is loaded do
# lsmod|grep lve
lve 46496 0
Enabling / Disabling LVE
You can toggle LVE on/ff by editing
/etc/sysconfig/lve and setting LVE_ENABLE variable to yes or no.
Setting it to yes will enable LVE, setting it to no will disable LVE.
You need to reboot the server, after you set this option to make the changes live
Adjusting LVE default values
Values set in /etc/container/ve.cfg are used by default when processes are running within LVE. After editing /etc/container/ve.cfg run the following command:
# /etc/init.d/lvectl restart
* This feature was disabled in current release due to the bug in the kernel that causes some sites to lock up if they had processes running while restart. Bellow you can find the work around:
# lvectl --set --ve default --cpu 30 --maxEntryProcs 20
# for i in `cat /proc/lve/list |cut -f1|grep -v veid`; do lvectl --set --ve $i --cpu 30 --maxEntryProcs 20; done
Adjusting LVE default values in config file
Example:
<lveconfig>
<defaults>
<cpu limit="100"/>
<io limit="100"/>
<other maxEntryProcs="10"/>
</defaults> <lve id="532"> <cpu limit="30"/> </lve> </lveconfig>
Sets CPU limit to 100%, IO limit to 100% and maximum entry processes to 10 for all LVEs. It also sets the limit of 30% for LVE with ID 532
Adjusting LVE settings on the fly
You can adjust LVE settings on the fly using command lvectl
lvectl accepts four parameters
# lvectl --set --ve 600 --cpu 70
Adjust CPU limit for LVE id 600 to 70%
Monitoring LVE
You can see a list of LVEs with their CPU usage and limits by doing cat /proc/lve/list
The umber of entry processes inside LVE will be the value under *refs" minus two. If refs = 4, it means LVE has 2 entry processes.
cpu_usage field shows the total amount CPU (in jiffies) all threads within LVE use up to this moment (since last restart).
[root@localhost tests]# cat /proc/lve/list
veid refs cpu io cpu_usage
900 22 100 100 229662988927
Additionally you can use tool lveps to see CPU usage, and processes within LVE
usage: lveps [-p] [-n] [-h] [-d]
-p to print per-process/per-thread statistics
-n to print uid/gid rather than user/group
-h to print this brief help message
-d displays CPU usage as a percentage
[root@localhost tests]# lveps
LVE UID GID CPU MEM I/O
900 root root 9 1440 N/A
[root@localhost tests]# lveps -p
LVE PID TID UID GID CPU MEM I/O
900 --- --- root root 39 1440 N/A
--- 14959 14959 root root 1 72 N/A
--- 14958 14958 root root 1 72 N/A
--- 14957 14957 root root 1 72 N/A
--- 14956 14956 root root 1 72 N/A
--- 14955 14955 root root 1 72 N/A
--- 14954 14954 root root 1 72 N/A
--- 14953 14953 root root 1 72 N/A
--- 14952 14952 root root 1 72 N/A
--- 14951 14951 root root 1 72 N/A
--- 14950 14950 root root 1 72 N/A
--- 14949 14949 root root 1 72 N/A
--- 14948 14948 root root 1 72 N/A
--- 14947 14947 root root 1 72 N/A
--- 14946 14946 root root 1 72 N/A
--- 14945 14945 root root 1 72 N/A
--- 14944 14944 root root 1 72 N/A
--- 14943 14943 root root 1 72 N/A
--- 14942 14942 root root 1 72 N/A
--- 14941 14941 root root 1 72 N/A
--- 14940 14940 root root 1 72 N/A
--- 14918 14918 root root 0 0 N/A
The CPU value shown is number of seconds all the threads used.
Apache and LVE
CloudLinux comes standard with the module mod_hostinglimits that puts apache processes into LVEs. For more information on this module see: mod_hostinglimits config info
Disabling that module should disable LVE on for apache.
Apache and LVE -- mod_sucgid
CloudLinux has optional mod_sucgid module that runs as a daemon (similar to mod_cgid). More info here: mod_sucgid
Apache and LVE -- mod_fcgid
CloudLinux packages modified version of mod_fcgid, that supports LVE. More info here: FastCGI