Blog

The confusion of CloudLinux memory limits

The memory limits in CloudLinux are confusing at best. First of all they count virtual memory allocated by processes, instead of physical memory. And virtual memory use can be much higher, as Linux is very efficient in using same physical memory for multiple processes. We plan to add physical memory limits in the future – yet, this is not the only issue with memory limits. 

No matter if we limit physical or virtual memory, there will always be some guess work in detecting if the script error was due to memory limit, or if it was due to permissions, configuration errors or errors in the script itself. Such errors is the primary reason for us to ship CloudLinux with memory limits disabled by default.  Memory limits are useful, and can often save server from overloading, swapping & going down. Yet, they can also add errors, that most sys admins don't connect to memory limits right away

When software (such as php interpretor or mod_fcgid daemon) tries to allocate memory from a system, LVE can prevent that from happening. It would do it same way OS would do it in the case when there not enough memory. Most applications when they try to allocate memory, and fail, they will fail as well. It would look pretty much as if failed due to bug, or some other error. The distinction is very small, and usually comes as part of cryptic error message and strange exit code. When it comes to website, such errors usually pop up as error 500 – which means that script used to serve the request failed due to some error. In this case it usually means that PHP interpretor failed (same way it would fail on bad php script). Basically – PHP or some other components fails, for whatever reason, and error 500 served. Not much for CL to do here.

Sometimes it gets even worth. Recently we got a customer who complained about mail() not working in php script. It was working before, but it stopped working after CloudLinux was installed. We knew that CloudLinux 'never' does something like that, and were totally baffled. It was verifiable error. Running php script that was trying to send email would come back with:
Quote
Warning: mail() [function.mail]: Could not execute mail delivery program '/usr/sbin/sendmail -t -i'

Switching back to CentOS kernel would solve the problem (that would disable LVE). It took us some time to stumble upon the fact that it might be memory limits. Once we did, it took a minute to verify it. There was enough memory to run php interpretor, but not enough for sendmail to run on top of it. Hence sendmail would fail, and php would deliver such message. Increasing memory limit removed the issue. There is an easy way to figure out if the issue relates to memory limits. All you need to do is to run:
Code
# lveinfo --by-fault=mem --display-username


If you see user for which script failed in the list, it means that some script for that user hit memory limit within the past 10 minutes. Run the script again, re-check lveinfo (note, it takes 1 minute for it to update) – and you know for sure. Same information can be taken out of /proc/lve/list

Of course this is not enough, and we plan to do more. We want to create sophisticated notification system, so that both admin & user would be notified in case memory limits are reached. Additionally, we are researching the possibility to detect run time, on webserver level, when one of the processes that was used to serve up the request hit memory limit – and if we can intervene & serve our own error message in such cases. We are still at researching it – and if that would be possible, it would create a nice way to take out the confusion.