Linux Load Average explained
load average: 0.00, 0.00, 0.00
How do you get this output?
To get your system’s Load Average, run the command uptime. It will show you the current time, how long your system has been powered on for, the number of users logged in, and finally the system’s load average.
What does it mean?
Simply, it is the number of blocking processes in the run queue averaged over a certain time period.
Time periods:
load average: 1min, 5min, 15min
What is a blocking process?
A blocking process is a process that is waiting for something to continue. Typically, a process is waiting for:
- CPU
- Disk I/O
- Network I/O
What does a high load average mean?
A high load average typically means that your server is under-specified for what it is being used for, or that something has failed (like an externally mounted disk).
How do I diagnose a high load average?
Typically, a server with a high load average is unresponsive and slow — and you want to reduce the load and increase responsiveness. But how do you go about working out what is causing your high load?
Lets start with the simplest one, are we waiting for CPU? Run the Linux command top.
Check the numbers above in the red circle. They are basically representing what percentage of its’ total time the CPU is spending processing stuff. If these numbers are constantly around 99-100% then chances are the problem is related to your CPU, almost certainly that it is under powered. Consider upgrading your CPU.
The next thing to look for is if the cpu is waiting on I/O. Now check the number around where the red circle is now. If this number is high (above 80% or so) then you have problems. This means that the CPU is spending a LOT of time waiting in I/O. This could mean that you have a failing Hard Disk, Failing Network Card, or that your applications are trying to access data on either of them at a rate significantly higher than the throughput that they are designed for.
To find out what applications are causing the load, run the command ps faux. This will list every process running on your system, and the state it is in.
You want to look in the STAT column. The common flags that you should be looking for are:
- R – Running
- S – Sleeping
- D – Waiting for something
So, look for any processes with a STAT of D, and you can go from there to diagnose the problem.
Further Diagnosis
To diagnose further, you can use the following programs
- strace – to trace what a program is doing
- iostat – to see the throughput of your disks
- bwmon – to see your network throughput
November 28th, 2007 at 5:49 am
Nice little article…. really useful. Can you please do a similar article on strace iostat and bwmon too.
Thanks,
May 3rd, 2008 at 2:29 pm
Excellent Article! Will use some information on my blog too. Waiting for Further Diagnosis Article :)
May 9th, 2008 at 2:22 pm
Thanks a lot for great article with step by step instructions. Very ease and very useful. Really will be great if you will continue about strace, iostats and similar utilities. Thank you again.
June 13th, 2008 at 11:21 am
Nice article and very informative for the basics of Linux.
Thx.
August 5th, 2008 at 7:47 am
Identified that claamscan is running curently for every host and need to change the scheduling of each host for clamscan for a single instance at a time.
The more hosts, and more concurrent instances of clamscan, the greater the load average and slower the response time.
Thanks
PHK Corporation
December 2nd, 2008 at 8:19 pm
Really helpful information.
December 6th, 2008 at 9:37 am
[…] Concise and clear explaination of linux load averages. […]
July 18th, 2009 at 3:35 am
Excuse me , but what if I have a high value of “cpu %id”? What does this mean?
July 30th, 2009 at 3:21 pm
cpu %id means idle, not doing anything
August 3rd, 2009 at 6:13 pm
Thanks for the clear, concise post on load averages — I used it as a reference for my own writeup: http://blog.scoutapp.com/articles/2009/07/31/understanding-load-averages. Cheers!
September 2nd, 2009 at 10:47 am
Good article, easy to spot any bottleneck which slows down the system. CPU, Network, Disk, Process, and RAM.
October 13th, 2009 at 3:23 pm
very nice, further these types of diagnosis articles would be very helpful. thanks.
October 24th, 2009 at 4:36 pm
hi, do you know why Linux load average counts uninterruptable process?
FreeBSD8 does not, after checking sys/kern/sched_4bsd.c, kern_synch.c.
However Kernel 2.6.27 does based on Linux/kernel/sched.c, and timer.c.
Also, if you know… is there any changes made the method of load average calculation after Kernel 2.6.16?
Unfortunately I could not find sorce code for kernel 2.6.16.
If you know anything about it, it is very helpful.
November 21st, 2009 at 2:43 am
Clear, concise and to the point…just what I was looking for. Thank you for taking the time.
February 4th, 2010 at 9:52 am
Dears,
Kindly correct me if i am mistaken.
I disagree with the above in the part of checking the SY and US , its just saying the total amount of time the user/system processes is talking from the CPU time (so,its simply a function of time) , you may have many process waiting for the processor to handle , and you can normally find the the SY and US is low.
Thank you.
October 3rd, 2010 at 7:32 am
I didn’t ever need this until I got a linux virtual instance with only 256 megs of RAM =\
November 6th, 2010 at 9:51 am
great article. This is exactly what I need: step by step guide doubled by exact explanations. Good job!
August 7th, 2011 at 6:11 am
you are awesome.
December 2nd, 2011 at 7:27 pm
Excellent article. Recently there was a question on load average in my company. Your article gave me exact steps to figure it out. Thanks for sharing.
December 6th, 2011 at 9:27 am
its a nice work…really helpfull to me to knw about load average
February 23rd, 2012 at 11:57 pm
<3
May 4th, 2012 at 3:56 am
Really Great Article and Very Useful to deal performance Problems.
June 5th, 2012 at 6:58 pm
A good article! Helped better my understanding of the load averages
October 4th, 2012 at 6:09 am
nice article………..Helped me can you please tell me how to delete process and which process i am going to delete…….!!!!
April 22nd, 2013 at 7:39 am
Nice. …
Thanks….
October 4th, 2013 at 7:33 am
Thanks for publishing the nice article …… looking forward for next one…….