Linux Load Average explained


load average: 0.00, 0.00, 0.00

How do you get this output?

To get your system’s Load Average, run the command uptime. It will show you the current time, how long your system has been powered on for, the number of users logged in, and finally the system’s load average.
What does it mean?

Simply, it is the number of blocking processes in the run queue averaged over a certain time period.

Time periods:

load average: 1min, 5min, 15min

What is a blocking process?

A blocking process is a process that is waiting for something to continue. Typically, a process is waiting for:

  • CPU
  • Disk I/O
  • Network I/O

What does a high load average mean?

A high load average typically means that your server is under-specified for what it is being used for, or that something has failed (like an externally mounted disk).

How do I diagnose a high load average?

Typically, a server with a high load average is unresponsive and slow — and you want to reduce the load and increase responsiveness. But how do you go about working out what is causing your high load?

Lets start with the simplest one, are we waiting for CPU? Run the Linux command top.
TOP output

Check the numbers above in the red circle. They are basically representing what percentage of its’ total time the CPU is spending processing stuff. If these numbers are constantly around 99-100% then chances are the problem is related to your CPU, almost certainly that it is under powered. Consider upgrading your CPU.

TOP wait

The next thing to look for is if the cpu is waiting on I/O. Now check the number around where the red circle is now. If this number is high (above 80% or so) then you have problems. This means that the CPU is spending a LOT of time waiting in I/O. This could mean that you have a failing Hard Disk, Failing Network Card, or that your applications are trying to access data on either of them at a rate significantly higher than the throughput that they are designed for.

To find out what applications are causing the load, run the command ps faux. This will list every process running on your system, and the state it is in.

PS FAUX

You want to look in the STAT column. The common flags that you should be looking for are:

  • R – Running
  • S – Sleeping
  • D – Waiting for something

So, look for any processes with a STAT of D, and you can go from there to diagnose the problem.

Further Diagnosis

To diagnose further, you can use the following programs

  • strace – to trace what a program is doing
  • iostat – to see the throughput of your disks
  • bwmon – to see your network throughput

26 Responses to “Linux Load Average explained”

  1. Gurjeet Singh Says:

    Nice little article…. really useful. Can you please do a similar article on strace iostat and bwmon too.

    Thanks,

  2. Vladimir Says:

    Excellent Article! Will use some information on my blog too. Waiting for Further Diagnosis Article :)

  3. Zip Driver Says:

    Thanks a lot for great article with step by step instructions. Very ease and very useful. Really will be great if you will continue about strace, iostats and similar utilities. Thank you again.

  4. Ash Says:

    Nice article and very informative for the basics of Linux.

    Thx.

  5. PHK Corporation Says:

    Identified that claamscan is running curently for every host and need to change the scheduling of each host for clamscan for a single instance at a time.

    The more hosts, and more concurrent instances of clamscan, the greater the load average and slower the response time.

    Thanks
    PHK Corporation

  6. Gil Otero Says:

    Really helpful information.

  7. Load average and Vmware Server Hang Fix — Somewhere out there! Says:

    […] Concise and clear explaination of linux load averages. […]

  8. freakrobot Says:

    Excuse me , but what if I have a high value of “cpu %id”? What does this mean?

  9. jdd0x01 Says:

    cpu %id means idle, not doing anything

  10. Andre Lewis Says:

    Thanks for the clear, concise post on load averages — I used it as a reference for my own writeup: http://blog.scoutapp.com/articles/2009/07/31/understanding-load-averages. Cheers!

  11. games Says:

    Good article, easy to spot any bottleneck which slows down the system. CPU, Network, Disk, Process, and RAM.

  12. Dileep Says:

    very nice, further these types of diagnosis articles would be very helpful. thanks.

  13. esc Says:

    hi, do you know why Linux load average counts uninterruptable process?

    FreeBSD8 does not, after checking sys/kern/sched_4bsd.c, kern_synch.c.
    However Kernel 2.6.27 does based on Linux/kernel/sched.c, and timer.c.

    Also, if you know… is there any changes made the method of load average calculation after Kernel 2.6.16?

    Unfortunately I could not find sorce code for kernel 2.6.16.
    If you know anything about it, it is very helpful.

  14. kudzu74 Says:

    Clear, concise and to the point…just what I was looking for. Thank you for taking the time.

  15. Wael ElMasry Says:

    Dears,

    Kindly correct me if i am mistaken.

    I disagree with the above in the part of checking the SY and US , its just saying the total amount of time the user/system processes is talking from the CPU time (so,its simply a function of time) , you may have many process waiting for the processor to handle , and you can normally find the the SY and US is low.

    Thank you.

  16. Mike Says:

    I didn’t ever need this until I got a linux virtual instance with only 256 megs of RAM =\

  17. Catalin Says:

    great article. This is exactly what I need: step by step guide doubled by exact explanations. Good job!

  18. mike Says:

    you are awesome.

  19. Garima Says:

    Excellent article. Recently there was a question on load average in my company. Your article gave me exact steps to figure it out. Thanks for sharing.

  20. vishnu Says:

    its a nice work…really helpfull to me to knw about load average

  21. Detonator316 Says:

    <3

  22. azmir Says:

    Really Great Article and Very Useful to deal performance Problems.

  23. Shveta Says:

    A good article! Helped better my understanding of the load averages

  24. rajendra parihar Says:

    nice article………..Helped me can you please tell me how to delete process and which process i am going to delete…….!!!!

  25. Bimal Says:

    Nice. …

    Thanks….

  26. Rizwan Saheb Says:

    Thanks for publishing the nice article …… looking forward for next one…….

Leave a Reply