# What is "Drive Space" on a linux server?



## Guest (Feb 25, 2007)

I've been getting these *Drive Space Warning* system emails for the last 2 days:

*Drive Warning: sda2 (/var) is 82% full*

I just checked *Current Disk Usage* and see this:

Filesystem Size Used Avail Use% Mounted on
/dev/sda5 2.9G 835M 1.9G 31% /
/dev/sda1 99M 24M 70M 26% /boot
/dev/shm 490M 0 490M 0% /dev/shm
/dev/sda8 84G 36G 44G 46% /home
/dev/sda7 965M 18M 898M 2% /tmp
/dev/sda3 9.5G 3.2G 5.9G 35% /usr
/dev/sda2 9.5G 6.2G 2.9G 69% /var
/tmp 965M 18M 898M 2% /var/tmp

/home is where the websites are. What is in /var and what can I do to reduce the space used by it?


----------



## Nevada (Sep 9, 2004)

Since log files are normally found in /var/log it's likely that you have a rogue log file that's growing. Take a look in that directory for a large file. If that's the case, you should setup log rotation for that service in the future.


----------



## MeanDean (Apr 16, 2002)

if you're running Fedora w/mail and mailman ... check the archives

they along with the logs Nevada correctly mentions, can be a source of space consumption


----------



## Guest (Feb 25, 2007)

MeanDean said:


> if you're running Fedora w/mail and mailman ... check the archives
> 
> they along with the logs Nevada correctly mentions, can be a source of space consumption


 Yep, Fedora:

WHM 10.8.0 cPanel 10.9.0-C117 
Fedora i686 - WHM X v3.1.0 

Now for a really stupid question (remember, I'm 100% self taught and only for the last 3 years): how do you check the archives?


----------



## Gary in ohio (May 11, 2002)

Var variables, This is where many log files go and can file usage can go up and down. Make sure you rotating your logs. by default many linux version dont every get rid of log files. For home use dump them monthly.


----------



## Nevada (Sep 9, 2004)

By the way Ladycat, filling the /var partition is a serious matter. If the /var partition is filled you won't be able to login, not even as root, since the login can't be written to the log file. It's imperative that you identify the problem and correct the situation before the partition is filled. Once the partition is filled your services will stop responding, since they will go into error due to inability to log.

The only way out of it that I'm aware of is to start a "linux single" session at boot time and delete the rogue file. Linux single doesn't require the session to be written to log. However, I'm guessing this is a server colocated in a remote server room (or perhaps a VPS). You won't be able to initiate a "linux single" session remotely.


----------



## Guest (Feb 25, 2007)

Nevada said:


> However, I'm guessing this is a server colocated in a remote server room (or perhaps a VPS). You won't be able to initiate a "linux single" session remotely.


 It's colocated, not a VPS.


----------



## Guest (Feb 25, 2007)

Before I get more confused, can someone give me idiot instructions, step by step, from the moment I log in to root? :stars:


----------



## Guest (Feb 25, 2007)

Here's where I am so far:

[email protected] [~]# cd /var
[email protected] [/var]# ls -a
./ clamd= lib/ [email protected] portsentry/ spool/
../ cpanel/ local/ named/ preserve/ tmp/
account/ crash/ lock/ netenberg/ quota.user* www/
aquota.user* db/ log/ nis/ racoon/ yp/
cache/ empty/ lost+found/ opt/ run/


[email protected] [/var]# cd log/
[email protected] [/var/log]# ls -a
./ exim_mainlog ppp/
../ exim_mainlog.1.gz prelink.log
anaconda.log exim_mainlog.2.gz rpmpkgs
anaconda.syslog exim_mainlog.3.gz rpmpkgs.1
audit/ exim_mainlog.4.gz rpmpkgs.2
bandwidth/ exim_paniclog rpmpkgs.3
boot.log exim_paniclog.1.gz rpmpkgs.4
boot.log.1 exim_paniclog.2.gz secure
boot.log.2 exim_paniclog.3.gz secure.1
boot.log.3 exim_paniclog.4.gz secure.2
boot.log.4 exim_rejectlog secure.3
btmp exim_rejectlog.1.gz secure.4
btmp.1 exim_rejectlog.2.gz spooler
checkphpv_error.log exim_rejectlog.3.gz spooler.1
checkphpv_output.log exim_rejectlog.4.gz spooler.2
chkservd.log lastlog spooler.3
clam-update.log mail/ spooler.4
cpanel-install-thread0.log maillog stunnel-4.04-build.log
cpanel-install-thread1.log maillog.1 stunnel-4.15-build.log
cpupdate.env maillog.2 vbox/
cron maillog.3 wtmp
cron.1 maillog.4 wtmp.1
cron.2 messages [email protected]
cron.3 messages.1 xferlog.offset
cron.4 messages.2 xferlog.offsetftpsep
cups/ messages.3 yum.log
dcpumon/ messages.4
dmesg munin/
[email protected] [/var/log]#


----------



## Nevada (Sep 9, 2004)

ladycat said:


> Before I get more confused, can someone give me idiot instructions, step by step, from the moment I log in to root?


The easiest way to do that is with webmin. It makes chores on a Linux server such as file maintenance, administrating users, monitoring mail servers, and most everything else as simple & graphical as working on a Windows machine.

It's free, stable, and uses practically no resources.

http://www.webmin.com/

That said, if you wish to work from the command prompt that will work too. When you list the contents of the directory you will need to specify verbose (long) listing, so it will also display the file sizes, like this:

ls -la /var/log

The "l" (small L, as in long) after the hyphen specifies the long listing, and the "a" tells it to list all files so you won't miss any. Give that a try.

******
Since file size is your primary concern, perhaps a better way to display would be:

ls -ash /var/log

The switches after the hyphen will display all files (a), display file sizes (s), and display sizes in human format (h), such as 4K, 16M, 1G, etc.


----------



## Guest (Feb 25, 2007)

Nevada said:


> ls -ash /var/log


 Ok, I did that.

[email protected] [/var/log]# ls -ash /var/log
total 1.5G
8.0K ./ 46M exim_mainlog 8.0K ppp/
8.0K ../ 151M exim_mainlog.1.gz 16K prelink.log
20K anaconda.log 134M exim_mainlog.2.gz 12K rpmpkgs
32K anaconda.syslog 67M exim_mainlog.3.gz 12K rpmpkgs.1
8.0K audit/ 69M exim_mainlog.4.gz 12K rpmpkgs.2
4.0K bandwidth/ 160K exim_paniclog 12K rpmpkgs.3
0 boot.log 676K exim_paniclog.1.gz 12K rpmpkgs.4
0 boot.log.1 988K exim_paniclog.2.gz 1.6M secure
0 boot.log.2 296K exim_paniclog.3.gz 2.2M secure.1
0 boot.log.3 408K exim_paniclog.4.gz 3.1M secure.2
0 boot.log.4 27M exim_rejectlog 2.1M secure.3
23M btmp 105M exim_rejectlog.1.gz 4.1M secure.4
32M btmp.1 87M exim_rejectlog.2.gz 0 spooler
0 checkphpv_error.log 28M exim_rejectlog.3.gz 0 spooler.1
4.0K checkphpv_output.log 49M exim_rejectlog.4.gz 0 spooler.2
7.3M chkservd.log 92K lastlog 0 spooler.3
36K clam-update.log 8.0K mail/ 0 spooler.4
656K cpanel-install-thread0.log 1.2M maillog 24K stunnel-4.04-build.log
396K cpanel-install-thread1.log 19M maillog.1 36K stunnel-4.15-build.log
4.0K cpupdate.env 18M maillog.2 8.0K vbox/
912K cron 16M maillog.3 4.0K wtmp
13M cron.1 14M maillog.4 48K wtmp.1
13M cron.2 8.1M messages 0 [email protected]
13M cron.3 114M messages.1 4.0K xferlog.offset
13M cron.4 87M messages.2 4.0K xferlog.offsetftpsep
8.0K cups/ 61M messages.3 16K yum.log
4.0K dcpumon/ 232M messages.4
16K dmesg 4.0K munin/


----------



## Guest (Feb 25, 2007)

A sceen shot is a lot easier to read


----------



## Nevada (Sep 9, 2004)

Evidently the culprit isn't in that directory, although I am concerned about a few entries. As you recall, your warning said that :

/dev/sda2 9.5G 6.2G 2.9G 69% /var

Which means that the partition is 9.5 gigs, of which 6.2 gigs is used, leaving 2.9 gigs free, so the used portion is 69%. The /var/log directory is only using 1.5 gigs, so you need to look elsewhere for the high usage.

Next I would check the subdirectories of the /var/log directory. The culprit may still may be a log file.

******
Just a suggestion, based on seeing your logs:

You should consider setting up a rotation schedule for some of your high usage logs that don't already have rotation schedule created, and modify the rotation schedule for some that already have a rotation schedule. Some of your log files are too big to view anyway. 

What I would do is to change the rotation schedule for some of them so they will rotate more frequently. For example, some of the cron, mail and messages logs are getting pretty big. You should probably rotate them more often (older logs are automatically emptied and overwritten). 

Again, the easiest way to modify the log rotation setup is with webmin. You'll love it! You are doing yourself a real disservice by not having it.


----------



## Guest (Feb 25, 2007)

Nevada said:


> Again, the easiest way to modify the log rotation setup is with webmin. You'll love it! You are doing yourself a real disservice by not having it.


 Can webmin be installed in addition to cpanel? I can't get rid of cpanel, my host would kill me. She uses unix commands and cpanel, and if she has to go into my server for any reason, that's what she would use.

Also, my own clients are accustomed to cpanel.

I can take a look in my WHM and see if the log rotation can be done through that interface.


----------



## Nevada (Sep 9, 2004)

ladycat said:


> A sceen shot is a lot easier to read


Yes, it is. After taking a closer look you don't really have anything that looks dangerously active and isn't already being rotated, although you have to admit that a log file over 200 megs is huge. I don't see any rotations that need to be created, but you may wish to increase rotation frequency on a few.

After installing webmin take a look at some of your logs. Webmin includes a log searching utility that will search logs for text strings (user names, email addresses, etc.) and displays them in a way you would expect to see on a Windows machine. I don't know what the limitations might be on searching logs, but I can't imagine needing text logs any bigger than 50 megs, then saving 4 or 5 rotation periods of that log. Of course, as the administrator that's up to you. You do have the disk space to keep 1.5 gigs, so you can keep things as the are if you think you need them.

Anyway, keep looking for the high usage file or directory.


----------



## Nevada (Sep 9, 2004)

ladycat said:


> Can webmin be installed in addition to cpanel? I can't get rid of cpanel, my host would kill me. She uses unix commands and cpanel, and if she has to go into my server for any reason, that's what she would use.
> 
> Also, my own clients are accustomed to cpanel.
> 
> I can take a look in my WHM and see if the log rotation can be done through that interface.


I don't see any reason why webmin should interfere with cpanel, but it does have some overlapping functions. For example, I wouldn't try to create system user accounts or email accounts with webmin, since the account ownship may become confusing to cpanel. Continue to use cpanel for the things you're accustomed to useing cpanel for, and only use webmin for system maintenence. Here is a link to a thread that addresses that question:

http://forums.serverbeach.com/archive/index.php/t-443.html


----------



## Gary in ohio (May 11, 2002)

do a ls -lstRr and that will give a full list of all files.


----------



## Guest (Feb 25, 2007)

Gary in ohio said:


> do a ls -lstRr and that will give a full list of all files.


 Oh my gosh, it's still going.


----------



## Guest (Feb 25, 2007)

Still going. It's showing every mysql table in every account on the server. ugh.


----------



## Nevada (Sep 9, 2004)

ladycat said:


> Still going. It's showing every mysql table in every account on the server. ugh.


You should be able to break it with a Ctrl-C.


----------



## Guest (Feb 26, 2007)

Nevada said:


> You should be able to break it with a Ctrl-C.


 That worked! I knew you could stop it with a Ctrl _something_. I just wasn't sure what.


----------



## Nevada (Sep 9, 2004)

ladycat said:


> That worked! I knew you could stop it with a Ctrl _something_. I just wasn't sure what.


----------



## Guest (Feb 26, 2007)

My host said this:

_Try deleting your mail queue and cleaning up any databases as well as log files. _ 

I know how to do the mail queue. How do I do the other 2? Idiot instructions, please.


----------



## Gary in ohio (May 11, 2002)

cd /var and do the ls command
You can re-direct it to a file
ls -lstRr > /tmp/myfile


----------



## Nevada (Sep 9, 2004)

ladycat said:


> My host said this:
> 
> _Try deleting your mail queue and cleaning up any databases as well as log files. _
> 
> I know how to do the mail queue. How do I do the other 2? Idiot instructions, please.


They're taking a shot in the dark. You already know the problem isn't with the log files, since you only have 1.5 gigs of logs. You might look in the directory where MySQL is located.


----------



## Guest (Feb 26, 2007)

I might have a clue. Since I already know how to do the exim stuff, I decided to do that first. My usual routine is to do the following, one at a time, in order:

/usr/sbin/exim_tidydb -t 1d /var/spool/exim retry
/usr/sbin/exim_tidydb -t 1d /var/spool/exim reject 
/usr/sbin/exim_tidydb -t 1d /var/spool/exim wait-remote_smtp
/usr/sbin/exim -qff 

I've only done the first one so far, and it's going and going like the energizer bunny. It's cleaning out about a gazillion old emails.

After I go through the entire routine, I'll hopefully figure out what to do next.


----------



## Guest (Feb 26, 2007)

I've been going through my very extensive collection of UNIX notes, and I found this:

*Clearing disk drive space:

cd /usr/local/apache/domlogs
rm -Rf *
cd /usr/local/apache/logs
rm -Rf error_log
rm -Rf access_log
rm -Rf suexec_log
/etc/rc.d/init.d/httpd restart*

I wonder if I should try that after it finishes all the exim stuff?


----------

