[clue-tech] Swapped memory with 1GB of RAM?
Jack Swope
jack.swope at bil.konicaminolta.us
Wed Feb 9 15:22:48 MST 2005
Your are not the first. Appears to be VM problems with many of RedHat's
Kernels. See
https://bugzilla.redhat.com/bugzilla/long_list.cgi?buglist=89226
And think about changing kernels (update or build your own).
On Wed, 2005-02-09 at 14:38, Alex Young wrote:
> I've got a memory problem, and I'm stumped.
>
> I have a webserver (RH 9.0) running 2.4.20-19.9smp, and after a few
> days of uptime it gets ungodly slow at serving pages due to swapping.
> The machine has a gig of ram and is not running x or anything like
> that.
>
> The system in question is serving a bunch of low-volume non-profit
> websites, so rebooting is an issue, but can be done late at night.
>
> I've looked at migrating to debian, but I'd like to try to get this
> machine working as the swap is potentially going to cause a lot of
> issues.
>
> Here is some output from vmstat:
>
> procs memory swap io system cpu
> r b w swpd free buff cache si so bi bo in cs us sy id
> 0 0 1 139464 16224 1580 35096 2 3 1 3 3 1 1 1 4
> 0 3 4 139464 16304 1600 34904 0 0 234 168 199 392 1 2 97
> 0 0 1 139464 16308 1536 34996 0 0 61 43 135 244 0 1 99
> 0 0 1 139464 16120 1684 35144 0 0 575 198 279 660 2 3 95
>
> Now while a page is loading:
>
> procs memory swap io system cpu
> r b w swpd free buff cache si so bi bo in cs us sy id
> 0 1 1 139464 13108 1632 34228 2 3 1 3 3 1 1 1 4
> 0 1 2 139464 10116 1740 35064 886 0 1494 243 290 525 13 3 84
> 0 3 1 139464 9040 1632 35436 430 0 874 174 280 562 7 4 89
> 1 0 1 139464 8604 1660 35036 1306 92 1353 217 265 409 13 3 84
> 0 0 1 139388 8728 1724 35224 338 180 496 244 239 497 15 1 84
>
> So, it's swapping hard. So I look at free:
>
> total used free shared buffers cached
> Mem: 1030304 1020468 9836 0 1444 35336
> -/+ buffers/cache: 983688 46616
> Swap: 2072376 139388 1932988
>
> That "-/+ buffers/cache:" line looks bad to me, but I'm not sure. I
> think the buffer line is high for this setup, it's filling most of the
> RAM. Also, there is a lot of swapped data.
>
> So I looked at ps aux:
>
> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> root 1 0.0 0.0 1384 440 ? S 2004 1:22 init
> root 2 0.0 0.0 0 0 ? SW 2004 0:00 [migration/0]
> root 3 0.0 0.0 0 0 ? SW 2004 0:00 [migration/1]
> root 4 0.0 0.0 0 0 ? SW 2004 13:21 [keventd]
> root 5 0.0 0.0 0 0 ? SWN 2004 0:00 [ksoftirqd_CPU0]
> root 6 0.0 0.0 0 0 ? SWN 2004 0:00 [ksoftirqd_CPU1]
> root 11 0.0 0.0 0 0 ? SW 2004 0:25 [bdflush]
> root 7 0.5 0.0 0 0 ? DW 2004 411:25 [kswapd]
> root 8 0.0 0.0 0 0 ? SW 2004 0:16 [kscand/DMA]
> root 9 0.0 0.0 0 0 ? SW 2004 57:26 [kscand/Normal]
> root 10 0.1 0.0 0 0 ? SW 2004 73:05 [kscand/HighMem]
> root 12 0.0 0.0 0 0 ? SW 2004 2:02 [kupdated]
> root 13 0.0 0.0 0 0 ? SW 2004 0:00 [mdrecoveryd]
> root 19 0.0 0.0 0 0 ? SW 2004 0:00 [scsi_eh_0]
> root 20 0.0 0.0 0 0 ? SW 2004 0:00 [scsi_eh_1]
> root 24 0.0 0.0 0 0 ? SW 2004 6:49 [kjournald]
> root 82 0.0 0.0 0 0 ? SW 2004 0:00 [khubd]
> root 188 0.0 0.0 0 0 ? SW 2004 0:40 [kjournald]
> root 189 0.0 0.0 0 0 ? SW 2004 0:00 [kjournald]
> root 190 0.0 0.0 0 0 ? SW 2004 0:29 [kjournald]
> root 191 0.0 0.0 0 0 ? SW 2004 12:37 [kjournald]
> root 192 0.0 0.0 0 0 ? SW 2004 7:16 [kjournald]
> root 692 0.0 0.0 1456 512 ? S 2004 1:01 syslogd -m 0
> root 696 0.0 0.0 1376 388 ? S 2004 0:00 klogd -x
> rpc 714 0.0 0.0 1560 440 ? S 2004 0:00 portmap
> rpcuser 733 0.0 0.0 1580 568 ? S 2004 0:00 rpc.statd -p 4000
> root 844 0.0 0.0 3720 484 ? S 2004 1:25 /usr/sbin/sshd
> root 860 0.0 0.0 2072 540 ? S 2004 0:00 xinetd -stayalive
> ntp 879 0.0 0.2 2412 2408 ? SL 2004 4:27 ntpd -U ntp
> lp 895 0.0 0.0 9136 160 ? S 2004 0:19 lpd Waiting
> root 906 0.0 0.0 1424 4 ? S 2004 0:00 gpm -t imps2 -m /
> bin 927 0.0 0.0 2064 40 ? S 2004 0:01 /usr/sbin/cannase
> root 938 0.0 0.0 1432 136 ? S 2004 0:38 crond
> xfs 971 0.0 0.0 5664 28 ? S 2004 0:00 xfs -droppriv -da
> daemon 1010 0.0 0.0 1420 112 ? S 2004 0:04 /usr/sbin/atd
> root 1020 0.0 0.0 3416 388 ? S 2004 0:00 rhnsd --interval
> root 1027 0.0 0.0 1360 0 tty2 SW 2004 0:00 /sbin/mingetty tt
> root 1028 0.0 0.0 1360 0 tty3 SW 2004 0:00 /sbin/mingetty tt
> root 1029 0.0 0.0 1360 0 tty4 SW 2004 0:00 /sbin/mingetty tt
> root 1030 0.0 0.0 1360 0 tty5 SW 2004 0:00 /sbin/mingetty tt
> root 1031 0.0 0.0 1360 0 tty6 SW 2004 0:00 /sbin/mingetty tt
> root 1032 0.0 0.0 2120 0 ? SW 2004 0:00 /bin/sh /command/
> root 1034 0.0 0.0 1400 88 ? S 2004 3:48 svscan /service
> root 1035 0.0 0.0 1352 0 ? SW 2004 0:00 readproctitle ser
> root 1036 0.0 0.0 1352 12 ? S 2004 0:00 supervise qmail-s
> root 1037 0.0 0.0 1352 24 ? S 2004 0:00 supervise log
> root 1038 0.0 0.0 1352 12 ? S 2004 0:00 supervise qmail-s
> root 1039 0.0 0.0 1352 12 ? S 2004 0:00 supervise log
> qmaild 1040 0.0 0.0 1428 4 ? S 2004 0:05 /usr/local/bin/tc
> qmaill 1041 0.0 0.0 1376 0 ? SW 2004 0:04 /usr/local/bin/mu
> qmails 1042 0.0 0.0 1580 216 ? S 2004 63:05 qmail-send
> qmaill 1043 0.0 0.0 1376 64 ? S 2004 26:29 /usr/local/bin/mu
> root 1046 0.0 0.0 1372 104 ? S 2004 55:27 qmail-lspawn |dot
> qmailr 1047 0.0 0.0 1368 0 ? SW 2004 0:05 qmail-rspawn
> qmailq 1048 0.0 0.0 1364 52 ? S 2004 1:39 qmail-clean
> root 6398 0.0 0.0 1364 4 tty1 S 2004 0:00 /sbin/mingetty tt
> root 13626 0.0 0.0 0 0 ? SW 2004 0:17 [kjournald]
> root 22547 0.0 0.0 0 0 ? SW Jan02 0:00 [nfsd]
> root 22549 0.0 0.0 0 0 ? SW Jan02 0:00 [lockd]
> root 22550 0.0 0.0 0 0 ? SW Jan02 0:18 [rpciod]
> rpcuser 22552 0.0 0.0 1580 4 ? S Jan02 0:00 /sbin/rpc.statd
> root 22554 0.0 0.0 1508 8 ? S Jan02 0:00 /usr/sbin/rpc.mou
> root 22556 0.0 0.0 3628 4 ? S Jan02 0:00 /usr/sbin/rpc.rqu
> rpcuser 30664 0.0 0.0 1580 28 ? S Jan02 0:00 /sbin/rpc.statd
> root 30666 0.0 0.0 1508 0 ? SW Jan02 0:00 /usr/sbin/rpc.mou
> root 30668 0.0 0.0 3636 0 ? SW Jan02 0:00 /usr/sbin/rpc.rqu
> rpcuser 15749 0.0 0.0 1576 0 ? SW Jan05 0:00 /sbin/rpc.statd
> root 15751 0.0 0.0 1508 0 ? SW Jan05 0:00 /usr/sbin/rpc.mou
> root 15753 0.0 0.0 3636 0 ? SW Jan05 0:00 /usr/sbin/rpc.rqu
> root 19011 0.0 0.0 18652 84 ? S Feb04 0:17 /usr/sbin/httpd -
> www-data 27260 0.0 0.4 26720 4760 ? S Feb06 2:21 /usr/sbin/httpd -
> www-data 27261 0.0 0.8 26472 8316 ? S Feb06 2:13 /usr/sbin/httpd -
> www-data 27267 0.0 0.5 26640 5168 ? S Feb06 2:24 /usr/sbin/httpd -
> www-data 27271 0.0 0.5 26724 5640 ? S Feb06 2:23 /usr/sbin/httpd -
> www-data 27272 0.0 0.1 27032 1344 ? S Feb06 2:26 /usr/sbin/httpd -
> www-data 29164 0.0 0.3 26296 3332 ? S Feb06 2:39 /usr/sbin/httpd -
> www-data 19753 0.0 0.0 26300 952 ? S Feb06 2:14 /usr/sbin/httpd -
> www-data 30495 0.0 0.2 26500 2504 ? S Feb06 2:21 /usr/sbin/httpd -
> www-data 30496 0.0 0.9 26544 9468 ? S Feb06 2:19 /usr/sbin/httpd -
> www-data 30497 0.0 0.5 26796 5504 ? S Feb06 2:22 /usr/sbin/httpd -
> www-data 30526 0.0 0.0 26904 612 ? S Feb06 2:20 /usr/sbin/httpd -
> www-data 8409 0.0 0.2 26520 2540 ? S Feb06 2:16 /usr/sbin/httpd -
> www-data 8555 0.0 0.6 26744 7204 ? S Feb06 2:26 /usr/sbin/httpd -
> www-data 26335 0.0 0.8 26544 8624 ? S Feb06 2:21 /usr/sbin/httpd -
> www-data 26256 0.0 0.7 26564 7716 ? S Feb07 1:37 /usr/sbin/httpd -
> root 18493 0.0 0.0 6388 0 ? SW 08:24 0:00 sshd: ayoung [pri
> ayoung 18510 0.0 0.0 6432 304 ? S 08:24 0:02 sshd: ayoung at pts/
> ayoung 18511 0.0 0.0 4604 0 pts/0 SW 08:24 0:00 -bash
> root 18965 0.0 0.0 4312 0 pts/0 SW 08:26 0:00 su
> root 18966 0.0 0.0 4624 672 pts/0 S 08:26 0:01 bash
> named 25776 0.0 0.1 39060 1544 ? S 08:59 0:00 /usr/sbin/named -
> root 13602 0.0 0.0 4200 108 pts/0 S 10:30 0:00 /bin/sh /usr/bin/
> mysql 13657 0.4 0.2 15544 2144 pts/0 S 10:30 0:08 /usr/libexec/mysq
> ayoung 20722 0.0 0.2 4044 2168 ? S 11:05 0:00 /usr/libexec/gcon
> root 20988 0.0 0.0 2700 760 pts/0 R 11:06 0:00 ps aux
>
> Nothing looks huge in the memory area. It hardly looks like the
> memory is being touched at all (7.2%).
>
> I don't get it. Where did my RAM go? What else can I do to troubleshoot?
>
> Thanks in advance,
>
> -Alex
> _______________________________________________
> CLUE-tech mailing list
> CLUE-tech at clue.denver.co.us
> http://clue.denver.co.us/mailman/listinfo/clue-tech
--
Jack Swope
Systems Administrator
Konica-Minolta Systems Laboratory
Core Technology Laboratory
Boulder, CO
(303)563-1323
More information about the clue-tech
mailing list