Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Linux Memory Management or 'Why is there no free RAM?'
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
imsdunn
n00b
n00b


Joined: 06 Sep 2003
Posts: 19

PostPosted: Wed Jun 09, 2004 9:19 pm    Post subject: Reply with quote

I am having problems as well with very slow performance. I thought it was my internet provider at first, but have come to realize that everything takes a while to open and load.

I tried the free -m command and it appears I have plenty of RAM available

Code:
 free -m
             total       used       free     shared    buffers     cached
Mem:           440        212        227          0         21        104
-/+ buffers/cache:         86        353
Swap:          494          0        494


I am still using a 2.4 kernel with gentoo and kde. It just slowed down this weekend. I have rebooted both the computer and modem several times trying to free it up. Any suggestions?

Quote:
I went to sleep waiting on mine to sync up yesterday.


I am also having to wait a terribly long time for sync to run. I emerged sync once each of the last two days.
_________________
---------------------
s.dunn
Back to top
View user's profile Send private message
beastmaster
Apprentice
Apprentice


Joined: 24 May 2004
Posts: 230

PostPosted: Wed Jun 09, 2004 11:38 pm    Post subject: Reply with quote

I think there are really a lot of possibilites for slow performance :roll:

1. internet clustered
2. cpu getting overheated, but if you just have a cold start, and it's still slow on loading, then it's not it.
3. kernel 2.4, if you want to have some memory advantage like the original author posted ealier, you should give kernel 2.6 a try.
4. harddisk DMA transfer is not turned on, or something, hdparm would help about 20%~30%
5. small L1, L2 cache, so have to live with it (like my Intel celeron's on my notebook)
6. the famous optimization flag tweaking :D, it will boost up performance for 30~40% with good and right CFLAGS (for those debian lovers who are lurking it, I dare you that, this is the fact, so don't argue)

and there are also so many other factors that cause a slow performance.
I'm no expert, so I'm out of idea now :P
but I would look into kernel, cflags, and harddisk hdparm first.
Back to top
View user's profile Send private message
nadamsieee
Guru
Guru


Joined: 30 May 2003
Posts: 340
Location: Atlanta, GA, USA

PostPosted: Wed Jun 09, 2004 11:49 pm    Post subject: Reply with quote

Don't forget this fix for Gnome/KDE performance:

Edit your /etc/hosts file such that 127.0.0.1 points to localhost.localdomain, localhost, and your actual hostname. So for me, its like this:

Code:
# cat /etc/hostname
alqualonde
# cat /etc/hosts
127.0.0.1       localhost.localdomain localhost
127.0.0.1       alqualonde

_________________
nadams (at) ieee (dot) org
Back to top
View user's profile Send private message
smiley
Tux's lil' helper
Tux's lil' helper


Joined: 03 Dec 2003
Posts: 127

PostPosted: Mon Jun 14, 2004 8:44 pm    Post subject: Re: Linux Memory Management or 'Why is there no free RAM?' Reply with quote

Evangelion wrote:
sapphirecat wrote:
2. The mysterious 800 MB limit on x86


related to this: do you need to enable high memory-support on AMD64-machines to take advantage of lots of RAM?


My understanding is that on 64bit processors, all memory is "low memory," so you wouldn't need high memory support. On the other hand, since amd64 also runs 32bit programs, maybe it's different than other 64bit processors.

smiley 8)
Back to top
View user's profile Send private message
flickerfly
l33t
l33t


Joined: 08 Nov 2002
Posts: 677
Location: Lanham, MD

PostPosted: Tue Jun 15, 2004 1:05 pm    Post subject: Reply with quote

A neat followup article on this might be a tweaking article including swappiness settings, hdparm, and all those other little things that can slow down Gentoo simply because they aren't set optimally. That might ought to even be official documentation tacked on the end of the Gentoo Handbook.
_________________
An Evil Genious' Guide to Sheeple and How To Avoid Becoming One | 0x4C9EF4A
Back to top
View user's profile Send private message
RAPUL
l33t
l33t


Joined: 29 Dec 2002
Posts: 664
Location: Valencia (SPAIN)

PostPosted: Tue Jun 15, 2004 1:17 pm    Post subject: Nice... Reply with quote

I already knew most of this thread says but it is nice someone wrote all these ideas joint for people who don't know them.

I also discovered swapiness concept. So as everyday i learnt a new thing today.

I discovered these some time ago as i have 1.5GB since 1-2 years ago.

Code:

crujido [15:09:56] gentoo-rsync # free -m
             total       used       free     shared    buffers     cached
Mem:          1514       1463         51          0        177        667
-/+ buffers/cache:        617        897
Swap:            0          0          0
crujido [15:22:20] gentoo-rsync # cat /proc/meminfo
        total:    used:    free:  shared: buffers:  cached:
Mem:  1588178944 1535356928 52822016        0 186544128 701300736
Swap:        0        0        0
MemTotal:      1550956 kB
MemFree:         51584 kB
MemShared:           0 kB
Buffers:        182172 kB
Cached:         684864 kB
SwapCached:          0 kB
Active:         466964 kB
Inactive:       715128 kB
HighTotal:      655280 kB
HighFree:         3868 kB
LowTotal:       895676 kB
LowFree:         47716 kB
SwapTotal:           0 kB
SwapFree:            0 kB


As you can see.
My machine has 182MB used in buffers and 682MB in cache which are really free memory. Also you can notice i don't use swap because i don't need it.

As I didn't know swapiness concept i disabled swap when i installed my system. :P

Wheeeee....!
_________________
Entropy rulz world.
Redundancy sux.
World is full of redundancy.
World sux.


Last edited by RAPUL on Tue Jun 15, 2004 1:28 pm; edited 1 time in total
Back to top
View user's profile Send private message
rich0
Developer
Developer


Joined: 15 Sep 2002
Posts: 161

PostPosted: Tue Jun 15, 2004 1:21 pm    Post subject: Re: Linux Memory Management or 'Why is there no free RAM?' Reply with quote

smiley wrote:
Evangelion wrote:
sapphirecat wrote:
2. The mysterious 800 MB limit on x86


related to this: do you need to enable high memory-support on AMD64-machines to take advantage of lots of RAM?


My understanding is that on 64bit processors, all memory is "low memory," so you wouldn't need high memory support. On the other hand, since amd64 also runs 32bit programs, maybe it's different than other 64bit processors.

smiley 8)


If you are running your kernel in 64-bit mode (ie you selected x86_64 as your arch type), then I'm pretty sure you are good up to whatever virtual memory size your CPU handles (I think they're up to 48-bit or something like that right now).

If you compile your kernel telling it you just have a regular x86, it will run just fine in 32-bit mode, but then it has the same limitations on memory size (plus it can't use all those extra registers).

I'm guessing that if you run 32-bit apps under the 64-bit kernel in emulation mode, you probably can give each app about 1-4GB of RAM or something like that (not sure exactly how linux maps memory - I believe the 386 can handle 4GB, but the 1GB limit is probably due to using some of the address bits for other kernel purposes). I'll have to check when I'm at home, but I'm guessing that there isn't even a compile option for big memory support on the amd64. So, you can have 10 copies of 32-bit gimp running with 1GB of RAM each if you have enough RAM/swap. Or, you can have a few copies of 64-bit gimp running with 100GB each...

Keep in mind that while AMD64 supports up to 64-bit addressing in theory, if you read the fine print I think most of the existing processors only support about 40-bit physical RAM, and 48-bit virtual RAM. That should tide us over for quite a while though, as long as future chips can go all the way to 64-bit.
Back to top
View user's profile Send private message
erezny
n00b
n00b


Joined: 20 Dec 2003
Posts: 48
Location: Wisconsin, USA

PostPosted: Tue Jun 15, 2004 4:35 pm    Post subject: Reply with quote

Thanks a lot. this is very useful and isn't explained much anywhere.
Back to top
View user's profile Send private message
routerguy
Guru
Guru


Joined: 07 Feb 2004
Posts: 462

PostPosted: Tue Jun 15, 2004 4:36 pm    Post subject: Reply with quote

Hi, I have 1024 MB of RAM, but linux says I only have 883. What's up with this?
Back to top
View user's profile Send private message
RAPUL
l33t
l33t


Joined: 29 Dec 2002
Posts: 664
Location: Valencia (SPAIN)

PostPosted: Tue Jun 15, 2004 4:46 pm    Post subject: Already answered. Reply with quote

Mystic0 wrote:
Hi, I have 1024 MB of RAM, but linux says I only have 883. What's up with this?


Solution to that problem was already said in this thread...

Enable this on your kernel.

Code:

Processor Type and Features ---->
 High Memory Support ---->
 (X) 4GB


and recompile it afterwards of course. :P
_________________
Entropy rulz world.
Redundancy sux.
World is full of redundancy.
World sux.
Back to top
View user's profile Send private message
routerguy
Guru
Guru


Joined: 07 Feb 2004
Posts: 462

PostPosted: Tue Jun 15, 2004 5:27 pm    Post subject: Reply with quote

I saw that, I just thought it was strange that I have 883, not 800 MBs of RAM.... I thought the limit was 800.
Back to top
View user's profile Send private message
Joe
n00b
n00b


Joined: 01 Jul 2002
Posts: 70
Location: Europe

PostPosted: Tue Jun 15, 2004 6:30 pm    Post subject: Re: Linux Memory Management or 'Why is there no free RAM?' Reply with quote

sapphirecat wrote:

Note that turning on high memory support does slow down accesses slightly, and the current belief is that for exactly 1GB of memory, it's better to leave high memory support off and sacrifice the space for speed.


Carefull, this depends heavily on what the machine is doing. If it's already swapping you better try to access every MB of RAM being available.

Regards,

Joe
Back to top
View user's profile Send private message
bssteph
l33t
l33t


Joined: 26 Feb 2003
Posts: 652
Location: Wisconsin

PostPosted: Tue Jun 15, 2004 6:33 pm    Post subject: Reply with quote

Just a note, Con Kolivas has said (I believe somewhere on Kernel Trap) that the overhead of enabling highmem has been mostly negated in 2.6. I did some really light "feel" testing and I didn't notice a difference between activated highmem or not when I tried on my GB RAM box, but I certainly did not do very rough benchmarks to stress the RAM system. ATM I am using highmem although this reminds me to look at it again.
Back to top
View user's profile Send private message
mem7
Tux's lil' helper
Tux's lil' helper


Joined: 17 Dec 2003
Posts: 85
Location: Spokane, Washington

PostPosted: Tue Jun 15, 2004 11:08 pm    Post subject: Reply with quote

Code:
MemTotal:       904296 kB


I just got a gig of ram 2 days ago and just assumed because I havent had the time to check untill now, that it was all recognized. Well, thanks once again to a Gentoo forum post another problem (one that I didn't even know had) is solved. But it dose make me wonder why it was detecting 900MB of ram instead of the supposed 800MB limit with out high memory support in the kernel.
_________________
NeoGeo, PC Engine, and NES for life!
Back to top
View user's profile Send private message
Abraxas
l33t
l33t


Joined: 25 May 2003
Posts: 814

PostPosted: Tue Jun 15, 2004 11:44 pm    Post subject: Re: Nice... Reply with quote

RAPUL wrote:
I already knew most of this thread says but it is nice someone wrote all these ideas joint for people who don't know them.

I also discovered swapiness concept. So as everyday i learnt a new thing today.

I discovered these some time ago as i have 1.5GB since 1-2 years ago.

Code:

crujido [15:09:56] gentoo-rsync # free -m
             total       used       free     shared    buffers     cached
Mem:          1514       1463         51          0        177        667
-/+ buffers/cache:        617        897
Swap:            0          0          0
crujido [15:22:20] gentoo-rsync # cat /proc/meminfo
        total:    used:    free:  shared: buffers:  cached:
Mem:  1588178944 1535356928 52822016        0 186544128 701300736
Swap:        0        0        0
MemTotal:      1550956 kB
MemFree:         51584 kB
MemShared:           0 kB
Buffers:        182172 kB
Cached:         684864 kB
SwapCached:          0 kB
Active:         466964 kB
Inactive:       715128 kB
HighTotal:      655280 kB
HighFree:         3868 kB
LowTotal:       895676 kB
LowFree:         47716 kB
SwapTotal:           0 kB
SwapFree:            0 kB


As you can see.
My machine has 182MB used in buffers and 682MB in cache which are really free memory. Also you can notice i don't use swap because i don't need it.

As I didn't know swapiness concept i disabled swap when i installed my system. :P

Wheeeee....!


I'd think twice about that setup. Everything I have read about swapspace in linux says that it is more effiecient to have a swap partition even if you have a lot or memory. I had a setup with 1GB of mem and no swap but after reading up on it more I decided it would be better off with some swap space.
_________________
Time makes more converts than reason. - Thomas Paine
Travel is fatal to prejudice, bigotry, and narrow-mindedness, and many of our people need it sorely on these accounts. - Mark Twain
Back to top
View user's profile Send private message
qal21
n00b
n00b


Joined: 06 Nov 2003
Posts: 18

PostPosted: Tue Jun 15, 2004 11:53 pm    Post subject: Reply with quote

mem7: actually it's 870MB or so it's detecting for you, not quite 900 (a MB is 1024 KB not 100), oh well, I'm probably getting too technical


Anyways, about the main topic. I already guessed that that (about the main explaination) based upon top results. It's a good feature but, my HD is NOT quiet and when the CPU is idle it goes cacheing some of the disk, and very annoying because for 15+ min it's being really noisy .
Back to top
View user's profile Send private message
stp_1993
Apprentice
Apprentice


Joined: 09 May 2004
Posts: 173
Location: Isle of Lesbos

PostPosted: Wed Jun 16, 2004 9:22 am    Post subject: Re: Linux Memory Management or 'Why is there no free RAM?' Reply with quote

Excellent writeup! This clears up a lot of confusion I've had.

sapphirecat wrote:

To see a better estimation of how much memory is really free for applications to use, run the command:
Code:
free -m

The -m option stands for megabytes, and the output will look something like this:
Code:
             total       used       free     shared    buffers     cached
Mem:           503        451         52          0         14        293
-/+ buffers/cache:        143        360
Swap:         1027          0       1027

The -/+ buffers/cache line shows how much memory is used and free from the perspective of the applications. Generally speaking, if little swap is being used, memory usage isn't impacting performance at all.


Okay. So in this case, 360MB of non-swap memory can be allocated by user programs?

Also, what do buffers=14mb and cached=293mb mean? I read your post very carefully, but I still do not understand. Why doesn't the used buffers/cache=143mb equal 14mb+293mb?
Back to top
View user's profile Send private message
meyerm
Veteran
Veteran


Joined: 27 Jun 2002
Posts: 1311
Location: Munich / Germany

PostPosted: Wed Jun 16, 2004 9:57 am    Post subject: Reply with quote

mem7 wrote:
Code:
MemTotal:       904296 kB

But it dose make me wonder why it was detecting 900MB of ram instead of the supposed 800MB limit with out high memory support in the kernel.


First: The Limit is not 800 MB but higher. (see below)
Second:
Code:
meyerm@yavin meyerm $ echo 904296/1024 | bc --
883
Back to top
View user's profile Send private message
rich0
Developer
Developer


Joined: 15 Sep 2002
Posts: 161

PostPosted: Wed Jun 16, 2004 5:21 pm    Post subject: Re: Linux Memory Management or 'Why is there no free RAM?' Reply with quote

stp_1993 wrote:

sapphirecat wrote:

To see a better estimation of how much memory is really free for applications to use, run the command:
Code:
free -m

The -m option stands for megabytes, and the output will look something like this:
Code:
             total       used       free     shared    buffers     cached
Mem:           503        451         52          0         14        293
-/+ buffers/cache:        143        360
Swap:         1027          0       1027



Okay. So in this case, 360MB of non-swap memory can be allocated by user programs?


Yes

Quote:
Also, what do buffers=14mb and cached=293mb mean? I read your post very carefully, but I still do not understand. Why doesn't the used buffers/cache=143mb equal 14mb+293mb?


143 = 451 - (14+293)
360 = 52 + (14+293)

143 is not the space used by buffer/cache. It is the amount used after buffers/cache are subtracted out.[/quote]
Back to top
View user's profile Send private message
nadamsieee
Guru
Guru


Joined: 30 May 2003
Posts: 340
Location: Atlanta, GA, USA

PostPosted: Wed Jun 16, 2004 7:09 pm    Post subject: Reply with quote

qal21 wrote:
mem7: actually it's 870MB or so it's detecting for you, not quite 900 (a MB is 1024 KB not 100), oh well, I'm probably getting too technical


Lets not rehash that debate! :wink:
_________________
nadams (at) ieee (dot) org
Back to top
View user's profile Send private message
log
Apprentice
Apprentice


Joined: 16 Jun 2003
Posts: 168
Location: Adelaide, Australia

PostPosted: Thu Jun 17, 2004 3:24 am    Post subject: Reply with quote

Thanks for explaining that :)

About 3 months ago I needed more ram for my PhD research and upgraded to 1GB, ran free and thought "WTF!?" :lol:

So i remembered that there was that option to enable 4GB in the kernel and tried it out. I think it would be a good idea to have a small description in the kernel that explains if you have exactly 1GB and want it all.

To me, I need the additional memory desperatly (im solving large non-sparse linear programming problems)
Back to top
View user's profile Send private message
rich0
Developer
Developer


Joined: 15 Sep 2002
Posts: 161

PostPosted: Thu Jun 17, 2004 12:07 pm    Post subject: Reply with quote

log wrote:
Thanks for explaining that :)

About 3 months ago I needed more ram for my PhD research and upgraded to 1GB, ran free and thought "WTF!?" :lol:

So i remembered that there was that option to enable 4GB in the kernel and tried it out. I think it would be a good idea to have a small description in the kernel that explains if you have exactly 1GB and want it all.

To me, I need the additional memory desperatly (im solving large non-sparse linear programming problems)


I got an AMD64 system with 512MB up and running for about $600 (some commodity parts like DVD-ROM and floppy drive were recycled, but all in all it would only cost about $700 from scratch). Once you are 64-bit you can expand the memory size quite a bit without penalty.

If this is for your PhD perhaps you can get a grant to boost your computing power? If 4GB is plenty then you may just want to stick with your x86. Also - if you only have 32-bit binaries of your application for some reason you may also just want to stay with x86. But if you have the source then most likely you can run it under AMD64 on gentoo just fine.

RAM isn't super-cheap, but if you really do need it then it is probably worth just buying. And a newer system with faster RAM may also help out for your application.
Back to top
View user's profile Send private message
log
Apprentice
Apprentice


Joined: 16 Jun 2003
Posts: 168
Location: Adelaide, Australia

PostPosted: Thu Jun 17, 2004 3:32 pm    Post subject: Reply with quote

a-ight, I would love to be 64bit....BUT the uni has provided me with a P4 3.2Ghz HT ("State of the art") and the losers they bought it from are "mr dickhead intel" boys....so unfortunatley there isnt much hope of me going the SUPERIOR AMD way.

All in all, I was luck to get the 512MB upgrade.

sorry, i just came home after having a few beers (im Aussie after all) so if there is gramatical errors...meh :P

life is fun, have a good one :)
Back to top
View user's profile Send private message
madmango
Guru
Guru


Joined: 15 Jul 2003
Posts: 507
Location: PA, USA

PostPosted: Thu Jun 17, 2004 4:57 pm    Post subject: Reply with quote

I'm pretty sure you do, memory management on x86 and x86_64 are identical. The only difference is where the controller is located :D
_________________
word.
Back to top
View user's profile Send private message
Joseph_sys
Advocate
Advocate


Joined: 08 Jun 2004
Posts: 2712
Location: Edmonton, AB

PostPosted: Thu Jun 17, 2004 10:40 pm    Post subject: Reply with quote

I took my old box with 1GB RAM to a test. I compile one kernel 2.6 with high mem. support and one without and here are some numbers:

1.) Without High Memory Support:

total used free shared buffers cached
Mem: 882 244 638 0 10 123
-/+ buffers/cache: 110 772
Swap: 956 0 956

2.) With High Memory Support

total used free shared buffers cached
Mem: 1009 357 652 0 13 224
-/+ buffers/cache: 119 890
Swap: 956 0 956


In addition I have run some "hdparm -Tt /dev/hda" test with and without hight memory support and to be honest I did not noticed any difference.
As I run a web-server I check my page_parse_time.log and did not noticed any difference either.
Though I like the number 652 free RAM with High Memory Access much better than 638 :-)

Though I must admit this article is very informative.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, 4  Next
Page 2 of 4

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum