| View previous topic :: View next topic |
| Author |
Message |
Varean
Guru
Joined: 03 Jul 2005
Posts: 436
Location: California, USA
|
 Posted: Sun Sep 11, 2005 3:26 am Post subject: Gentoo Speed |
 |
|
So far, my Gentoo has been a bit slow, and I am wondering if there is a way to speed it up. I have been un emerging a lot of apps lately and I wonder if it would leave my harddrive fragmented(Kinda like Windows), and if it does, is there a way to do a defragmentation on it. Also, I would like to clean up my HD, so if anyone knows any temp files I can delete, that would be helpful.
_________________
Registered Linux User #387568
|
|
| Back to top |
|
 |
Dlareh
Advocate
Joined: 06 Aug 2005
Posts: 2102
|
| Posted: Sun Sep 11, 2005 4:02 am Post subject: |
|
|
you can delete things in /usr/portage/distfiles, /tmp , and /var/tmp
_________________
"Mr Thomas Edison has been up on the two previous nights discovering 'a bug' in his phonograph." --Pall Mall Gazette (1889)
Are we THERE yet? |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20099
|
| Posted: Mon Sep 12, 2005 5:09 pm Post subject: |
|
|
What are your CFLAGS settings? Some, that people think will make them l33t $p33d |>3|\/|0|\|$, can actually slow your box down.
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
Taladar
Guru
Joined: 09 Oct 2004
Posts: 458
Location: Bielefeld, Germany
|
| Posted: Mon Sep 12, 2005 7:55 pm Post subject: |
|
|
| There is a Guide somewhere in the Forums called "Flying with Gentoo" or something similar that contains most serious speed tips (not CFLAGS, but prelinking,...) |
|
| Back to top |
|
|
loki99
Advocate
Joined: 10 Oct 2003
Posts: 2056
Location: Vienna, €urope
|
| Posted: Mon Sep 12, 2005 8:29 pm Post subject: |
|
|
*) Check whether you have Dma turned on by looking at the output of:
*) Post the output of: | Code: | | hdparm -tT /dev/hda |
*) You could activate NPTL.
Installing and uninstalling software does not slow down your system under linux and, AFAIK, there is no such thing as fragmentation of your HD either.
What filesystem do you use, BTW? |
|
| Back to top |
|
|
Varean
Guru
Joined: 03 Jul 2005
Posts: 436
Location: California, USA
|
| Posted: Mon Sep 12, 2005 11:00 pm Post subject: |
|
|
| Dlareh wrote: | | you can delete things in /usr/portage/distfiles, /tmp , and /var/tmp |
Did, that seems to work better now but I am gonna Hdpram as soon as I get home. Thx for the help.
_________________
Registered Linux User #387568
|
|
| Back to top |
|
|
neuromancerzero
n00b
Joined: 17 Aug 2005
Posts: 63
|
| Posted: Tue Sep 13, 2005 12:30 am Post subject: |
|
|
| You might want to change the file system |
|
| Back to top |
|
|
Dlareh
Advocate
Joined: 06 Aug 2005
Posts: 2102
|
| Posted: Tue Sep 13, 2005 12:34 am Post subject: |
|
|
to reiserfs, if you are using something else...
rsync -ax everything somewhere safe, make a new filesystem, and rsync -a everything back
note that rsync accepts scp-style paths, e.g. "username@server:/some/path"
_________________
"Mr Thomas Edison has been up on the two previous nights discovering 'a bug' in his phonograph." --Pall Mall Gazette (1889)
Are we THERE yet? |
|
| Back to top |
|
|
lotw
Guru
Joined: 09 Jan 2005
Posts: 342
Location: Palmdale
|
| Posted: Wed Sep 14, 2005 4:13 pm Post subject: |
|
|
| Your drive is probably really fragmented and to all you people who say filesystems don't really get fragmented are wrong. My server has 50g left and is up to 46.7% fragmented. I currently don't have a spare drive with enough free space to copy everything over, reformat and put everything back. Gentoo and Linux needs a defrag for the various filesystems. |
|
| Back to top |
|
|
madbiker
Guru
Joined: 12 Oct 2003
Posts: 439
Location: Victoria, BC, Canada
|
| Posted: Wed Sep 14, 2005 5:00 pm Post subject: |
|
|
| lotw wrote: | | Your drive is probably really fragmented and to all you people who say filesystems don't really get fragmented are wrong. My server has 50g left and is up to 46.7% fragmented. I currently don't have a spare drive with enough free space to copy everything over, reformat and put everything back. Gentoo and Linux needs a defrag for the various filesystems. |
Exactly how are you checking the percent fragmentation on a *nix drive? And what filesystem is it using? |
|
| Back to top |
|
|
Cinder6
l33t
Joined: 05 Aug 2004
Posts: 767
Location: California
|
| Posted: Wed Sep 14, 2005 11:10 pm Post subject: |
|
|
| madbiker wrote: | | lotw wrote: | | Your drive is probably really fragmented and to all you people who say filesystems don't really get fragmented are wrong. My server has 50g left and is up to 46.7% fragmented. I currently don't have a spare drive with enough free space to copy everything over, reformat and put everything back. Gentoo and Linux needs a defrag for the various filesystems. |
Exactly how are you checking the percent fragmentation on a *nix drive? And what filesystem is it using? |
IIRC, fsck programs usually tell you.
_________________
Knowledge is power.
Power corrupts.
Study hard.
Be evil.
Ugly Overload |
|
| Back to top |
|
|
loki99
Advocate
Joined: 10 Oct 2003
Posts: 2056
Location: Vienna, €urope
|
| Posted: Thu Sep 15, 2005 12:50 am Post subject: |
|
|
I found an interesting link about disk fragmentation.
For the lazy  , here are some quotes:
| Quote: | Unix file systems (and there are many kinds of file systems for Unix)
reside on all the cylinders within a given partition. They allocate
files in the blocks that make up these cylinders, trying to make all
the blocks contiguous and within the same cylinder. This means file
I/O is made more efficient because the heads do not need to seek to
different locations on the disc to read or write all the blocks within
a given cylinder. Many files can then all reside in the same cylinder
and the heads will not need to seek to read/write any of them. (It
should be noted that disc I/O is also speeded by file buffer caching
in RAM so that write operations can be optimized in relation to other
file I/O requests and CPU utilization, although file reads/writes that
are not in, or are too large for, the cache end up being limited by
the raw filesystem speed.) |
| Quote: | Unix, on the other hand, has much less likelihood of having situations
of high disk fragmentation that will impact I/O performance.
("Better" is not the same as "perfect". Some file systems, like the
Berkeley Fast File System, try to allocate block in at least a
rotationally efficient, if not perfectly contiguous, manner. You can
also get some improvements in fragment allocation via "tunefs". See
"man tunefs" and _The Design an Implementation of the 4.3 BSD Unix
Operating System_, listed in the Library section.) |
So, yes, there is some fragmentation possible under linux file systems, but it is rather unlikely.
And what can you do to avoid fragmentation of your hd?
| Quote: | The most likely time for fragmentation problems to crop up is when the
disk partitions near 100% capacity. This is a good reason to monitor
disk usage and ensure plenty of free space is generally available.
|
But what do you do when it still did happen to you?
| Quote: | A common way to defragment Unix file systems is to do a backup, remake
the file system, then restore the files. Note that you must do the
backup using a program that operates at the level of directories and
files (e.g., "cpio", "tar", "dump"), rather than dealing with raw
partitions themselves (e.g., "dd"). |
|
|
| Back to top |
|
|
alphonce
Apprentice
Joined: 10 Oct 2004
Posts: 184
Location: Sweden
|
| Posted: Thu Sep 15, 2005 4:02 pm Post subject: |
|
|
try recompile your gentoo system with latest unstable gcc (3.4.4-r1):
here is a link you could try:
https://forums.gentoo.org/viewtopic-t-345229.html
it is aimed for fresh installations, but if you skip the first part and go directly to point "6.6.2 Package Keywords - Enabling GCC 3.4.4 in the Stable Branch" and recompile your toolchain i think you will notice a system boost.
but it's a little unstabe so do it on your own risk.
_________________
no computer at the moment!  |
|
| Back to top |
|
|
Dlareh
Advocate
Joined: 06 Aug 2005
Posts: 2102
|
| Posted: Thu Sep 15, 2005 4:11 pm Post subject: |
|
|
| Quote: | | but it's a little unstabe so do it on your own risk. |
Calling it a little unstable is an exageration. It is probably one of the most stable packages currently marked ~x86.
_________________
"Mr Thomas Edison has been up on the two previous nights discovering 'a bug' in his phonograph." --Pall Mall Gazette (1889)
Are we THERE yet? |
|
| Back to top |
|
|
Blood Fluke
Apprentice
Joined: 15 Sep 2005
Posts: 224
|
| Posted: Thu Sep 15, 2005 11:34 pm Post subject: |
|
|
| Varean wrote: | | So far, my Gentoo has been a bit slow, and I am wondering if there is a way to speed it up. I have been un emerging a lot of apps lately and I wonder if it would leave my harddrive fragmented(Kinda like Windows), and if it does, is there a way to do a defragmentation on it. |
If you're using ReiserFS, then yes, it's very likely that your filesystem is highly fragmented. I'm afraid that the only way to defragment it is "dump" followed by "restore." You can avoid fragmentation in the future by omitting the "notail" option, but that will cripple performance.
I suggest you use ext2, ext3, or XFS in the future if you wish to avoid fragmentation.
| loki99 wrote: | I found an interesting link about disk fragmentation.
For the lazy , here are some quotes:
(quotes)
But what do you do when it still did happen to you? |
This article is talking about UFS variants, specifically the 4BSD Berkeley FFS and the old DEC AdvFS. It's totally irrelevant to Linux, whose filesystems do not even superficially resemble either one.
| Quote: | A common way to defragment Unix file systems is to do a backup, remake
the file system, then restore the files. Note that you must do the
backup using a program that operates at the level of directories and
files (e.g., "cpio", "tar", "dump"), rather than dealing with raw
partitions themselves (e.g., "dd"). |
This was a common way to do things. In 1985. In the modern world, we use non-fragmenting filesystems (XFS) or we use fsck (ext2).
Of course, if you're using ReiserFS, which fragments incredibly quickly, you're pretty fucked. The only way to defragment a ReiserFS filesystem is the old dump and restore trick. Translation: Make backups and cross your fingers. |
|
| Back to top |
|
|
alphonce
Apprentice
Joined: 10 Oct 2004
Posts: 184
Location: Sweden
|
| Posted: Fri Sep 16, 2005 6:55 am Post subject: |
|
|
| Dlareh wrote: | | Quote: | | but it's a little unstabe so do it on your own risk. |
Calling it a little unstable is an exageration. It is probably one of the most stable packages currently marked ~x86. |
when you say most stable, it's a relative judgement. it't doesn't mean it's stable, but most stable Of those packages in the ~x86 branch. But would it stable, it would be in the x86 branch, which it's not. Then it's Little unstable, but not complete stabil.
_________________
no computer at the moment! |
|
| Back to top |
|
|
mdshort
Apprentice
Joined: 06 Dec 2004
Posts: 157
|
| Posted: Wed Nov 16, 2005 1:04 pm Post subject: |
|
|
Actually all packages are required to go though ~x86, so even though it might be completely stable, and usually is, it must at some point be in ~x86 for quite some time. (this is why I use ~x86 religiously)
_________________
"With every rise, there is a fall." |
|
| Back to top |
|
|
omp
Retired Dev
Joined: 10 Sep 2005
Posts: 1018
Location: Glendale, California
|
| Posted: Wed Nov 16, 2005 5:58 pm Post subject: |
|
|
| alphonce wrote: | | when you say most stable, it's a relative judgement. it't doesn't mean it's stable, but most stable Of those packages in the ~x86 branch. But would it stable, it would be in the x86 branch, which it's not. Then it's Little unstable, but not complete stabil. |
A package being in ~arch usually doesn't mean it's unstable, but it means it has not gone through the necessary testing yet.
_________________
meow. |
|
| Back to top |
|
|
mikegpitt
Advocate
Joined: 22 May 2004
Posts: 3224
|
| Posted: Wed Nov 16, 2005 6:07 pm Post subject: |
|
|
| Reiserfs is journaled, and I thought journaled filesystems don't fragment, unless like stated above you are almost at disk capacity. Correct me if I'm wrong... |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Wed Nov 16, 2005 6:39 pm Post subject: |
|
|
| mikegpitt wrote: | | Reiserfs is journaled, and I thought journaled filesystems don't fragment, unless like stated above you are almost at disk capacity. Correct me if I'm wrong... |
My understanding is that reiserFS, to maximize disk space usage, will place a file fragment with another fragement on another block that is only partyly-used. The notail option is supposed to disable this, which means that ReiserFS will attempt to layout the files contiguously instead (at the cost of some disk space due to not having every block on the system full of data). I'm going to try to find some detailed explanations of these. This intrigues me as well.
Edit: Personally, I'm a big fan of Ext3. With the dir_index and full journalling options, it's generally quite fast, it's very stable, and I have that warm fuzzy feeling that I know my data is being protected. 
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
mrness
Retired Dev
Joined: 17 Feb 2004
Posts: 375
Location: bucharest.ro
|
| Posted: Sun Nov 20, 2005 4:56 am Post subject: |
|
|
If you have directories like /usr/portage, there is no fs like reiserfs! In my experience, an emerge --sync is x times faster ( 2 < x < 5 ) on reiserfs than on ext2/ext3/xfs.
Maybe xfs is non-fragmenting fs but the price is too big - the system responsiveness is very low when comes the time to flush the cache on HDD. |
|
| Back to top |
|
|
curtis119
Bodhisattva
Joined: 10 Mar 2003
Posts: 2160
Location: Toledo, Ohio,USA, North America, Earth, SOL System, Milky Way, The Universe, The Cosmos, and Beyond.
|
| Posted: Sun Nov 20, 2005 11:07 am Post subject: |
|
|
| codergeek42 wrote: |
Edit: Personally, I'm a big fan of Ext3. With the dir_index and full journalling options, it's generally quite fast, it's very stable, and I have that warm fuzzy feeling that I know my data is being protected. |
I second this. I've tested out all the various filesystems and ext2 with a journal (ext3) and dir_index can't be beat. It's rock solid stable and fast as hell.
ext2/3 reserves a percentage of the filesystem (5% by default but you can increase/decrease this with the -m option when you mke2fs) that is used to prevent fragmentation. In other words, it reports the disk as being 5% smaller than it actually is and it uses that extra 5% to keep files contiguous instead of fragmenting them. It doesn't actually move files around if they do get fragemented, you have to back up the partition, remake it and restore the backup. There are a few defragmenters available but they are so slow it's just easier to do it the manual way.
Another way ext3 avoids fragmentation is using reservation. Sometime last year they added a patch that allows the file system to "guess" how big the file is going to be and reserve a contiguos space on the disk for that file thus avoiding fragmentation. So if you create a new Office doc and save it but keep the file open and keep adding to it the filesystem will "guess" how big your going t make the file and reserve a block for it. I can't be bothered to find a link for this bit but I clearly remember reading about it on the LKML.
I have never run into heavy fragmentation in the 10+ years I've been using linux by the way.
_________________
Gentoo: it's like wiping your ass with silk. |
|
| Back to top |
|
|
Zepp
Veteran
Joined: 15 Mar 2004
Posts: 1246
Location: Ontario, Canada
|
| Posted: Mon Nov 21, 2005 5:10 am Post subject: |
|
|
| Can you even check for fragmentation with reiserfs (3 that is). Looked at fsck.reiserfs but didn't see it or an option... |
|
| Back to top |
|
|
Enlight
Advocate
Joined: 28 Oct 2004
Posts: 3519
Location: Alsace (France)
|
| Posted: Mon Nov 21, 2005 8:26 am Post subject: |
|
|
| mrness wrote: | If you have directories like /usr/portage, there is no fs like reiserfs! In my experience, an emerge --sync is x times faster ( 2 < x < 5 ) on reiserfs than on ext2/ext3/xfs.
Maybe xfs is non-fragmenting fs but the price is too big - the system responsiveness is very low when comes the time to flush the cache on HDD. |
Could you please explain how you performed the test and wich version of reiserfs you're talking about? |
|
| Back to top |
|
|
mrness
Retired Dev
Joined: 17 Feb 2004
Posts: 375
Location: bucharest.ro
|
| Posted: Mon Nov 21, 2005 9:17 am Post subject: |
|
|
| Enlight wrote: | | mrness wrote: | If you have directories like /usr/portage, there is no fs like reiserfs! In my experience, an emerge --sync is x times faster ( 2 < x < 5 ) on reiserfs than on ext2/ext3/xfs.
Maybe xfs is non-fragmenting fs but the price is too big - the system responsiveness is very low when comes the time to flush the cache on HDD. |
Could you please explain how you performed the test and wich version of reiserfs you're talking about? |
- reiserfs 3.6
- I did the test some time ago (6 to 9 months):
mount a reiser filesystem on /var/cache/edb and /usr/portage with defaults (without notail or noatime)
run emerge --sync
Previously I had xfs, on which emerge --sync completed in about 20+ minutes. On reiser, it took less than 10 minutes.
On older machines, the improvement was bigger.
Also, during emerge --sync, xfs had the nasty habit to write the cache at every 5 (or so) seconds, for about 1+ second. When cache is written on HDD, the system is unresponsive.
Of course, YMMV. |
|
| Back to top |
|
|
|
Gentoo Chat
