[SOLVED]frustrations using SDHC solid state disks -EeePC

stardotstar · Last edited by stardotstar on Tue Apr 29, 2008 4:36 am; edited 1 time in total

I have snipped this from a post on the eeeusers forum because it is not getting any response - not a crit of their community; but maybe someone would be interested in this experience from a pure gentooers point of view and maybe someone would be able to comment whether they have eee experience or not - I have heard of many running gentoo happily on the eee... And I have not given up... currently I can get it to work fine on an external hdd...

---snip -->
Hi all,

I am rather out of time and patience for getting a stable Gentoo build onto my EeePC; a task to which I committed myself with great enthusiasm.

I am far from incompetent with Gentoo Linux, Debian and the *buntu distros although I freely admit that hardware level troubleshooting when it comes to disks and low level drivers is quite beyond my expertise.

Allow me to put my circumstances in perspective and perhaps someone will see the flaw in how I am proceeding and all will become clear.

The fact of the matter is that I can't get a Gentoo build to complete on my Eee - not on the internal SD card, not on the SDHC in the slot and/or any combination of partitions between the two and even attempts to mount bind the write/space intensive directories to an external HDD.

The frustrating thing is that the machine really does seem to be fine; it behaves very predictably until the dreaded "No space left on device" or "Cannot open: no such file or directory" errors - usually when copying (decompressing tarballs) and occasionally when simply trying to execute a command from (for example) /bin or /usr/bin

I have the 4G internal solid state disk and have read extensively about the pros and cons, FUDs and Opinions about how likely one distro or one approach to utilising the disk space vs another can impact device life. In fact to begin with I left the default install in tact and worked entirely off an 8G SDHC in the internal bay. I was booting successfully in via a 4GB Toshiba U3 USB Stick using the Gentoo based System Rescue CD 1.0.1 and found that everything I needed to begin a Gentoo install simply worked out of the box:

Networking on the wired LAN worked on boot; the display was more than adequete, RAM sufficient, touchpad for GDM fine, the keyboard I have quickly become used to.

So I enthusiastically launched into building a TinyGentoo (embedded) system on the box since I figured that I would only be struggling with a slightly unfamiliar install of Gentoo and its related packages and toolsets - I did not expect to be frustrated at so many stages with unfamiliar hardware issues appearing apparently out of nowhere to completely bulldoze the progress.

After getting so far into a TinyGentoo (uClibc) install - to the point of having had the little box successfully compile GCC4.1.2 from source over several hours of uninterrupted operation monitored and managed from an ssh session on my Gentoo based Intel MacBook Pro I suddenly found the machine unresponsive and the system disintegrating before my eyes:

I detailed one instance of this in this thread here:
http://forum.eeeuser.com/viewtopic.php?id=25172

Following on from several attempts I began to see a pattern; the solid state disks would simply not cope with the demand of installation and appear full or display a directory of files that were inaccessible when called.

I decided to get bold and do a vanilla gentoo install using the minimal installer etc; ie take the uClibc issues out of the picture as well as simplifying the partitioning and structure of the build. This met with the same fate.

So I gave the Debian installer a try. I use Debian Stable on my web servers and was quite satisfied that it was a reasonable intermediate step for my Eee since I was beginning to want a working install on it at all let alone my distro of first choice!

Debian Lenny installed fine on the main disk and wireless worked out of the box as promised and I was once again enthused about proceeding. Since I had by this time elected to install over the Xandros internal build I was committed to rewriting the internal SDD so I figured I'd have another crack at gentoo using the fundamental elements from the debian terminal and gentoo installers to prep a usb installer.

This was complicated by the fact that I could only get one of my usb drives to work correctly as a boot disk.

I figured that I was getting distracted and returned to building a simple clean gentoo on the internal disk.

I partitioned with ext2 for boot, ext3 for root and everything else to keep it simple.

The install progressed as though it would just go straight through - formatting and initialising the disks, mounting the partitions and proc and binding dev. Unpacking the stage3 tarball worked perfectly (I extracted it onto the internal disk after copying it to there to try and avoid speed/cache issues with pulling it over from another disk (esp the much faster external HDD). I was able to unpack the initial portage snapshot as well - which was a very exciting and encouraging breakthrough - usually - even with over 1.8G free the disk would baulk and say it was full or no such file or directory - couldn't open etc... So I proceeded with the chroot and the env-update worked fine; I also found all my chrooted binaries worked and the path statements were sane and fully operational.

This was the furthest I had got in so far (save from the gcc compile I had managed to get to earlier) and it was proceeding like a normal gentoo install (I have been doing these for several years - on lots of notebooks and desktops).

I had the proxy configured to get through my corp firewall and successfully used emerge-webrsync to pull down the latest portage to proceed when... mid way through unpacking the portage tree - same problem - no space left on device followed by Cannot open: no such file or directory ...

Disapointment, frustration.

I will return to Debian - to see if that works well a second time and then stays stable and operational and perhaps will put it down to something I simply don't have the time or expertise to fathom at the moment. But I am sorely dissapointed that a lightweight Gentoo build was not as easy for me to achieve as I fully expected.

I have detailed my (lack of) progress at a forum I dedicated to my sort of hacks at getting Gentoo fully sorted and you may find a link to that in another of my posts (I don't want to draw audience and participation away from this excellent site - especially to a narrative of failure!

If anyone can provide some insight or suggestions as to why I might be suffering this amount of trouble I would appreciate it.

The current state of the device is quoted below to show the basics of my situtation. As you can see there is no space issue.

NeddySeagoon · Posted: Fri Apr 25, 2008 3:18 pm Post subject:

stardotstar,

gcc wants a fair amount of space in /var/tmp/portage to build, as that is on your /, I suspect its filling up.
With such a small amount of space (4G), you will need to flush /usr/portage/distfiles fairly frequently and make sure no broken build are left in /var/tmp/portage.

You may be out of inodes, rather than disk space. When your root fills up, what does

hielvc · Posted: Sat Apr 26, 2008 12:26 am Post subject:

stardotstar,

I have a 4g and I'm thinking about putting gentoo on it also. I would not try to build on the eeepc though unless I used a usb drive orlarge usb or SD drives. Your going to wipe out that SSD memory using it to build gentoo. Unless your desktop is reallly old and slow use a partition or a dir to build it in and maintain it.
_________________
An A-Z Index of the Linux BASH command line

stardotstar · Posted: Sat Apr 26, 2008 12:12 pm Post subject:

Hi Guys,

Firstly, Neddy, fantastic advice as always; I am as always indebted to you for the time you take to understand and respond to so many varied questions.

I think I begin to understand what else may be causing this kind of behavior after your points about the need for inodes and root on tmpfs. I will attempt to grok this in fullness and begin again. Furthermore I will follow your advice and post df -i and h as soon as I can.

I was certainly working to minimize the impact on the internal disk but your insight into the implications of the tmpfs and inodes etc is something that gives me a great deal of encouragement!

Hielvc I understand the problematic issue of thrashing out sdhc cards, (there are a number of very interesting if not controversial articles doing the rounds) I have been trying to avoid using it intensively as much as possible and it is quite possible that this has contributed to my problems. When gave up and tried to do a full install on the internal disk and it failed and then I repeated the exercise on the external hdd I was perplexed but I am beginning to understand the full implications of using a source based meta-distro.

Your responses are very much appreciated guys.

Will
_________________
\\'
ab extra
ab aeterno
a posse ad esse
ad fontes
\\'

cwr · Veteran Joined: 17 Dec 2005 Posts: 1288

For lightweight machines (in my case a _very_ old Pentium) it was easiest for
me to keep all the portage sources off the machine on a local NAS box; ftp
could pick them up very quickly. I expect you could set up one of your other
boxes as an ftp server, which would save a lot of space.

And I've usually found that I need to unpack a portage snapshot on an ext3
filesystem on a separate partition built with extra inodes,
ie. mkfs -t ext3 -T news.

An awful lot of junk can accumulate in /var/spool/portage, too, after a couple
of build failures; more than enough to put a small filesystem over the top.
Then when portage fails most of the files are deleted, and you never find out
what happened. I tend to work from the command line, and open a separate
terminal to check current usage with "df".

Good luck - Will

stardotstar · Posted: Sun Apr 27, 2008 12:41 am Post subject:

Guys, this is a fantastic truly Gentoo set of replies

I am going to forge ahead as suggested.

In my original notes which if anyone could be bothered to read a tale of aborted and misconceived first attempts is here

(not to mention the gransiose claim of howto "gentoo fast on an eeePC" :lol:

http://www.sourcepoint.com.au/a3gis/forum/viewforum.php?f=2

Anyway I am researching the filesystem fundamentals of inodes and will again try to work up a stage 4 install similar to my original intentions:

stardotstar · Posted: Sun Apr 27, 2008 9:06 am Post subject:

Sadly it seems that in my frustration I have wiped the offending partitions - probably in a frenzy of "try another approach" on Friday afternoon.

I may give some recovery tools a go having done a thorough read and research on ext2/3 and perhaps, if I can recover the filesystem/s I can discover if it was an inode problem or something else worth reporting for analysis here.

Once again thanks for the help.

WIll
_________________
\\'
ab extra
ab aeterno
a posse ad esse
ad fontes
\\'

NeddySeagoon · Posted: Sun Apr 27, 2008 12:04 pm Post subject:

stardotstar,

testdisk will scan for what looks like partitions and offer to rebuild the partition table for you.
However, if you have already done a mkfs your data will be gone, even if you recover the partition table(s).
More exactly, you data will still be there but all the pointers to it in the file system meta data will be cleared.

Heres a quick guide ... do this on another system with Gentoo already installed. The CPU in this host system needs to be upwards compatible with the target eeepc CPU.

First, make an empty file on the host for what will be your eeepc install. You will do maintainace here for the eeepc in the future.
We then format the file as if it were a partition and mount it, so we need a mount point too.

stardotstar · Posted: Sun Apr 27, 2008 9:42 pm Post subject:

NeddySeagoon, thank you for that fantastic tutorial.

What I propose is that I attempt a new build with your approach in mind with one small change.

I will boot the E from the live CD/USB and go about an install as you recommend but on my external HDD.

I know this works because I managed it last time as long as I didn't use the solid state disks.

I would use another gentoo system but when I am at work I only have the E available next to my OSX Mac.

I take it that the final step in this would be to format the E's internal disk - or an SDHC in the slot for that matter; with a boot part (ext2 around 32MB) and the remainder
with ext3 and then copy everything over from the build on the external disk (the 10G loopback FS you defined) to the ext3 root on the E. - or would I do it with a dd?

Then complete the install by switching into the E (chroot) and mounting the internal boot partition; and installing grub to the MBR.

Since Dev and Proc are auto populated will I need to have anything other than an empty directory there in the tree or are there special copying switches I will need to use -

(I will go off and research this myself in the mean time.)

Only other question at this time is to understand how mount bind survives chroot in tact - obviously I have found that sym links get broken ...

Thanks again.
Will
_________________
\\'
ab extra
ab aeterno
a posse ad esse
ad fontes
\\'

stardotstar · Posted: Sun Apr 27, 2008 10:14 pm Post subject:

BTW - the problem is indeed inode exhaustion:

the testdisk tool recovered my partition(s) and upon mounting the old build and doing

df -i and df -h I saw that despite there being over 2G free the inode usage was 100% - 241920 - what a great lesson in filesystem handling

_________________
\\'
ab extra
ab aeterno
a posse ad esse
ad fontes
\\'

stardotstar · Posted: Tue Apr 29, 2008 4:41 am Post subject:

Following the guidance I have had here I ended up with the following solution:

Build the gentoo system for the E from the E on an external USB-HDD using two partitions
One partition was what would ultimately be the main root partition of the E.
The second is a mount bind location for /var/tmp and /usr/portage that is used whenever maintanance has to happen

I proceeded with a normal handbook build did some custom E-centric kernel stuff and then moved the root over to the second partition of the internal sdhc and made the necessary changes to fstab and grub.

Boots fast and clean and shouldn't be thrashing the internal fixed solid state disk because I have got home, an experimental swap and /var/tmp and /tmp on the 8G sdhc in the slot.

Now to kernel optimisation, madwifi-ng and X

uvesafb first :grin:

Thanks for all kthe help guys!

Will
_________________
\\'
ab extra
ab aeterno
a posse ad esse
ad fontes
\\'

NeddySeagoon · Posted: Tue Apr 29, 2008 7:04 pm Post subject:

stardotstar,

-o bind does not survive a chroot. With -o bind, you actually mount the same thing in two places.
Once inside and once outside the chroot.

If the eeepc has a flash drive, you should not use ext3 on it. The journal causes a lot of extra writes which age the flash and slow things down.

You can save more space on the eeepc - you don't need gcc and friends. It can stay in the maintainence partition on your USB hard drive. Its all a matter of how much you need the space or inodes.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

stardotstar · Posted: Tue Apr 29, 2008 10:53 pm Post subject:

Hi Neddy Seagoon, I am working hard to grok this fully and onto my fifth gentoo install with this little beast.

Until I really have my head around the mount binds and the implications of traversing bindings for maintenance both during and after install I think I will continue to build and build again.

Here is the initial steps for my current approach:

NeddySeagoon · Posted: Wed Apr 30, 2008 6:21 pm Post subject:

stardotstar,

An ASCII diagram may help, Its intended to be a part of the file system tree during a gentoo install, just before the chroot.

stardotstar · Posted: Wed Apr 30, 2008 9:32 pm Post subject:

Thank you sir. I feel much more confident with these concepts now. I guess that is why the mtab shows multiple instances of mount binds if one mistakenly initiates the mount more than once...

(having read something of your background I think I'll just go with ext2

)

Best Regards,
Will
_________________
\\'
ab extra
ab aeterno
a posse ad esse
ad fontes
\\'

cfriedt · Posted: Sat May 10, 2008 1:47 pm Post subject:

Your best bet with the EEE PC is to use a binary gentoo distribution. If you're not comfortable building a binary distribution on your own, then you should probably just use tinderbox.

First, you'll need to modify the installed grub configuration in the EEE pc (so that you can roll your own kernel, if you're into that) - search around on google for 'advanced mode' and 'persistent write' if you haven't done that already.

Next, extract a stage3 filesystem to a 4GB USB stick, and try using the following in /etc/make.conf

USE="" # edit your use flags as you see fit, but they'll be ignored by
# the binary package installation
FEATURES="-ccache getbinpkg"
EMERGE_DEFAULT_OPTS="-K"
PORTAGE_BINHOST="http://tinderbox.dev.gentoo.org/default-linux/x86"
INSTALL_MASK="*.h *.a *.la *.pc /usr/share/doc /usr/share/info /usr/lib/pkgconfig /usr/share/locale /usr/share/info /usr/share/gtk-doc /usr/include /usr/share/gnome/help /usr/share/aclocal* /usr/share/automake* /usr/share/autoconf* /usr/share/gnuconfig"

Note that INSTALL_MASK will likely need to be set on a per-package basis - another feature still missing from portage.

Emerge is still a bit annoying even when using a pure bindist, often requiring the portage tree to be present - otherwise "(masked by:)" will often appear.

You should be able to do emerge -av1 xorg-x11 gnome gdm, mozilla-firefox-bin, etc, etc.

The EEE PC and other ultra-mobile platforms are where Gentoo is noticibly lacking - portage really doesn't have the best binary package support ... eveeentually, I still think it will be a superior binary package manager too, but there is some distance to go.

stardotstar · Posted: Sun May 11, 2008 10:03 pm Post subject:

Thanks for the suggestions.

So far so good. I have successfully made a minimalist gentoo using portage mount bound when needed on an external HDD

My root system is 1.9G and includes Xorg minimal and fvwm, firefox, vim, ncftp and some various tools so far. I want to get the wireless and powermanagement sorted next but on the whole I am verry happy with the outcome of this little box.

I will sus out tinderbox - not had a lot of experience with gentoo on boxes with really constrained resources before

I'll build one on the internal slot sd card and have a go - I like the idea of a remote portage.

Will
_________________
\\'
ab extra
ab aeterno
a posse ad esse
ad fontes
\\'