View previous topic :: View next topic |
Author |
Message |
joanandk Apprentice
Joined: 12 Feb 2017 Posts: 169
|
Posted: Mon Sep 18, 2017 2:27 pm Post subject: Tiny Gentoo for sort of embedded systems |
|
|
Hi,
I am using x86 and x86_64 systems (Geode, Pentium, Core 2, etc.) to create servants which have a specific task. One of the tasks is to provide a owserver, another is to automatically start xorg and vlc and play IPTV stream.
I have managed to create a working environment and have been using it for sometime. I have started to strip the system down to strip the system down in order to start the system faster and use less flash space as possible.
I make use of Virtualbox and Gentoo inside the Virtualbox to create a working system. Using a script I create a squashfs file with the root in it, but stripped off everything unnecessary (for example /usr/src and /usr/portage).
I have managed to bring down the squashfs file to 313MB (with xorg and vlc on x86_64) and 162MB (with owserver and no xorg). The directories which are excluded:
- /etc/portage
- /usr/bin/x86_64-pc-linux-gnu-*
- /usr/include
- /usr/lib64/*.a
- /usr/lib64/*.la
- /usr/lib54/gcc
- /usr/portage
- /usr/share/automake*
- /usr/lib/python[23]*
Is it possible to strip down further? The target system does not need emerge, gcc, etc.
BR |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9679 Location: almost Mile High in the USA
|
Posted: Wed Sep 20, 2017 9:56 am Post subject: |
|
|
I was trying to build a really tiny Gentoo but unhappy with it because it made updating it even harder than it already is. At least with a complete portage build system it can still self update in a bind, and not depend on another system that somehow maintains the correct USE flags to copy over...
*shrug* I guess it depends on the person and expected update schedule. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
Telemin l33t
Joined: 25 Aug 2005 Posts: 753 Location: Glasgow, UK
|
Posted: Wed Sep 20, 2017 10:07 am Post subject: |
|
|
Hi guys, I assume you are aware that portage supports installing from binary packages for this kind of situation?
The idea is that you can have a full buildroot somewhere, e.g on a VM or a chroot in another machine, and then save all the packages you build as binaries and install just the bits you need to a deploy image. You can filter the installed files from each package to minimise bloat and either use a list of individual packages with --nodeps or create your own profile in order to avoid installing anything except the bits you need. It is a lot of work for a one-shot deployment, but if you are going to use it long-term or deploy many similar images it can be a very useful approach.
This is mostly within the realm of the embedded project so their IRC and mailing list will probably be the places to go to get specific help.
-Telemin- _________________ The Geek formerly known as -Freestyling-
When you feel your problem has been solved please add [Solved] to the topic title.
Please adopt an unanswered post |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54242 Location: 56N 3W
|
Posted: Wed Sep 20, 2017 12:52 pm Post subject: |
|
|
joanandk,
If you need a single package network appliance build the single package that you need, the way you want it, then run lddtree on it.
That will give a tree of everything the package needs to run.
Minor things like config files, bootloaders and kernels will not be listed.
Test by installing just those things into a chroot.
I guess, you have found the FEATURES settnigs to suppress the install of documentation? _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9679 Location: almost Mile High in the USA
|
Posted: Wed Sep 20, 2017 4:44 pm Post subject: |
|
|
Also don't forget about
app-admin/localepurge
There's a bunch of locale/i18n stuff that also can be deleted that I forget where in /usr/share/i18n and /usr/share/locale.
In any case going through the exercise of deleting stuff is a pain after each update is also miserable.
---
oh gosh, soapbox here about
FEATURES="nodoc noinfo"
As far as I can tell, it seems that portage installs the doc/info files and then *slowly* deletes them... As I watch portage clean up the doc/info files, I gave up watching and let it go ahead and install the doc and info files and I manually delete them afterwards because it seems faster... _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54242 Location: 56N 3W
|
Posted: Wed Sep 20, 2017 5:04 pm Post subject: |
|
|
and noman.
--- edit ---
The man, docs and info pages are packed into the tarballs, they are not installed on the host.
emerge still goes through all the motions.
Than means that if you make tarballs then install them without using portage, you get the man, doc and info pages too. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
szatox Advocate
Joined: 27 Aug 2013 Posts: 3137
|
Posted: Wed Sep 20, 2017 7:15 pm Post subject: |
|
|
joanandk, I've tried your approach, failed miserably and then went the opposite direction.
I was much more successful using emerge to build those tiny images bottom-up instead of striping down all the cruft from stage3.
Running emerge --root=/path/to/new/root goes a long way. It's not perfect, but it certainly is much easier and faster. |
|
Back to top |
|
|
joanandk Apprentice
Joined: 12 Feb 2017 Posts: 169
|
Posted: Fri Sep 22, 2017 9:07 am Post subject: |
|
|
Thanks for your inputs.
@eccerr0r
Quote: | I was trying to build a really tiny Gentoo but unhappy with it because it made updating it even harder |
I think you are missing my setup: I use a VM to create a read only filesystem which is executed on the target device. There is no need for the target device to update via Gentoo portage. If I need something, I update the VM and add the packages I need and reprogram the target.
I have a script in the squashfs file which can be executed. This uses wget to download newer Kernel and Gentoo-root (as squashfs), copy it to the flash. At reboot, the initrd script which is in charge for mounting the squashfs root and overlaying it with overlayfs, chooses the new squashfs file. Should the new system fail to boot, initrd will start the old squashfs file at next reboot.
Quote: | Also don't forget about
app-admin/localepurge |
I am using localepurge, it has brought down the squashfs around 200MB down.
@Telemin
Quote: | It is a lot of work for a one-shot deployment, but if you are going to use it long-term or deploy many similar images it can be a very useful approach. |
About your approach, I was not successful at the first shot and I gave up due to the initial "lot of work". Further I was not well aware how to get the dependencies correct.
@NeddySeagoon
Quote: | If you need a single package network appliance build the single package that you need, the way you want it, then run lddtree on it. |
I was not aware of this, I will give this a try with the solution shown by Telemin
Quote: | I guess, you have found the FEATURES settnigs to suppress the install of documentation? |
Yes I have and I am using it.
@szatox
Quote: | joanandk, I've tried your approach, failed miserably and then went the opposite direction. |
Here is where the VM comes in handy: Before I make any major stripping, I create a snapshot. If I see that something went horribly wrong, I still have a fallback.
Quote: | I was much more successful using emerge to build those tiny images bottom-up instead of striping down all the cruft from stage3. |
Yes you are correct not to use stage3. But as I now have a running stage3 installation (which I have deployed and running on over 4 devices), I was just thinking if the stripping down further.
Thanks again. I will try the two solutions mentionned: Installing from binary packages and starting from stage1 or stage2. I will leave my working development system intact until I have some expriences on the binary package installation method. |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9679 Location: almost Mile High in the USA
|
Posted: Wed Oct 11, 2017 7:23 am Post subject: |
|
|
Well, I went back to two of my "small" installs - one is on a 512M CF card, and another on a 2GB CF card.
512M card... It has about 256MB worth of Gentoo stuff on it, including compiler, full glibc, but no portage toolkit. When I tried to ROOT= emerge to it, first thing I saw: it wanted to install systemd and full of blockers...
NTY.
Forget about it, I'm not upgrading this one. Hopeless (had a lot of customizations that it likely would overwrite as well). Alas it is a fairly old install I did a VERY long time ago, so it's kind of expected.
The 2GB CF card: has full portage system (but no /usr/portage, intended to be mounted via NFS), uses ~860MB including a 64MB swap file and a bunch of router and tools (hostapd, wireless-tools, ethtool, wpa_supplicant, gnu screen, lynx, cron, pcmciautils,)... This one doesn't look too bad to upgrade, in fact emerge pretend came back clean including a portage upgrade. So, I'm pushing an upgrade to it now. Unfortunately it's bloating about 20MB so far from the upgrades, hopefully I can get rid of some of it from depclean and cleaning up locales. Hopefully it will stay below 1GB...though I'm not sure how long I can hold back anything past gcc 4.9.3 (manually masked...) before it becomes obsolete. I might try another whack at getting this Elan 520 to be a Gentoo router, however, need to get it to have wifi and another Ethernet port, else I may have to resort to using this on the Geode GX1. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
MarginallyStable n00b
Joined: 15 Oct 2017 Posts: 3
|
Posted: Sun Oct 15, 2017 5:42 am Post subject: Buildroot |
|
|
I use buildroot for the embedded case. |
|
Back to top |
|
|
joanandk Apprentice
Joined: 12 Feb 2017 Posts: 169
|
Posted: Wed Dec 20, 2017 7:21 am Post subject: |
|
|
szatox wrote: | joanandk, I've tried your approach, failed miserably and then went the opposite direction.
I was much more successful using emerge to build those tiny images bottom-up instead of striping down all the cruft from stage3.
Running emerge --root=/path/to/new/root goes a long way. It's not perfect, but it certainly is much easier and faster. |
I am having hard time finding stage1 tarball. Can some one help me find it?
BR |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54242 Location: 56N 3W
|
Posted: Wed Dec 20, 2017 11:02 am Post subject: |
|
|
joanandk,
The stage 1 is no longer published. What would you do with it?
What is your plan to build your tiny system? _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
joanandk Apprentice
Joined: 12 Feb 2017 Posts: 169
|
Posted: Thu Dec 21, 2017 6:51 am Post subject: |
|
|
NeddySeagoon wrote: | The stage 1 is no longer published. What would you do with it?
What is your plan to build your tiny system? |
Hi Neddy,
I have a chance to get my hands on an old AMD-Geode board with 128MB flash. My stage 3 installation was squeezed with squashfs down to 196MB + 2.7MB Kernel. As you have said you were more successful with bottom-up, I wanted to try that approach.
If I am successful with the AMD-Geode, there is an old Fortigate 50B waiting for its alternative Firmware
BR |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54242 Location: 56N 3W
|
Posted: Thu Dec 21, 2017 11:22 am Post subject: |
|
|
joanandk,
A long time ago I wrote a page about unpicking the liveDVD and using the binaries to install Gentoo.
Not many people used it but I was told that it wasn't possible. :)
As a part of that, I needed to discover what the minimum package set was for a chroot.
The original XML is still there but it won't render any longer.
The good news is that the Wayback machine has it.
It prepares a minimal chroot from nothing and chroots into it. That guide unpicks a liveDVD. You can just as well unpick a stage3
When Code: | Code Listing 5.3: Entering the chroot | works you have a chroot that is a lot smaller than a stage1
The package versions in Code: | Code Listing 5.2: Unpacking some key tarballs | are from 2009. They may have changed :)
In particular, python-3.x should be enough. Almost nothing needs python-2 any more.
The tar command will need thein addition to Code: | tar -xjpf sys-apps/portage-2.1.6.13.tbz2 -C /mnt/gentoo |
Random thought, you don't need portage nor the snapshot inside the chroot. That's properly called the target root. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|