Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Tiny Gentoo for sort of embedded systems
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
joanandk
Apprentice
Apprentice


Joined: 12 Feb 2017
Posts: 169

PostPosted: Mon Sep 18, 2017 2:27 pm    Post subject: Tiny Gentoo for sort of embedded systems Reply with quote

Hi,

I am using x86 and x86_64 systems (Geode, Pentium, Core 2, etc.) to create servants which have a specific task. One of the tasks is to provide a owserver, another is to automatically start xorg and vlc and play IPTV stream.
I have managed to create a working environment and have been using it for sometime. I have started to strip the system down to strip the system down in order to start the system faster and use less flash space as possible.

I make use of Virtualbox and Gentoo inside the Virtualbox to create a working system. Using a script I create a squashfs file with the root in it, but stripped off everything unnecessary (for example /usr/src and /usr/portage).

I have managed to bring down the squashfs file to 313MB (with xorg and vlc on x86_64) and 162MB (with owserver and no xorg). The directories which are excluded:
- /etc/portage
- /usr/bin/x86_64-pc-linux-gnu-*
- /usr/include
- /usr/lib64/*.a
- /usr/lib64/*.la
- /usr/lib54/gcc
- /usr/portage
- /usr/share/automake*
- /usr/lib/python[23]*

Is it possible to strip down further? The target system does not need emerge, gcc, etc.

BR
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9646
Location: almost Mile High in the USA

PostPosted: Wed Sep 20, 2017 9:56 am    Post subject: Reply with quote

I was trying to build a really tiny Gentoo but unhappy with it because it made updating it even harder than it already is. At least with a complete portage build system it can still self update in a bind, and not depend on another system that somehow maintains the correct USE flags to copy over...

*shrug* I guess it depends on the person and expected update schedule.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Telemin
l33t
l33t


Joined: 25 Aug 2005
Posts: 753
Location: Glasgow, UK

PostPosted: Wed Sep 20, 2017 10:07 am    Post subject: Reply with quote

Hi guys, I assume you are aware that portage supports installing from binary packages for this kind of situation?

The idea is that you can have a full buildroot somewhere, e.g on a VM or a chroot in another machine, and then save all the packages you build as binaries and install just the bits you need to a deploy image. You can filter the installed files from each package to minimise bloat and either use a list of individual packages with --nodeps or create your own profile in order to avoid installing anything except the bits you need. It is a lot of work for a one-shot deployment, but if you are going to use it long-term or deploy many similar images it can be a very useful approach.

This is mostly within the realm of the embedded project so their IRC and mailing list will probably be the places to go to get specific help.

-Telemin-
_________________
The Geek formerly known as -Freestyling-
When you feel your problem has been solved please add [Solved] to the topic title.
Please adopt an unanswered post
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54099
Location: 56N 3W

PostPosted: Wed Sep 20, 2017 12:52 pm    Post subject: Reply with quote

joanandk,

If you need a single package network appliance build the single package that you need, the way you want it, then run lddtree on it.
That will give a tree of everything the package needs to run.

Minor things like config files, bootloaders and kernels will not be listed.

Test by installing just those things into a chroot.

I guess, you have found the FEATURES settnigs to suppress the install of documentation?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9646
Location: almost Mile High in the USA

PostPosted: Wed Sep 20, 2017 4:44 pm    Post subject: Reply with quote

Also don't forget about

app-admin/localepurge

There's a bunch of locale/i18n stuff that also can be deleted that I forget where in /usr/share/i18n and /usr/share/locale.

In any case going through the exercise of deleting stuff is a pain after each update is also miserable.

---

oh gosh, soapbox here about

FEATURES="nodoc noinfo"

As far as I can tell, it seems that portage installs the doc/info files and then *slowly* deletes them... As I watch portage clean up the doc/info files, I gave up watching and let it go ahead and install the doc and info files and I manually delete them afterwards because it seems faster...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54099
Location: 56N 3W

PostPosted: Wed Sep 20, 2017 5:04 pm    Post subject: Reply with quote

and noman.

--- edit ---

The man, docs and info pages are packed into the tarballs, they are not installed on the host.
emerge still goes through all the motions.

Than means that if you make tarballs then install them without using portage, you get the man, doc and info pages too.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
szatox
Advocate
Advocate


Joined: 27 Aug 2013
Posts: 3106

PostPosted: Wed Sep 20, 2017 7:15 pm    Post subject: Reply with quote

joanandk, I've tried your approach, failed miserably and then went the opposite direction.
I was much more successful using emerge to build those tiny images bottom-up instead of striping down all the cruft from stage3.
Running emerge --root=/path/to/new/root goes a long way. It's not perfect, but it certainly is much easier and faster.
Back to top
View user's profile Send private message
joanandk
Apprentice
Apprentice


Joined: 12 Feb 2017
Posts: 169

PostPosted: Fri Sep 22, 2017 9:07 am    Post subject: Reply with quote

Thanks for your inputs.

@eccerr0r
Quote:
I was trying to build a really tiny Gentoo but unhappy with it because it made updating it even harder

I think you are missing my setup: I use a VM to create a read only filesystem which is executed on the target device. There is no need for the target device to update via Gentoo portage. If I need something, I update the VM and add the packages I need and reprogram the target.
I have a script in the squashfs file which can be executed. This uses wget to download newer Kernel and Gentoo-root (as squashfs), copy it to the flash. At reboot, the initrd script which is in charge for mounting the squashfs root and overlaying it with overlayfs, chooses the new squashfs file. Should the new system fail to boot, initrd will start the old squashfs file at next reboot.
Quote:
Also don't forget about

app-admin/localepurge

I am using localepurge, it has brought down the squashfs around 200MB down.

@Telemin
Quote:
It is a lot of work for a one-shot deployment, but if you are going to use it long-term or deploy many similar images it can be a very useful approach.

About your approach, I was not successful at the first shot and I gave up due to the initial "lot of work". Further I was not well aware how to get the dependencies correct.

@NeddySeagoon
Quote:
If you need a single package network appliance build the single package that you need, the way you want it, then run lddtree on it.

I was not aware of this, I will give this a try with the solution shown by Telemin
Quote:
I guess, you have found the FEATURES settnigs to suppress the install of documentation?

Yes I have and I am using it.

@szatox
Quote:
joanandk, I've tried your approach, failed miserably and then went the opposite direction.

Here is where the VM comes in handy: Before I make any major stripping, I create a snapshot. If I see that something went horribly wrong, I still have a fallback.
Quote:
I was much more successful using emerge to build those tiny images bottom-up instead of striping down all the cruft from stage3.

Yes you are correct not to use stage3. But as I now have a running stage3 installation (which I have deployed and running on over 4 devices), I was just thinking if the stripping down further.

Thanks again. I will try the two solutions mentionned: Installing from binary packages and starting from stage1 or stage2. I will leave my working development system intact until I have some expriences on the binary package installation method.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9646
Location: almost Mile High in the USA

PostPosted: Wed Oct 11, 2017 7:23 am    Post subject: Reply with quote

Well, I went back to two of my "small" installs - one is on a 512M CF card, and another on a 2GB CF card.

512M card... It has about 256MB worth of Gentoo stuff on it, including compiler, full glibc, but no portage toolkit. When I tried to ROOT= emerge to it, first thing I saw: it wanted to install systemd and full of blockers...

NTY.

Forget about it, I'm not upgrading this one. Hopeless (had a lot of customizations that it likely would overwrite as well). Alas it is a fairly old install I did a VERY long time ago, so it's kind of expected.



The 2GB CF card: has full portage system (but no /usr/portage, intended to be mounted via NFS), uses ~860MB including a 64MB swap file and a bunch of router and tools (hostapd, wireless-tools, ethtool, wpa_supplicant, gnu screen, lynx, cron, pcmciautils,)... This one doesn't look too bad to upgrade, in fact emerge pretend came back clean including a portage upgrade. So, I'm pushing an upgrade to it now. Unfortunately it's bloating about 20MB so far from the upgrades, hopefully I can get rid of some of it from depclean and cleaning up locales. Hopefully it will stay below 1GB...though I'm not sure how long I can hold back anything past gcc 4.9.3 (manually masked...) before it becomes obsolete. I might try another whack at getting this Elan 520 to be a Gentoo router, however, need to get it to have wifi and another Ethernet port, else I may have to resort to using this on the Geode GX1.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
MarginallyStable
n00b
n00b


Joined: 15 Oct 2017
Posts: 3

PostPosted: Sun Oct 15, 2017 5:42 am    Post subject: Buildroot Reply with quote

I use buildroot for the embedded case.
Back to top
View user's profile Send private message
joanandk
Apprentice
Apprentice


Joined: 12 Feb 2017
Posts: 169

PostPosted: Wed Dec 20, 2017 7:21 am    Post subject: Reply with quote

szatox wrote:
joanandk, I've tried your approach, failed miserably and then went the opposite direction.
I was much more successful using emerge to build those tiny images bottom-up instead of striping down all the cruft from stage3.
Running emerge --root=/path/to/new/root goes a long way. It's not perfect, but it certainly is much easier and faster.


I am having hard time finding stage1 tarball. Can some one help me find it?

BR
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54099
Location: 56N 3W

PostPosted: Wed Dec 20, 2017 11:02 am    Post subject: Reply with quote

joanandk,

The stage 1 is no longer published. What would you do with it?

What is your plan to build your tiny system?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
joanandk
Apprentice
Apprentice


Joined: 12 Feb 2017
Posts: 169

PostPosted: Thu Dec 21, 2017 6:51 am    Post subject: Reply with quote

NeddySeagoon wrote:
The stage 1 is no longer published. What would you do with it?

What is your plan to build your tiny system?


Hi Neddy,

I have a chance to get my hands on an old AMD-Geode board with 128MB flash. My stage 3 installation was squeezed with squashfs down to 196MB + 2.7MB Kernel. As you have said you were more successful with bottom-up, I wanted to try that approach.

If I am successful with the AMD-Geode, there is an old Fortigate 50B waiting for its alternative Firmware ;-)

BR
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54099
Location: 56N 3W

PostPosted: Thu Dec 21, 2017 11:22 am    Post subject: Reply with quote

joanandk,

A long time ago I wrote a page about unpicking the liveDVD and using the binaries to install Gentoo.
Not many people used it but I was told that it wasn't possible. :)

As a part of that, I needed to discover what the minimum package set was for a chroot.
The original XML is still there but it won't render any longer.
The good news is that the Wayback machine has it.

It prepares a minimal chroot from nothing and chroots into it. That guide unpicks a liveDVD. You can just as well unpick a stage3

When
Code:
Code Listing 5.3: Entering the chroot
works you have a chroot that is a lot smaller than a stage1

The package versions in
Code:
Code Listing 5.2: Unpacking some key tarballs
are from 2009. They may have changed :)
In particular, python-3.x should be enough. Almost nothing needs python-2 any more.

The tar command will need the
Code:
--xattrs
in addition to
Code:
tar -xjpf sys-apps/portage-2.1.6.13.tbz2 -C /mnt/gentoo


Random thought, you don't need portage nor the snapshot inside the chroot. That's properly called the target root.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum