Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
raspberry pi, distcc, and cc1
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
njcwotx
Guru
Guru


Joined: 25 Feb 2005
Posts: 587
Location: Texas

PostPosted: Fri Oct 26, 2012 4:50 pm    Post subject: raspberry pi, distcc, and cc1 Reply with quote

I have setup distcc on my RaspberryPi, and it is working. But there is a build process called cc1 that still gets run on the Pi locally. Is this something that can be offloaded through distcc by adding a symlink or something.

Also, Is it possible to build for this architecture within a chroot on an x86 box then copy the files onto the SD card?
_________________
Drinking from the fountain of knowldege.
Sometimes sipping.
Sometimes gulping.
Always thirsting.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54030
Location: 56N 3W

PostPosted: Fri Oct 26, 2012 5:18 pm    Post subject: Reply with quote

njcwotx,

cc1 is gcc. Some things cannot be distrubuted - they are known to break with distcc, so the ebuilds won't use gistcc.
Other things are run locally anyway.

What command are you using to emerge things on the Pi?


Yes you can cross compile on x86 for Pi. Some things won't build as they are not cross compile friendly.
Python and Perl are by far the worst. If you want to cross compile and link against things that won't build in the cross environment, you need to get the binary package off the Pi and install it in the cross environment.
Be careful to get that right. Your i686 install won't like an ARM binary being installed.

Set FEATURES=buildpkg on both the Pi and in the cross environment. This saves binary tarballs of all of the packages you build
Use quickpkg to build tarballs of already installed packages. Mske sure the USE flags are the same in both places.
Move the tarballs around - they are like single package stage3 files.

On the Pi use emerge -K <package>
In the cross environment use
emerge-wrapper --init
/armv6j-hardfloat-linux-gnueabi-emerge -K <package>
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
njcwotx
Guru
Guru


Joined: 25 Feb 2005
Posts: 587
Location: Texas

PostPosted: Fri Oct 26, 2012 7:00 pm    Post subject: Reply with quote

I added some USE flags and rebuilt system before I began emerging xorg-server and fluxbox. The gtk flag triggered a rebuild of gcc of all things. Took 5 hours to rebuild glibc yesterday. I suppose Ill just let it run. Once I get the base install with X and a window manger. I can stage it an not have to mess with it.

I did take an SSD and mounted distfiles/ var/tmp/portage there so I could lift some activity off the SD card. I still have the defaults set for frequencies.

So, how long did it take for your Pi to build?
_________________
Drinking from the fountain of knowldege.
Sometimes sipping.
Sometimes gulping.
Always thirsting.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54030
Location: 56N 3W

PostPosted: Fri Oct 26, 2012 7:25 pm    Post subject: Reply with quote

njcwotx,

About four days. Mine is a server, so it has little or no Xorg.

I also mount root over nfs to avoid SD card wear and tear.
In service, swap, /home and /boot are on the SD card.
Swap is only provided for emergencies - its better then the OOM kicking in.

For building I used a USB 'spinny' HDD and cross distcc. Only /boot was on the SD card.
I also left gcc in a mess with regards to USE gtk.

I also use ~arm
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
njcwotx
Guru
Guru


Joined: 25 Feb 2005
Posts: 587
Location: Texas

PostPosted: Sat Oct 27, 2012 3:27 am    Post subject: Reply with quote

GCC finally finished, took about 12 hours. The next few pkgs build reasonably quick. Will see how far along I am in 12 more.
_________________
Drinking from the fountain of knowldege.
Sometimes sipping.
Sometimes gulping.
Always thirsting.
Back to top
View user's profile Send private message
njcwotx
Guru
Guru


Joined: 25 Feb 2005
Posts: 587
Location: Texas

PostPosted: Sat Oct 27, 2012 7:41 am    Post subject: Reply with quote

Yeah. About 5 more hours and the rest completed. now I am starting to build Xorg-server and fluxbox.
_________________
Drinking from the fountain of knowldege.
Sometimes sipping.
Sometimes gulping.
Always thirsting.
Back to top
View user's profile Send private message
njcwotx
Guru
Guru


Joined: 25 Feb 2005
Posts: 587
Location: Texas

PostPosted: Sat Oct 27, 2012 3:19 pm    Post subject: Reply with quote

xorg-server took 4 hours and fluxbox took less than 2. Hopefully, with just a little bit of configuration I can get flux to start up and then Ill stage4 the SD card and use that as a base.

did a pretend on firefox(v16) - 250M of packages. Not sure I wish to tackle that one yet. Ill have to do a little research into finding a smaller footprint browser to build.

Firefox may not be the best on this little guy. I tried the fedora RPi and firefox was very slow and practically unsuable.

epiphany -230M, Opera 140M, xxxterm-193M, conkeror - 242M, cromium 496M! = cant really go by download size, but I bet they do roughly correlate to a build time. Probably much better looking though :)

dillow came back as 5M, never used it but its an option to try.

I know 'links -g' is a simple option for basic tasks and its really small.

Not trying to build a modern desktop. However, one of the first experiments will be to replace the PC's we use to mount in ceiling that just open up a browser and display Nagios on the TV's on the wall in the office. Would be nice to have a fast functional browser, but reasonably fast is key to being practical here.
_________________
Drinking from the fountain of knowldege.
Sometimes sipping.
Sometimes gulping.
Always thirsting.
Back to top
View user's profile Send private message
john.newman
Tux's lil' helper
Tux's lil' helper


Joined: 17 Oct 2009
Posts: 85

PostPosted: Sat Dec 01, 2012 5:59 am    Post subject: Reply with quote

hi there

I finally got one of these (that's all I could get... :evil: ) and am basically doing the exact same thing as you describe.

Distcc is working, but really it's still quite slow during all of the other phases of the build. The grunt work of gcc is really all I've offloaded - is there a way to have my x86 desktop run the entire build in favor of the pi, and just ship it a package that will work on the pi, and for that pi's particular install and configuration? The fetch/extract/configure/link/etc phases.. All of this takes very long as that's where the SD card is used.

If not, can anything be done about /var/tmp/portage? On the desktop that is a ramdisk, and 512 is not going to be enough for that. I could use a USB jump drive or hard drive, but I think the speeds would be about the same. I've moved /usr/portage to NFS, that saves a lot of time, so would NFS for tmp dirs be faster than the SD card? I doubt it ... but maybe :?:

Also I will try to get a faster card, the only one I had on hand is probably not very good. sudo and it's 3 deps just took almost 30 minutes. (which reminds me I skipped the linpack test & need to do that.)

Has anyone bothered to look at the debian install they provide and found any good bits to borrow off of it? Gentoo + fluxbox is on every machine I have, I can't use some other distro on this, but that's where they've been adding the polish so we might be missing out on something.

Also I am going to be using Synergy to control the pi on screen through the keyboard of some other laptop, seems like a great setup over it's own usb keyboard. :idea:

How about the extra pins ... anyone here using them for anything good?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54030
Location: 56N 3W

PostPosted: Sat Dec 01, 2012 2:42 pm    Post subject: Reply with quote

john.newman,

You must have run crossdev to get your cross compiler for distcc to work.
That also provides you with a otherwise empty ARM toot filesystem on your helper system(s). Mine is at /usr/armv6j-hardfloat-linux-gnueabi/
You can run a cross emerge there. Read the Gentoo embedded handbook.

There are a few things that don't work this way, so you can't do the equivelent of emerge system through the cross environment but you can emerge -K packages there once they have been built on the Pi with the aid of disctcc.
Set up the cross make.conf and the Pi make.conf with the same USE flags. Make sure both have FEATURES="buildpkg" set, so you save binary packages at build time.

You can freely move these binary packages around but you may want to use rsync. Thats a few pointers.

Almost anything is faster than an SD card. On my Pi webserver, /var/tmp/portage is mounted over NFS. On my Pi build system, its on a USB 'spinny'.

Are you using distcc in pump mode, e.g.
Code:
pump emerge world
this allows more then jsu the building to be off loaded.
What do you have MAKEOPS= on the Pi?
I use -j8 to keep my helper busy.

Luxury ... a 512Mb Pi ... I have one on order.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
john.newman
Tux's lil' helper
Tux's lil' helper


Joined: 17 Oct 2009
Posts: 85

PostPosted: Sat Dec 01, 2012 7:41 pm    Post subject: Reply with quote

thanks for all the tips. Yeah, I ordered this thing way back in June and it just arrived the other day. Because it took so long they threw in the 512 upgrade out of good will. Very nice of them, I didn't mind the wait so much, I just need a few more than one unit.

I have -j5 on the pi, the helper box is an i7. I have never seen the "pump" command before, I will look into it. I'll also try moving /var/tmp/portage to nfs, typically the network is the slowest thing a machine can use for IO but in this case you're probably right - anything is faster than those cards..

I've done distcc cross compiling before so that was no sweat really, crossdev is working very well. I'll check into build pkg and the embedded book, that should probably cover it. Thanks much.\
Back to top
View user's profile Send private message
john.newman
Tux's lil' helper
Tux's lil' helper


Joined: 17 Oct 2009
Posts: 85

PostPosted: Sun Dec 02, 2012 8:00 am    Post subject: Reply with quote

I tried using the chroot steps in the embedded guide, but I'm not sure it's working. When I run the two chroot w/ busybox & bash commands, they complete successfully but it doesn't actually "chroot" into the dir as chroot normally does. E.g. / still points to my real / and not the chroot dir. Is it supposed to do that? If so that seems dangerous, I don't want arm code accidentally going to the real /bin (which would eventually happen with me at the controls :twisted: ) And anyway I'm not sure if I like this approach as it would just be for emerge -K, which.. for some reason I don't care for with gentoo. Just the wording "bin package" puts me off, perhaps I should try to adjust my beliefs.

So, what do you think about using an arm VM through qemu-user? On the VM you'd just build the entire pi file system out and transfer it to the card (minus /boot and any other pi specifics). rsync could be used for updates. I think this could be a good fit for how I'd like to do it, but I'm not sure how the performance of qemu-user arm running on a powerful i7 rig would compare to the just using the pi itself (with as much of the i7 as I can with pump distcc etc). Running emerge through qemu-kvm is great on this machine, but that's still native code right on the cpu, arm qemu-user is totally different story. Has anyone done it and is the performance any good to make it worth me trying?


xorg-server on the pi is _finally almost finished ... patience patience
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54030
Location: 56N 3W

PostPosted: Sun Dec 02, 2012 11:41 am    Post subject: Reply with quote

john.newman,

I'm not sure what you are chrooting into but it doesn't sound right.
The idea of the cross build system in a chroot is to stop exactly the sort of accident you describe, putting ARM code into your host filesystem.

You need to understand Gentoo binary packages. Provided you use the same version of gcc with the same USE and CFLAGS settings in the cross build environment, you get the same code as you would have done building on the Pi ... but much faster, so when you move the binary over to the target and run emerge -K, you have saved some time. If the USE flags in the binary package are not the same as the target, emerge -K will complain.

There is one difference - the cross build environment builds code with a different name for ld, so the target needs an extra symlink to make that code work.

With the need to move binary packages backwards and forwards both ways, I've settled on using cross distcc in pump mode for system maintainence, since the urgency to get the Pi goings has worn off.
As I have two 256Mb Pis and a 512Mb one on order, I do save binary packages and share them among the Pis.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
john.newman
Tux's lil' helper
Tux's lil' helper


Joined: 17 Oct 2009
Posts: 85

PostPosted: Tue Dec 04, 2012 5:28 am    Post subject: Reply with quote

well i switched from one of the cheap micro 4gb cards to one of these http://www.newegg.com/Product/Product.aspx?Item=N82E16820220721

and the speed difference is HUGe. My first performance tip would be to get a good fast SD card first. It seems that this card is faster than NFS, while NFS was faster than the micro card. I should measure...

Unfortunately I had gcc building overnight (not sure why portage selected it as [R] for -uDN world), but it failed at some point. So again today .. same thing.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54030
Location: 56N 3W

PostPosted: Tue Dec 04, 2012 1:19 pm    Post subject: Reply with quote

john.newman,

Thats a class 10 SD card, so its faster.

gcc is being rebuild because of a flag change in the profile, its being caught by the -N option.
Don't worry about the rebuild. If its crashed
Code:
pump emerge --resume --skipfirst
will skip it and continue.
Or use the --keep-going option and don't wait for a build to crash. Portage will pick up the pieces.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
_______0
Guru
Guru


Joined: 15 Oct 2012
Posts: 521

PostPosted: Sat Dec 22, 2012 9:50 am    Post subject: Reply with quote

is portage prefix ready for prime time? I also want to cross-compile for slower systems but it's way to complicated.

what's the simples way to do this?

powerful machine compile for -----> different arch slower machine

one example:

8 core processor -> arduino

thnx
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54030
Location: 56N 3W

PostPosted: Sat Dec 22, 2012 7:11 pm    Post subject: Reply with quote

________________________0,

Please choose a shorter nick. If you email me I will set it for you.

Prefix is mostly used for installing on systems where you don't have root.
It allows you to install into your home, for example. Its not needed for cross compiling.

Code:
emerge crossdev
is a good place to start, then use crossdev to build the cross toolchain of your choice.
If you are nervous about installing arduino code on the host, which would be a very bad thing, you might want to make a chroot to contain crossdev.

The embedded handbook is worth reqding.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
KdC
n00b
n00b


Joined: 14 Jul 2003
Posts: 13
Location: Germany

PostPosted: Sun Dec 23, 2012 1:52 pm    Post subject: Reply with quote

@john.newman:
I ran into the same error. Did you copy qemu-static-arm and qemu-static-arm-binfmt into your chroot-environment?
If yes then try qemu-user-9999, it worked better for me (after editing /etc/init.d/qemu-binfmt).
The cool thing about this is that you can speedup everything if you configure your chroot-environment to use distcc.
This setup is working very well for me but some java related packages need to be compiled on the pi. I'm trying to get icedtea installed for weeks now and I've got the feeling that i got stuck in java-hell ;).
Back to top
View user's profile Send private message
JanR
Tux's lil' helper
Tux's lil' helper


Joined: 21 Jan 2007
Posts: 78

PostPosted: Sun Dec 23, 2012 4:29 pm    Post subject: Reply with quote

Hi,

Quote:

Has anyone done it and is the performance any good to make it worth me trying?


I actually did that and set up a complete ARM system in qemu-system-arm that I (among other things) also use for compiling packages (using -b while emerging) for small real ARM systems. With respect to the Raspberry Pi, it is not really faster. The major advantage is that you can have significantly more RAM enabling building on tmpfs. In order to do that, use the "highbank" machine (needs SVN qemu, at least a few month ago when I created that setup) that simulates a Calxeda Energy Core with 4 GB RAM (all the other simulated ARM machines allow significantly less).

Some examples on my RPI:

Code:

 * sys-libs/glibc-2.15-r3

        Emerged at: Sun Dec  2 10:46:04 2012
        Build time: 4 hours, 28 minutes, and 30 seconds

 * sys-devel/gcc-4.5.4

        Emerged at: Mon Dec  3 00:02:54 2012
        Build time: 14 hours, 49 minutes, and 53 seconds

 * app-portage/eix-0.25.5

        Emerged at: Sun Dec  2 05:40:17 2012
        Build time: 40 minutes, and 39 seconds


On the simulated Cortex A9 running at my Xeon X3470 (2.93 GHz Nehalem, Turbo up to 3.6 GHz):

Code:

 * sys-libs/glibc-2.15-r3

        Emerged at: Mon Dec 17 23:34:52 2012
        Build time: 6 hours, 5 minutes, and 55 seconds

 * sys-devel/gcc-4.6.3

        Emerged at: Wed Dec 19 05:05:17 2012
        Build time: 1 day, 14 minutes, and 34 seconds

 * app-portage/eix-0.25.5

        Emerged at: Sat Sep 22 15:31:42 2012
        Build time: 17 minutes, and 33 seconds


Note the different gcc versions. Furthermore, the run with more than one day was on a heavily loaded machine, so usually it is faster. Overall, I would say that C code compiles faster on the RPi while C++ code is faster on Qemu executed on a modern desktop machine. My Xeon is Nehalem, so Sandy or Ivy bridge would give you around 15-30% more, depending on clock speed.

Btw., overclocking the RPi (especially SDRAM and Core) gives you another boost: I tested this with a kernel compilation on the RPi. Using the default 700 MHz, it requires 244 minutes (class 10 SD card). Running the RPi at 920 MHz (RAM and Core at 460, this is possible for my board in a stable manner without increasing voltage), it only needs 175 min.

Greetings,

Jan
Back to top
View user's profile Send private message
GENKILL
n00b
n00b


Joined: 09 Dec 2012
Posts: 7

PostPosted: Sat Jan 26, 2013 10:36 pm    Post subject: Reply with quote

Hello Rpi friends

I'm trying to do the best configuration for my rpidev environment but not everything is working as i'd like to. I've set up distcc with crossdev and everything is working much faster but not gcc-4.6.3 update - only localhost (rpi) is used for this and I don't know why. Strange thing is that at the begining of gcc compilation there are some cc1 processes on my Core I5 machine but now I'm waiting 5th hour and maybe I should go sleep :)
Btw, I've started my Rpi-gentoo optimisations with fstab :)
/dev/mmcblk0p1 /boot auto noauto,noatime 1 2
/dev/mmcblk0p2 / ext4 defaults 1 1
/dev/mmcblk0p3 /usr/portage reiserfs defaults 1 1
/dev/mmcblk0p4 /var ext2 defaults 1 1
#tmpfs /var/tmp/portage tmpfs size=128M 0 0
/dev/sda1 /var/tmp/portage reiserfs defaults 1 1
tmpfs /tmp tmpfs size=32M 0 0
tmpfs /var/log tmpfs size=64M 0 0

How can i make my Rpi env faster? Someone said about tbz2 on crossdev machine and installing this package on Rpi?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54030
Location: 56N 3W

PostPosted: Sat Jan 26, 2013 11:33 pm    Post subject: Reply with quote

GENKILL,

I've given up using the SD card for anything except boot and swap, if everything is NFS mounted.
Everything else is either a USB HDD or mounted over NFS.

I'm having build problems with gcc now too. The poor wee thing has 108Mb in swap, so I guess -j8 was OK until distcc pump mode fell over, now its just bogged down swapping.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
GENKILL
n00b
n00b


Joined: 09 Dec 2012
Posts: 7

PostPosted: Sun Jan 27, 2013 9:25 am    Post subject: Reply with quote

heh, I woke up and hmm still compiling :) In previous post i gave my fstab in which You can see I'm using external drive for /var/tmp/portage (compiling gcc needs nearly 1GB!) but NeddySeagoon You are right, i'll try to see if whole root partition will do much faster, just for my dev env :)
Back to top
View user's profile Send private message
njcwotx
Guru
Guru


Joined: 25 Feb 2005
Posts: 587
Location: Texas

PostPosted: Sat May 18, 2013 1:08 am    Post subject: Reply with quote

I did some benchmarks and any external storage is no faster than the SD card. I gave up on flash and external ssd and just go with the SD card.
_________________
Drinking from the fountain of knowldege.
Sometimes sipping.
Sometimes gulping.
Always thirsting.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum