Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[geki-overlay] split build of boost.org libraries
View unanswered posts
View posts from last 24 hours

Goto page 1, 2, 3, 4, 5, 6, 7  Next  
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
geki
Advocate
Advocate


Joined: 13 May 2004
Posts: 2115
Location: Germania

PostPosted: Sun Mar 14, 2010 8:13 am    Post subject: [geki-overlay] split build of boost.org libraries Reply with quote

___
overlay
http://code.google.com/p/gekis-playground/source/checkout

___
howto
http://code.google.com/p/gekis-playground/wiki/Boost

___
hi there!
I have started splitting up the boost build.
I hope the gentoo boost maintainers may see the use of it somewhen. ;)

general improvements - can be achieved in various ways!

[ - move code from ebuild to eclass
[ => single source to maintain
[ => ebuilds ( and if it is just of the one boost package ) just include the minimal necessary descriptions

[ - add something like boost-utils eclass ( yes, the eselect module could do it, too, and ... )
[ => for packages to get specific version(s) - see: 306151 & hackerly ebuild diff

[ - make libraries optional by ...

optional ideas

[ - create tarballs for patches
[ => clean filesdir
[ => uniform way to handle patches in eclass - apply all patches in directory instead of filling src_prepare with 'epatch $patch'

[ - split headers for RDEPEND <-> DEPEND of packages

well, a quick collection.

___
integrated into gentoo packaging thanks to a wrapper ebuild.
Quote:
dev-libs/boost-1.42.0 USE="date_time filesystem signals thread -graph -graph_parallel -headers -iostreams -math -mpi -program_options -python -random -regex -serialization -system -test -wave"
though, you are free to shoot yourself and fix you up with correct useflags set!

not supported is
- unversioned libraries in LIBDIR
- no eselect necessary; either you add include/library path to your projects or use boost-utils eclass.

=> automagic is evil(tm) ...

you may remove boost & eselect-boost and its left-overs located at
Quote:
/usr/include/boost
/usr/lib/libboost_*

( the eselect code from the ebuild is commented out in the eclass. )

now, you may check your system for boost libraries used.
Quote:
# libraries
for f in $(find /usr/lib/ -type f -name '*.so*'); do echo $f; ldd $f | grep boost; done
# binaries
for f in $(find /usr/bin/ -type f -perm -111); do echo $f; ldd $f | grep boost; done
# binaries under LIBDIR
for f in $(find /usr/lib/ -type f -perm -111 ! -name '*.so*'); do echo $f; ldd $f | grep boost; done

___
an example from my overlay.
Quote:
ana portage # emerge -qpv --oneshot net-p2p/lince
[ebuild N ] dev-libs/boost-headers-1.42.0
[ebuild N ] dev-libs/boost-date_time-1.42.0 USE="threads -debug -doc -static-libs -test -tools"
[ebuild N ] dev-libs/boost-signals-1.42.0 USE="threads -debug -doc -static-libs -test -tools"
[ebuild N ] dev-libs/boost-thread-1.42.0 USE="-debug -doc -static-libs -test -tools"
[ebuild N ] dev-libs/boost-system-1.42.0 USE="threads -debug -doc -static-libs -test -tools"
[ebuild N ] dev-libs/boost-filesystem-1.42.0 USE="threads -debug -doc -static-libs -test -tools"
[ebuild N ] dev-libs/boost-1.42.0 USE="date_time filesystem signals thread -graph -graph_parallel -headers -iostreams -math -mpi -program_options -python -random -regex -serialization -system -test -wave"
[ebuild R ] net-p2p/lince-1.2 USE="dbus"

a line of RDEPEND from lince ebuild.
Quote:
dev-libs/boost[date_time,filesystem,signals,thread]

add 'boost-utils' to 'inherit' at the top of the ebuild
add this line to src_configure before configure ; it checks for the respective library path of the boost-headers version!
Quote:
add_boost_paths
truely fixes these bugs: 306151 & hackerly ebuild diff

even runtime is resolved properly!
Quote:
ana portage # ldd /usr/lib64/liblince-1.so.0.0.5|grep boost
libboost_date_time-mt-1_42.so.1.42.0 => /usr/lib/libboost_date_time-mt-1_42.so.1.42.0 (0x00007ff922bbf000)
libboost_signals-mt-1_42.so.1.42.0 => /usr/lib/libboost_signals-mt-1_42.so.1.42.0 (0x00007ff9229ac000)
libboost_system-mt-1_42.so.1.42.0 => /usr/lib/libboost_system-mt-1_42.so.1.42.0 (0x00007ff921c8c000)
libboost_filesystem-mt-1_42.so.1.42.0 => /usr/lib/libboost_filesystem-mt-1_42.so.1.42.0 (0x00007ff921a7b000)
libboost_thread-mt-1_42.so.1.42.0 => /usr/lib/libboost_thread-mt-1_42.so.1.42.0 (0x00007ff92186a000)

voila!

the compile-time of boost libraries for lince. the seven packages take about ~4 minutes.
Quote:
real 4m11.065s
user 3m22.201s
sys 0m43.303s

compared to the full boost distribution.
Quote:
[ebuild N ] dev-libs/boost-1.42.0 USE="eselect -debug -doc -icu -mpi -python -static-libs -test -tools"

and its time.
Quote:
real 9m16.908s
user 12m43.668s
sys 1m8.532s

the source tarball has already been fetched for both.
remember, portage idles now and then at some phases and thrashes cpu-time,
especially at the install phase it takes its time.

___
this has always been on my mind. I just do not need the complete distribution.
it even takes a while to compile on my aging core2duo e6600.

another reason I have is that I am interested in porting java-config to c++ and, yes, with boost.
( I just know my tools, sorry. ;) )

but even I do not want java-config to depend on the complete boost ditribution.
that would surely be silly. still, one useful library is program_options for its 'program option parser' & 'help message generator'.
you just do not write that on your own, do you? 8O

AFAIK date_time, filesystem, program_options & thread are the ones I am using.

some try to stay header-only.
OpenOffice.org was using header-only boost library until recently where they started using boost libraries.

one thing may already be said, there is no increase in compile-time.
there are two or three intra-dependencies within boost libraries from ~15 libraries available.
- they compile that fast it does not hurt.

the list
Quote:
filesystem :: system
graph_parallel :: mpi
wave :: thread, system, filesystem, date_time
so, the only bad one would be the wave library. :roll:

well, what I have been doing so far:
- a straight copy from boost ebuild to boost eclass
- change 'use ...' in PN checks
- filter unpack stage: just unpack necessary things
- added useflag [threads]
- reuse boost-headers package instead of unpack every time

that is roughly it. simple & straight forward.

what I do not test
- boost-mpi based packages

- useflags [debug,doc,static-libs,test,tools]

=> an analytic check yields no errors; paths are separated by PF;

please help testing! thank you. :o

note: gentoo boost maintainers have done a great ebuild. nearly 1:1 copy as eclass for splitted version. thanks to go them!

___
p.s.: I am still tired. time to make a coffee. 8)
p.s.2: yes, I am an emoticon fanatic.
_________________
boost|trans-follow xcb|instruction set analyzer
___
the self obscures the truth. I am no one, and so my words alone come from the truth of this world. amen.


Last edited by geki on Sat Feb 05, 2011 2:03 pm; edited 21 times in total
Back to top
View user's profile Send private message
genstorm
Advocate
Advocate


Joined: 05 Apr 2007
Posts: 2463
Location: Austria

PostPosted: Sun Mar 14, 2010 10:07 pm    Post subject: Reply with quote

How much time did you spend on achieving that? :D

Although I'm impressed, this is too hot for me - no way (or, time ;)) I'm going to figure out which parts of the boost distribution are needed by the various packages that depend on it. Could you please mask it for the not-so-experimental guys? ;)
_________________
backend.cpp:92:2: warning: #warning TODO - this error message is about as useful as a cooling unit in the arctic
Back to top
View user's profile Send private message
Ant P.
Advocate
Advocate


Joined: 18 Apr 2009
Posts: 2333
Location: UK

PostPosted: Sun Mar 14, 2010 10:17 pm    Post subject: Reply with quote

If I've got enough time I'll try breaking my system with this.

I noticed you don't have the icu useflag (possibly others) that's in gentoo's boost ebuild - is that intentional?
Back to top
View user's profile Send private message
geki
Advocate
Advocate


Joined: 13 May 2004
Posts: 2115
Location: Germania

PostPosted: Sun Mar 14, 2010 11:00 pm    Post subject: Reply with quote

genstorm
yesterday 5 hours thrashing by getting into it.
today, too, thrashing about 5 hours; testing eselect; failed; added boost-utils eclass with nice effect.

like I said - simple & straight forward for I am not a boost packager.

something like this to see boost dependencies
Quote:
for f in $(find /usr/lib/ -type f); do echo $f; ldd $f | grep boost; done
for f in $(find /usr/bin -type f -perm -111); do echo $f; ldd $f | grep boost; done

masking, hmm... another day - it is late already.

Ant_P
the useflag [icu] is only interesting for 'dev-libs/boost-regex'.

yes, only some libraries need these useflags.
_________________
boost|trans-follow xcb|instruction set analyzer
___
the self obscures the truth. I am no one, and so my words alone come from the truth of this world. amen.


Last edited by geki on Mon Mar 15, 2010 11:14 am; edited 1 time in total
Back to top
View user's profile Send private message
geki
Advocate
Advocate


Joined: 13 May 2004
Posts: 2115
Location: Germania

PostPosted: Sun Mar 14, 2010 11:14 pm    Post subject: Reply with quote

btw, one should update the overlay, it has just been repackaged with latest fixes!
_________________
boost|trans-follow xcb|instruction set analyzer
___
the self obscures the truth. I am no one, and so my words alone come from the truth of this world. amen.
Back to top
View user's profile Send private message
DaggyStyle
Advocate
Advocate


Joined: 22 Mar 2006
Posts: 4924

PostPosted: Mon Mar 15, 2010 8:55 am    Post subject: Reply with quote

question, what is the pros over the current way?
also, if current state is one block, why not use one build with useflags?
_________________
Only two things are infinite, the universe and human stupidity and I'm not sure about the former - Albert Einstein
ProjectFootball
Back to top
View user's profile Send private message
geki
Advocate
Advocate


Joined: 13 May 2004
Posts: 2115
Location: Germania

PostPosted: Mon Mar 15, 2010 10:51 am    Post subject: Reply with quote

DaggyStyle
I think I have stated why it is useful in the first post already. most of the libraries are not used.

talking about system packages, right now, paludis, and when I start on java-config that one, too, would depend on the full blown boost package.
that is self-explaining: silly for that it uses only a minimal subset. ;)

you may also check a desktop installation for boost libraries used.
Quote:
for f in $(find /usr/lib/ -type f -name '*.so*'); do echo $f; ldd $f | grep boost; done
for f in $(find /usr/bin/ -type f -perm -111); do echo $f; ldd $f | grep boost; done
or something like this.

like I said already, I just want to show its use to split. I do not say that it has to be this way.
so, another way is to add useflags to the gentoo boost package.
though, instead of adding useflags, I could think of another idea.

what I have in '/etc/make.conf':
Quote:
APACHE2_MODULES="actions alias auth_basic ..."
how about that way? :o

but then again, how do you fix the two bugs I have mentioned?

and whenever you need a new boost library you have to merge the boost package, instead of selectively adding the boost library package. hmm ...
_________________
boost|trans-follow xcb|instruction set analyzer
___
the self obscures the truth. I am no one, and so my words alone come from the truth of this world. amen.


Last edited by geki on Mon Mar 15, 2010 11:14 am; edited 1 time in total
Back to top
View user's profile Send private message
DaggyStyle
Advocate
Advocate


Joined: 22 Mar 2006
Posts: 4924

PostPosted: Mon Mar 15, 2010 11:01 am    Post subject: Reply with quote

sorry, you post was too long and I've lost concentration.
in that case,why not use one ebuild with multiple use flags?
_________________
Only two things are infinite, the universe and human stupidity and I'm not sure about the former - Albert Einstein
ProjectFootball
Back to top
View user's profile Send private message
yoshi314
l33t
l33t


Joined: 30 Dec 2004
Posts: 819
Location: PL

PostPosted: Mon Mar 15, 2010 2:47 pm    Post subject: Reply with quote

great idea. i think it's only natural for gentoo to split up big multi-feature packages like that.

i wish we had the same thing for openoffice (it actually distributes its apps in separate tarballs, which suggests that upstream is supporting such builds).
_________________
~amd64, ~x86
shrink your /usr/portage with squashfs+aufs
Back to top
View user's profile Send private message
geki
Advocate
Advocate


Joined: 13 May 2004
Posts: 2115
Location: Germania

PostPosted: Mon Mar 15, 2010 3:46 pm    Post subject: Reply with quote

yoshi314
well, the way I do it is like qt is handled. though, there are other ways to split. each has its pros & cons.

openoffice may be splitted in 5 to 10 years, maybe. first ideas started some years ago.
though, it moves slowly, managing and re-organizing its modules is a pita.
_________________
boost|trans-follow xcb|instruction set analyzer
___
the self obscures the truth. I am no one, and so my words alone come from the truth of this world. amen.
Back to top
View user's profile Send private message
geki
Advocate
Advocate


Joined: 13 May 2004
Posts: 2115
Location: Germania

PostPosted: Tue Mar 16, 2010 8:06 am    Post subject: Reply with quote

I did some minor fixes and cleanup.

if anyone sees the DEBUG message, tell me, otherwise I am going to bin it.
Quote:
# FIXME: this is not triggered!
# Remove (unversioned) symlinks
# And check for what we remove to catch bugs
# got a better idea how to do it? tell me!
for f in $(ls -1 ${LIBRARY_TARGETS} | grep -v "${MAJOR_PV}") ; do
echo "DEBUG: $f"

if [ ! -h "${f}" ] ; then
eerror "Ups, tried to remove '${f}' which is a a real file instead of a symlink"
die "slotting/naming of the libs broken!"
fi

rm "${f}" || die
done

_________________
boost|trans-follow xcb|instruction set analyzer
___
the self obscures the truth. I am no one, and so my words alone come from the truth of this world. amen.
Back to top
View user's profile Send private message
Kingoftherings
Guru
Guru


Joined: 04 May 2008
Posts: 328

PostPosted: Wed Mar 17, 2010 5:48 am    Post subject: Reply with quote

This is a great idea. About 80% of the Boost libraries are headers only anyway, but the remaining parts that need to be built take forever.

Now we just need ebuilds that specify which parts of Boost they need.
Back to top
View user's profile Send private message
SpectreDragon
Tux's lil' helper
Tux's lil' helper


Joined: 22 Aug 2008
Posts: 92
Location: Germany

PostPosted: Wed Mar 17, 2010 2:38 pm    Post subject: Reply with quote

hm I get something like this for apparently every boost ebuild except boost-headers, am I doing something wrong? :?

Code:
>>> Emerging (2 of 9) dev-libs/boost-system-1.42.0 from geki-overlay
 * boost_1_42_0.tar.bz2 RMD160 SHA1 SHA256 size ;-) ... [ ok ]
 * checking ebuild checksums ;-) ... [ ok ]
 * checking auxfile checksums ;-) ... [ ok ]
 * checking miscfile checksums ;-) ... [ ok ]
 * CPV:  dev-libs/boost-system-1.42.0
 * REPO: geki-overlay
 * USE:  amd64 elibc_glibc kernel_linux threads userland_GNU
>>> Unpacking source...
tar xjpf /var/tmp/portage/dev-libs/boost-system-1.42.0/distdir/boost_1_42_0.tar.bz2 --exclude=boost_1_42_0/boost --exclude=boost_1_42_0/libs
tar xjpf /var/tmp/portage/dev-libs/boost-system-1.42.0/distdir/boost_1_42_0.tar.bz2 boost_1_42_0/libs/system
>>> Source unpacked in /var/tmp/portage/dev-libs/boost-system-1.42.0/work
>>> Preparing source in /var/tmp/portage/dev-libs/boost-system-1.42.0/work/boost_1_42_0 ...
 * Applying remove-toolset-1.42.0.patch ... [ ok ]
 * Applying skip-headers.diff ... [ ok ]
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/dev-libs/boost-system-1.42.0/work/boost_1_42_0 ...
 * Writing new user-config.jam
 * found boost version 1_42
 * ERROR: dev-libs/boost-system-1.42.0 failed:
 *   path not found!
 *
 * Call stack:
 *     ebuild.sh, line   48:  Called src_configure
 *   environment, line 5094:  Called boost_src_configure
 *   environment, line  945:  Called add_boost_paths '=142'
 *   environment, line  826:  Called add_boost_library_path '1_42'
 *   environment, line  818:  Called die
 * The specific snippet of code:
 *           die "path not found!";
 *
 * If you need support, post the output of 'emerge --info =dev-libs/boost-system-1.42.0',
 * the complete build log and the output of 'emerge -pqv =dev-libs/boost-system-1.42.0'.
 * This ebuild used the following eclasses from overlays:
 *   /var/lib/layman/openoffice-geki/eclass/boost.eclass
 *   /var/lib/layman/openoffice-geki/eclass/boost-utils.eclass
 * This ebuild is from an overlay named 'geki-overlay': '/var/lib/layman/openoffice-geki/'
 * The complete build log is located at '/var/tmp/portage/dev-libs/boost-system-1.42.0/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/dev-libs/boost-system-1.42.0/temp/environment'.
 * S: '/var/tmp/portage/dev-libs/boost-system-1.42.0/work/boost_1_42_0'
Back to top
View user's profile Send private message
i92guboj
Moderator
Moderator


Joined: 30 Nov 2004
Posts: 9833
Location: Córdoba (Spain)

PostPosted: Wed Mar 17, 2010 2:47 pm    Post subject: Reply with quote

DaggyStyle wrote:
sorry, you post was too long and I've lost concentration.
in that case,why not use one ebuild with multiple use flags?


You can always do that later. Just wrap all the boost-* ebuilds into a boost-meta ebuild and add lots of use flags if that's your call. Not that your are going to gain anything doing that, though. The modular approach has only advantages, just like in kde or qt to name a couple of examples. That way, other ebuilds can depend only on the needed split package, instead of depending upon the whole monster that boost is.
_________________
Gentoo Handbook | My website
Back to top
View user's profile Send private message
geki
Advocate
Advocate


Joined: 13 May 2004
Posts: 2115
Location: Germania

PostPosted: Wed Mar 17, 2010 2:57 pm    Post subject: Reply with quote

SpectreDragon
I was adding include & library path in boost_src_configure. so the first time were failing like that.

thanks for report and fixed! :o

i92guboj
I just want to point out the other way around. have a look at dbus.
it was split-packaged and then merged into one package again.
many ways possible.

but, yes, having the boost headers split for DEPEND over RDEPEND and selective libraries is just sweet. :twisted:
_________________
boost|trans-follow xcb|instruction set analyzer
___
the self obscures the truth. I am no one, and so my words alone come from the truth of this world. amen.


Last edited by geki on Wed Mar 17, 2010 2:59 pm; edited 1 time in total
Back to top
View user's profile Send private message
mv
Advocate
Advocate


Joined: 20 Apr 2005
Posts: 4070

PostPosted: Wed Mar 17, 2010 2:59 pm    Post subject: Reply with quote

i92guboj wrote:
DaggyStyle wrote:
sorry, you post was too long and I've lost concentration.
in that case,why not use one ebuild with multiple use flags?


You can always do that later. Just wrap all the boost-* ebuilds into a boost-meta ebuild and add lots of use flags if that's your call.

This would make much more sense and be easier to maintain. For instance, poppler (and some others which I forgot now) have been changed from split ebuild into an ebuild with useflag.
Quote:
Not that your are going to gain anything doing that, though.

For instance, you gain that an
Code:
emerge -f @world
(e.g. with the trickyfetch script) would be much faster, because there is no need to checksum the same tarball everytime. Also maintenance (revbump) is much easier, since one revbump changes everything and you do not have the problem that you need to check whether different versions are compatible (or even have to block each other which can cause unnecessary headache for the user).
Quote:
The modular approach has only advantages, just like in kde or qt to name a couple of examples.

I can see only disadvantages. It only makes sense if there are reasons to recompile rather often only some of the split ebuilds and not any others. For a mega-project like kde this makes sense (revdep-rebuild), for qt the approach is already doubtfully.
Quote:
That way, other ebuilds can depend only on the needed split package, instead of depending upon the whole monster that boost is.

Which is a completely meaningless argument now that portage has USE-dependencies.
Back to top
View user's profile Send private message
geki
Advocate
Advocate


Joined: 13 May 2004
Posts: 2115
Location: Germania

PostPosted: Wed Mar 17, 2010 3:16 pm    Post subject: Reply with quote

mv wrote:
For instance, you gain that an
Code:
emerge -f @world
(e.g. with the trickyfetch script) would be much faster, because there is no need to checksum the same tarball everytime.
doing multiple checksum and portage idle my split-boost is 1/3 of merge time compared to full boost. what do I miss in your statement?

mv wrote:
Also maintenance (revbump) is much easier, since one revbump changes everything and you do not have the problem that you need to check whether different versions are compatible (or even have to block each other which can cause unnecessary headache for the user)
I still have SLOTs and instead of one ebuild I have one eclass. revbump causes one library to remerge over complete boost. hmm ....

mv wrote:
I can see only disadvantages. It only makes sense if there are reasons to recompile rather often only some of the split ebuilds and not any others. ... for qt the approach is already doubtfully.
to the latter, you must have a high-end cpu. I get sick each time I have to update qt4. I am glad it is split. to the former, yes, I use just some. and the general desktop will. remember, I just have to maintain one eclass instead of one ebuild. no difference in maintenance.
mv wrote:
i92guboj wrote:
That way, other ebuilds can depend only on the needed split package, instead of depending upon the whole monster that boost is.
Which is a completely meaningless argument now that portage has USE-dependencies.
o ha, try that on a not so brand-new hardware. :wink:
just have a look at bjam analysing the headers on install stage. if you got that time, you can do it the other way around, too. :P
_________________
boost|trans-follow xcb|instruction set analyzer
___
the self obscures the truth. I am no one, and so my words alone come from the truth of this world. amen.
Back to top
View user's profile Send private message
SpectreDragon
Tux's lil' helper
Tux's lil' helper


Joined: 22 Aug 2008
Posts: 92
Location: Germany

PostPosted: Wed Mar 17, 2010 3:23 pm    Post subject: Reply with quote

geki wrote:
SpectreDragon
I was adding include & library path in boost_src_configure. so the first time were failing like that.

thanks for report and fixed! :o
...

thanks. next would be.. :)

Code:
...

libs/program_options/src/utf8_codecvt_facet.cpp:15:47: error: ../../detail/utf8_codecvt_facet.cpp: No such file or directory
gcc.compile.c++ bin.v2/libs/program_options/build/gcc-4.4/gentoorelease/pch-off/value_semantic.o

    "x86_64-pc-linux-gnu-g++"  -ftemplate-depth-128 -march=core2 -O2 -pipe -msse3 -mssse3 -fno-strict-aliasing -I/usr/include/boost-1_42 -finline-functions -Wno-inline -Wall -fPIC  -DBOOST_ALL_NO_LIB=1 -DBOOST_PROGRAM_OPTIONS_DYN_LINK=1 -DNDEBUG  -I"." -c -o "bin.v2/libs/program_options/build/gcc-4.4/gentoorelease/pch-off/value_semantic.o" "libs/program_options/src/value_semantic.cpp"

...failed updating 1 target...
 * ERROR: dev-libs/boost-program_options-1.42.0 failed:
 *   building boost failed
 *
 * Call stack:
 *     ebuild.sh, line   48:  Called src_compile
 *   environment, line 5096:  Called boost_src_compile
 *   environment, line  923:  Called die
 * The specific snippet of code:
 *       ${BJAM} ${NUMJOBS} -q -d+2 gentoorelease ${OPTIONS} threading=${threading} ${LINK_OPTS} runtime-link=shared || die "building boost failed";
 *
 * If you need support, post the output of 'emerge --info =dev-libs/boost-program_options-1.42.0',
 * the complete build log and the output of 'emerge -pqv =dev-libs/boost-program_options-1.42.0'.
 * This ebuild used the following eclasses from overlays:
 *   /var/lib/layman/openoffice-geki/eclass/boost.eclass
 *   /var/lib/layman/openoffice-geki/eclass/boost-utils.eclass
 * This ebuild is from an overlay named 'geki-overlay': '/var/lib/layman/openoffice-geki/'
 * The complete build log is located at '/var/tmp/portage/dev-libs/boost-program_options-1.42.0/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/dev-libs/boost-program_options-1.42.0/temp/environment'.
 * S: '/var/tmp/portage/dev-libs/boost-program_options-1.42.0/work/boost_1_42_0'


looks like if there is something missing?
Back to top
View user's profile Send private message
mv
Advocate
Advocate


Joined: 20 Apr 2005
Posts: 4070

PostPosted: Wed Mar 17, 2010 4:24 pm    Post subject: Reply with quote

geki wrote:
mv wrote:
For instance, you gain that an
Code:
emerge -f @world
(e.g. with the trickyfetch script) would be much faster, because there is no need to checksum the same tarball everytime.
doing multiple checksum and portage idle my split-boost is 1/3 of merge time compared to full boost. what do I miss in your statement?

You mean emerging the related split ebuilds is three times faster than emerging one monolithic ebuild (with the corresponding USE-flags set, of course)? In this case, I would say that the build system of the monolithic ebuild is broken.
However, it is more likely that something in your measurement is false (e.g. you have used ccache and did not empty it before emerging the split ebuilds or something similar).
Quote:
I still have SLOTs and instead of one ebuild I have one eclass. revbump causes one library to remerge over complete boost. hmm ....

So you install redundant identical data in different slots. And the eselect of course changes slots of every split ebuild simultaneously so that it makes absolutely no sense to have one of the split ebuild installed in one version but not in another. Once again: What is the advantage of not installing all in one ebuild?
Quote:
mv wrote:
I can see only disadvantages. It only makes sense if there are reasons to recompile rather often only some of the split ebuilds and not any others. ... for qt the approach is already doubtfully.
to the latter, you must have a high-end cpu. I get sick each time I have to update qt4. I am glad it is split.

For upgrading qt, it would need the same (actually even less) time if qt were not split (but of course only those parts corresponding to the USE-flags are compiled). The only case where you see a difference is if you have to reemerge the same version of a split ebuild but not the other installed ones - this usually can only happen during revdep-rebuild, if only one of the split ebuilds has a dependency on some library but the others not. But how often does this really occur?
Quote:
remember, I just have to maintain one eclass instead of one ebuild. no difference in maintenance.

There will be enormous difference if you want to avoid installing redundant data. If you do not want to avoid this, split ebuild makes no sense - USE-flags-ebuild is preferrable.
Quote:
mv wrote:
i92guboj wrote:
That way, other ebuilds can depend only on the needed split package, instead of depending upon the whole monster that boost is.
Which is a completely meaningless argument now that portage has USE-dependencies.
o ha, try that on a not so brand-new hardware. :wink:

What are you talking about? Do you seriously believe that portage needs considerable more CPU-time to track USE-dependencies instead of a split ebuild?
Quote:
just have a look at bjam analysing the headers on install stage. if you got that time, you can do it the other way around, too. :P

I do not understand how this is related with the question of whether split ebuilds or a monolithic ebuild with USE-flags (to determine the parts to be built) is preferrable.
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9011
Location: beyond the rim

PostPosted: Wed Mar 17, 2010 5:36 pm    Post subject: Reply with quote

Quote:
'inherit boost-utils' - add this line to src_configure before configure ; it checks for the minimum version required!

You might want to rephrase that, in the current form it sounds (to me at least) like the inherit call should be added in src_configure, I had to read it three times until I realized you meant the following line. Fortunately I couldn't believe anyone would really call inherit inside a function ;)
Back to top
View user's profile Send private message
geki
Advocate
Advocate


Joined: 13 May 2004
Posts: 2115
Location: Germania

PostPosted: Wed Mar 17, 2010 5:48 pm    Post subject: Reply with quote

mv
well, boost in portage is what is.

and to most of the other things I states my pov already. I know the advantages and disadvantages of various ways. how it is done it does not really matter but for the headers being DEPEND or RDEPEND, then. just thinking about intranet distribution. and yes, we have the bandwidth it does not really matter. so we may turn around and round and ... in the end it is how the maintainers want it to be. ;)

just one final thing.
mv wrote:
There will be enormous difference if you want to avoid installing redundant data. If you do not want to avoid this, split ebuild makes no sense - USE-flags-ebuild is preferrable.
that you should show me. at the moment I avoid more than the other way. ;)

Genone
let's see.
_________________
boost|trans-follow xcb|instruction set analyzer
___
the self obscures the truth. I am no one, and so my words alone come from the truth of this world. amen.
Back to top
View user's profile Send private message
geki
Advocate
Advocate


Joined: 13 May 2004
Posts: 2115
Location: Germania

PostPosted: Wed Mar 17, 2010 6:31 pm    Post subject: Reply with quote

and another round of dependency fixes. :o
_________________
boost|trans-follow xcb|instruction set analyzer
___
the self obscures the truth. I am no one, and so my words alone come from the truth of this world. amen.
Back to top
View user's profile Send private message
mv
Advocate
Advocate


Joined: 20 Apr 2005
Posts: 4070

PostPosted: Wed Mar 17, 2010 6:35 pm    Post subject: Reply with quote

geki wrote:
mv wrote:
There will be enormous difference if you want to avoid installing redundant data. If you do not want to avoid this, split ebuild makes no sense - USE-flags-ebuild is preferrable.
that you should show me.

Let us say you have boost-filesystem:1.41 and boost-filesystem:1.42 but actually the filesystem-related code did not change in the upgrade boost-1.41 ->boost-1.42 (i.e. only other parts of boost were changed). Then you could omit boost-filesystem:1.42 and depend instead on boost-filesystem:1.41: This way, the user would neither have to upgrade to boost-filesystem:1.42 nor would he have to install boost-filesystem:1.42 redundantly if he wants to have both slots. Of course, eselect-boost would need to be smart enough to set correct symlinks.
If ebuilds are really maintained this way, this is a lot of work for the maintainer, but it really saves compilation time for the user: The maintainer has to check for every upgrade which packages really changed. Moreover, it would be sane to change the boost-filesystem:1.41 ebuild (without revbump) for the upgrade to use the tarball of boost-1.42: This way, the user who wants only the most current boost would not have to keep the old boost-1.41 tarball around (or download it again) when he needs to re-emerge boost-filesystem:1.41.
Quote:
at the moment I avoid more than the other way.

I do not see anything which you avoid in the moment - I mean compared to ebuild with USE-flags, of course. Just in case that this was a misunderstanding: I do not argue, of course, that a split ebuild has advantages over a monolithic ebuild without USE-flags. However, a monolithic ebuild with USE-flags has even more advantages.
This is what happened e.g. to poppler: First, it was split, but then the maintainer decided that it is more reasonable to have a monolothic ebuild with USE-flags instead and to have USE-dependencies. I can understand this both, from a user perspective (concerning problems with upgrade blockers and the mentioned repeated checksumming) as well as from the maintainer's perspective (you have to bump all ebuilds, and unless you do the mentioned work with comparing versions for every bump, there is no real point in having split ebuilds). Of course, this is only possible since USE-dependencies are available - otherwise it would be a nightmare for users.
In my opinion there is just a few cases where - in the presence of USE-dependencies - it really makes sense to have split ebuilds: This is mainly if a package has many split ebuilds which in turn need to have separate USE-flags or are such that it is not an exception that a user wants to install or uninstall one of the split ebuilds independently from the others. A collection of programs like KDE is an example where both cases apply. A collection of libraries like boost/qt/poppler does not belong to this class in my opinion, since most users will install these only by means of dependencies, and these are not likely to change frequently.
Back to top
View user's profile Send private message
geki
Advocate
Advocate


Joined: 13 May 2004
Posts: 2115
Location: Germania

PostPosted: Wed Mar 17, 2010 7:07 pm    Post subject: Reply with quote

well, you talk about my implementation. that is just not the point in this thread. what I want to show is what can be improved. full stop.
the implementation is up to the maintainer alone and his/her liking. :)


p.s.: I added some notes to the first post. anything else is uninteresting to discuss here.
_________________
boost|trans-follow xcb|instruction set analyzer
___
the self obscures the truth. I am no one, and so my words alone come from the truth of this world. amen.
Back to top
View user's profile Send private message
SpectreDragon
Tux's lil' helper
Tux's lil' helper


Joined: 22 Aug 2008
Posts: 92
Location: Germany

PostPosted: Thu Mar 18, 2010 6:11 pm    Post subject: Reply with quote

using the splitted boost every ebuild depending on some boost libraries fails to detect it :-/
is there anything I have to do in order to get this fixed?

some examples:

from your openoffice ebuild
Code:
...
checking which boost to use... external
checking boost/shared_ptr.hpp usability... no
checking boost/shared_ptr.hpp presence... no
checking for boost/shared_ptr.hpp... no
configure: error: boost/shared_ptr.hpp not found. install boost
make: *** [stamp/build] Error 1


kde-base/akonadi
Code:
...
-----------------------------------------------------------------------------
-- The following OPTIONAL packages could NOT be located on your system.
-- Consider installing them to enable more features from this software.
-----------------------------------------------------------------------------
   * Boost (1.34.0 or higher)  <http://www.boost.org>
     Boost C++ Libraries
     Required by several critical KDEPIM apps.
   * The Boost Topological_sort header  <http://www.boost.org/>
     topological_sort.hpp (part of Boost_graph)
     Necessary to build kleopatra

-----------------------------------------------------------------------------

-- <<< Gentoo configuration >>>
Build type: Gentoo
Install path: /usr

CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
Boost_INCLUDE_DIR (ADVANCED)
   used as include directory in directory /var/tmp/portage/kde-base/akonadi-4.4.1/work/akonadi-4.4.1/akonadi/kcal
   used as include directory in directory /var/tmp/portage/kde-base/akonadi-4.4.1/work/akonadi-4.4.1/akonadi/akonadi_next
   used as include directory in directory /var/tmp/portage/kde-base/akonadi-4.4.1/work/akonadi-4.4.1/akonadiconsole
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Goto page 1, 2, 3, 4, 5, 6, 7  Next
Page 1 of 7

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum