Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
TIP: Compressing portage using squashfs: initscript method
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6 ... 9, 10, 11  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
Dont Panic
Guru
Guru


Joined: 20 Jun 2007
Posts: 322
Location: SouthEast U.S.A.

PostPosted: Wed Apr 01, 2009 7:54 am    Post subject: Reply with quote

I'm using a kernel from Sabayon.

We're using an aufs patch for 2.6.29 that can be found here:

http://svn.sabayonlinux.org/filedetails.php?repname=Sabayon+Linux+Overlay&path=%2Fsys-kernel%2Flinux-sabayon%2Ffiles%2F2.6.29%2Flinux-sabayon-2.6.29-aufs.patch.bz2

Unfortunately, based on what I'm seeing so far, you don't need to be in a hurry to get a 2.6.29 kernel going.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Wed Apr 01, 2009 11:57 am    Post subject: Reply with quote

Dont Panic wrote:
I've just upgraded to a 2.6.29 kernel
When you use the 2.6.29 kernel you must use >=sys-fs/squashfs-tools-4.0_pre20090324: Only this generates the format which can be used by that kernel. Unfortunately, its format cannot be used with earlier kernel. So upgrading an existing sqfs-file is a bit cumbersome: You have to boot with an old kernel to mount the old sqfs file, then upgrade squashfs-tools and then make sure to re-squash the directory before you boot with the 2.6.29 kernel. An alternative way (if you cannot switch back to an older kernel temporarily) is to install an earlier version of squashfs-tools and use "unsquashfs" to decompress the directory manually to some free space on your harddisk; then upgrade squashfs-tools and compress the directory manually.
Quote:
And I also see the entire set of sys-fs/squashfs-tools ebuilds have been hard-masked (as of the time of this post).
I cannot confirm this. As far as I know, squashfs-tools was never hard-masked, and I did not hear of plans to do this.
Back to top
View user's profile Send private message
Dont Panic
Guru
Guru


Joined: 20 Jun 2007
Posts: 322
Location: SouthEast U.S.A.

PostPosted: Wed Apr 01, 2009 3:32 pm    Post subject: Reply with quote

mv wrote:
Dont Panic wrote:
And I also see the entire set of sys-fs/squashfs-tools ebuilds have been hard-masked (as of the time of this post).
I cannot confirm this. As far as I know, squashfs-tools was never hard-masked, and I did not hear of plans to do this.

Oops, I just realized what I was looking at. Eix was showing all the sys-fs/squashfs-tools in Red, which I interpreted as being hard-masked. But what eix was really telling me was that it couldn't find my /usr/portage/*, which never mounted because of the kernel upgrade. :oops:

Good Times! :D
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Sun Apr 05, 2009 10:39 am    Post subject: Reply with quote

A rather current live ebuild for aufs (together with ebuilds for some other union file systems) can now be found on the same place as the init-script: unionfs.tar.gz

Please be aware that if you use other ebuilds you should remove /sbin/mount.aufs and /sbin/umount.aufs for security reasons (at least, they seem to be subject to insecure temporary file creation). New versions of the init-script pass the options -i to mount/umount to avoid executing these scripts even if they are installed.
Back to top
View user's profile Send private message
R.Aven
n00b
n00b


Joined: 04 Mar 2008
Posts: 51

PostPosted: Sun Apr 05, 2009 5:48 pm    Post subject: Reply with quote

Dont Panic wrote:
I'm using a kernel from Sabayon.

We're using an aufs patch for 2.6.29 that can be found here:

http://svn.sabayonlinux.org/filedetails.php?repname=Sabayon+Linux+Overlay&path=%2Fsys-kernel%2Flinux-sabayon%2Ffiles%2F2.6.29%2Flinux-sabayon-2.6.29-aufs.patch.bz2

Unfortunately, based on what I'm seeing so far, you don't need to be in a hurry to get a 2.6.29 kernel going.

Does anybody have an ebuild which actually includes this patch or could tell me how to apply it to the work-dir of aufs?
I just tried several -p* options, but it won't apply the patch.
Back to top
View user's profile Send private message
Dont Panic
Guru
Guru


Joined: 20 Jun 2007
Posts: 322
Location: SouthEast U.S.A.

PostPosted: Sun Apr 05, 2009 6:09 pm    Post subject: Reply with quote

Have you tried building aufs/unionfs/funionfs as a module using the ebuilds in unionfs.tar.gz mentioned directly above in mv's post?
Back to top
View user's profile Send private message
R.Aven
n00b
n00b


Joined: 04 Mar 2008
Posts: 51

PostPosted: Sun Apr 05, 2009 6:22 pm    Post subject: Reply with quote

Dont Panic wrote:
Have you tried building aufs/unionfs/funionfs as a module using the ebuilds in unionfs.tar.gz mentioned directly above in mv's post?

Yes, of course. But the aufs-999999.ebuild uses the current aufs dev tree, which doesn't support the 2.6.29 kernel branch, afaik not even officially the 2.6.28 version. So it won't compile without the applied patch mentioned above as dentry_open(...) seems to be changed within 2.6.29.

Code:
laptop /var/portage/overlay/personal/sys-fs/aufs $ emerge -av aufs

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U ] sys-fs/aufs-99999999 [20081208-r1] USE="fuse -debug -hinotify -nfsexport -robr -utils% (-sec_perm%)" 0 kB [1=>2]

Total: 1 package (1 upgrade), Size of downloads: 0 kB
Portage tree and overlays:
 [0] /var/portage/repository
 [1] /var/portage/overlay/sunrise
 [2] /var/portage/overlay/personal

Would you like to merge these packages? [Yes/No]

>>> Verifying ebuild manifests

>>> Emerging (1 of 1) sys-fs/aufs-99999999 from gentoo-overlay
 * checking ebuild checksums ;-) ...                                     [ ok ]
 * checking auxfile checksums ;-) ...                                    [ ok ]
 * checking miscfile checksums ;-) ...                                   [ ok ]
 * Determining the location of the kernel source code
 * Found kernel source directory:
 *     /usr/src/linux
 * Found kernel object directory:
 *     /lib/modules/2.6.29/build
 * Found sources for kernel version:
 *     2.6.29
>>> Unpacking source...
 * Fetching CVS module aufs into /var/portage/distfiles/cvs-src ...
 * Running  cvs -q -f -z1 -d ":pserver:anonymous:@aufs.cvs.sourceforge.net:/cvsroot/aufs" login
Logging in to :pserver:anonymous@aufs.cvs.sourceforge.net:2401/cvsroot/aufs
 * Running  cvs -q -f -z1 -d ":pserver:anonymous@aufs.cvs.sourceforge.net:/cvsroot/aufs" update -dP aufs
 * Copying aufs from /var/portage/distfiles/cvs-src ...
 * CVS module aufs is now in /var/tmp/portage/sys-fs/aufs-99999999/work
>>> Source unpacked in /var/tmp/portage/sys-fs/aufs-99999999/work
>>> Preparing source in /var/tmp/portage/sys-fs/aufs-99999999/work/aufs ...
 * patching local.mk
 * disabling sysaufs
 * setting workaround_fuse
 * unsetting debug
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/sys-fs/aufs-99999999/work/aufs ...
>>> Source configured.
>>> Compiling source in /var/tmp/portage/sys-fs/aufs-99999999/work/aufs ...
make -j3 KDIR=/usr/src/linux SUBLEVEL=29 -f local.mk
fs/aufs25
make CONFIG_AUFS=m AUFS_EXTRA_CFLAGS="-I/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/include -DCONFIG_AUFS_BRANCH_MAX_127 -DCONFIG_AUFS_RR_SQUASHFS -DCONFIG_AUFS_BR_XFS -DCONFIG_AUFS_WORKAROUND_FUSE -DCONFIG_AUFS_GETATTR -DCONFIG_AUFS_LOCAL -DCONFIG_AUFS_MODULE -UCONFIG_AUFS -DLKTRHidePrePath=\\\"/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25\\\"" -C /usr/src/linux M=/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25 modules
make CONFIG_AUFS=m AUFS_EXTRA_CFLAGS="-I/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/include -DCONFIG_AUFS_BRANCH_MAX_127 -DCONFIG_AUFS_RR_SQUASHFS -DCONFIG_AUFS_BR_XFS -DCONFIG_AUFS_WORKAROUND_FUSE -DCONFIG_AUFS_GETATTR -DCONFIG_AUFS_LOCAL -DCONFIG_AUFS_MODULE -UCONFIG_AUFS -DLKTRHidePrePath=\\\"/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25\\\"" -C util
make[1]: Entering directory `/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/util'
cc -O2 -march=prescott -O2 -pipe -msse3 -fomit-frame-pointer -O2 -Wall -I/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/include  -DCONFIG_AUFS_BRANCH_MAX_127 -DCONFIG_AUFS_RR_SQUASHFS -DCONFIG_AUFS_BR_XFS -DCONFIG_AUFS_WORKAROUND_FUSE -DCONFIG_AUFS_GETATTR -DCONFIG_AUFS_LOCAL -DCONFIG_AUFS_MODULE -UCONFIG_AUFS -DLKTRHidePrePath=\"/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25\"   -Wl,-O1  c2tmac.c   -o c2tmac
cc -O2 -march=prescott -O2 -pipe -msse3 -fomit-frame-pointer -O2 -Wall -I/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/include  -DCONFIG_AUFS_BRANCH_MAX_127 -DCONFIG_AUFS_RR_SQUASHFS -DCONFIG_AUFS_BR_XFS -DCONFIG_AUFS_WORKAROUND_FUSE -DCONFIG_AUFS_GETATTR -DCONFIG_AUFS_LOCAL -DCONFIG_AUFS_MODULE -UCONFIG_AUFS -DLKTRHidePrePath=\"/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25\"   -Wl,-O1 -s  aulchown.c   -o aulchown
cc -O2 -march=prescott -O2 -pipe -msse3 -fomit-frame-pointer -O2 -Wall -I/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/include  -DCONFIG_AUFS_BRANCH_MAX_127 -DCONFIG_AUFS_RR_SQUASHFS -DCONFIG_AUFS_BR_XFS -DCONFIG_AUFS_WORKAROUND_FUSE -DCONFIG_AUFS_GETATTR -DCONFIG_AUFS_LOCAL -DCONFIG_AUFS_MODULE -UCONFIG_AUFS -DLKTRHidePrePath=\"/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25\"   -Wl,-O1  c2sh.c   -o c2sh
rm -f aufs.5
./c2tmac > aufs.5
awk '{ \
      gsub(/\140[^\047]*\047/, "\\[oq]&\\[cq]"); \
      gsub(/\\\[oq\]\140/, "\\[oq]"); \
      gsub(/\047\\\[cq\]/, "\\[cq]"); \
      gsub(/\047/, "\\[aq]"); \
      print; \
   }' aufs.in.5 >> aufs.5
rm -f etc_default_aufs
sed -e '/Id: /,$d' aufs.shlib > etc_default_aufs
chmod a-w aufs.5
echo '# aufs variables for shell scripts' >> etc_default_aufs
./c2sh >> etc_default_aufs
sed -e '0,/Id: /d' aufs.shlib >> etc_default_aufs
chmod a-w etc_default_aufs
test -x ./mount.aufs || chmod a+x ./mount.aufs
rm c2tmac c2sh
make[1]: Leaving directory `/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/util'
test -e aufs.5 || ln -s util/aufs.5 aufs.5
test -e aufind.sh || ln -s util/aufind.sh aufind.sh
test -e mount.aufs || ln -s util/mount.aufs mount.aufs
test -e auplink || ln -s util/auplink auplink
test -e aulchown || ln -s util/aulchown aulchown
test -e umount.aufs || ln -s util/umount.aufs umount.aufs
test -e etc_default_aufs || ln -s util/etc_default_aufs etc_default_aufs
test -e auchk || ln -s util/auchk auchk
make[1]: Entering directory `/usr/src/linux-2.6.29'
  CC [M]  /var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/module.o
  CC [M]  /var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/super.o
  CC [M]  /var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/sbinfo.o
/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/super.c: In Funktion »au_show_wbr_create«:
/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/super.c:153: Warnung: Format ist kein Zeichenkettenliteral, und keine Formatargumente
  CC [M]  /var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/branch.o
  CC [M]  /var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/xino.o
  CC [M]  /var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/sysaufs.o
/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/xino.c: In Funktion »au_xino_create2«:
/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/xino.c:624: Fehler: Zu wenige Argumente für Funktion »dentry_open«
make[2]: *** [/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/xino.o] Fehler 1
make[2]: *** Warte auf noch nicht beendete Prozesse...
make[1]: *** [_module_/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25] Fehler 2
make[1]: Leaving directory `/usr/src/linux-2.6.29'
make: *** [fs/aufs25/aufs.ko] Fehler 2
 *
 * ERROR: sys-fs/aufs-99999999 failed.
 * Call stack:
 *               ebuild.sh, line   48:  Called src_compile
 *             environment, line 3535:  Called die
 * The specific snippet of code:
 *       emake KDIR=${KV_DIR} SUBLEVEL=${KV_PATCH} -f local.mk || die "emake failed"
 *  The die message:
 *   emake failed
 *
 * If you need support, post the topmost build error, and the call stack if relevant.
 * A complete build log is located at '/var/tmp/portage/sys-fs/aufs-99999999/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/sys-fs/aufs-99999999/temp/environment'.
 * This ebuild is from an overlay named 'gentoo-overlay': '/var/portage/overlay/personal/'
 *

>>> Failed to emerge sys-fs/aufs-99999999, Log file:

>>>  '/var/tmp/portage/sys-fs/aufs-99999999/temp/build.log'

 * Messages for package sys-fs/aufs-99999999:

 *
 * ERROR: sys-fs/aufs-99999999 failed.
 * Call stack:
 *               ebuild.sh, line   48:  Called src_compile
 *             environment, line 3535:  Called die
 * The specific snippet of code:
 *       emake KDIR=${KV_DIR} SUBLEVEL=${KV_PATCH} -f local.mk || die "emake failed"
 *  The die message:
 *   emake failed
 *
 * If you need support, post the topmost build error, and the call stack if relevant.
 * A complete build log is located at '/var/tmp/portage/sys-fs/aufs-99999999/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/sys-fs/aufs-99999999/temp/environment'.
 * This ebuild is from an overlay named 'gentoo-overlay': '/var/portage/overlay/personal/'
 *
 *
 * The following package has failed to build or install:
 *
 *    ('ebuild', '/', 'sys-fs/aufs-99999999', 'merge')
 *

Back to top
View user's profile Send private message
Dont Panic
Guru
Guru


Joined: 20 Jun 2007
Posts: 322
Location: SouthEast U.S.A.

PostPosted: Sun Apr 05, 2009 8:53 pm    Post subject: Reply with quote

The Hitchhiker Sources also have aufs.

If you don't want to use Hitchhiker sources or Sabayon sources, you may want to still download one or the other, and just copy their /fs/aufs/ directory, and overwrite whatever you have for your /fs/aufs/ directory.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Sun Apr 05, 2009 9:02 pm    Post subject: Reply with quote

R.Aven wrote:
But the aufs-999999.ebuild uses the current aufs dev tree, which doesn't support the 2.6.29 kernel branch, afaik not even officially the 2.6.28 version.

I will not try 2.6.29 until a corresponding hardened-sources is out, but hardened-sources-2.6.28 works smoothly. However, shouldn't one better use aufs2-standalone for 2.6.29? So far, I wasn't able to compile it here: It seems that the build system has troubles with different KERNEL_DIR and KBUILD_OUTPUT directories. Did anybody else have success with it?
Back to top
View user's profile Send private message
R.Aven
n00b
n00b


Joined: 04 Mar 2008
Posts: 51

PostPosted: Sun Apr 05, 2009 10:23 pm    Post subject: Reply with quote

mv wrote:
R.Aven wrote:
But the aufs-999999.ebuild uses the current aufs dev tree, which doesn't support the 2.6.29 kernel branch, afaik not even officially the 2.6.28 version.

I will not try 2.6.29 until a corresponding hardened-sources is out, but hardened-sources-2.6.28 works smoothly. However, shouldn't one better use aufs2-standalone for 2.6.29? So far, I wasn't able to compile it here: It seems that the build system has troubles with different KERNEL_DIR and KBUILD_OUTPUT directories. Did anybody else have success with it?

I just downloaded the patch and the corresponding fs and include files from aufs2-standalone-29, applied them manually and now it seems to compile correctly. Thanks for your help.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Mon Apr 06, 2009 5:04 pm    Post subject: Reply with quote

There is now also an experimental ebuild for aufs2 on unionfs.tar.gz.
However, unless I completely misunderstood something, at least the aufs2-28 version (for 2.6.28) is completely broken: In fact, the module is not even compiled, because the symbols are not passed to the sub-makes by the buildsystem. The experimental ebuild patches the Makefile to fix this problem, but even then it dies soon with a compile error (tested against the patched hardened-sources-2.6.28-r7).
R.Aven wrote:
just downloaded the patch and the corresponding fs and include files from aufs2-standalone-29, applied them manually and now it seems to compile correctly. Thanks for your help.

Does this mean you managed to compile aufs2-standalone-29 against the aufs2-patched kernel, or did you compile aufs1 against this kernel? If you did the former: how? Does the aufs2 ebuild work for you? (Is perhaps even the my_makefile_patch in the ebuild unnecessary for aufs2-29?)
Back to top
View user's profile Send private message
R.Aven
n00b
n00b


Joined: 04 Mar 2008
Posts: 51

PostPosted: Mon Apr 06, 2009 7:25 pm    Post subject: Reply with quote

mv wrote:
There is now also an experimental ebuild for aufs2 on unionfs.tar.gz.
However, unless I completely misunderstood something, at least the aufs2-28 version (for 2.6.28) is completely broken: In fact, the module is not even compiled, because the symbols are not passed to the sub-makes by the buildsystem. The experimental ebuild patches the Makefile to fix this problem, but even then it dies soon with a compile error (tested against the patched hardened-sources-2.6.28-r7).
R.Aven wrote:
just downloaded the patch and the corresponding fs and include files from aufs2-standalone-29, applied them manually and now it seems to compile correctly. Thanks for your help.

Does this mean you managed to compile aufs2-standalone-29 against the aufs2-patched kernel, or did you compile aufs1 against this kernel? If you did the former: how? Does the aufs2 ebuild work for you? (Is perhaps even the my_makefile_patch in the ebuild unnecessary for aufs2-29?)

I've done nothing with ebuild.The aufs version from the sunrise-overlay is currently installed, but won't compile on a 2.6.29 kernel. Therefore I used the patch mentioned above and applied them manually to my 2.6.29.1 kernel - which works here on my system. The utils installed with the aufs-2008* ebuild are also working with the aufs module I build using my patched kernel - at least I were able to mount my compressed portage tree with squashfs+aufs.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Mon Apr 06, 2009 8:04 pm    Post subject: Reply with quote

R.Aven wrote:
I've done nothing with ebuild.The aufs version from the sunrise-overlay is currently installed, but won't compile on a 2.6.29 kernel. Therefore I used the patch mentioned above and applied them manually to my 2.6.29.1 kernel - which works here on my system.

But the patch alone (at least that from aufs2-standalone) will not build the actual module for you. My question is how you managed to generate /lib/modules/2.6.29-*/misc/aufs.ko
Quote:
The utils installed with the aufs-2008* ebuild are also working with the aufs module I build using my patched kernel

Did I understand correctly that you built the aufs.ko module still with the aufs-2008* ebuild (only after applying the aufs2-standalone patch to the kernel)? This is somewhat strange, because it means that you mix aufs version: You use the aufs2-kernel patch (meant for the aufs2 module) with the aufs1 module. So it is somewhat just accidental that this works.
Back to top
View user's profile Send private message
R.Aven
n00b
n00b


Joined: 04 Mar 2008
Posts: 51

PostPosted: Mon Apr 06, 2009 8:18 pm    Post subject: Reply with quote

The patch adds aufs functionality to the kernel itself. So I was able to build it - as I would build all kernel modules which are actually part of the official tree - with selecting the corresponding module via "make menuconfig", make and make modules_install. That's all I did. Now I've got the aufs.ko and can modprobe, and use it as before switching to 2.6.29.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Tue Apr 07, 2009 7:33 am    Post subject: Reply with quote

R.Aven wrote:
The patch adds aufs functionality to the kernel itself.

The patch alone does not. I suppose then that besides applying the patch you also copied the other files of aufs2-standalone (except the Makefile) into your kernel tree?
Then there are of course no problems with the broken Makefile (because it is simply not copied). However, the new kernel didn't compile either with hardened-sources-2.6.28-r7 (the reason might be some of the gentoo or hardened patches, I didn't check yet).

But maybe you are right: Since - in contrast to aufs1 - the kernel patch is nontrivial anyway for aufs2, it does not make much sense to require that the user applies this patch manually and then only to install the module by an ebuild. Indeed, the new version of the aufs2 ebuild on the mentioned web page only installs all required files into the kernel source directory and asks the user to apply the patches. Also this solution is not optimal, but there is no really good solution for a module which cannot be compiled standalone but requires in addition nontrivial kernel patches.

Edit: I retried again with hardened-sources-2.6.28-r7, and now aufs2 works as it should. (Due to a mistake in using aufs with my kernel sources, I had probably patched by mistake sources which had already been patched for aufs1).

Conclusion: The new aufs2 ebuild (which is essentially the aufs2-standalone kernel patch with corresponding files in the kernel source directory) works without problems with hardened-sources-2.6.28-r7 and (I suppose) *-sources-2.6.29-*
Back to top
View user's profile Send private message
shgadwa
Guru
Guru


Joined: 12 Mar 2009
Posts: 327

PostPosted: Tue Apr 07, 2009 8:07 pm    Post subject: Reply with quote

I'm also trying to emerge aufs...

I'm using the Zen-Sources kernel-2.6.29-rc8-zen1 kernel. I can configure it for aufs support, and I've done that. But I am not able to emerge aufs (from sunrise overlay) or aufs2 (from ebuild that I downloaded)...

It says that its not going to be installed because of file collisions.

here is the build errors:

Code:
 * This package will overwrite one or more files that may belong to other
 * packages (see list below). You can use a command such as `portageq
 * owners / <filename>` to identify the installed package that owns a
 * file. If portageq reports that only one package owns a file then do
 * NOT file a bug report. A bug report is only useful if it identifies at
 * least two or more packages that are known to install the same file(s).
 * If a collision occurs and you can not explain where the file came from
 * then you should simply ignore the collision since there is not enough
 * information to determine if a real problem exists. Please do NOT file
 * a bug report at http://bugs.gentoo.org unless you report exactly which
 * two packages install the same file(s). Once again, please do NOT file
 * a bug report unless you have completely understood the above message.
 *
 * Detected file collision(s):
 *
 *    /usr/src/linux-2.6.29-rc8-zen1/include/linux/aufs_type.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/branch.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/f_op.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/module.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/opts.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dcsub.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/vfsub.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/fstype.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/inode.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/aufs.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dcsub.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op_ren.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/whout.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/loop.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dentry.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysaufs.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/vfsub.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/file.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dir.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/hinotify.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op_add.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/Kconfig
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/iinfo.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysaufs.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/file.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysrq.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/Makefile
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysfs.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op_del.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/super.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dir.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/wbr_policy.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/spl.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/cpup.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/branch.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/debug.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/inode.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/cpup.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/ioctl.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/rwsem.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/loop.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dinfo.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dentry.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/finfo.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/super.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/xino.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/whout.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/plink.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/module.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/opts.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/magic.mk
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/debug.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sbinfo.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/wkq.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/vdir.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/wkq.c
 *
 * Searching all installed packages for file collisions...
 *
 * Press Ctrl-C to Stop
 *
 * sys-kernel/zen-sources-2.6.29_rc8-r1
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/Kconfig
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/Makefile
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/aufs.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/branch.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/branch.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/cpup.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/cpup.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dcsub.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dcsub.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/debug.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/debug.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dentry.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dentry.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dinfo.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dir.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dir.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/f_op.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/file.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/file.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/finfo.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/fstype.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/hinotify.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op_add.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op_del.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op_ren.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/iinfo.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/inode.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/inode.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/ioctl.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/loop.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/loop.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/magic.mk
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/module.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/module.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/opts.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/opts.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/plink.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/rwsem.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sbinfo.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/spl.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/super.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/super.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysaufs.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysaufs.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysfs.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysrq.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/vdir.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/vfsub.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/vfsub.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/wbr_policy.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/whout.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/whout.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/wkq.c
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/wkq.h
 *    /usr/src/linux-2.6.29-rc8-zen1/fs/aufs/xino.c
 *    /usr/src/linux-2.6.29-rc8-zen1/include/linux/aufs_type.h
 *
 * Package 'sys-fs/aufs2-99999999' NOT merged due to file collisions. If
 * necessary, refer to your elog messages for the whole content of the
 * above message.


EDIT:

I already solved the problem... I had to add COLLISIONS_IGNORE="/usr/src/linux-2.6.29-zen1" to /etc/make.conf

Now it installed, no problems. :)
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Tue Apr 07, 2009 8:30 pm    Post subject: Reply with quote

belikeyeshua wrote:
I'm using the Zen-Sources kernel-2.6.29-rc8-zen1 kernel. I can configure it for aufs support, and I've done that. But I am not able to emerge aufs (from sunrise overlay) or aufs2 (from ebuild that I downloaded)...

If Zen-Sources already contain aufs, then why do you want to emerge the ebuild? The purpose of the ebuild is to give aufs support for kernels which do not contain it.
Back to top
View user's profile Send private message
shgadwa
Guru
Guru


Joined: 12 Mar 2009
Posts: 327

PostPosted: Tue Apr 07, 2009 9:19 pm    Post subject: Reply with quote

Oooh.... good point.

I was wondering about that. So, should I uninstall aufs2?
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Tue Apr 07, 2009 9:27 pm    Post subject: Reply with quote

belikeyeshua wrote:
So, should I uninstall aufs2?

Probably it will do no harm, but I am not sure whether it is exactly the same as that version which is contained in zen-source. (If you use zen-sources anyway, you should better also use the aufs contained there).

So it is probably safer if you uninstall aufs2 (sorry for my false suggestion in the other thread: I was not aware that zen-sources contain aufs).

However, be aware that uninstalling aufs2 will now remove the files you had overridden in zen-sources. To fix this you should re-emerge zen-sources afterwards.
Back to top
View user's profile Send private message
crater2150
n00b
n00b


Joined: 04 Mar 2008
Posts: 22

PostPosted: Mon Apr 27, 2009 10:13 pm    Post subject: Reply with quote

Hi,
I tried to set up the script for the portage tree, but when the script stops, it says:
Code:
File /usr/portage/portage-sqfs.XXXXXX changed size while reading filesystem, attempting to re-read
, and repeats this, until it is killed manually.
My configuration:
Code:

DIRECTORY="/usr/portage"
DIR_CHANGE="/tmp/portage.XXXXXX"
DIR_SQUASH="/tmp/portage.sqfs"
TMP_SQFS="/tmp/portage-sqfs.XXXXXX"
FILE_SQFS="/var/portage/portage-current.sqfs"
FILE_SQFS_OLD="/var/portage/portage-old.sqfs"
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Tue Apr 28, 2009 7:47 am    Post subject: Reply with quote

crater2150 wrote:
Code:
File /usr/portage/portage-sqfs.XXXXXX changed size while reading filesystem, attempting to re-read

This filename is certainly not invented by the script, but nothing in the configuration you posted contains it. I suspect that you have a symbolic link pointing to that file (e.g. from one of the files in your configuration): Of course none of your files in the configuration should be inside the directory you want to squash (also not by symbolic links).

Another note: I just observed that if one uses the temporary name feature on DIR_CHANGE, the corresponding directory cannot be erased on stop. Probably some rewrite of the script is necessary to support the temporary DIR_CHANGE feature properly.

Edit: Important security note: Do not use fixed names in world-writable directories! E.g. your "/tmp/portage.sqfs" is a very bad idea! Such things can be subject to symlink-attacks!
Back to top
View user's profile Send private message
crater2150
n00b
n00b


Joined: 04 Mar 2008
Posts: 22

PostPosted: Tue Apr 28, 2009 8:37 pm    Post subject: Reply with quote

there weren't any symlinks to /usr/portage/portage-sqfs.XXXXXX, but removing the temporary name part from DIR_CHANGE solved the problem, thx
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Wed Apr 29, 2009 5:55 pm    Post subject: Reply with quote

There is now a new version of the script which should solve the problem with the temporary name feature in DIR_CHANGE. Morever, it brings also back the temporary name feature for DIR_SQUASH (in a non-hackish manner).
Back to top
View user's profile Send private message
ocbMaurice
Tux's lil' helper
Tux's lil' helper


Joined: 14 Feb 2003
Posts: 90
Location: Switzerland

PostPosted: Mon May 11, 2009 10:36 pm    Post subject: Reply with quote

[url]Hi here,

I came across this problem/solution because I own an EEE 701 with 4GB SSD (and gentoo) :-)
I guess mv is the developer of the script at uni-wuerzburg?

Somehow I didn't really find this script before I wrote my own.
I think your script is more mature than mine, but is missing one or two features I implemented.
You'll find them in my response in the above linked thread.
It's mostly about the exclusion of ie. distfiles from /usr/portage.
The way I do it, you don't have to reconfig your distfiles dir ...

I didn't really yet test you script, but IMO it's lacking this feature ;)
Additionaly implemented some functions for the init script (ie. init, sync, info, todo: uninit)

Hope you like my additional ideas and will incorporate them into your script.
This should also go in the portage tree, once it's mature enough.

Sorry for crossposting, IMO the two threads should be merged!
Maurice
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Thu May 14, 2009 5:18 pm    Post subject: Reply with quote

The new version of the init-script supports a THRESHOLD and a new magic file IGNORE_THRESHOLD which can be used to override the default and thus force compression even if the THRESHOLD is not reached. The other suggestions (like init function or exclusion of directories by mount --bind) are not implemented due to security considerations: Such a delicate task as compressing and removing a whole directory tree should be done manually and manually be checked twice, since there are too many setups where this can cause problems (e.g. when root has not all permissions like in SELinux or a virtual box, in --make-shared/--make-slaved directories, automatic submounts, problems with symbolic or non-symbolic links in/out the compressed tree etc.). Without an init function, there is no proper way to implement the directory-exclusion-feature with mount --bind from within the script. As mentioned in the other thread, I would suggest a separate init-file like
Code:
#!/sbin/runscript
depend () { need squash_....; }
start () { mount --bind ...; }
stop () { umount ...; }
if you really need this. However, I cannot think of cases where this makes sense and where you cannot just use a symbolic link instead. For PORTDIR, it is certainly safer to put DISTDIR outside the PORTDIR tree. I would suggest the latter anyway for security reasons (independently whether some script is used) - this is meanwhile even suggested in make.conf.example. Even layman already has changed the default (although here it might make sense to include it in the compressed PORTDIR directory).
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6 ... 9, 10, 11  Next
Page 5 of 11

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum