TIP: Compressing portage using squashfs: initscript method

Message

Dont Panic · Post by **Dont Panic** » Wed Apr 01, 2009 7:54 am

I'm using a kernel from Sabayon.

We're using an aufs patch for 2.6.29 that can be found here:

http://svn.sabayonlinux.org/filedetails ... .patch.bz2

Unfortunately, based on what I'm seeing so far, you don't need to be in a hurry to get a 2.6.29 kernel going.

mv · Post by mv » Wed Apr 01, 2009 11:57 am

Dont Panic wrote:I've just upgraded to a 2.6.29 kernel

When you use the 2.6.29 kernel you must use >=sys-fs/squashfs-tools-4.0_pre20090324: Only this generates the format which can be used by that kernel. Unfortunately, its format cannot be used with earlier kernel. So upgrading an existing sqfs-file is a bit cumbersome: You have to boot with an old kernel to mount the old sqfs file, then upgrade squashfs-tools and then make sure to re-squash the directory before you boot with the 2.6.29 kernel. An alternative way (if you cannot switch back to an older kernel temporarily) is to install an earlier version of squashfs-tools and use "unsquashfs" to decompress the directory manually to some free space on your harddisk; then upgrade squashfs-tools and compress the directory manually.

And I also see the entire set of sys-fs/squashfs-tools ebuilds have been hard-masked (as of the time of this post).

I cannot confirm this. As far as I know, squashfs-tools was never hard-masked, and I did not hear of plans to do this.

Dont Panic · Post by **Dont Panic** » Wed Apr 01, 2009 3:32 pm

mv wrote:
Dont Panic wrote:And I also see the entire set of sys-fs/squashfs-tools ebuilds have been hard-masked (as of the time of this post).
I cannot confirm this. As far as I know, squashfs-tools was never hard-masked, and I did not hear of plans to do this.

Oops, I just realized what I was looking at. Eix was showing all the sys-fs/squashfs-tools in Red, which I interpreted as being hard-masked. But what eix was really telling me was that it couldn't find my /usr/portage/*, which never mounted because of the kernel upgrade.

Good Times!

mv · Post by mv » Sun Apr 05, 2009 10:39 am

A rather current live ebuild for aufs (together with ebuilds for some other union file systems) can now be found on the same place as the init-script: unionfs.tar.gz

Please be aware that if you use other ebuilds you should remove /sbin/mount.aufs and /sbin/umount.aufs for security reasons (at least, they seem to be subject to insecure temporary file creation). New versions of the init-script pass the options -i to mount/umount to avoid executing these scripts even if they are installed.

R.Aven · Post by **R.Aven** » Sun Apr 05, 2009 5:48 pm

Dont Panic wrote:I'm using a kernel from Sabayon.

We're using an aufs patch for 2.6.29 that can be found here:

http://svn.sabayonlinux.org/filedetails ... .patch.bz2

Unfortunately, based on what I'm seeing so far, you don't need to be in a hurry to get a 2.6.29 kernel going.

Does anybody have an ebuild which actually includes this patch or could tell me how to apply it to the work-dir of aufs?
I just tried several -p* options, but it won't apply the patch.

Dont Panic · Post by **Dont Panic** » Sun Apr 05, 2009 6:09 pm

Have you tried building aufs/unionfs/funionfs as a module using the ebuilds in unionfs.tar.gz mentioned directly above in mv's post?

R.Aven · Post by **R.Aven** » Sun Apr 05, 2009 6:22 pm

Dont Panic wrote:Have you tried building aufs/unionfs/funionfs as a module using the ebuilds in unionfs.tar.gz mentioned directly above in mv's post?

Yes, of course. But the aufs-999999.ebuild uses the current aufs dev tree, which doesn't support the 2.6.29 kernel branch, afaik not even officially the 2.6.28 version. So it won't compile without the applied patch mentioned above as dentry_open(...) seems to be changed within 2.6.29.

Code: Select all

laptop /var/portage/overlay/personal/sys-fs/aufs $ emerge -av aufs

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U ] sys-fs/aufs-99999999 [20081208-r1] USE="fuse -debug -hinotify -nfsexport -robr -utils% (-sec_perm%)" 0 kB [1=>2]

Total: 1 package (1 upgrade), Size of downloads: 0 kB
Portage tree and overlays:
 [0] /var/portage/repository
 [1] /var/portage/overlay/sunrise
 [2] /var/portage/overlay/personal

Would you like to merge these packages? [Yes/No] 

>>> Verifying ebuild manifests

>>> Emerging (1 of 1) sys-fs/aufs-99999999 from gentoo-overlay
 * checking ebuild checksums ;-) ...                                     [ ok ]
 * checking auxfile checksums ;-) ...                                    [ ok ]
 * checking miscfile checksums ;-) ...                                   [ ok ]
 * Determining the location of the kernel source code
 * Found kernel source directory:
 *     /usr/src/linux
 * Found kernel object directory:
 *     /lib/modules/2.6.29/build
 * Found sources for kernel version:
 *     2.6.29
>>> Unpacking source...
 * Fetching CVS module aufs into /var/portage/distfiles/cvs-src ...
 * Running  cvs -q -f -z1 -d ":pserver:anonymous:@aufs.cvs.sourceforge.net:/cvsroot/aufs" login
Logging in to :pserver:anonymous@aufs.cvs.sourceforge.net:2401/cvsroot/aufs
 * Running  cvs -q -f -z1 -d ":pserver:anonymous@aufs.cvs.sourceforge.net:/cvsroot/aufs" update -dP aufs
 * Copying aufs from /var/portage/distfiles/cvs-src ...
 * CVS module aufs is now in /var/tmp/portage/sys-fs/aufs-99999999/work
>>> Source unpacked in /var/tmp/portage/sys-fs/aufs-99999999/work
>>> Preparing source in /var/tmp/portage/sys-fs/aufs-99999999/work/aufs ...
 * patching local.mk
 * disabling sysaufs
 * setting workaround_fuse
 * unsetting debug
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/sys-fs/aufs-99999999/work/aufs ...
>>> Source configured.
>>> Compiling source in /var/tmp/portage/sys-fs/aufs-99999999/work/aufs ...
make -j3 KDIR=/usr/src/linux SUBLEVEL=29 -f local.mk 
fs/aufs25
make CONFIG_AUFS=m AUFS_EXTRA_CFLAGS="-I/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/include -DCONFIG_AUFS_BRANCH_MAX_127 -DCONFIG_AUFS_RR_SQUASHFS -DCONFIG_AUFS_BR_XFS -DCONFIG_AUFS_WORKAROUND_FUSE -DCONFIG_AUFS_GETATTR -DCONFIG_AUFS_LOCAL -DCONFIG_AUFS_MODULE -UCONFIG_AUFS -DLKTRHidePrePath=\\"/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25\\"" -C /usr/src/linux M=/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25 modules
make CONFIG_AUFS=m AUFS_EXTRA_CFLAGS="-I/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/include -DCONFIG_AUFS_BRANCH_MAX_127 -DCONFIG_AUFS_RR_SQUASHFS -DCONFIG_AUFS_BR_XFS -DCONFIG_AUFS_WORKAROUND_FUSE -DCONFIG_AUFS_GETATTR -DCONFIG_AUFS_LOCAL -DCONFIG_AUFS_MODULE -UCONFIG_AUFS -DLKTRHidePrePath=\\"/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25\\"" -C util
make[1]: Entering directory `/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/util'
cc -O2 -march=prescott -O2 -pipe -msse3 -fomit-frame-pointer -O2 -Wall -I/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/include  -DCONFIG_AUFS_BRANCH_MAX_127 -DCONFIG_AUFS_RR_SQUASHFS -DCONFIG_AUFS_BR_XFS -DCONFIG_AUFS_WORKAROUND_FUSE -DCONFIG_AUFS_GETATTR -DCONFIG_AUFS_LOCAL -DCONFIG_AUFS_MODULE -UCONFIG_AUFS -DLKTRHidePrePath="/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25"   -Wl,-O1  c2tmac.c   -o c2tmac
cc -O2 -march=prescott -O2 -pipe -msse3 -fomit-frame-pointer -O2 -Wall -I/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/include  -DCONFIG_AUFS_BRANCH_MAX_127 -DCONFIG_AUFS_RR_SQUASHFS -DCONFIG_AUFS_BR_XFS -DCONFIG_AUFS_WORKAROUND_FUSE -DCONFIG_AUFS_GETATTR -DCONFIG_AUFS_LOCAL -DCONFIG_AUFS_MODULE -UCONFIG_AUFS -DLKTRHidePrePath="/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25"   -Wl,-O1 -s  aulchown.c   -o aulchown
cc -O2 -march=prescott -O2 -pipe -msse3 -fomit-frame-pointer -O2 -Wall -I/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/include  -DCONFIG_AUFS_BRANCH_MAX_127 -DCONFIG_AUFS_RR_SQUASHFS -DCONFIG_AUFS_BR_XFS -DCONFIG_AUFS_WORKAROUND_FUSE -DCONFIG_AUFS_GETATTR -DCONFIG_AUFS_LOCAL -DCONFIG_AUFS_MODULE -UCONFIG_AUFS -DLKTRHidePrePath="/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25"   -Wl,-O1  c2sh.c   -o c2sh
rm -f aufs.5
./c2tmac > aufs.5
awk '{ \
		gsub(/\140[^\047]*\047/, "\\[oq]&\\[cq]"); \
		gsub(/\\\[oq\]\140/, "\\[oq]"); \
		gsub(/\047\\\[cq\]/, "\\[cq]"); \
		gsub(/\047/, "\\[aq]"); \
		print; \
	}' aufs.in.5 >> aufs.5
rm -f etc_default_aufs
sed -e '/Id: /,$d' aufs.shlib > etc_default_aufs
chmod a-w aufs.5
echo '# aufs variables for shell scripts' >> etc_default_aufs
./c2sh >> etc_default_aufs
sed -e '0,/Id: /d' aufs.shlib >> etc_default_aufs
chmod a-w etc_default_aufs
test -x ./mount.aufs || chmod a+x ./mount.aufs
rm c2tmac c2sh
make[1]: Leaving directory `/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/util'
test -e aufs.5 || ln -s util/aufs.5 aufs.5
test -e aufind.sh || ln -s util/aufind.sh aufind.sh
test -e mount.aufs || ln -s util/mount.aufs mount.aufs
test -e auplink || ln -s util/auplink auplink
test -e aulchown || ln -s util/aulchown aulchown
test -e umount.aufs || ln -s util/umount.aufs umount.aufs
test -e etc_default_aufs || ln -s util/etc_default_aufs etc_default_aufs
test -e auchk || ln -s util/auchk auchk
make[1]: Entering directory `/usr/src/linux-2.6.29'
  CC [M]  /var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/module.o
  CC [M]  /var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/super.o
  CC [M]  /var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/sbinfo.o
/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/super.c: In Funktion »au_show_wbr_create«:
/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/super.c:153: Warnung: Format ist kein Zeichenkettenliteral, und keine Formatargumente
  CC [M]  /var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/branch.o
  CC [M]  /var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/xino.o
  CC [M]  /var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/sysaufs.o
/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/xino.c: In Funktion »au_xino_create2«:
/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/xino.c:624: Fehler: Zu wenige Argumente für Funktion »dentry_open«
make[2]: *** [/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25/xino.o] Fehler 1
make[2]: *** Warte auf noch nicht beendete Prozesse...
make[1]: *** [_module_/var/tmp/portage/sys-fs/aufs-99999999/work/aufs/fs/aufs25] Fehler 2
make[1]: Leaving directory `/usr/src/linux-2.6.29'
make: *** [fs/aufs25/aufs.ko] Fehler 2
 * 
 * ERROR: sys-fs/aufs-99999999 failed.
 * Call stack:
 *               ebuild.sh, line   48:  Called src_compile
 *             environment, line 3535:  Called die
 * The specific snippet of code:
 *       emake KDIR=${KV_DIR} SUBLEVEL=${KV_PATCH} -f local.mk || die "emake failed"
 *  The die message:
 *   emake failed
 * 
 * If you need support, post the topmost build error, and the call stack if relevant.
 * A complete build log is located at '/var/tmp/portage/sys-fs/aufs-99999999/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/sys-fs/aufs-99999999/temp/environment'.
 * This ebuild is from an overlay named 'gentoo-overlay': '/var/portage/overlay/personal/'
 * 

>>> Failed to emerge sys-fs/aufs-99999999, Log file:

>>>  '/var/tmp/portage/sys-fs/aufs-99999999/temp/build.log'

 * Messages for package sys-fs/aufs-99999999:

 * 
 * ERROR: sys-fs/aufs-99999999 failed.
 * Call stack:
 *               ebuild.sh, line   48:  Called src_compile
 *             environment, line 3535:  Called die
 * The specific snippet of code:
 *       emake KDIR=${KV_DIR} SUBLEVEL=${KV_PATCH} -f local.mk || die "emake failed"
 *  The die message:
 *   emake failed
 * 
 * If you need support, post the topmost build error, and the call stack if relevant.
 * A complete build log is located at '/var/tmp/portage/sys-fs/aufs-99999999/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/sys-fs/aufs-99999999/temp/environment'.
 * This ebuild is from an overlay named 'gentoo-overlay': '/var/portage/overlay/personal/'
 * 
 * 
 * The following package has failed to build or install:
 * 
 * 	('ebuild', '/', 'sys-fs/aufs-99999999', 'merge')
 *

Dont Panic · Post by **Dont Panic** » Sun Apr 05, 2009 8:53 pm

The Hitchhiker Sources also have aufs.

If you don't want to use Hitchhiker sources or Sabayon sources, you may want to still download one or the other, and just copy their /fs/aufs/ directory, and overwrite whatever you have for your /fs/aufs/ directory.

mv · Post by mv » Sun Apr 05, 2009 9:02 pm

R.Aven wrote:But the aufs-999999.ebuild uses the current aufs dev tree, which doesn't support the 2.6.29 kernel branch, afaik not even officially the 2.6.28 version.

I will not try 2.6.29 until a corresponding hardened-sources is out, but hardened-sources-2.6.28 works smoothly. However, shouldn't one better use aufs2-standalone for 2.6.29? So far, I wasn't able to compile it here: It seems that the build system has troubles with different KERNEL_DIR and KBUILD_OUTPUT directories. Did anybody else have success with it?

R.Aven · Post by **R.Aven** » Sun Apr 05, 2009 10:23 pm

mv wrote:
R.Aven wrote:But the aufs-999999.ebuild uses the current aufs dev tree, which doesn't support the 2.6.29 kernel branch, afaik not even officially the 2.6.28 version.
I will not try 2.6.29 until a corresponding hardened-sources is out, but hardened-sources-2.6.28 works smoothly. However, shouldn't one better use aufs2-standalone for 2.6.29? So far, I wasn't able to compile it here: It seems that the build system has troubles with different KERNEL_DIR and KBUILD_OUTPUT directories. Did anybody else have success with it?

I just downloaded the patch and the corresponding fs and include files from aufs2-standalone-29, applied them manually and now it seems to compile correctly. Thanks for your help.

mv · Post by mv » Mon Apr 06, 2009 5:04 pm

There is now also an experimental ebuild for aufs2 on unionfs.tar.gz.
However, unless I completely misunderstood something, at least the aufs2-28 version (for 2.6.28) is completely broken: In fact, the module is not even compiled, because the symbols are not passed to the sub-makes by the buildsystem. The experimental ebuild patches the Makefile to fix this problem, but even then it dies soon with a compile error (tested against the patched hardened-sources-2.6.28-r7).

R.Aven wrote:just downloaded the patch and the corresponding fs and include files from aufs2-standalone-29, applied them manually and now it seems to compile correctly. Thanks for your help.

Does this mean you managed to compile aufs2-standalone-29 against the aufs2-patched kernel, or did you compile aufs1 against this kernel? If you did the former: how? Does the aufs2 ebuild work for you? (Is perhaps even the my_makefile_patch in the ebuild unnecessary for aufs2-29?)

R.Aven · Post by **R.Aven** » Mon Apr 06, 2009 7:25 pm

mv wrote:There is now also an experimental ebuild for aufs2 on unionfs.tar.gz.
However, unless I completely misunderstood something, at least the aufs2-28 version (for 2.6.28) is completely broken: In fact, the module is not even compiled, because the symbols are not passed to the sub-makes by the buildsystem. The experimental ebuild patches the Makefile to fix this problem, but even then it dies soon with a compile error (tested against the patched hardened-sources-2.6.28-r7).
R.Aven wrote:just downloaded the patch and the corresponding fs and include files from aufs2-standalone-29, applied them manually and now it seems to compile correctly. Thanks for your help.
Does this mean you managed to compile aufs2-standalone-29 against the aufs2-patched kernel, or did you compile aufs1 against this kernel? If you did the former: how? Does the aufs2 ebuild work for you? (Is perhaps even the my_makefile_patch in the ebuild unnecessary for aufs2-29?)

I've done nothing with ebuild.The aufs version from the sunrise-overlay is currently installed, but won't compile on a 2.6.29 kernel. Therefore I used the patch mentioned above and applied them manually to my 2.6.29.1 kernel - which works here on my system. The utils installed with the aufs-2008* ebuild are also working with the aufs module I build using my patched kernel - at least I were able to mount my compressed portage tree with squashfs+aufs.

mv · Post by mv » Mon Apr 06, 2009 8:04 pm

R.Aven wrote:I've done nothing with ebuild.The aufs version from the sunrise-overlay is currently installed, but won't compile on a 2.6.29 kernel. Therefore I used the patch mentioned above and applied them manually to my 2.6.29.1 kernel - which works here on my system.

But the patch alone (at least that from aufs2-standalone) will not build the actual module for you. My question is how you managed to generate /lib/modules/2.6.29-*/misc/aufs.ko

The utils installed with the aufs-2008* ebuild are also working with the aufs module I build using my patched kernel

Did I understand correctly that you built the aufs.ko module still with the aufs-2008* ebuild (only after applying the aufs2-standalone patch to the kernel)? This is somewhat strange, because it means that you mix aufs version: You use the aufs2-kernel patch (meant for the aufs2 module) with the aufs1 module. So it is somewhat just accidental that this works.

R.Aven · Post by **R.Aven** » Mon Apr 06, 2009 8:18 pm

The patch adds aufs functionality to the kernel itself. So I was able to build it - as I would build all kernel modules which are actually part of the official tree - with selecting the corresponding module via "make menuconfig", make and make modules_install. That's all I did. Now I've got the aufs.ko and can modprobe, and use it as before switching to 2.6.29.

mv · Post by mv » Tue Apr 07, 2009 7:33 am

R.Aven wrote:The patch adds aufs functionality to the kernel itself.

The patch alone does not. I suppose then that besides applying the patch you also copied the other files of aufs2-standalone (except the Makefile) into your kernel tree?
Then there are of course no problems with the broken Makefile (because it is simply not copied). However, the new kernel didn't compile either with hardened-sources-2.6.28-r7 (the reason might be some of the gentoo or hardened patches, I didn't check yet).

But maybe you are right: Since - in contrast to aufs1 - the kernel patch is nontrivial anyway for aufs2, it does not make much sense to require that the user applies this patch manually and then only to install the module by an ebuild. Indeed, the new version of the aufs2 ebuild on the mentioned web page only installs all required files into the kernel source directory and asks the user to apply the patches. Also this solution is not optimal, but there is no really good solution for a module which cannot be compiled standalone but requires in addition nontrivial kernel patches.

Edit: I retried again with hardened-sources-2.6.28-r7, and now aufs2 works as it should. (Due to a mistake in using aufs with my kernel sources, I had probably patched by mistake sources which had already been patched for aufs1).

Conclusion: The new aufs2 ebuild (which is essentially the aufs2-standalone kernel patch with corresponding files in the kernel source directory) works without problems with hardened-sources-2.6.28-r7 and (I suppose) *-sources-2.6.29-*

shgadwa · Post by **shgadwa** » Tue Apr 07, 2009 8:07 pm

I'm also trying to emerge aufs...

I'm using the Zen-Sources kernel-2.6.29-rc8-zen1 kernel. I can configure it for aufs support, and I've done that. But I am not able to emerge aufs (from sunrise overlay) or aufs2 (from ebuild that I downloaded)...

It says that its not going to be installed because of file collisions.

here is the build errors:

Code: Select all

 * This package will overwrite one or more files that may belong to other
 * packages (see list below). You can use a command such as `portageq
 * owners / <filename>` to identify the installed package that owns a
 * file. If portageq reports that only one package owns a file then do
 * NOT file a bug report. A bug report is only useful if it identifies at
 * least two or more packages that are known to install the same file(s).
 * If a collision occurs and you can not explain where the file came from
 * then you should simply ignore the collision since there is not enough
 * information to determine if a real problem exists. Please do NOT file
 * a bug report at http://bugs.gentoo.org unless you report exactly which
 * two packages install the same file(s). Once again, please do NOT file
 * a bug report unless you have completely understood the above message.
 * 
 * Detected file collision(s):
 * 
 * 	/usr/src/linux-2.6.29-rc8-zen1/include/linux/aufs_type.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/branch.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/f_op.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/module.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/opts.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dcsub.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/vfsub.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/fstype.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/inode.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/aufs.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dcsub.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op_ren.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/whout.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/loop.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dentry.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysaufs.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/vfsub.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/file.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dir.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/hinotify.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op_add.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/Kconfig
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/iinfo.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysaufs.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/file.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysrq.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/Makefile
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysfs.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op_del.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/super.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dir.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/wbr_policy.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/spl.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/cpup.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/branch.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/debug.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/inode.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/cpup.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/ioctl.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/rwsem.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/loop.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dinfo.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dentry.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/finfo.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/super.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/xino.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/whout.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/plink.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/module.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/opts.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/magic.mk
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/debug.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sbinfo.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/wkq.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/vdir.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/wkq.c
 * 
 * Searching all installed packages for file collisions...
 * 
 * Press Ctrl-C to Stop
 * 
 * sys-kernel/zen-sources-2.6.29_rc8-r1
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/Kconfig
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/Makefile
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/aufs.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/branch.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/branch.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/cpup.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/cpup.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dcsub.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dcsub.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/debug.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/debug.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dentry.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dentry.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dinfo.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dir.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/dir.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/f_op.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/file.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/file.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/finfo.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/fstype.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/hinotify.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op_add.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op_del.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/i_op_ren.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/iinfo.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/inode.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/inode.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/ioctl.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/loop.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/loop.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/magic.mk
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/module.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/module.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/opts.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/opts.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/plink.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/rwsem.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sbinfo.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/spl.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/super.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/super.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysaufs.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysaufs.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysfs.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/sysrq.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/vdir.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/vfsub.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/vfsub.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/wbr_policy.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/whout.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/whout.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/wkq.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/wkq.h
 * 	/usr/src/linux-2.6.29-rc8-zen1/fs/aufs/xino.c
 * 	/usr/src/linux-2.6.29-rc8-zen1/include/linux/aufs_type.h
 * 
 * Package 'sys-fs/aufs2-99999999' NOT merged due to file collisions. If
 * necessary, refer to your elog messages for the whole content of the
 * above message.

EDIT:

I already solved the problem... I had to add COLLISIONS_IGNORE="/usr/src/linux-2.6.29-zen1" to /etc/make.conf

Now it installed, no problems.

mv · Post by mv » Tue Apr 07, 2009 8:30 pm

belikeyeshua wrote:I'm using the Zen-Sources kernel-2.6.29-rc8-zen1 kernel. I can configure it for aufs support, and I've done that. But I am not able to emerge aufs (from sunrise overlay) or aufs2 (from ebuild that I downloaded)...

If Zen-Sources already contain aufs, then why do you want to emerge the ebuild? The purpose of the ebuild is to give aufs support for kernels which do not contain it.

shgadwa · Post by **shgadwa** » Tue Apr 07, 2009 9:19 pm

Oooh.... good point.

I was wondering about that. So, should I uninstall aufs2?

mv · Post by mv » Tue Apr 07, 2009 9:27 pm

belikeyeshua wrote:So, should I uninstall aufs2?

Probably it will do no harm, but I am not sure whether it is exactly the same as that version which is contained in zen-source. (If you use zen-sources anyway, you should better also use the aufs contained there).

So it is probably safer if you uninstall aufs2 (sorry for my false suggestion in the other thread: I was not aware that zen-sources contain aufs).

However, be aware that uninstalling aufs2 will now remove the files you had overridden in zen-sources. To fix this you should re-emerge zen-sources afterwards.

crater2150 · Post by **crater2150** » Mon Apr 27, 2009 10:13 pm

Hi,
I tried to set up the script for the portage tree, but when the script stops, it says:

Code: Select all

File /usr/portage/portage-sqfs.XXXXXX changed size while reading filesystem, attempting to re-read

, and repeats this, until it is killed manually.
My configuration:

Code: Select all

DIRECTORY="/usr/portage"
DIR_CHANGE="/tmp/portage.XXXXXX"
DIR_SQUASH="/tmp/portage.sqfs"
TMP_SQFS="/tmp/portage-sqfs.XXXXXX"
FILE_SQFS="/var/portage/portage-current.sqfs"
FILE_SQFS_OLD="/var/portage/portage-old.sqfs"

mv · Post by mv » Tue Apr 28, 2009 7:47 am

crater2150 wrote:

Code: Select all

File /usr/portage/portage-sqfs.XXXXXX changed size while reading filesystem, attempting to re-read

This filename is certainly not invented by the script, but nothing in the configuration you posted contains it. I suspect that you have a symbolic link pointing to that file (e.g. from one of the files in your configuration): Of course none of your files in the configuration should be inside the directory you want to squash (also not by symbolic links).

Another note: I just observed that if one uses the temporary name feature on DIR_CHANGE, the corresponding directory cannot be erased on stop. Probably some rewrite of the script is necessary to support the temporary DIR_CHANGE feature properly.

Edit: Important security note: Do not use fixed names in world-writable directories! E.g. your "/tmp/portage.sqfs" is a very bad idea! Such things can be subject to symlink-attacks!

crater2150 · Post by **crater2150** » Tue Apr 28, 2009 8:37 pm

there weren't any symlinks to /usr/portage/portage-sqfs.XXXXXX, but removing the temporary name part from DIR_CHANGE solved the problem, thx

mv · Post by mv » Wed Apr 29, 2009 5:55 pm

There is now a new version of the script which should solve the problem with the temporary name feature in DIR_CHANGE. Morever, it brings also back the temporary name feature for DIR_SQUASH (in a non-hackish manner).

ocbMaurice · Post by **ocbMaurice** » Mon May 11, 2009 10:36 pm

Hi here, I came across this problem/so ... -wuerzburg?

Somehow I didn't really find this script before I wrote my own.
I think your script is more mature than mine, but is missing one or two features I implemented.
You'll find them in my response in the above linked thread.
It's mostly about the exclusion of ie. distfiles from /usr/portage.
The way I do it, you don't have to reconfig your distfiles dir ...

I didn't really yet test you script, but IMO it's lacking this feature

Additionaly implemented some functions for the init script (ie. init, sync, info, todo: uninit)

Hope you like my additional ideas and will incorporate them into your script.
This should also go in the portage tree, once it's mature enough.

Sorry for crossposting, IMO the two threads should be merged!
Maurice

mv · Post by mv » Thu May 14, 2009 5:18 pm

The new version of the init-script supports a THRESHOLD and a new magic file IGNORE_THRESHOLD which can be used to override the default and thus force compression even if the THRESHOLD is not reached. The other suggestions (like init function or exclusion of directories by mount --bind) are not implemented due to security considerations: Such a delicate task as compressing and removing a whole directory tree should be done manually and manually be checked twice, since there are too many setups where this can cause problems (e.g. when root has not all permissions like in SELinux or a virtual box, in --make-shared/--make-slaved directories, automatic submounts, problems with symbolic or non-symbolic links in/out the compressed tree etc.). Without an init function, there is no proper way to implement the directory-exclusion-feature with mount --bind from within the script. As mentioned in the other thread, I would suggest a separate init-file like

Code: Select all

#!/sbin/runscript
depend () { need squash_....; }
start () { mount --bind ...; }
stop () { umount ...; }

if you really need this. However, I cannot think of cases where this makes sense and where you cannot just use a symbolic link instead. For PORTDIR, it is certainly safer to put DISTDIR outside the PORTDIR tree. I would suggest the latter anyway for security reasons (independently whether some script is used) - this is meanwhile even suggested in make.conf.example. Even layman already has changed the default (although here it might make sense to include it in the compressed PORTDIR directory).