Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
TIP: Compressing portage using squashfs: initscript method
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3 ... 9, 10, 11  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
Khumarahn
Apprentice
Apprentice


Joined: 17 Apr 2009
Posts: 162

PostPosted: Mon Sep 24, 2018 10:57 pm    Post subject: Reply with quote

I am interested in squashfs primarily for a system with little memory, namely Teres 1 arm laptop. It has 2G ram and 16G MMC. With these little resources, optimizing portage from 1G to 50M is important. Then, the sync should ideally be also efficient... May not have enough ram free to keep whole portage tree.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 17509

PostPosted: Tue Sep 25, 2018 3:54 am    Post subject: Reply with quote

tmpfs is optional. I apparently edited that out of my previous post. My squashfs images are ~59M.
_________________
It is what it is out there. So whatever it is, it is.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6242

PostPosted: Tue Sep 25, 2018 4:45 am    Post subject: Reply with quote

Khumarahn wrote:
What is the easiest way to keep portage in squashfs, and recompress in on emerge --sync?

You could use squashmount and call
Code:
squashmount remount [repos mount point]
in an /etc/portage/postsync.d hook.
Back to top
View user's profile Send private message
Khumarahn
Apprentice
Apprentice


Joined: 17 Apr 2009
Posts: 162

PostPosted: Tue Sep 25, 2018 4:04 pm    Post subject: Reply with quote

mv wrote:
You could use squashmount and call
Code:
squashmount remount [repos mount point]
in an /etc/portage/postsync.d hook.


Thanks! This worked beautifully. I get a strange error on squashmount remount:
Code:
mount: /usr/portage: wrong fs type, bad option, bad superblock on overlayfs, missing codepage or helper program, or other error.

But despite the error, it works correctly. Is there an easy way to see where the error comes from?

My config:
/etc/squashmount.pl:
#!/usr/bin/perl (this is only for editors)

@order = qw(overlay overlayfs aufs! unionfs-fuse! unionfs??# funionfs??#);

$rm_changes = $rm_workdir = $rm_readonly = 0;

my $defaults = {
   COMPRESSION => 'xz',
   COMPOPT_XZ => ['-Xbcj', 'arm'],
};

@mounts = (
   standard_mount('portage', '/usr/portage', $defaults),
);

1;  # The last executed command in this file should be a true expression


/etc/portage/repo.postsync.d/20-teres-squash-gentoo:
#!/bin/sh
# This file remounts the squashmount mount-point "gentoo" after each syncing
# of the "gentoo" repository.
set -u

repository_name=$1
sync_uri=$2
repository_path=$3

# Run only for repository gentoo:
[ x"$repository_name" = x'gentoo' ] || exit 0

# Run only if mount-point portage is configured
[ x"$(squashmount --quiet print-tag portage 2>/dev/null)" = x'portage' ] || exit 0

squashmount remount portage
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6242

PostPosted: Wed Sep 26, 2018 6:35 am    Post subject: Reply with quote

Khumarahn wrote:
I get a strange error on squashmount remount

squashmount has its own thread and a github bugtracker, so we should perhaps not clutter the discussion here.
My guess is that either your kernel is too old (older than 3.18 requires $obsolete_overlayfs being set, see squasmount --man) or some options are not compiled in your kernel (cf. MOUNT_OVERLAY in squashmount --man) and therefore mounting overlay(fs) fails and something else (probably unionfs-fuse) is used as a fallback. With
Code:
squashmount list
you can see what was effectively used for mounting. If you pass squashmount remount the option(s) -vvv you can see which commands/options are actually executed to understand which options overlayfs does not like. If you need further help or suspect a bug, I suggest we move discussion to either the above mentioned thread or (preferrably) to the github bug tracker.
Back to top
View user's profile Send private message
Khumarahn
Apprentice
Apprentice


Joined: 17 Apr 2009
Posts: 162

PostPosted: Wed Sep 26, 2018 6:44 pm    Post subject: Reply with quote

@mv, thank you! Running squashmount -vvv, I found that on the first attempt to mount overlayfs, the mount command does not accept the --workdir option. Probably because I am on a very old kernel with whatever custom patches by Allwinner and sunxi. By the next mount, without --workdir, it mounts and works.

I found that it is important to disable hardlinks for portage sync (`sync-allow-hardlinks = no`). Otherwise the sync is ugly, slow and takes lots of space.

I will ask more questions in the other thread or github, if I have any. So far, I am rather satisfied!
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6242

PostPosted: Wed Sep 26, 2018 7:54 pm    Post subject: Reply with quote

Khumarahn wrote:
--workdir option

This option (and the working directory) are used for the variant in newer kernels; squashmount calls this variant "overlay".
The variant in older kernels (without a working directory) is by squashmount called "overlayfs".
Since the default value of @order is qw(overlay!? overlayfs!? aufs! unionfs-fuse! unionfs! funionfs!), squashmount will first attempt 'overlay' and in case of failure 'overlayfs'. Thus, to suppress the error message, you might exchange the first two entries in your configuration file:
/etc/squashmount.pl wrote:
@order=qw(overlayfs!? overlay!? aufs! unionfs-fuse! unionfs! funionfs!);

This way, first the variant without a working directory is attempted. However, you will have to remove that line once you upgrade to >=linux-3.18.
Back to top
View user's profile Send private message
Khumarahn
Apprentice
Apprentice


Joined: 17 Apr 2009
Posts: 162

PostPosted: Wed Sep 26, 2018 9:10 pm    Post subject: Reply with quote

I already put overlayfs first... I get this that error message with
Code:
@order = qw(overlayfs? overlay)
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6242

PostPosted: Wed Sep 26, 2018 9:20 pm    Post subject: Reply with quote

Khumarahn wrote:
I already put overlayfs first..

You are right; I misremembered. The working directory was introduced in linux-3.15. The correct recommendation would probably have been instead to set
/etc/squashmount.pl wrote:
$obsolete_overlayfs = 1;
Back to top
View user's profile Send private message
Khumarahn
Apprentice
Apprentice


Joined: 17 Apr 2009
Posts: 162

PostPosted: Wed Sep 26, 2018 9:22 pm    Post subject: Reply with quote

/etc/squashmount.pl:
$obsolete_overlayfs = 1;

That worked :-)))) Thanks!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3 ... 9, 10, 11
Page 11 of 11

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum