Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
TIP: Compressing portage using squashfs: initscript method
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8, 9  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
gringo
Advocate
Advocate


Joined: 27 Apr 2003
Posts: 3505

PostPosted: Tue Oct 18, 2011 9:40 am    Post subject: Reply with quote

mv wrote:
The patch should be fixed now to build in all cases.


nice, thanks a lot !

cheers
Back to top
View user's profile Send private message
costel78
Apprentice
Apprentice


Joined: 20 Apr 2007
Posts: 165

PostPosted: Sat Jan 21, 2012 6:20 pm    Post subject: Reply with quote

Is it possible to port the init script to systemd ?
I looked over actual openrc script, but it is too far complex to simple migrating the commands :D
_________________
Sorry for my English. I'm still learning this language.
Back to top
View user's profile Send private message
mv
Advocate
Advocate


Joined: 20 Apr 2005
Posts: 2589

PostPosted: Sun Jan 22, 2012 8:20 am    Post subject: Reply with quote

costel78 wrote:
Is it possible to port the init script to systemd?

I don't know. AFAIK systemd does not have proper dependencies and probably also not a standard route during shutdown which is crucial for the script.
Actually, I consider the whole systemd a very bad idea (running permanently daemons which take resources only to save 1-2 seconds during booting is not sane), so I hope that I will never be forced to use it, and I am not at all interested in porting something to a system with a bad concept.
Quote:
I looked over actual openrc script, but it is too far complex to simple migrating the commands :D

Essentially, it is the "start" and "stop" which would have to be called at the proper place. The other provided command are only for wrapper scripts to check the current state (e.g. whether it would be saved). Probably one could put a part in a "library" which is sourced.
Back to top
View user's profile Send private message
costel78
Apprentice
Apprentice


Joined: 20 Apr 2007
Posts: 165

PostPosted: Sun Jan 22, 2012 9:53 am    Post subject: Reply with quote

Thank you for your answer!
Yes, I already tried to simulate the logics behind and, so far, I found no way to start tar and bzip compress of the portage tree BEFORE /usr/portage was umounted.
What was the simulation:
- a separate partition for /usr/portage (without distfiles, of course) mounted automatically at boot via initscript (noauto in fstab)
- during shutdown compress it (tar -cjpf ....) and umount it
As you said, I found no way to prevent systemd to try to umount the partition and kill tar process.
It seems that all thing it's to risky.
_________________
Sorry for my English. I'm still learning this language.
Back to top
View user's profile Send private message
mv
Advocate
Advocate


Joined: 20 Apr 2005
Posts: 2589

PostPosted: Sun Jan 22, 2012 10:19 am    Post subject: Reply with quote

costel78 wrote:
As you said, I found no way to prevent systemd to try to umount the partition and kill tar process.

At least the mount and umount commands from sys-apps/util-linux provide a way to call helper scripts for mount and umount of certain filesystem types. I have never dealt with them and do not know how compatible this setting is. But it might be a possible to invent a "new" filesystem type and to do the actual action in these helper scripts. However, this needs to be well-tested: e.g. aufs also uses such helper scripts, so "recursive" callings of such scripts must be possible.
The advantage of this approach would be that it works with any init-system, the disadvantage is that it probably works only with util-linux.
Currently, I have no time to implement such a thing.
Back to top
View user's profile Send private message
slycordinator
Advocate
Advocate


Joined: 31 Jan 2004
Posts: 3050
Location: Houston, TX

PostPosted: Tue Feb 28, 2012 8:58 pm    Post subject: Reply with quote

I'm having a problem with mv's squash_dir initscript.

My portage squashfs file is at /squashed_dirs/portage.sqfs and the portage.readonly and portage.changes are placed inside that same directory. And I named the portage initscript squash_portage

My portage.changes directory has a whole bunch of subdirs in it currently, since I did an "emerge --sync" earlier, but if I do "/etc/init.d/squash_portage restart" I simply get:
Code:
/etc/init.d/squash_portage restart
 * Unmounting /usr/portage ...                                                                   [ ok ]
 * Mounting /squashed_dirs/portage.sqfs as /usr/portage ...                                      [ ok ]

And the portage.sqfs is not recompressed with the portage.changes directory containing the same stuff it did previously.
_________________
My political stance/bias
slycordinator != slycoordinator
Back to top
View user's profile Send private message
mv
Advocate
Advocate


Joined: 20 Apr 2005
Posts: 2589

PostPosted: Wed Feb 29, 2012 9:55 am    Post subject: Reply with quote

slycordinator wrote:
And the portage.sqfs is not recompressed

This is strange. I cannot reproduce it here. Are you sure that you do not have a .no-save file and not set any threshold?
If yes, please help me debugging:
  1. Which version of squash_dir are you using? To simplify debugging, please try with the latest from the mv overlay (from today).
  2. Is the output of
    Code:
    /etc/init.d/squash_portage print_dir_change
    indeed your portage.changes which contains data? In particular, does
    Code:
    ls $(/etc/init.d/squash_portage print_dir_change)
    show that content?
  3. Does some of the following two commands output "1" to stderr?
    Code:
    /etc/init.d/squash_portage need_squash
    /etc/init.d/squash_portage will_squash
Back to top
View user's profile Send private message
slycordinator
Advocate
Advocate


Joined: 31 Jan 2004
Posts: 3050
Location: Houston, TX

PostPosted: Wed Feb 29, 2012 7:59 pm    Post subject: Reply with quote

Note: This happens sporadically. Like when I shutdown the computer last night, the portage squashfs file got recompressed and the ones I've got for /var/db and /usr/share/texmf-dist didn't, but at least now (using version 11.6) doing a "/etc/init.d/squash_db restart" made it recompress on the first try. And the need_squash and will_squash variables showed that the tex directory will be recompressed. Guess it's fixed but not sure, given how sporadic it would happen.

edit: PEBKAC. Think it was the threshold setting.
_________________
My political stance/bias
slycordinator != slycoordinator
Back to top
View user's profile Send private message
js08
n00b
n00b


Joined: 04 Mar 2008
Posts: 41

PostPosted: Sat Mar 24, 2012 12:36 am    Post subject: Reply with quote

Hi,

I'm no longer able to update my portage tree (squash_dir, unionfs_fuse [everything from mv-overlay])
sys-fs/fuse-2.8.7, kernel 3.2.11-gentoo or 3.3-gentoo, net-misc/rsync-3.0.9-r1


emerge --sync produces the following error:

Code:

app-backup/backup-manager/files/
app-backup/backupninja/
app-backup/backuppc/
app-backup/backuppc/files/
app-backup/backuppc/files/3.2.0/
app-backup/bacula/
app-backup/bacula/ChangeLog
      40.52K 100%   59.51kB/s    0:00:00 (xfer#24, to-check=1056/4230)
app-backup/bacula/Manifest
       8.54K 100%   12.22kB/s    0:00:00 (xfer#25, to-check=1055/4230)
app-backup/bacula/bacula-5.2.5.ebuild
      10.09K 100%   14.42kB/s    0:00:00 (xfer#26, to-check=1048/4230)
app-backup/bacula/bacula-5.2.6.ebuild
      10.09K 100%   14.09kB/s    0:00:00 (xfer#27, to-check=1047/4230)
rsync: rename "/usr/portage/app-backup/bacula/.bacula-5.2.6.ebuild.rJDBEZ" -> "app-backup/bacula/bacula-5.2.6.ebuild": Software caused connection abort (103)
app-backup/bacula/files/
app-backup/bacula/files/5.0.2/
app-backup/bacula/files/5.0.3/
app-backup/bacula/files/5.2.3/
app-backup/boxbackup/
app-backup/boxbackup/files/
app-backup/ccollect/
app-backup/cdbackup/
app-backup/cdbkup/
app-backup/cpdup/
app-backup/cpdup/files/
app-backup/dar/
app-backup/dar/files/
rsync: failed to set times on "/usr/portage/app-crypt": Software caused connection abort (103)
rsync: opendir "/usr/portage/app-crypt" failed: Transport endpoint is not connected (107)
rsync: recv_generator: failed to stat "/usr/portage/app-crypt/metadata.xml": Transport endpoint is not connected (107)
rsync: recv_generator: mkdir "/usr/portage/app-crypt/WiRouterKeyRec" failed: Transport endpoint is not connected (107)
*** Skipping any contents from this failed directory ***
rsync: recv_generator: mkdir "/usr/portage/app-crypt/acr38u" failed: Transport endpoint is not connected (107)
...

_________________
Train Hard Or Don't Train At All
Back to top
View user's profile Send private message
mv
Advocate
Advocate


Joined: 20 Apr 2005
Posts: 2589

PostPosted: Sat Mar 24, 2012 10:25 am    Post subject: Reply with quote

js08 wrote:
Hi,

I'm no longer able to update my portage tree (squash_dir, unionfs_fuse [everything from mv-overlay])
sys-fs/fuse-2.8.7, kernel 3.2.11-gentoo or 3.3-gentoo, net-misc/rsync-3.0.9-r1


emerge --sync produces the following error:

And it worked before (i.e. with other kernels)? Then it looks to me like a race condition problem in unionfs-fuse or an incompatibility of fuse with the newest kernels.
If the start of the error message is not reproducible, I would guess on the former and suggest that you report the problem upstream at unionfs-fuse.
Back to top
View user's profile Send private message
js08
n00b
n00b


Joined: 04 Mar 2008
Posts: 41

PostPosted: Sat Mar 24, 2012 7:46 pm    Post subject: Reply with quote

mv wrote:
js08 wrote:
Hi,

I'm no longer able to update my portage tree (squash_dir, unionfs_fuse [everything from mv-overlay])
sys-fs/fuse-2.8.7, kernel 3.2.11-gentoo or 3.3-gentoo, net-misc/rsync-3.0.9-r1


emerge --sync produces the following error:

And it worked before (i.e. with other kernels)? Then it looks to me like a race condition problem in unionfs-fuse or an incompatibility of fuse with the newest kernels.
If the start of the error message is not reproducible, I would guess on the former and suggest that you report the problem upstream at unionfs-fuse.


yes, everything worked fine until 3-21-2012.
and yes the start of the error message is not reproducible. but everything else on two different hosts and 1 virtual machine.
_________________
Train Hard Or Don't Train At All
Back to top
View user's profile Send private message
mv
Advocate
Advocate


Joined: 20 Apr 2005
Posts: 2589

PostPosted: Mon Apr 02, 2012 10:41 pm    Post subject: Reply with quote

js08 wrote:
and yes the start of the error message is not reproducible. but everything else on two different hosts and 1 virtual machine.

As I mentioned, please report upstream at unionfs-fuse - this needs help by the corresponding specialist (it probably is some race issue).
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8, 9
Page 9 of 9

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum