Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Portage TMPDIR on tmpfs & per-package choice on sshfs fails
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
Yminus
Apprentice
Apprentice


Joined: 06 Jan 2008
Posts: 184

PostPosted: Sun Nov 11, 2012 10:24 am    Post subject: Portage TMPDIR on tmpfs & per-package choice on sshfs fa Reply with quote

To reduce SSD wear in general my Portage TMPDIR resides on tmpfs. For bigger packages there is:

Code:
cat /etc/portage/env/notmpfs.conf
PORTAGE_TMPDIR="/var/no-tmpfs"


Code:
cat /etc/portage/package.env/no-tmpfs
app-office/libreoffice notmpfs.conf
mail-client/mozilla-thunderbird notmpfs.conf
dev-lang/mono notmpfs.conf


Code:
sshfs -o nonempty,umask=0002,uid=250,gid=250 nas_portage:/mnt/user/gentoo/var/no-tmpfs /var/no-tmpfs


When compiling dev-lang/mono I get this error:
Code:
>>> Emerging (1 of 84) dev-lang/mono-2.10.9-r2
 * Adjusting permissions for FEATURES=userpriv: '/var/no-tmpfs/portage/.distcc'
 * Adjusting permissions for FEATURES=userpriv: '/var/no-tmpfs/portage/.distcc/lock'
 * Adjusting permissions for FEATURES=userpriv: '/var/no-tmpfs/portage/.distcc/state'
 * Adjusting permissions for FEATURES=userpriv: '/var/no-tmpfs/ccache'
 * mono-2.10.9.tar.bz2 SHA256 SHA512 WHIRLPOOL size ;-) ...                      [ ok ]
 * Adjusting permissions for FEATURES=userpriv: '/var/no-tmpfs/portage/.distcc'
 * Adjusting permissions for FEATURES=userpriv: '/var/no-tmpfs/portage/.distcc/lock'
 * Adjusting permissions for FEATURES=userpriv: '/var/no-tmpfs/portage/.distcc/state'
 * Adjusting permissions for FEATURES=userpriv: '/var/no-tmpfs/ccache'
Traceback (most recent call last):
  File "/usr/bin/emerge", line 48, in <module>
    retval = emerge_main()
  File "/usr/lib64/portage/pym/_emerge/main.py", line 1021, in emerge_main
    gc_locals=locals().clear)
  File "/usr/lib64/portage/pym/_emerge/actions.py", line 3837, in run_action
    myopts, myaction, myfiles, spinner)
  File "/usr/lib64/portage/pym/_emerge/actions.py", line 463, in action_build
    retval = mergetask.merge()
  File "/usr/lib64/portage/pym/_emerge/Scheduler.py", line 1006, in merge
    rval = self._merge()
  File "/usr/lib64/portage/pym/_emerge/Scheduler.py", line 1390, in _merge
    self._main_loop()
  File "/usr/lib64/portage/pym/_emerge/Scheduler.py", line 1361, in _main_loop
    self._event_loop.iteration()
  File "/usr/lib64/portage/pym/portage/util/_eventloop/EventLoop.py", line 180, in iteration
    if not x.callback(f, event, *x.args):
  File "/usr/lib64/portage/pym/portage/util/_async/PipeLogger.py", line 119, in _output_handler
    self._unregister_if_appropriate(event)
  File "/usr/lib64/portage/pym/_emerge/AbstractPollTask.py", line 129, in _unregister_if_appropriate
    self.wait()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait
    self._wait_hook()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 161, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib64/portage/pym/_emerge/SpawnProcess.py", line 142, in _pipe_logger_exit
    self.wait()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait
    self._wait_hook()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 161, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib64/portage/pym/_emerge/EbuildPhase.py", line 257, in _ebuild_exit
    self.wait()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait
    self._wait_hook()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 161, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib64/portage/pym/_emerge/EbuildBuild.py", line 200, in _pre_clean_exit
    self._fetch_exit(fetcher)
  File "/usr/lib64/portage/pym/_emerge/EbuildBuild.py", line 254, in _fetch_exit
    self._start_task(build, self._build_exit)
  File "/usr/lib64/portage/pym/_emerge/CompositeTask.py", line 151, in _start_task
    task.start()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 30, in start
    self._start()
  File "/usr/lib64/portage/pym/_emerge/EbuildExecuter.py", line 53, in _start
    self.scheduler.scheduleSetup(setup_phase)
  File "/usr/lib64/portage/pym/_emerge/Scheduler.py", line 551, in _schedule_setup
    self._task_queues.merge.add(setup_phase)
  File "/usr/lib64/portage/pym/_emerge/SequentialTaskQueue.py", line 23, in add
    self.schedule()
  File "/usr/lib64/portage/pym/_emerge/SequentialTaskQueue.py", line 45, in schedule
    task.start()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 30, in start
    self._start()
  File "/usr/lib64/portage/pym/_emerge/EbuildPhase.py", line 132, in _start
    self._start_lock()
  File "/usr/lib64/portage/pym/_emerge/EbuildPhase.py", line 149, in _start_lock
    self._start_task(self._ebuild_lock, self._lock_exit)
  File "/usr/lib64/portage/pym/_emerge/CompositeTask.py", line 151, in _start_task
    task.start()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 30, in start
    self._start()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousLock.py", line 52, in _start
    self.wait()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait
    self._wait_hook()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 161, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib64/portage/pym/_emerge/EbuildPhase.py", line 158, in _lock_exit
    self._start_ebuild()
  File "/usr/lib64/portage/pym/_emerge/EbuildPhase.py", line 181, in _start_ebuild
    self._start_task(ebuild_process, self._ebuild_exit)
  File "/usr/lib64/portage/pym/_emerge/CompositeTask.py", line 151, in _start_task
    task.start()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 30, in start
    self._start()
  File "/usr/lib64/portage/pym/_emerge/AbstractEbuildProcess.py", line 75, in _start
    self._start_ipc_daemon()
  File "/usr/lib64/portage/pym/_emerge/AbstractEbuildProcess.py", line 155, in _start_ipc_daemon
    input_fifo, output_fifo = self._init_ipc_fifos()
  File "/usr/lib64/portage/pym/_emerge/AbstractEbuildProcess.py", line 124, in _init_ipc_fifos
    os.mkfifo(p)
OSError: [Errno 1] Operation not permitted


It works when sshfs is not mounted so that /var/no-tmpfs resides on my SSD.

What's wrong with my setup?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54098
Location: 56N 3W

PostPosted: Sun Nov 11, 2012 12:11 pm    Post subject: Reply with quote

Yminus,

I suspect the permissions options are not honored by sshfs.
Mounts over sshfs are not intended to be shared between users. That you set uid=250,gid=250 suggests that the user doing the mount is not portage.
It might work if the portage user did the sshfs mount.

For build space why not use nfs?
It should be faster as there is no encryption overhead.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Yminus
Apprentice
Apprentice


Joined: 06 Jan 2008
Posts: 184

PostPosted: Sun Nov 11, 2012 9:15 pm    Post subject: Reply with quote

NeddySeagoon wrote:
For build space why not use nfs?

Well, because "if all you have is a hammer, everything looks like a nail" ;)

I never used nfs, but now I tried without success:

Code:
>>> Emerging (1 of 84) dev-lang/mono-2.10.9-r2
Permission denied: 'makedirs('/var/notmpfs/portage')'


Code:
mount -t nfs lars-server:notmpfs /var/notmpfs/


On the client:
Code:
mount
...
lars-server:notmpfs on /var/notmpfs type nfs (rw,vers=4,addr=192.168.17.8,clientaddr=192.168.17.4)


Code:
ls -al /var/
total 80
...
drwxrwxr-x  2 portage portage  4096 11. Nov 21:45 notmpfs
...


On the server:
Code:
ls -al /mnt/backup/nfs/
total 12
drwxr-xr-x 3 root    root    4096 11. Nov 18:10 .
drwxr-xr-x 6 root    root    4096 11. Nov 20:25 ..
drwxrwxr-x 2 portage portage 4096 11. Nov 21:45 notmpfs


Surprisingly this works:
Code:
sudo -u portage -g portage mkdir /var/notmpfs/portage


The 'portage' directory is created on client and server with user and group set to portage.

As I said, I am new to nfs. I used Gentoo Wiki: NFS (without the "binding to specific ports" stuff) as guide. Maybe I missed something?
Back to top
View user's profile Send private message
Yminus
Apprentice
Apprentice


Joined: 06 Jan 2008
Posts: 184

PostPosted: Sun Nov 11, 2012 9:31 pm    Post subject: Reply with quote

I have some more infos.

The local portage directory on tmpfs contains '.distcc' which belongs to user root:
Code:
ls -al /var/tmp/portage/
total 0
drwxrwxr-x 4 portage portage  80 11. Nov 22:11 .
drwxrwxrwt 5 root    root    100 11. Nov 20:25 ..
drwxrwxr-x 2 portage portage  40 11. Nov 16:05 ._unmerge_
drwxrwsr-x 4 root    portage  80 11. Nov 15:15 .distcc


Creating a directory in the nfs-mount as user root fails:
Code:
sudo -u root -g portage mkdir /var/notmpfs/portage
Sorry, user root is not allowed to execute '/bin/mkdir /var/notmpfs/portage' as root:portage on lars-laptop.


Maybe that's the problem, but I don't know how to fix it.


Last edited by Yminus on Sun Nov 11, 2012 9:41 pm; edited 1 time in total
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54098
Location: 56N 3W

PostPosted: Sun Nov 11, 2012 9:33 pm    Post subject: Reply with quote

Yminus,

nfs version 4 is more demanding to make work. version 3 is relatively easy.

I have a few systems with root over nfs. On the server, /etc/exports contains
Code:
# /etc/exports: NFS file systems being exported.  See exports(5).
# root fs for Raspberry Pi
/var/armv6j                             192.168.10.0/24(rw,no_root_squash,no_all_squash,async,no_subtree_check)

# shared portage and distfiles between host and Raspberry Pi
# packages is in /usr/packages to avoid arch collisions
/usr/portage                            192.168.10.0/24(rw,no_root_squash,no_all_squash,async,no_subtree_check)


On the client, /etc/fstab is
Code:

# <fs>                                  <mountpoint>    <type>          <opts>          <dump/pass>

# NOTE: If your BOOT partition is ReiserFS, add the notail option to opts.
/dev/mmcblk0p1                          /boot           vfat            noauto,noatime  1 2
192.168.10.30:/var/armv6j               /               nfs             noatime         0 0
192.168.10.30:/usr/portage              /usr/portage    nfs             noatime         0 0
/dev/mmcblk0p2                          none            swap            sw              0 0
/dev/mmcblk0p3                          /home           ext2            noatime,nodev   0 0


It all works by user IDs not user names. You must make sure that the user IDs on the server and client are identical. The userIDs you want to use anyway.
The file /etc/idmapd.conf needs to have the same domain set on all participants or you will have strange permission issues on the client.

As you don't want root access, your exports can use root_squash where I have no_root_squash.


NFS Version 4 demands that you create a fake root directory that you will export everything relative to. This means you need to bind mount the exports inside this dir the export its contents.
If you set up the exports file to allow the exported filesystems to be nfs mounted on the server, you can test that it works as expected without all the other complications. NFS mount the export somewhere and look around.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Yminus
Apprentice
Apprentice


Joined: 06 Jan 2008
Posts: 184

PostPosted: Sun Nov 11, 2012 10:58 pm    Post subject: Reply with quote

Thank you, NeddySeagoon, for your support!

I have to correct my previous post:
Yminus wrote:
Creating a directory in the nfs-mount as user root fails:
Code:
sudo -u root -g portage mkdir /var/notmpfs/portage
Sorry, user root is not allowed to execute '/bin/mkdir /var/notmpfs/portage' as root:portage on lars-laptop.


This is not related to the nfs problem, maybe it's a sudo restriction, but I did not take a closer look.

NeddySeagoon wrote:
It all works by user IDs not user names. You must make sure that the user IDs on the server and client are identical. The userIDs you want to use anyway.

The userID of the portage user is 250 on both machines, so this is ok.

NeddySeagoon wrote:
The file /etc/idmapd.conf needs to have the same domain set on all participants or you will have strange permission issues on the client.

Thank you for this hint, I forgot to set the domain on the client.

NeddySeagoon wrote:
As you don't want root access, your exports can use root_squash where I have no_root_squash.

NFS Version 4 demands that you create a fake root directory that you will export everything relative to. This means you need to bind mount the exports inside this dir the export its contents.

My exports are:
Code:
cat /etc/exports
# /etc/exports: NFS file systems being exported.  See exports(5).
/mnt/backup/nfs/ 192.168.17.0/24(rw,fsid=0,no_subtree_check,async,no_root_squash)
/mnt/backup/nfs/notmpfs 192.168.17.0/24(rw,no_subtree_check,async,no_root_squash)


But now there is another issue:
Code:
>>> Installing (1 of 84) dev-lang/mono-2.10.9-r2
 * checking 1068 files for package collisions
1000 files checked ...
>>> Merging dev-lang/mono-2.10.9-r2 to /
--- /usr/
--- /usr/include/
--- /usr/include/mono-2.0/
--- /usr/include/mono-2.0/mono/
--- /usr/include/mono-2.0/mono/metadata/
!!! Failed to copy extended attributes. In order to avoid this error,
!!! set FEATURES="-xattr" in make.conf.
!!! copy /var/notmpfs/portage/dev-lang/mono-2.10.9-r2/image/usr/include/mono-2.0/mono/metadata/debug-mono-symfile.h -> /usr/include/mono-2.0/mono/metadata/debug-mono-symfile.h failed.
!!! Filesystem containing file '/usr/include/mono-2.0/mono/metadata/debug-mono-symfile.h#new' does not support extended attributes


Last edited by Yminus on Tue Nov 13, 2012 10:21 pm; edited 1 time in total
Back to top
View user's profile Send private message
Yminus
Apprentice
Apprentice


Joined: 06 Jan 2008
Posts: 184

PostPosted: Mon Nov 12, 2012 1:10 am    Post subject: Reply with quote

Yminus wrote:
But now there is another issue:
Code:
>>> Installing (1 of 84) dev-lang/mono-2.10.9-r2
 * checking 1068 files for package collisions
1000 files checked ...
>>> Merging dev-lang/mono-2.10.9-r2 to /
--- /usr/
--- /usr/include/
--- /usr/include/mono-2.0/
--- /usr/include/mono-2.0/mono/
--- /usr/include/mono-2.0/mono/metadata/
!!! Failed to copy extended attributes. In order to avoid this error,
!!! set FEATURES="-xattr" in make.conf.
!!! copy /var/notmpfs/portage/dev-lang/mono-2.10.9-r2/image/usr/include/mono-2.0/mono/metadata/debug-mono-symfile.h -> /usr/include/mono-2.0/mono/metadata/debug-mono-symfile.h failed.
!!! Filesystem containing file '/usr/include/mono-2.0/mono/metadata/debug-mono-symfile.h#new' does not support extended attributes


Fixed it by using nfs3:

Code:
mount -t nfs -o nfsvers=3 lars-server:/mnt/backup/nfs /var/notmpfs/


Added this line to the client's /etc/fstab:
Code:
lars-server:/mnt/backup/nfs/notmpfs    /var/notmpfs    nfs    noatime,noauto,nfsvers=3    0 0

In future I just mount it before emerging memory hogs.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum