Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Mounting /var/tmp/portage thru sshfs
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
destroyedlolo
l33t
l33t


Joined: 17 Jun 2011
Posts: 846
Location: Close to Annecy (France)

PostPosted: Tue Apr 09, 2013 1:51 pm    Post subject: Mounting /var/tmp/portage thru sshfs Reply with quote

Hello,

Due to space constraint, I'm trying to mount /var/tmp/portage from another machine using sshfs with the following command :
Code:
sshfs -o nonempty,allow_root root@PiV.local:/var/tmp/portage /var/tmp/portage


On PiV machine, /var/tmp/portage exists and is a tmpfs.

But when I'm trying to emerge something, I got :
Code:
>> Emerging (1 of 1) www-client/firefox-17.0.4
 * firefox-17.0.4esr-fr.xpi SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                       [ ok ]
 * firefox-17.0-patches-0.5.tar.xz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                [ ok ]
 * firefox-17.0.4esr.source.tar.bz2 SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                               [ ok ]
Traceback (most recent call last):
  File "/usr/bin/emerge", line 51, in <module>
    retval = emerge_main()
  File "/usr/lib/portage/pym/_emerge/main.py", line 1044, in emerge_main
    gc_locals=locals().clear)
  File "/usr/lib/portage/pym/_emerge/actions.py", line 3920, in run_action
    myopts, myaction, myfiles, spinner)
  File "/usr/lib/portage/pym/_emerge/actions.py", line 478, in action_build
    retval = mergetask.merge()
  File "/usr/lib/portage/pym/_emerge/Scheduler.py", line 1010, in merge
    rval = self._merge()
  File "/usr/lib/portage/pym/_emerge/Scheduler.py", line 1395, in _merge
    self._main_loop()
  File "/usr/lib/portage/pym/_emerge/Scheduler.py", line 1372, in _main_loop
    self._event_loop.iteration()
  File "/usr/lib/portage/pym/portage/util/_eventloop/EventLoop.py", line 200, in iteration
    if self._run_timeouts():
  File "/usr/lib/portage/pym/portage/util/_eventloop/EventLoop.py", line 461, in _run_timeouts
    if self._run_idle_callbacks():
  File "/usr/lib/portage/pym/portage/util/_eventloop/EventLoop.py", line 423, in _run_idle_callbacks
    if not x.callback(*x.args):
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 74, in _async_wait_cb
    self.wait()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait
    self._wait_hook()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib/portage/pym/_emerge/EbuildPhase.py", line 158, in _lock_exit
    self._start_ebuild()
  File "/usr/lib/portage/pym/_emerge/EbuildPhase.py", line 181, in _start_ebuild
    self._start_task(ebuild_process, self._ebuild_exit)
  File "/usr/lib/portage/pym/_emerge/CompositeTask.py", line 151, in _start_task
    task.start()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 30, in start
    self._start()
  File "/usr/lib/portage/pym/_emerge/AbstractEbuildProcess.py", line 75, in _start
    self._start_ipc_daemon()
  File "/usr/lib/portage/pym/_emerge/AbstractEbuildProcess.py", line 155, in _start_ipc_daemon
    input_fifo, output_fifo = self._init_ipc_fifos()
  File "/usr/lib/portage/pym/_emerge/AbstractEbuildProcess.py", line 124, in _init_ipc_fifos
    os.mkfifo(p)
OSError: [Errno 1] Operation not permitted


Is sftp mounting too limited for this kind of things ?
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9601
Location: almost Mile High in the USA

PostPosted: Tue Apr 09, 2013 3:13 pm    Post subject: Reply with quote

I've had problems with nfs even... sometimes it works, sometimes it doesn't, I end up having to use a USB HDD...
I'd imagine if sometimes nfs has issues, sshfs would have more... Usually it's permissions/file ownership issue, but timestamps could also affect it.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7470

PostPosted: Tue Apr 09, 2013 7:09 pm    Post subject: Reply with quote

you could try that, man make.conf for explain

Code:
FEATURES="-distlocks" emerge -1 firefox
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21431

PostPosted: Wed Apr 10, 2013 2:05 am    Post subject: Reply with quote

krinn wrote:
you could try that, man make.conf for explain

Code:
FEATURES="-distlocks" emerge -1 firefox
Although the OP might have a distlock problem later, the current problem is clearly shown to be Operation not permitted when preparing the IPC FIFOs.

OP: is there no way to run the build in a tmpfs locally? Could you have PiV build the package and then just install it prebuilt on the local machine?
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9501
Location: beyond the rim

PostPosted: Wed Apr 10, 2013 7:38 am    Post subject: Reply with quote

Generally speaking: for each abstraction/projection layer involved you're going to loose features. In case of sshfs that involves the actual sshfs module, fuse, vfs and tmpfs, and my guess is that sshfs doesn't implement mkfifo or another syscall involved (see http://sourceforge.net/mailarchive/message.php?msg_id=24025449).
Back to top
View user's profile Send private message
destroyedlolo
l33t
l33t


Joined: 17 Jun 2011
Posts: 846
Location: Close to Annecy (France)

PostPosted: Wed Apr 10, 2013 3:22 pm    Post subject: Reply with quote

Hu wrote:
OP: is there no way to run the build in a tmpfs locally?

This machine is the compile box for all my 32b machines, and has only 2Gb which is not enough to build Firefox (that's the only thing that is not compiling for the moment). I can't do it on local disk as well.
Hu wrote:
Could you have PiV build the package and then just install it prebuilt on the local machine?

PiV is 64b machine ...

Well, for the moment, it seems I have to give up until I'm able to change the harddisk ... never-mind, if it's become really blocking, I'll directly emerge a binary.

Thanks anyway for replies.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9601
Location: almost Mile High in the USA

PostPosted: Wed Apr 10, 2013 4:11 pm    Post subject: Reply with quote

What you could do is setup a 32-bit chroot on the 64-bit machine and build there (or just copy your 32-bit install straight to the 64-bit machine and chroot into it). Or set up a virtual machine on the 64-bit machine.

I don't know if nfsv4 would do the trick too, that may be the other workaround.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21431

PostPosted: Thu Apr 11, 2013 2:03 am    Post subject: Reply with quote

I think that 2Gb of physical is adequate to build Firefox without thrashing. You will probably need swap available and you should not try to keep the build in a tmpfs.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9601
Location: almost Mile High in the USA

PostPosted: Thu Apr 11, 2013 2:41 am    Post subject: Reply with quote

I found firefox 7 needed at least 1.5GB of virtual ram to complete (I was compiling it with 384MB physical + 1.125GB swap) , but PORTAGE_TMP needs to be 4GB... I suspect firefox 17 may need more...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21431

PostPosted: Fri Apr 12, 2013 2:07 am    Post subject: Reply with quote

Right, that is why I stated he would need swap available. Having 2GB of physical should avoid horrible thrashing, but there is no way he can keep the entire compilation in memory with only 2GB of physical.
Back to top
View user's profile Send private message
Viorel
Developer
Developer


Joined: 13 Feb 2012
Posts: 16

PostPosted: Fri Apr 19, 2013 4:06 pm    Post subject: Reply with quote

Never tried it, but here's an idea: create a file with dd, export it via nfs or sshfs, create a filesystem inside it and mount it with -o loop. I suppose it's gonna be very slow, if it even works.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9601
Location: almost Mile High in the USA

PostPosted: Fri Apr 19, 2013 6:35 pm    Post subject: Reply with quote

Viorel, interesting idea. So I tried a loop mount of a file over NFSV4 (!) and this appears to be causing an endless loop in portage... odd...
more information as it becomes available...

I think though all the filesystem features are available I have a feeling that file (actually, since it's one file, individual block) locking isn't working right for nfs with default mount options... :(
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum