Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[SOLVED]emerge-webrsync fails
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
n8V8r
n00b
n00b


Joined: 28 Jun 2018
Posts: 17

PostPosted: Sat Sep 14, 2019 9:40 am    Post subject: [SOLVED]emerge-webrsync fails Reply with quote

Hitting a roadblock at the installation https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Base stage where
Code:
emerge-webrsync
is to be excuted

Code:
!!! Section 'gentoo' in repos.conf has location attribute set to nonexistent directory: '/var/db/repos/gentoo'
!!! SYNC setting found in make.conf.
    This setting is Deprecated and no longer used.  Please ensure your 'sync-type' and 'sync-uri' are set correctly in /etc/portage/repos.conf/gentoo.conf
!!! Invalid Repository Location (not a dir): '/var/db/repos/gentoo'
Fetching most recent snapshot ...
Trying to retrieve 20190913 snapshot from rsync://mirror.eu.oneandone.net/gentoo ...
Fetching file portage-20190913.tar.xz.md5sum ...
Fetching file portage-20190913.tar.bz2.md5sum ...
Fetching file portage-20190913.tar.gz.md5sum ...
Trying to retrieve 20190913 snapshot from http://mirror.netcologne.de/gentoo ...
Fetching file portage-20190913.tar.xz.md5sum ...
Fetching file portage-20190913.tar.xz.gpgsig ...
Fetching file portage-20190913.tar.xz ...
Checking digest ...
Getting snapshot timestamp ...
Syncing local tree ...
tar: portage/sci-visualization/pythonprop/Manifest: Cannot open: No space left on device


Suppose the warnings about Invalid Repository Location and SYNC settings are negligible but being stymied by the no space left issue. Looked up online about potential culprits but nothing pointed out remedies the matter

Code:
/dev/mapper/vg_main-lv_os on / type ext4 (rw,relatime,errors=remount-ro)
/dev/mapper/vg_boot-lv_boot on /boot type ext4 (rw,nosuid,nodev,noexec,noatime)
/proc on /proc type proc (rw,nosuid,nodev,noexec,noatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
cgroup_root on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,relatime,size=10240k,mode=755)
openrc on /sys/fs/cgroup/openrc type cgroup (rw,nosuid,nodev,noexec,relatime,release_agent=/lib/rc/sh/cgroup-release-agent.sh,name=openrc)
none on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=10240k,nr_inodes=996123,mode=755)
devpts on /dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=000)
shm on /dev/shm type tmpfs (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/debug/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)


Code:
Filesystem                  Type     Inodes IUsed  IFree IUse% Mounted on
/dev/mapper/vg_main-lv_os   ext4     196608 53077 143531   27% /
cgroup_root                 tmpfs    997428     3 997425    1% /sys/fs/cgroup
udev                        devtmpfs 996123   604 995519    1% /dev
shm                         tmpfs    997428     1 997427    1% /dev/shm
/dev/mapper/vg_boot-lv_boot ext4      65024    10  65014    1% /boot

Code:

Filesystem                  Type     1K-blocks    Used Available Use% Mounted on
/dev/mapper/vg_main-lv_os   ext4       3030800 1590328   1266804  56% /
cgroup_root                 tmpfs        10240       0     10240   0% /sys/fs/cgroup
udev                        devtmpfs     10240       4     10236   1% /dev
shm                         tmpfs      3989712       0   3989712   0% /dev/shm
/dev/mapper/vg_boot-lv_boot ext4       1007640    2528    936712   1% /boot


Tried
Code:
env-update
but that did not remedy the matter.

It would seems inexplicable thus that
Code:
tar
stakes the claim about
Code:
No space left on device


[Moderator edit: changed [quote] tags to [code] tags to preserve output layout. -Hu]


Last edited by n8V8r on Sun Sep 15, 2019 5:31 pm; edited 1 time in total
Back to top
View user's profile Send private message
Prof. Frink
Apprentice
Apprentice


Joined: 07 Jan 2017
Posts: 176

PostPosted: Sat Sep 14, 2019 9:54 am    Post subject: Re: emerge-webrsync fails Reply with quote

n8V8r wrote:
Code:
tar
stakes the claim about
Quote:
No space left on device


Is there space left on the device? Otherwise this is probably the problem. You can check this using

Code:
df -h
Back to top
View user's profile Send private message
n8V8r
n00b
n00b


Joined: 28 Jun 2018
Posts: 17

PostPosted: Sat Sep 14, 2019 10:06 am    Post subject: Re: emerge-webrsync fails Reply with quote

Prof. Frink wrote:


Is there space left on the device? Otherwise this is probably the problem. You can check this using

Code:
df -h


The output of dh been quoted in the initial post, according to which there is no issue with space nor inodes availability.
Back to top
View user's profile Send private message
Prof. Frink
Apprentice
Apprentice


Joined: 07 Jan 2017
Posts: 176

PostPosted: Sat Sep 14, 2019 10:15 am    Post subject: Reply with quote

Okay, sry, I did not read it carefully. Then this may be the problem:

Code:
!!! Section 'gentoo' in repos.conf has location attribute set to nonexistent directory: '/var/db/repos/gentoo'


I do not have this dir too. You may either create it or even better change the repos.conf the way, that it uses another dir. Even the "tar-error" may make sens, if it tries to write to nonexisting dir, but I do not know, if this is the case. What does your repos.conf look like?
Back to top
View user's profile Send private message
n8V8r
n00b
n00b


Joined: 28 Jun 2018
Posts: 17

PostPosted: Sat Sep 14, 2019 10:23 am    Post subject: Reply with quote

Prof. Frink wrote:
Okay, sry, I did not read it carefully. Then this may be the problem:

Code:
!!! Section 'gentoo' in repos.conf has location attribute set to nonexistent directory: '/var/db/repos/gentoo'


I do not have this dir too. You may either create it or even better change the repos.conf the way, that it uses another dir. Even the "tar-error" may make sens, if it tries to write to nonexisting dir, but I do not know, if this is the case. What does your repos.conf look like?


That is not the issue since the warning is only printed on the first run and the directory then being created on the fly. On the second run the warning is absent since the directory is in residence by then.

I am wondering which path tar is trying to expand the compressed files to.
Back to top
View user's profile Send private message
Prof. Frink
Apprentice
Apprentice


Joined: 07 Jan 2017
Posts: 176

PostPosted: Sat Sep 14, 2019 10:44 am    Post subject: Reply with quote

n8V8r wrote:

Prof. Frink wrote:
Okay, sry, I did not read it carefully. Then this may be the problem:

That is not the issue since the warning is only printed on the first run and the directory then being created on the fly. On the second run the warning is absent since the directory is in residence by then.



Ah okay, sry.
n8V8r wrote:

I am wondering which path tar is trying to expand the compressed files to.


Me too, but here are probably some experts, who will know :)
Back to top
View user's profile Send private message
freke
Guru
Guru


Joined: 23 Jan 2003
Posts: 491
Location: Somewhere in Denmark

PostPosted: Sat Sep 14, 2019 10:49 am    Post subject: Reply with quote

You're in the chroot - right?
Back to top
View user's profile Send private message
Prof. Frink
Apprentice
Apprentice


Joined: 07 Jan 2017
Posts: 176

PostPosted: Sat Sep 14, 2019 10:50 am    Post subject: Reply with quote

Maybe it is

Code:
/usr/portage/sci-visualization/pythonprop/



Or maybe just

Code:
portage/sci-visualization/pythonprop/Manifest


Then this is the problem, but I do not know how to fix it.
Back to top
View user's profile Send private message
n8V8r
n00b
n00b


Joined: 28 Jun 2018
Posts: 17

PostPosted: Sat Sep 14, 2019 11:18 am    Post subject: Reply with quote

n8V8r wrote:
I am wondering which path tar is trying to expand the compressed files to.


Code:
emerge-webrsync -x -v
provides the inisght

Quote:
++ stat -c %U:%G /var/db/repos/gentoo
+ ownership=root:root
+ type -P tarsync
+ do_tar /var/tmp/portage/webrsync-0LNECV/portage-20190913.tar.xz xf -
+ local file=/var/tmp/portage/webrsync-0LNECV/portage-20190913.tar.xz
+ shift
+ local decompressor
+ case ${file} in
+ decompressor=xzcat
+ xzcat /var/tmp/portage/webrsync-0LNECV/portage-20190913.tar.xz
+ tar xf -
tar: portage/sci-visualization/paraview/metadata.xml: Cannot open: No space left on device
tar: portage/sci-visualization/paraview/paraview-5.6.1.ebuild: Cannot open: No space left on device


Sill the df output for the specific path/directory(/var/db/repos/gentoo) appears to be ok

Code:
Filesystem                1K-blocks    Used Available Use% Mounted on
/dev/mapper/vg_main-lv_os   3030800 1833060   1024072  65% /

Code:
Filesystem                Inodes IUsed  IFree IUse% Mounted on
/dev/mapper/vg_main-lv_os 196608 53079 143529   27% /


Entirely baffling/inexplicable

[Moderator edit: changed [quote] tags to [code] tags to preserve output layout. Left one as quote to keep its bold active. -Hu]
Back to top
View user's profile Send private message
Prof. Frink
Apprentice
Apprentice


Joined: 07 Jan 2017
Posts: 176

PostPosted: Sat Sep 14, 2019 11:22 am    Post subject: Reply with quote

Hmm, still I have no good idea. But maybe emerge-webrsync is not executed by root and can therefore not write do the dir. And what does your repos.conf look like?
Back to top
View user's profile Send private message
Prof. Frink
Apprentice
Apprentice


Joined: 07 Jan 2017
Posts: 176

PostPosted: Sat Sep 14, 2019 11:24 am    Post subject: Reply with quote

And another question: Why do you use emerge-webrsync? Is emerge --sync not enough? The manual says:

Quote:
NOTE: The emerge-webrsync program will download the entire ebuild repository as a tarball, which is much faster than emerge --sync for first time syncs.
Back to top
View user's profile Send private message
freke
Guru
Guru


Joined: 23 Jan 2003
Posts: 491
Location: Somewhere in Denmark

PostPosted: Sat Sep 14, 2019 11:28 am    Post subject: Reply with quote

Prof. Frink wrote:
And another question: Why do you use emerge-webrsync? Is emerge --sync not enough? The manual says:

Quote:
NOTE: The emerge-webrsync program will download the entire ebuild repository as a tarball, which is much faster than emerge --sync for first time syncs.


This is a first time sync as I read OP?
Back to top
View user's profile Send private message
Prof. Frink
Apprentice
Apprentice


Joined: 07 Jan 2017
Posts: 176

PostPosted: Sat Sep 14, 2019 11:30 am    Post subject: Reply with quote

freke wrote:
[quote="Prof. Frink" ]

This is a first time sync as I read OP?


What is "OP"? :)
Back to top
View user's profile Send private message
n8V8r
n00b
n00b


Joined: 28 Jun 2018
Posts: 17

PostPosted: Sat Sep 14, 2019 11:34 am    Post subject: Reply with quote

Prof. Frink wrote:
But maybe emerge-webrsync is not executed by root and can therefore not write do the dir.


Checked, it runs as root, also tar, and root has write access
Code:

stat /var/db/repos/gentoo

Code:
  File: /var/db/repos/gentoo
  Size: 4096            Blocks: 8          IO Block: 4096   directory
Device: fc07h/64519d    Inode: 153361      Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)


[Moderator edit: changed [quote] tags to [code] tags to preserve output layout. -Hu]
Back to top
View user's profile Send private message
n8V8r
n00b
n00b


Joined: 28 Jun 2018
Posts: 17

PostPosted: Sat Sep 14, 2019 11:36 am    Post subject: Reply with quote

freke wrote:
This is a first time sync as I read OP?

indeed, it is
Back to top
View user's profile Send private message
n8V8r
n00b
n00b


Joined: 28 Jun 2018
Posts: 17

PostPosted: Sat Sep 14, 2019 11:44 am    Post subject: Reply with quote

Prof. Frink wrote:
And another question: Why do you use emerge-webrsync? Is emerge --sync not enough? The manual says:

Does not matter since producing similar errors, e.g.
Code:
rsync: recv_generator: mkdir "/var/db/repos/gentoo/.tmp-unverified-download-quarantine/sys-auth" failed: No space left on device

____
Prof. Frink wrote:
What is "OP"?

Original Post(er)

[Moderator edit: changed [quote] tags to [code] tags to preserve output layout. -Hu]
Back to top
View user's profile Send private message
Prof. Frink
Apprentice
Apprentice


Joined: 07 Jan 2017
Posts: 176

PostPosted: Sat Sep 14, 2019 11:53 am    Post subject: Reply with quote

n8V8r wrote:
Prof. Frink wrote:
And another question: Why do you use emerge-webrsync? Is emerge --sync not enough? The manual says:

Does not matter since producing similar errors, e.g.
Quote:
rsync: recv_generator: mkdir "/var/db/repos/gentoo/.tmp-unverified-download-quarantine/sys-auth" failed: No space left on device


What does manual execution of

Code:
mkdir "/var/db/repos/gentoo/.tmp-unverified-download-quarantine/sys-auth"


say?
Back to top
View user's profile Send private message
Prof. Frink
Apprentice
Apprentice


Joined: 07 Jan 2017
Posts: 176

PostPosted: Sat Sep 14, 2019 11:55 am    Post subject: Reply with quote

And if the path does not exists: Which dir is the first one in the path, which does not exist?
Back to top
View user's profile Send private message
n8V8r
n00b
n00b


Joined: 28 Jun 2018
Posts: 17

PostPosted: Sat Sep 14, 2019 12:00 pm    Post subject: Reply with quote

Prof. Frink wrote:
What does manual execution of

Code:
mkdir "/var/db/repos/gentoo/.tmp-unverified-download-quarantine/sys-auth"


say?


That is out of context since the parent dir .tmp-unverified-download-quarantine need to be created as well, anyway

Code:
mkdir -pv "/var/db/repos/gentoo/.tmp-unverified-download-quarantine/sys-auth"
from the cli produces
Code:
mkdir: created directory '/var/db/repos/gentoo/.tmp-unverified-download-quarantine'
mkdir: created directory '/var/db/repos/gentoo/.tmp-unverified-download-quarantine/sys-auth'


[Moderator edit: changed [quote] tags to [code] tags to preserve output layout. -Hu]
Back to top
View user's profile Send private message
Prof. Frink
Apprentice
Apprentice


Joined: 07 Jan 2017
Posts: 176

PostPosted: Sat Sep 14, 2019 12:02 pm    Post subject: Reply with quote

Okay, I do not have any ideas anymore.

Maybe what does emerge-webrsync say now (after you created the path)? And still: What is the output of

Code:
cat /etc/portage/repos.conf


or whatever your repos.conf-path is.
Back to top
View user's profile Send private message
freke
Guru
Guru


Joined: 23 Jan 2003
Posts: 491
Location: Somewhere in Denmark

PostPosted: Sat Sep 14, 2019 12:09 pm    Post subject: Reply with quote

What happens if you download a snapshot manually?

ie. from here https://gentoo.osuosl.org/releases/snapshots/current/

or go here for other mirrors: https://www.gentoo.org/downloads/mirrors/

and try and untar it?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14290

PostPosted: Sat Sep 14, 2019 4:36 pm    Post subject: Reply with quote

Based on the output provided, it looks to me like (1) there are insufficient inodes and (2) this is masked by the process cleaning up after itself. For my most recent sync:
Code:
$ find /usr/portage/ -print | wc
 163379  163379 8601681
Assuming no hard links, that means you need 163379 free inodes to store that tree. That exceeds the IFree shown for / in the first post. If that df -i output is collected at any time other than when tar has just failed, and no cleanup has been done, then it will show as free inodes that were used during the unpack. Perhaps more interestingly, if I truncate the find at the specified bad file, I get 143840 lines, which is suspiciously close to the IFree shown in the first post. Throw in that there may be a bit of variance between the tree I checked and the tree that the OP used, and it could line up.

OP: could you sync to a filesystem with more free inodes, as a test?
Back to top
View user's profile Send private message
n8V8r
n00b
n00b


Joined: 28 Jun 2018
Posts: 17

PostPosted: Sat Sep 14, 2019 7:24 pm    Post subject: Reply with quote

The issue seems with mounting LVM Logical Volumes for stacked mount points, e.g.

Code:
mount -vt ext4 LABEL=usr /mnt/gentoo/usr -o defaults,journal_checksum,nodev,relatime
mount -vt ext4 LABEL=usr_local /mnt/gentoo/usr/local -o defaults,journal_checksum,nodev,relatime


In which case /usr/local is not accessible and thus lacking the underlying space provided by the physical volume. Subsequently then the root volume is probably too small to proceed with the installation since lacking the necessary space.

Previously I did not use LVM and mounting physical volumes to stacked mount points, e.g.

Code:
/var
/var/log
/var/mail
/var/tmp
/var/spool
/var/usr


never been an issue. Not sure what makes LVM Logical Volumes different.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14290

PostPosted: Sat Sep 14, 2019 8:22 pm    Post subject: Reply with quote

LVM does not matter here. What does matter is the capacity of the filesystem receiving the writes. If you created a tiny filesystem on an LVM volume, then that filesystem will be tiny and unable to receive the tree. You would get the same problem if you made a tiny filesystem directly on a partition.
Back to top
View user's profile Send private message
n8V8r
n00b
n00b


Joined: 28 Jun 2018
Posts: 17

PostPosted: Sat Sep 14, 2019 8:33 pm    Post subject: Reply with quote

Surely concur about the size but that is not the matter really. The issue is that stacked mount points are not working with LVM the same way as it does with with physical volumes and that shrinks the available space if those logical volumes are not available/accessible.

Mounting e.g. physical volumes to

Code:
/var
/var/tmp


works as expcted, i.e. both directories are accessible. But mounting Logical Volumes exhibits differently, e.g. mounting in the aforementioned order /var/tmp would be inaccessible as mount point and mounting to /var/tmp fails. Reversing the mount order to

Code:
/var/tmp
/var


both LV getting mounted but any subdirectory of /var is not accessible (as in hidden).
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum