Problem with the FETCHCOMMAND

Message

galanom · Post by **galanom** » Mon Oct 24, 2011 6:47 pm

Hi!
I try to make my emerge use aria2c to enable segmented downloading.

My make.conf
GENTOO_MIRRORS="http://files.gentoo.gr http://ftp.ntua.gr/pub/linux/gentoo http://ftp.cc.uoc.gr/mirrors/linux/gentoo http://mirrors.linuxant.fr/distfiles.gentoo.org http://gentoo.wheel.sk "
FETCHCOMMAND="/usr/bin/aria2c -j5 --connect-timeout=5 -t5 --lowest-speed-limit=100K -x5 -d \${DISTDIR} -o \"\${FILE}\" ${GENTOO_MIRRORS// //\${FILE} }"
RESUMECOMMAND="/usr/bin/aria2c -c -j5 --connect-timeout=5 -t5 --lowest-speed-limit=100K -x5 -d \"\${DISTDIR}\" -o \"\${FILE}\" ${GENTOO_MIRRORS// //\${FILE} }"

Notice that in GENTOO_MIRRORS I did not escape the $, because it would not find the variable at all.

As for the trick with GENTOO_MIRRORS:

Code: Select all

localhost ~ # echo $U
mirror1 mirror2 mirror3 mirror4 mirror5 
localhost ~ # echo ${U// //file }
mirror1/file mirror2/file mirror3/file mirror4/file mirror5/file

I get this:

Code: Select all

>>> Emerging (1 of 1) x11-base/xorg-server-1.11.1-r1
!!! FETCHCOMMAND does not contain the required ${FILE} parameter.
!!! RESUMECOMMAND does not contain the required ${FILE} parameter.
!!! Refer to the make.conf(5) man page for information about how to
!!! correctly specify FETCHCOMMAND and RESUMECOMMAND.
>>> Downloading 'http://files.gentoo.gr/distfiles/xorg-server-1.11.1.tar.bz2'
Traceback (most recent call last):
  File "/usr/lib64/portage/pym/_emerge/EbuildFetcher.py", line 186, in _spawn
    allow_missing_digests=allow_missing):
  File "/usr/lib64/portage/pym/portage/package/ebuild/fetch.py", line 957, in fetch
    myret = _spawn_fetch(mysettings, myfetch)
  File "/usr/lib64/portage/pym/portage/package/ebuild/fetch.py", line 89, in _spawn_fetch
    rval = spawn_func(args, env=settings.environ(), **kwargs)
  File "/usr/lib64/portage/pym/portage/process.py", line 203, in spawn
    binary = mycommand[0]
IndexError: list index out of range
 * Fetch failed for 'x11-base/xorg-server-1.11.1-r1', Log file:
 *  '/var/log/portage/x11-base:xorg-server-1.11.1-r1:20111024-184210.log'

>>> Failed to emerge x11-base/xorg-server-1.11.1-r1, Log file:

>>>  '/var/log/portage/x11-base:xorg-server-1.11.1-r1:20111024-184210.log'

But my code DOES contain ${FILE}. If I remove the ${GENTOO_MIRRORS... var, then emerge accepts it, but aria2c fails as it does not know from where to download.

I could certainly make a shell script that accepts $FILE and $GENTOO_MIRRORS but I want to understand why my solution is wrong.

My system

Code: Select all

portage 2.1.10.30
python 2.7.2-r3, 3.1.3-r1, 3.2.2 (default)
bash 4.2p10
kernel 3.1.0-rc8 (git-sources)

mikegpitt · Post by **mikegpitt** » Mon Oct 24, 2011 7:00 pm

I'm guessing it must be related to some sort of parsing error. Take a look at your `emerge --info` to see what the command parses to (also post here). What do you actually want the final command to look like?

galanom · Post by **galanom** » Mon Oct 24, 2011 7:15 pm

Thank you.
First of all one correction: in FETCHCOMMAND should be ${GENTOO_MIRRORS// //${FILE} } (without escaping $ of FILE
But all remains the same

What I expect (run in bash):

Code: Select all

localhost galanom # GENTOO_MIRRORS="http://files.gentoo.gr http://ftp.ntua.gr/pub/linux/gentoo http://ftp.cc.uoc.gr/mirrors/linux/gentoo http://mirrors.linuxant.fr/distfiles.gentoo.org http://gentoo.wheel.sk "
localhost galanom # FETCHCOMMAND="/usr/bin/aria2c -j5 --connect-timeout=5 -t5 --lowest-speed-limit=100K -x5 -d \${DISTDIR} -o \"\${FILE}\" ${GENTOO_MIRRORS// //${FILE} }"
FILE=lala.tgz
localhost galanom # echo $FETCHCOMMAND
/usr/bin/aria2c -j5 --connect-timeout=5 -t5 --lowest-speed-limit=100K -x5 -d ${DISTDIR} -o "${FILE}" http://files.gentoo.gr/lala.tgz http://ftp.ntua.gr/pub/linux/gentoo/lala.tgz http://ftp.cc.uoc.gr/mirrors/linux/gentoo/lala.tgz http://mirrors.linuxant.fr/distfiles.gentoo.org/lala.tgz http://gentoo.wheel.sk/lala.tgz

My emerge --info

Code: Select all

Portage 2.1.10.30 (default/linux/amd64/10.0, gcc-4.6.1, glibc-2.13-r4, 3.1.0-rc8 x86_64)
=================================================================
System uname: Linux-3.1.0-rc8-x86_64-Intel-R-_Core-TM-2_Duo_CPU_T9600_@_2.80GHz-with-gentoo-2.1
Timestamp of tree: Sun, 23 Oct 2011 12:30:01 +0000
ccache version 3.1.6 [enabled]
app-shells/bash:          4.2_p10
dev-java/java-config:     2.1.11-r3
dev-lang/python:          2.6.6-r1, 2.7.2-r3, 3.1.3-r1, 3.2.2
dev-util/ccache:          3.1.6
dev-util/cmake:           2.8.6-r1
dev-util/pkgconfig:       0.26
sys-apps/baselayout:      2.1
sys-apps/openrc:          0.9.4
sys-apps/sandbox:         2.5
sys-devel/autoconf:       2.13, 2.68
sys-devel/automake:       1.9.6-r3, 1.10.3, 1.11.1-r1
sys-devel/binutils:       2.21.1-r1
sys-devel/gcc:            4.5.2, 4.6.1-r1
sys-devel/gcc-config:     1.4.1-r1
sys-devel/libtool:        2.4-r3
sys-devel/make:           3.82-r3
sys-kernel/linux-headers: 2.6.39 (virtual/os-headers)
sys-libs/glibc:           2.13-r4
Repositories:

gentoo
    location: /usr/portage
    sync: rsync://88.198.83.250/gentoo-portage
    priority: -1000

x-portage
    location: /usr/local/portage
    masters: gentoo
    priority: 0

ABI="amd64"
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
ACCEPT_PROPERTIES="*"
ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci"
ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol"
ANT_HOME="/usr/share/ant"
APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias"
ARCH="amd64"
AUTOCLEAN="yes"
CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump"
CAMERAS="ptp2"
CBUILD="x86_64-pc-linux-gnu"
CCACHE_DIR="/var/cache/ccache"
CCACHE_SIZE="2G"
CFLAGS="-O2 -march=native -pipe"
CFLAGS_x86="-m32"
CHOST="x86_64-pc-linux-gnu"
CHOST_amd64="x86_64-pc-linux-gnu"
CHOST_x86="i686-pc-linux-gnu"
CLEAN_DELAY="0"
COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog"
COLLISION_IGNORE="/lib/modules"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/openvpn/easy-rsa"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -march=native -pipe"
DEFAULT_ABI="amd64"
DISPLAY=":0.0"
DISTDIR="/usr/portage/distfiles"
EDITOR="/bin/nano"
ELIBC="glibc"
EMERGE_DEFAULT_OPTS="-v --exclude x11-terms/guake"
EMERGE_WARNING_DELAY="10"
EPREFIX=""
EROOT="/"
FCFLAGS=""
FEATURES="assume-digests binpkg-logs ccache distlocks ebuild-locks fixlafiles news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FETCHCOMMAND=""
FETCHCOMMAND_RSYNC="rsync -avP "${URI}" "${DISTDIR}/${FILE}""
FETCHCOMMAND_SFTP="bash -c "x=\${2#sftp://} ; host=\${x%%/*} ; port=\${host##*:} ; host=\${host%:*} ; [[ \${host} = \${port} ]] && port=22 ; exec sftp -P \${port} \"\${host}:/\${x#*/}\" \"\$1\"" sftp "${DISTDIR}/${FILE}" "${URI}""
FETCHCOMMAND_SSH="bash -c "x=\${2#ssh://} ; host=\${x%%/*} ; port=\${host##*:} ; host=\${host%:*} ; [[ \${host} = \${port} ]] && port=22 ; exec rsync --rsh=\"ssh -p\${port}\" -avP \"\${host}:/\${x#*/}\" \"\$1\"" rsync "${DISTDIR}/${FILE}" "${URI}""
FFLAGS=""
GCC_SPECS=""
GDK_USE_XFT="1"
GENTOO_MIRRORS="http://files.gentoo.gr http://ftp.ntua.gr/pub/linux/gentoo http://ftp.cc.uoc.gr/mirrors/linux/gentoo http://mirrors.linuxant.fr/distfiles.gentoo.org http://gentoo.wheel.sk "
GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx"
GRUB_PLATFORMS=""
GSETTINGS_BACKEND="gconf"
HISTFILE=""
HOME="/root"
INFOPATH="/usr/share/info:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.21.1/info:/usr/share/gcc-data/x86_64-pc-linux-gnu/4.6.1/info"
INPUT_DEVICES="evdev keyboard mouse synaptics"
JAVAC="/etc/java-config-2/current-system-vm/bin/javac"
JAVA_HOME="/etc/java-config-2/current-system-vm"
JDK_HOME="/etc/java-config-2/current-system-vm"
KERNEL="linux"
LANG="el_GR.UTF-8"
LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text"
LC_ADDRESS="el_GR.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_MEASUREMENT="el_GR.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="el_GR.UTF-8"
LC_NAME="el_GR.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_PAPER="el_GR.UTF-8"
LC_TELEPHONE="el_GR.UTF-8"
LC_TIME="en_DK.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LDFLAGS_x86="-m elf_i386"
LESS="-R -M --shift 5"
LESSOPEN="|lesspipe.sh %s"
LIBDIR_amd64="lib64"
LIBDIR_amd64_fbsd="lib64"
LIBDIR_ppc="lib32"
LIBDIR_ppc64="lib64"
LIBDIR_s390="lib32"
LIBDIR_s390x="lib64"
LIBDIR_sparc32="lib32"
LIBDIR_sparc64="lib64"
LIBDIR_x86="lib32"
LIBDIR_x86_fbsd="lib32"
LOGNAME="root"
LS_COLORS="rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.pdf=00;32:*.ps=00;32:*.txt=00;32:*.patch=00;32:*.diff=00;32:*.log=00;32:*.tex=00;32:*.doc=00;32:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:"
MAIL="/var/mail/root"
MAKEOPTS="-j3"
MANPATH="/etc/java-config-2/current-system-vm/man:/usr/local/share/man:/usr/share/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.21.1/man:/usr/share/gcc-data/x86_64-pc-linux-gnu/4.6.1/man:/etc/java-config/system-vm/man/:/usr/lib64/php5.3/man/"
MULTILIB_ABIS="amd64 x86"
MULTILIB_STRICT_DENY="64-bit.*shared object"
MULTILIB_STRICT_DIRS="/lib32 /lib /usr/lib32 /usr/lib /usr/kde/*/lib32 /usr/kde/*/lib /usr/qt/*/lib32 /usr/qt/*/lib /usr/X11R6/lib32 /usr/X11R6/lib"
MULTILIB_STRICT_EXEMPT="(perl5|gcc|gcc-lib|binutils|eclipse-3|debug|portage|udev)"
NETBEANS="apisupport cnd groovy gsf harness ide identity j2ee java mobility nb php profiler soa visualweb webcommon websvccommon xml"
NOCOLOR="true"
OPENGL_PROFILE="xorg-x11"
PACKAGE_MANAGER="portage"
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/4.6.1"
PHP_TARGETS="php5-3"
PKGDIR="/usr/portage/packages"
PORTAGE_ARCHLIST="ppc sparc64-freebsd ppc-openbsd x86-openbsd ppc64 x86-winnt x86-fbsd ppc-aix alpha arm x86-freebsd s390 amd64 arm-linux x86-macos x64-openbsd ia64-hpux hppa x86-netbsd x86-cygwin amd64-linux ia64-linux x86 sparc-solaris x64-freebsd sparc64-solaris x86-linux x64-macos sparc m68k-mint ia64 mips ppc-macos x86-interix hppa-hpux amd64-fbsd x64-solaris mips-irix m68k sh x86-solaris sparc-fbsd"
PORTAGE_BINHOST_CHUNKSIZE="3000"
PORTAGE_BIN_PATH="/usr/lib64/portage/bin"
PORTAGE_COMPRESS_EXCLUDE_SUFFIXES="css gif htm[l]? jp[e]?g js pdf png"
PORTAGE_CONFIGROOT="/"
PORTAGE_DEBUG="0"
PORTAGE_DEPCACHEDIR="/var/cache/edb/dep"
PORTAGE_ELOG_CLASSES="warn error log"
PORTAGE_ELOG_MAILFROM="portage@localhost"
PORTAGE_ELOG_MAILSUBJECT="[portage] ebuild log for ${PACKAGE} on ${HOST}"
PORTAGE_ELOG_MAILURI="root"
PORTAGE_ELOG_SYSTEM="save"
PORTAGE_FETCH_CHECKSUM_TRY_MIRRORS="5"
PORTAGE_FETCH_RESUME_MIN_SIZE="1M"
PORTAGE_GID="250"
PORTAGE_GPG_SIGNING_COMMAND="gpg --sign --clearsign --yes --default-key "${PORTAGE_GPG_KEY}" --homedir "${PORTAGE_GPG_DIR}" "${FILE}""
PORTAGE_INST_GID="0"
PORTAGE_INST_UID="0"
PORTAGE_NICENESS="19"
PORTAGE_PYM_PATH="/usr/lib64/portage/pym"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_RSYNC_RETRIES="-1"
PORTAGE_SANDBOX_COMPAT_LEVEL="16"
PORTAGE_SYNC_STALE="30"
PORTAGE_TMPDIR="/var/tmp"
PORTAGE_VERBOSE="1"
PORTAGE_WORKDIR_MODE="0700"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
PORT_LOGDIR="/var/log/portage"
PORT_LOGDIR_CLEAN="find "${PORT_LOGDIR}" -type f ! -name "summary.log*" -mtime +7 -delete"
PRELINK_PATH_MASK="/usr/lib64/libfreebl3.so:/usr/lib64/libnssdbm3.so:/usr/lib64/libsoftokn3.so"
PROFILE_ONLY_VARIABLES="ARCH ELIBC KERNEL USERLAND"
PWD="/home/galanom"
PYTHONDONTWRITEBYTECODE="1"
QT_PLUGIN_PATH="/usr/lib64/kde4/plugins"
RESUMECOMMAND=""
RESUMECOMMAND_RSYNC="rsync -avP "${URI}" "${DISTDIR}/${FILE}""
RESUMECOMMAND_SSH="bash -c "x=\${2#ssh://} ; host=\${x%%/*} ; port=\${host##*:} ; host=\${host%:*} ; [[ \${host} = \${port} ]] && port=22 ; exec rsync --rsh=\"ssh -p\${port}\" -avP \"\${host}:/\${x#*/}\" \"\$1\"" rsync "${DISTDIR}/${FILE}" "${URI}""
ROOT="/"
ROOTPATH="/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/4.6.1"
RPMDIR="/usr/portage/rpm"
RUBYOPT="-rauto_gem"
RUBY_TARGETS="ruby18"
R_HOME="/usr/lib64/R"
SHELL="/bin/bash"
SHLVL="1"
STAGE1_USE="multilib nptl nptlonly unicode"
SUDO_COMMAND="/bin/su"
SUDO_GID="100"
SUDO_UID="10000"
SUDO_USER="galanom"
SYMLINK_LIB="yes"
SYNC="rsync://88.198.83.250/gentoo-portage"
TERM="xterm"
USE="X aalib acl acpi aiglx aim alsa amd64 avahi bash-completion berkdb bzip2 cdr cjk cli cracklib crypt cups cxx dbus djvu dri dvd dvdr dvi firefox fortran g110 g15 gdbm gif gnutls gpm gtk iconv icq imagemagick imlib ipv6 irc jabber java jingle jpeg jpeg2k lame latex latex3 lcd libcaca lm_sensors mad mmx modules mp3 mplayer msn mudflap multilib ncurses networkmanager nls nptl nptlonly nsplugin ogg opengl openmp pam pcre pdf png ppds pppd python3 rdesktop readline sdl semantic-desktop session sip skype slang smp spell sse sse2 sse3 sse4_1 ssl ssse3 startup-notification static-ppds svg sysfs tcpd threads thunar tiff truetype udev unicode usb vorbis xcb xcomposite xinetd xml xmpp xorg xvmc yahoo zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev keyboard mouse synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
USER="root"
USERLAND="GNU"
USERNAME="root"
USE_EXPAND="ALSA_CARDS ALSA_PCM_PLUGINS APACHE2_MODULES APACHE2_MPMS CALLIGRA_FEATURES CAMERAS COLLECTD_PLUGINS CROSSCOMPILE_OPTS DRACUT_MODULES DVB_CARDS ELIBC FCDSL_CARDS FOO2ZJS_DEVICES FRITZCAPI_CARDS GPSD_PROTOCOLS GRUB_PLATFORMS INPUT_DEVICES KERNEL LCD_DEVICES LINGUAS LIRC_DEVICES MISDN_CARDS NETBEANS_MODULES NGINX_MODULES_HTTP NGINX_MODULES_MAIL OFED_DRIVERS PHP_TARGETS QEMU_SOFTMMU_TARGETS QEMU_USER_TARGETS RUBY_TARGETS SANE_BACKENDS USERLAND VIDEO_CARDS XFCE_PLUGINS XTABLES_ADDONS"
USE_EXPAND_HIDDEN="CROSSCOMPILE_OPTS ELIBC KERNEL USERLAND"
USE_ORDER="env:pkg:conf:defaults:pkginternal:repo:env.d"
VIDEO_CARDS="nouveau"
XAUTHORITY="/root/.xauth71Tbk2"
XDG_CONFIG_DIRS="/etc/xdg"
XDG_DATA_DIRS="/usr/local/share:/usr/share:/usr/share/gdm"
XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
_="/usr/bin/emerge"

Strange:

Code: Select all

localhost galanom # emerge --info |grep COMMAND
FETCHCOMMAND=""
FETCHCOMMAND_RSYNC="rsync -avP "${URI}" "${DISTDIR}/${FILE}""
FETCHCOMMAND_SFTP="bash -c "x=\${2#sftp://} ; host=\${x%%/*} ; port=\${host##*:} ; host=\${host%:*} ; [[ \${host} = \${port} ]] && port=22 ; exec sftp -P \${port} \"\${host}:/\${x#*/}\" \"\$1\"" sftp "${DISTDIR}/${FILE}" "${URI}""
FETCHCOMMAND_SSH="bash -c "x=\${2#ssh://} ; host=\${x%%/*} ; port=\${host##*:} ; host=\${host%:*} ; [[ \${host} = \${port} ]] && port=22 ; exec rsync --rsh=\"ssh -p\${port}\" -avP \"\${host}:/\${x#*/}\" \"\$1\"" rsync "${DISTDIR}/${FILE}" "${URI}""
PORTAGE_GPG_SIGNING_COMMAND="gpg --sign --clearsign --yes --default-key "${PORTAGE_GPG_KEY}" --homedir "${PORTAGE_GPG_DIR}" "${FILE}""
RESUMECOMMAND=""
RESUMECOMMAND_RSYNC="rsync -avP "${URI}" "${DISTDIR}/${FILE}""
RESUMECOMMAND_SSH="bash -c "x=\${2#ssh://} ; host=\${x%%/*} ; port=\${host##*:} ; host=\${host%:*} ; [[ \${host} = \${port} ]] && port=22 ; exec rsync --rsh=\"ssh -p\${port}\" -avP \"\${host}:/\${x#*/}\" \"\$1\"" rsync "${DISTDIR}/${FILE}" "${URI}""
SUDO_COMMAND="/bin/su"
localhost galanom #

Look at the FETCHCOMMAND and RESUMECOMMAND, they're empty! I didn't do that!

mikegpitt · Post by **mikegpitt** » Mon Oct 24, 2011 8:18 pm

I also noticed that the FETCHCOMMAND var is blank in your `emerge --info`. Have you placed the revised FETCHCOMMAND variable in your /etc/make.conf?

Can you post your entire /etc/make.conf?

EDIT: How about try this version of the command?

Code: Select all

FETCHCOMMAND="/usr/bin/aria2c -j5 --connect-timeout=5 -t5 --lowest-speed-limit=100K -x5 -d \${DISTDIR} -o \${FILE} ${GENTOO_MIRRORS// //\$\{FILE\} }"

galanom · Post by **galanom** » Mon Oct 24, 2011 8:38 pm

Hi!

It does the same thing...
Here is my /etc/make.conf
You may notice different FETCHCOMMAND and RESUMECOMMAND. Both don't work, I'm just playing.
I also tried to use sed, gain the same.

Code: Select all

# These settings were set by the catalyst build script that automatically built this stage
# Please consult /etc/make.conf.example for a more detailed example

CCACHE_DIR="/var/cache/ccache"
PORT_LOGDIR="/var/log/portage"


CFLAGS="-O2 -march=native -pipe"
CHOST="x86_64-pc-linux-gnu"
CXXFLAGS="${CFLAGS}"
#CHECKREQS_ACTION="error"
GENTOO_MIRRORS="http://files.gentoo.gr http://ftp.ntua.gr/pub/linux/gentoo http://ftp.cc.uoc.gr/mirrors/linux/gentoo http://mirrors.linuxant.fr/distfiles.gentoo.org http://gentoo.wheel.sk "
SYNC="rsync://88.198.83.250/gentoo-portage"
PORTAGE_NICENESS="19"
###FETCHCOMMAND="/usr/bin/curl -Y 50000 -y 10 --connect-timeout 10 -f --output \${DISTDIR}/\${FILE} \${URI}"
###RESUMECOMMAND="/usr/bin/curl -Y 50000 -y 10 -C - --connect-timeout 10 -f --output \${DISTDIR}/\${FILE} \${URI}"
DISTDIR=/usr/portage/distfiles


FETCHCOMMAND="/usr/bin/aria2c -j5 --connect-timeout=5 -t5 --lowest-speed-limit=100K -x5 -d \${DISTDIR} -o \${FILE} ${GENTOO_MIRRORS// //\$\{FILE\} }"

RESUMECOMMAND="/usr/bin/aria2c -c -j5 --connect-timeout=5 -t5 --lowest-speed-limit=100K -x5 -d \"\${DISTDIR}\" -o \"\${FILE}\" \\\${GENTOO_MIRRORS// //\${FILE} }"

PORTAGE_FETCH_RESUME_MIN_SIZE="1M"
CLEAN_DELAY="0"

VIDEO_CARDS="nouveau"
ALSA_CARDS=""
INPUT_DEVICES="evdev keyboard mouse synaptics"

CLEAN_DELAY="0"

MAKEOPTS="-j3"

LINGUAS=""

CCACHE_SIZE="2G"
EMERGE_DEFAULT_OPTS="-v --exclude x11-terms/guake"

ACCEPT_KEYWORDS="~amd64"
ACCEPT_LICENSE="*"

# This sets what to log
PORTAGE_ELOG_CLASSES="warn error log"
# And this is how to do it
PORTAGE_ELOG_SYSTEM="save"


PORTDIR_OVERLAY="/usr/local/portage"
#source /var/lib/layman/make.conf

FEATURES="parallel-fetch ccache"

# Hardware 
USE="mmx sse sse2 sse3 ssse3 sse4_1 acpi -32bit usb g15 g110 lcd -nvidia xvmc dri cdr dvdr lm_sensors"

# System
USE="${USE} udev -hal ppds static-ppds dbus nptl smp threads cups"

# Multimedia
USE="${USE} -oss -arts -esd lame dvd dvdr alsa mad mp3 ogg vorbis opengl mplayer"
# Image
USE="${USE} imlib imagemagick svg tiff png jpeg jpeg2k gif"

# Graphics Libraries
USE="${USE} sdl aalib libcaca -system-libCg"

# X Windows
USE="${USE} -accessibility X -gnome -kde -qt -qt3 -qt3support -qt4 semantic-desktop gtk -tk -tcltk xcomposite aiglx"

# Net
USE="${USE} ipv6 avahi xinetd xml networkmanager firefox rdesktop"

# Text Processing
USE="${USE} -tetex unicode cjk truetype spell latex latex3 djvu pdf dvi readline gpm ncurses slang"

# Various
USE="${USE} -doc -handbook thunar java python3 startup-notification bash-completion xcb -symlink nsplugin gnutls irc jabber jingle xmpp skype icq msn yahoo aim sip"

galanom · Post by **galanom** » Mon Oct 24, 2011 9:11 pm

Edited line binary = mycommand[0] (/usr/lib64/portage/pym/portage/process.py:203)
and added print(mycommand)

With defaults, it prints: ['/usr/bin/wget', '-t', '3', '-T', '60', '--passive-ftp', '-O', '/usr/rep/gentoo/distfiles/xorg-server-1.11.1.tar.bz2', 'http://files.gentoo.gr/distfiles/xorg-s ... .1.tar.bz2']
while with mine it is empty []

Genone · Post by **Genone** » Mon Oct 24, 2011 9:30 pm

make.conf does not support bash variable expansions other than the basic ${foo} syntax.

Post by **floppymaster** » Mon Oct 24, 2011 9:37 pm

I would just throw it into a shell script.

Code: Select all

#!/bin/bash

set -e

ACTION=$1
DISTDIR=$2
FILE=$3
URI=$4

MIRRORS="http://distfiles.gentoo.org/ http://lug.mtu.edu/gentoo/"

SOURCES=
for x in $MIRRORS; do
		SOURCES+=" ${x}distfiles/$FILE"
done

if [[ $ACTION == resume ]]; then
		aria2c -c -d "$DISTDIR" -o $FILE $SOURCES
else
		aria2c -d "$DISTDIR" -o $FILE $SOURCES
fi

Code: Select all

FETCHCOMMAND="/usr/local/bin/aria2-fetch fetch \"\${DISTDIR}\" \${FILE} \${URI}"
RESUMECOMMAND="/usr/local/bin/aria2-fetch resume \"\${DISTDIR}\" \${FILE} \${URI}"

Post by **floppymaster** » Mon Oct 24, 2011 9:45 pm

You should also add some code to fall back to fetching from $URI if aria2c fails to grab it from the mirrors.

galanom · Post by **galanom** » Mon Oct 24, 2011 9:52 pm

Ah, ok! Thank you guys!