| View previous topic :: View next topic |
| Author |
Message |
janbielanski
n00b
Joined: 06 Jul 2019
Posts: 32
Location: Krakow
|
 Posted: Sun Jun 27, 2021 8:36 am Post subject: tensorflow - build problem |
 |
|
I had another build problem, this time with sci-libs/tensorflow-2.5.0 on new computer HP Zbook Fury G7.
My system specification are:
| Code: |
CPU Model: Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz
|
GPU with CUDA version
| Code: |
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.42.01 Driver Version: 470.42.01 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Quadro RTX 4000... Off | 00000000:01:00.0 Off | N/A |
| N/A 57C P8 7W / N/A | 215MiB / 7982MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 4020 G /usr/bin/X 214MiB |
+-----------------------------------------------------------------------------+
* Setting CUDA version: 11.2
* Setting CUDNN version: 8.2
You have bazel 3.7.2- (@non-git) installed.
Found CUDA 11.2 in:
/opt/cuda/targets/x86_64-linux/lib
/opt/cuda/targets/x86_64-linux/include
Found cuDNN 8 in:
/opt/cuda/targets/x86_64-linux/lib
/opt/cuda/targets/x86_64-linux/include
|
GCC
| Code: |
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-pc-linux-gnu/10.3.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /var/tmp/portage/sys-devel/gcc-10.3.0/work/gcc-10.3.0/configure --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/10.3.0 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/10.3.0/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/10.3.0 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/10.3.0/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/10.3.0/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/10.3.0/include/g++-v10 --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/10.3.0/python --enable-languages=c,c++,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 10.3.0 p1' --disable-esp --enable-libstdcxx-time --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-multilib --with-multilib-list=m32,m64 --disable-fixed-point --enable-targets=all --enable-libgomp --disable-libssp --disable-libada --disable-systemtap --disable-vtable-verify --disable-libvtv --without-zstd --enable-lto --without-isl --enable-default-pie --enable-default-ssp
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 10.3.0 (Gentoo 10.3.0 p1)
|
flags for sci-libs/tensorflow
| Code: |
[ Legend : U - final flag setting for installation]
[ : I - package is installed with flag ]
[ Colors : set, unset ]
* Found these USE flags for sci-libs/tensorflow-2.5.0:
U I
+ + cpu_flags_x86_avx : Adds support for Advanced Vector Extensions instructions
+ + cpu_flags_x86_avx2 : Adds support for Advanced Vector Extensions 2 instructions
+ + cpu_flags_x86_fma3 : Use the Fused Multiply Add 3 instruction set ([fma] in cpuinfo)
- - cpu_flags_x86_fma4 : Use the Fused Multiply Add 4 instruction set
+ + cpu_flags_x86_sse : Use the SSE instruction set
+ + cpu_flags_x86_sse2 : Use the SSE2 instruction set
+ + cpu_flags_x86_sse3 : Use the SSE3 instruction set ([pni] in cpuinfo, NOT ssse3)
+ + cpu_flags_x86_sse4_1 : Enable SSE4.1 instruction support
+ + cpu_flags_x86_sse4_2 : Enable SSE4.2 instruction support
+ - cuda : Enable support for nVidia CUDA
- - mpi : Add MPI (Message Passing Interface) layer to the apps that support it
+ + python : Add optional support/bindings for the Python language
+ + python_targets_python3_8 : Build with Python 3.8
+ - xla : XLA (Accelerated Linear Algebra) Optimizing Compiler for TensorFlow
|
Portage make.conf
| Code: |
# These settings were set by the catalyst build script that automatically
# built this stage.
# Please consult /usr/share/portage/config/make.conf.example for a more
# detailed example.
COMMON_FLAGS="-march=skylake -O2 -pipe"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"
MAKEOPTS="-j5"
# NOTE: This stage was built with the bindist Use flag enabled
PORTDIR="/var/db/repos/gentoo"
DISTDIR="/var/cache/distfiles"
PKGDIR="/var/cache/binpkgs"
# These are the USE flags that were used in addition to what is provided by the
# profile used for building.
USE="bash-completion unicode udev udisks icu posix policykit lzma alsa pulseaudio ogg openal gstreamer ffmpeg threads openmp opencl python qt5 qt4 X"
# CPU FLAGS
# AVX - Advanced Vector Extensions
# AVX2 - Advanced Vector Extensions 2
# AES - Advanced Encryption Standard
# F16C - Half-Precision instructions
# MMX - single instruction, multiple data
# MMXEXT - single instruction, multiple data extended instruction set
# SSE / SSE2 / SSE3 / SSE4.1 / SSE4.2 / SSSE3 - single instruction, multiple data
CPU_FLAGS_X86="aes avx avx2 f16c mmx mmxext sse sse2 sse3 sse4_1 sse4_2 ssse3 fma fma3"
# Python
USE_PYTHON="2.7 3.8 3.9"
PYTHON_TARGETS="python2_7 python3_8 python3_9"
PYTHON_SINGLE_TARGET="python3_9"
# CUDA capability
TF_CUDA_COMPUTE_CAPABILITIES="7.5"
# This sets the language of build output to English.
# Please keep this setting intact when reporting bugs.
LC_MESSAGES=C
# Servers
GENTOO_MIRRORS="http://ftp.vectranet.pl/gentoo/"
# Languages
LINGUAS="pl"
L10N="pl"
# EFI
GRUB_PLATFORMS="emu efi-32 efi-64 pc"
# X DRIVERS/VIDEO/INPUT
VIDEO_CARDS="intel i965 iris nvidia"
INPUT_DEVICES="evdev synaptics libinput wacom"
|
For following configuration i got this errors:
| Code: |
^M^M^M^M^M^M[11,207 / 19,406] 5 actions running
Compiling tensorflow/core/kernels/image/colorspace_op_gpu.cu.cc; 9s local
Compiling tensorflow/core/kernels/concat_lib_gpu_impl.cu.cc; 8s local
Compiling tensorflow/core/kernels/fill_functor.cu.cc; 5s local
Compiling tensorflow/core/kernels/fill_functor.cc; 2s local
Compiling tensorflow/core/kernels/concat_lib_gpu.cc; 1s local
^M^M^M^M^M^MERROR: /var/tmp/portage/sci-libs/tensorflow-2.5.0/work/tensorflow-2.5.0-python3_8/tensorflow/core/kernels/image/BUILD:191:18: C++ compilation of rule '//tensorflow/core/kernels/image:colorspace_op_gpu' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command
(cd /var/tmp/portage/sci-libs/tensorflow-2.5.0/work/tensorflow-2.5.0-python3_8-bazel-base/execroot/org_tensorflow && \
exec env - \
CUDA_TOOLKIT_PATH=/opt/cuda \
GCC_HOST_COMPILER_PATH=/usr/x86_64-pc-linux-gnu/gcc-bin/10.3.0/x86_64-pc-linux-gnu-gcc \
HOME=/var/tmp/portage/sci-libs/tensorflow-2.5.0/homedir \
KERAS_HOME=/var/tmp/portage/sci-libs/tensorflow-2.5.0/temp/.keras \
PATH=/var/tmp/portage/sci-libs/tensorflow-2.5.0/temp/python3.8/bin:/usr/lib/portage/python3.9/ebuild-helpers/xattr:/usr/lib/portage/python3.9/ebuild-helpers:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/lib/llvm/12/bin:/usr/lib/llvm/11/bin:/usr/lib/llvm/10/bin:/opt/cuda/bin \
PWD=/proc/self/cwd \
PYTHON_BIN_PATH=/usr/bin/python3.8 \
PYTHON_LIB_PATH=/usr/lib/python3.8/site-packages \
TF2_BEHAVIOR=1 \
TF_CUDA_COMPUTE_CAPABILITIES=7.5 \
TF_CUDA_PATHS=/opt/cuda \
TF_CUDA_VERSION=11.2 \
TF_CUDNN_VERSION=8.2 \
TF_SYSTEM_LIBS=absl_py,astor_archive,astunparse_archive,boringssl,com_github_googlecloudplatform_google_cloud_cpp,com_github_grpc_grpc,com_google_protobuf,curl,cython,dill_archive,double_conversion,enum34_archive,flatbuffers,functools32_archive,gast_archive,gif,hwloc,icu,jsoncpp_git,libjpeg_turbo,lmdb,nasm,nsync,opt_einsum_archive,org_sqlite,pasta,pcre,png,pybind11,six_archive,snappy,tblib_archive,termcolor_archive,typing_extensions_archive,wrapt,zlib \
external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -MD -MF bazel-out/k8-opt/bin/tensorflow/core/kernels/image/_objs/colorspace_op_gpu/colorspace_op_gpu.cu.pic.d '-frandom-seed=bazel-out/k8-opt/bin/tensorflow/core/kernels/image/_objs/colorspace_op_gpu/colorspace_op_gpu.cu.pic.o' -DEIGEN_MPL2_ONLY '-DEIGEN_MAX_ALIGN_BYTES=64' -iquote . -iquote bazel-out/k8-opt/bin -iquote external/eigen_archive -iquote bazel-out/k8-opt/bin/external/eigen_archive -iquote external/com_google_absl -iquote bazel-out/k8-opt/bin/external/com_google_absl -iquote external/nsync -iquote bazel-out/k8-opt/bin/external/nsync -iquote external/gif -iquote bazel-out/k8-opt/bin/external/gif -iquote external/libjpeg_turbo -iquote bazel-out/k8-opt/bin/external/libjpeg_turbo -iquote external/com_google_protobuf -iquote bazel-out/k8-opt/bin/external/com_google_protobuf -iquote external/com_googlesource_code_re2 -iquote bazel-out/k8-opt/bin/external/com_googlesource_code_re2 -iquote external/farmhash_archive -iquote bazel-out/k8-opt/bin/external/farmhash_archive -iquote external/fft2d -iquote bazel-out/k8-opt/bin/external/fft2d -iquote external/highwayhash -iquote bazel-out/k8-opt/bin/external/highwayhash -iquote external/zlib -iquote bazel-out/k8-opt/bin/external/zlib -iquote external/local_config_cuda -iquote bazel-out/k8-opt/bin/external/local_config_cuda -iquote external/local_config_rocm -iquote bazel-out/k8-opt/bin/external/local_config_rocm -iquote external/local_config_tensorrt -iquote bazel-out/k8-opt/bin/external/local_config_tensorrt -iquote external/png -iquote bazel-out/k8-opt/bin/external/png -Ibazel-out/k8-opt/bin/external/local_config_cuda/cuda/_virtual_includes/cuda_headers_virtual -Ibazel-out/k8-opt/bin/external/local_config_tensorrt/_virtual_includes/tensorrt_headers -isystem third_party/eigen3/mkl_include -isystem bazel-out/k8-opt/bin/third_party/eigen3/mkl_include -isystem external/eigen_archive -isystem bazel-out/k8-opt/bin/external/eigen_archive -isystem external/farmhash_archive/src -isystem bazel-out/k8-opt/bin/external/farmhash_archive/src -isystem external/local_config_cuda/cuda -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda -isystem external/local_config_cuda/cuda/cuda/include -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda/cuda/include -isystem external/local_config_rocm/rocm -isystem bazel-out/k8-opt/bin/external/local_config_rocm/rocm -isystem external/local_config_rocm/rocm/rocm/include -isystem bazel-out/k8-opt/bin/external/local_config_rocm/rocm/rocm/include -isystem external/local_config_rocm/rocm/rocm/include/rocrand -isystem bazel-out/k8-opt/bin/external/local_config_rocm/rocm/rocm/include/rocrand -isystem external/local_config_rocm/rocm/rocm/include/roctracer -isystem bazel-out/k8-opt/bin/external/local_config_rocm/rocm/rocm/include/roctracer -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -fPIC -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -fno-omit-frame-pointer -no-canonical-prefixes -fno-canonical-system-headers -DNDEBUG -g0 -O2 -ffunction-sections -fdata-sections -w -DAUTOLOAD_DYNAMIC_KERNELS -I/usr/include/jsoncpp '-std=c++14' '-march=skylake' -O2 -pipe -msse -msse2 -msse3 -msse4.1 -msse4.2 -mavx -mavx2 -mfma -x cuda '-DGOOGLE_CUDA=1' '-Xcuda-fatbinary=--compress-all' '--no-cuda-include-ptx=all' '--cuda-include-ptx=sm_75' '--cuda-gpu-arch=sm_75' -DEIGEN_AVOID_STL_ARRAY -Iexternal/gemmlowp -Wno-sign-compare '-ftemplate-depth=900' -fno-exceptions '-DGOOGLE_CUDA=1' '-DTENSORFLOW_USE_NVCC=1' '-DTENSORFLOW_USE_XLA=1' -DINTEL_MKL -msse3 -pthread '-nvcc_options=relaxed-constexpr' '-nvcc_options=ftz=true' -c tensorflow/core/kernels/image/colorspace_op_gpu.cu.cc -o bazel-out/k8-opt/bin/tensorflow/core/kernels/image/_objs/colorspace_op_gpu/colorspace_op_gpu.cu.pic.o)
Execution platform: @local_execution_config_platform//:platform
[11,208 / 19,406] 4 actions running
Compiling tensorflow/core/kernels/concat_lib_gpu_impl.cu.cc; 9s local
Compiling tensorflow/core/kernels/fill_functor.cu.cc; 6s local
Compiling tensorflow/core/kernels/fill_functor.cc; 3s local
Compiling tensorflow/core/kernels/concat_lib_gpu.cc; 2s local
^M^M^M^M^M/usr/lib/gcc/x86_64-pc-linux-gnu/10.3.0/include/g++-v10/chrono: In substitution of ‘template<class _Rep, class _Period> template<class _Period2> using __is_harmonic = std::__bool_constant<(std::ratio<((_Period2::num / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::num, _Period::num)) * (_Period::den / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::den, _Period::den))), ((_Period2::den / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::den, _Period::den)) * (_Period::num / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::num, _Period::num)))>::den == 1)> [with _Period2 = _Period2; _Rep = _Rep; _Period = _Period]’:
/usr/lib/gcc/x86_64-pc-linux-gnu/10.3.0/include/g++-v10/chrono:473:154: required from here
/usr/lib/gcc/x86_64-pc-linux-gnu/10.3.0/include/g++-v10/chrono:428:27: internal compiler error: Segmentation fault
428 | _S_gcd(intmax_t __m, intmax_t __n) noexcept
| ^~~~~~
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://bugs.gentoo.org/> for instructions.
[11,208 / 19,406] 4 actions running
Compiling tensorflow/core/kernels/concat_lib_gpu_impl.cu.cc; 9s local
Compiling tensorflow/core/kernels/fill_functor.cu.cc; 6s local
Compiling tensorflow/core/kernels/fill_functor.cc; 3s local
Compiling tensorflow/core/kernels/concat_lib_gpu.cc; 2s local
^M^M^M^M^MINFO: Elapsed time: 2355.612s, Critical Path: 103.10s
[11,212 / 19,406] checking cached actions
^MINFO: 11212 processes: 5949 internal, 5263 local.
[11,212 / 19,406] checking cached actions
^MFAILED: Build did NOT complete successfully
^MFAILED: Build did NOT complete successfully
* ERROR: sci-libs/tensorflow-2.5.0::gentoo failed (compile phase):
* ebazel failed
*
* Call stack:
* ebuild.sh, line 127: Called src_compile
* environment, line 4212: Called ebazel 'build' '//tensorflow:libtensorflow_framework.so' '//tensorflow:libtensorflow.so'
* environment, line 2507: Called die
* The specific snippet of code:
* "${@}" || die "ebazel failed"
*
* If you need support, post the output of `emerge --info '=sci-libs/tensorflow-2.5.0::gentoo'`,
* the complete build log and the output of `emerge -pqv '=sci-libs/tensorflow-2.5.0::gentoo'`.
* The complete build log is located at '/var/tmp/portage/sci-libs/tensorflow-2.5.0/temp/build.log'.
* The ebuild environment file is located at '/var/tmp/portage/sci-libs/tensorflow-2.5.0/temp/environment'.
* Working directory: '/var/tmp/portage/sci-libs/tensorflow-2.5.0/work/tensorflow-2.5.0-python3_8'
* S: '/var/tmp/portage/sci-libs/tensorflow-2.5.0/work/tensorflow-2.5.0'
|
First error is related to crosstool_wrapper_driver:
| Code: |
C++ compilation of rule '//tensorflow/core/kernels/image:colorspace_op_gpu' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command
|
Second error is related to chrono and make a segmentation fault:
| Code: |
^M^M^M^M^M/usr/lib/gcc/x86_64-pc-linux-gnu/10.3.0/include/g++-v10/chrono: In substitution of ‘template<class _Rep, class _Period> template<class _Period2> using __is_harmonic = std::__bool_constant<(std::ratio<((_Period2::num / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::num, _Period::num)) * (_Period::den / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::den, _Period::den))), ((_Period2::den / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::den, _Period::den)) * (_Period::num / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::num, _Period::num)))>::den == 1)> [with _Period2 = _Period2; _Rep = _Rep; _Period = _Period]’:
/usr/lib/gcc/x86_64-pc-linux-gnu/10.3.0/include/g++-v10/chrono:473:154: required from here
/usr/lib/gcc/x86_64-pc-linux-gnu/10.3.0/include/g++-v10/chrono:428:27: internal compiler error: Segmentation fault
428 | _S_gcd(intmax_t __m, intmax_t __n) noexcept
|
|
|
| Back to top |
|
 |
alamahant
Advocate
Joined: 23 Mar 2019
Posts: 3882
|
| Posted: Sun Jun 27, 2021 8:59 am Post subject: |
|
|
Not related probably but
according to
https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
your skylake supports
| Code: |
movbe mmx sse sse2 sse3 ssse3 sse4_1 sse4_2 popcnt avx avx2 aes pclmul fsgsbase rdrnd fma bmi bmi2 f16c rdseed adcx prefetchw clflushopt xsavec xsaves
|
Try with these?
Maybe it helps?
OR try
| Code: |
USE="-cpu_flags_x86_fma3" emerge -av tensorflow
|
IN FACT according to the above gcc link,
gcc only supports
There is NO mention of fma3 in ANY cpu.I checked.
Maybe this is an obsolete USE flag and should be retired?
_________________
 |
|
| Back to top |
|
|
janbielanski
n00b
Joined: 06 Jul 2019
Posts: 32
Location: Krakow
|
| Posted: Mon Jun 28, 2021 8:39 am Post subject: |
|
|
Use FMA3 flag was not a problem.
This flag is supported by Intel CPU since Haswell CPUs:
| Quote: |
en.wikipedia.org/wiki/FMA_instruction_set:
The FMA instruction set is an extension to the 128 and 256-bit Streaming SIMD Extensions instructions in the x86 microprocessor instruction set to perform fused multiply–add (FMA) operations.[1] There are two variants:
FMA4 is supported in AMD processors starting with the Bulldozer architecture. FMA4 was performed in hardware before FMA3 was. Support for FMA4 was removed since Zen 1.
FMA3 is supported in AMD processors starting with the Piledriver architecture and Intel starting with Haswell processors and Broadwell processors since 2014.
|
Source of problem is gcc in version 10.3.0 , when I updated gcc to version 10.3.0-r1 problem disappear.
My current gcc:
| Code: |
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-pc-linux-gnu/10.3.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /var/tmp/portage/sys-devel/gcc-10.3.0-r1/work/gcc-10.3.0/configure --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/10.3.0 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/10.3.0/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/10.3.0 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/10.3.0/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/10.3.0/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/10.3.0/include/g++-v10 --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/10.3.0/python --enable-languages=c,c++,d,go,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 10.3.0-r1 p2' --disable-esp --enable-libstdcxx-time --with-build-config=bootstrap-lto --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-multilib --with-multilib-list=m32,m64 --disable-fixed-point --enable-targets=all --enable-libgomp --disable-libssp --disable-libada --disable-systemtap --disable-vtable-verify --disable-libvtv --with-zstd --enable-lto --with-isl --disable-isl-version-check --enable-default-pie --enable-default-ssp
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.3.0 (Gentoo 10.3.0-r1 p2)
|
and gcc flags:
| Code: |
* Found these USE flags for sys-devel/gcc-10.3.0-r1:
U I
+ + d : Enable support for the D programming language
- - debug : Enable extra debug codepaths, like asserts and extra output. If you want to get meaningful backtraces see https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces
- - doc : Add extra documentation (API, Javadoc, etc). It is recommended to enable per package instead of globally
+ + fortran : Add support for fortran
+ + go : Build the GCC Go language frontend.
+ + graphite : Add support for the framework for loop optimizations based on a polyhedral intermediate representation
- - jit : Enable libgccjit so other applications can embed gcc for Just-In-Time compilation. This will slow down the compiler a bit as it forces all of the toolchain to be shared libs.
+ + lto : Build using Link Time Optimizations (LTO)
+ + nls : Add Native Language Support (using gettext - GNU locale utilities)
+ + nptl : Enable support for Native POSIX Threads Library, the new threading module (requires linux-2.6 or better usually)
- - objc : Build support for the Objective C code language
- - objc++ : Build support for the Objective C++ language
- - objc-gc : Build support for the Objective C code language Garbage Collector
+ + openmp : Build support for the OpenMP (support parallel computing), requires >=sys-devel/gcc-4.2 built with USE="openmp"
+ + pch : Enable precompiled header support for faster compilation at the expense of disk space and memory (>=sys-devel/gcc-3.4 only)
+ + pgo : Build GCC using Profile Guided Optimization (PGO)
+ + sanitize : Build support for various sanitizer functions (ASAN/TSAN/etc...)
+ + ssp : Build packages with stack smashing protector on by default
- - systemtap : enable systemtap static probe points
- - test : Enable dependencies and/or preparations necessary to run tests (usually controlled by FEATURES=test but can be toggled independently)
- - vanilla : Do not add extra patches which change default behaviour; DO NOT USE THIS ON A GLOBAL SCALE as the severity of the meaning changes drastically
- - vtv : Build support for virtual table verification (a C++ hardening feature)
+ + zstd : Enable support for ZSTD compression
|
Problem SOLVED. |
|
| Back to top |
|
|
janbielanski
n00b
Joined: 06 Jul 2019
Posts: 32
Location: Krakow
|
| Posted: Sat Jul 17, 2021 9:45 pm Post subject: |
|
|
Problem return after update dev-libs/protobuf-3.15.8 to dev-libs/protobuf-3.17.3, new package is needed by OpenCV so
I can not go back to earlier version. Compilation of tensorflow finish with error:
| Code: | ERROR: /var/tmp/portage/sci-libs/tensorflow-2.5.0/work/tensorflow-2.5.0-python3_8/tensorflow/core/kernels/BUILD:4393:18: C++ compilation of rule '//tensorflow/core/kernels:example_parsing_ops' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command
(cd /var/tmp/portage/sci-libs/tensorflow-2.5.0/work/tensorflow-2.5.0-python3_8-bazel-base/execroot/org_tensorflow && \
exec env - \
CUDA_TOOLKIT_PATH=/opt/cuda \
GCC_HOST_COMPILER_PATH=/usr/x86_64-pc-linux-gnu/gcc-bin/10.3.0/x86_64-pc-linux-gnu-gcc \
HOME=/var/tmp/portage/sci-libs/tensorflow-2.5.0/homedir \
KERAS_HOME=/var/tmp/portage/sci-libs/tensorflow-2.5.0/temp/.keras \
PATH=/var/tmp/portage/sci-libs/tensorflow-2.5.0/temp/python3.8/bin:/usr/lib/portage/python3.9/ebuild-helpers/xattr:/usr/lib/portage/python3.9/ebuild-helpers:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/lib/llvm/12/bin:/usr/lib/llvm/11/bin:/usr/lib/llvm/10/bin:/opt/cuda/bin \
PWD=/proc/self/cwd \
PYTHON_BIN_PATH=/usr/bin/python3.8 \
PYTHON_LIB_PATH=/usr/lib/python3.8/site-packages \
TF2_BEHAVIOR=1 \
TF_CUDA_COMPUTE_CAPABILITIES=7.5 \
TF_CUDA_PATHS=/opt/cuda \
TF_CUDA_VERSION=11.2 \
TF_CUDNN_VERSION=8.2 \
TF_SYSTEM_LIBS=absl_py,astor_archive,astunparse_archive,boringssl,com_github_googlecloudplatform_google_cloud_cpp,com_github_grpc_grpc,com_google_protobuf,curl,cython,dill_archive,double_conversion,enum34_archive,flatbuffers,functools32_archive,gast_archive,gif,hwloc,icu,jsoncpp_git,libjpeg_turbo,lmdb,nasm,nsync,opt_einsum_archive,org_sqlite,pasta,pcre,png,pybind11,six_archive,snappy,tblib_archive,termcolor_archive,typing_extensions_archive,wrapt,zlib \
external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -MD -MF bazel-out/k8-opt/bin/tensorflow/core/kernels/_objs/example_parsing_ops/example_parsing_ops.pic.d '-frandom-seed=bazel-out/k8-opt/bin/tensorflow/core/kernels/_objs/example_parsing_ops/example_parsing_ops.pic.o' -DTF_USE_SNAPPY -DEIGEN_MPL2_ONLY '-DEIGEN_MAX_ALIGN_BYTES=64' -iquote . -iquote bazel-out/k8-opt/bin -iquote external/com_google_absl -iquote bazel-out/k8-opt/bin/external/com_google_absl -iquote external/nsync -iquote bazel-out/k8-opt/bin/external/nsync -iquote external/eigen_archive -iquote bazel-out/k8-opt/bin/external/eigen_archive -iquote external/gif -iquote bazel-out/k8-opt/bin/external/gif -iquote external/libjpeg_turbo -iquote bazel-out/k8-opt/bin/external/libjpeg_turbo -iquote external/com_google_protobuf -iquote bazel-out/k8-opt/bin/external/com_google_protobuf -iquote external/com_googlesource_code_re2 -iquote bazel-out/k8-opt/bin/external/com_googlesource_code_re2 -iquote external/farmhash_archive -iquote bazel-out/k8-opt/bin/external/farmhash_archive -iquote external/fft2d -iquote bazel-out/k8-opt/bin/external/fft2d -iquote external/highwayhash -iquote bazel-out/k8-opt/bin/external/highwayhash -iquote external/zlib -iquote bazel-out/k8-opt/bin/external/zlib -iquote external/local_config_cuda -iquote bazel-out/k8-opt/bin/external/local_config_cuda -iquote external/local_config_rocm -iquote bazel-out/k8-opt/bin/external/local_config_rocm -iquote external/local_config_tensorrt -iquote bazel-out/k8-opt/bin/external/local_config_tensorrt -iquote external/double_conversion -iquote bazel-out/k8-opt/bin/external/double_conversion -iquote external/snappy -iquote bazel-out/k8-opt/bin/external/snappy -iquote external/curl -iquote bazel-out/k8-opt/bin/external/curl -iquote external/boringssl -iquote bazel-out/k8-opt/bin/external/boringssl -iquote external/jsoncpp_git -iquote bazel-out/k8-opt/bin/external/jsoncpp_git -Ibazel-out/k8-opt/bin/external/local_config_cuda/cuda/_virtual_includes/cuda_headers_virtual -Ibazel-out/k8-opt/bin/external/local_config_tensorrt/_virtual_includes/tensorrt_headers -Ibazel-out/k8-opt/bin/external/local_config_cuda/cuda/_virtual_includes/cudnn_header -isystem third_party/eigen3/mkl_include -isystem bazel-out/k8-opt/bin/third_party/eigen3/mkl_include -isystem external/eigen_archive -isystem bazel-out/k8-opt/bin/external/eigen_archive -isystem external/farmhash_archive/src -isystem bazel-out/k8-opt/bin/external/farmhash_archive/src -isystem external/local_config_cuda/cuda -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda -isystem external/local_config_cuda/cuda/cuda/include -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda/cuda/include -isystem external/local_config_rocm/rocm -isystem bazel-out/k8-opt/bin/external/local_config_rocm/rocm -isystem external/local_config_rocm/rocm/rocm/include -isystem bazel-out/k8-opt/bin/external/local_config_rocm/rocm/rocm/include -isystem external/local_config_rocm/rocm/rocm/include/rocrand -isystem bazel-out/k8-opt/bin/external/local_config_rocm/rocm/rocm/include/rocrand -isystem external/local_config_rocm/rocm/rocm/include/roctracer -isystem bazel-out/k8-opt/bin/external/local_config_rocm/rocm/rocm/include/roctracer -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -fPIC -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -fno-omit-frame-pointer -no-canonical-prefixes -fno-canonical-system-headers -DNDEBUG -g0 -O2 -ffunction-sections -fdata-sections -w -DAUTOLOAD_DYNAMIC_KERNELS -I/usr/include/jsoncpp '-std=c++14' '-march=skylake' -O2 -pipe -msse -msse2 -msse3 -msse4.1 -msse4.2 -mavx -mavx2 -mfma -DEIGEN_AVOID_STL_ARRAY -Iexternal/gemmlowp -Wno-sign-compare '-ftemplate-depth=900' -fno-exceptions '-DGOOGLE_CUDA=1' '-DTENSORFLOW_USE_NVCC=1' '-DTENSORFLOW_USE_XLA=1' -DINTEL_MKL -msse3 -pthread -DNV_CUDNN_DISABLE_EXCEPTION '-DGOOGLE_CUDA=1' -DNV_CUDNN_DISABLE_EXCEPTION '-DTENSORFLOW_USE_XLA=1' '-DINTEL_MKL=1' -c tensorflow/core/kernels/example_parsing_ops.cc -o bazel-out/k8-opt/bin/tensorflow/core/kernels/_objs/example_parsing_ops/example_parsing_ops.pic.o)
Execution platform: @local_execution_config_platform//:platform
[6,828 / 19,431] 8 actions running
Compiling .../service/hlo_evaluator_typed_visitor_complex64.cc; 18s local
Compiling .../service/hlo_evaluator_typed_visitor_complex128.cc; 16s local
Compiling .../xla/service/hlo_evaluator_typed_visitor_bool.cc; 16s local
Compiling .../xla/service/hlo_evaluator_typed_visitor_uint8.cc; 13s local
Compiling .../service/hlo_evaluator_typed_visitor_bfloat16.cc; 9s local
Compiling tensorflow/compiler/xla/service/hlo_evaluator.cc; 8s local
Compiling tensorflow/core/kernels/linalg/lu_op.cc; 2s local
Compiling tensorflow/compiler/xla/shape_util.cc; 0s local
^M^M^M^M^M^M^M^M^MIn file included from ./tensorflow/core/framework/op_kernel.h:35,
from ./tensorflow/core/framework/numeric_op.h:19,
from tensorflow/core/kernels/example_parsing_ops.cc:27:
tensorflow/core/kernels/example_parsing_ops.cc: In member function ‘virtual void tensorflow::DecodeJSONExampleOp::Compute(tensorflow::OpKernelContext*)’:
tensorflow/core/kernels/example_parsing_ops.cc:1221:57: error: ‘class google::protobuf::util::status_internal::Status’ has no member named ‘error_message’; did you mean ‘error_message_’?
1221 | string(status.error_message())));
| ^~~~~~~~~~~~~
./tensorflow/core/framework/op_requires.h:45:46: note: in definition of macro ‘OP_REQUIRES’
45 | (CTX)->CtxFailure(__FILE__, __LINE__, (STATUS)); \
| ^~~~~~
tensorflow/core/kernels/example_parsing_ops.cc:1221:57: error: ‘std::string google::protobuf::util::status_internal::Status::error_message_’ is private within this context
1221 | string(status.error_message())));
| ^~~~~~~~~~~~~
./tensorflow/core/framework/op_requires.h:45:46: note: in definition of macro ‘OP_REQUIRES’
45 | (CTX)->CtxFailure(__FILE__, __LINE__, (STATUS)); \
| ^~~~~~
In file included from /usr/include/google/protobuf/stubs/logging.h:36,
from /usr/include/google/protobuf/io/coded_stream.h:150,
from bazel-out/k8-opt/bin/tensorflow/core/protobuf/error_codes.pb.h:23,
from ./tensorflow/core/platform/status.h:30,
from ./tensorflow/core/lib/core/status.h:19,
from ./tensorflow/core/lib/monitoring/counter.h:37,
from ./tensorflow/core/framework/metrics.h:19,
from ./tensorflow/core/common_runtime/metrics.h:22,
from tensorflow/core/kernels/example_parsing_ops.cc:23:
/usr/include/google/protobuf/stubs/status.h:97:15: note: declared private here
97 | std::string error_message_;
[6,828 / 19,431] 8 actions running
Compiling .../service/hlo_evaluator_typed_visitor_complex64.cc; 18s local
Compiling .../service/hlo_evaluator_typed_visitor_complex128.cc; 16s local
Compiling .../xla/service/hlo_evaluator_typed_visitor_bool.cc; 16s local
Compiling .../xla/service/hlo_evaluator_typed_visitor_uint8.cc; 13s local
Compiling .../service/hlo_evaluator_typed_visitor_bfloat16.cc; 9s local
Compiling tensorflow/compiler/xla/service/hlo_evaluator.cc; 8s local
Compiling tensorflow/core/kernels/linalg/lu_op.cc; 2s local
Compiling tensorflow/compiler/xla/shape_util.cc; 0s local
^M^M^M^M^M^M^M^M^MINFO: Elapsed time: 467.344s, Critical Path: 129.09s
[6,836 / 19,431] checking cached actions
^MINFO: 6836 processes: 5960 internal, 876 local.
[6,836 / 19,431] checking cached actions
^MFAILED: Build did NOT complete successfully
^MFAILED: Build did NOT complete successfully
* ERROR: sci-libs/tensorflow-2.5.0::gentoo failed (compile phase):
* ebazel failed
*
* Call stack:
* ebuild.sh, line 127: Called src_compile
* environment, line 4154: Called ebazel 'build' '//tensorflow:libtensorflow_framework.so' '//tensorflow:libtensorflow.so'
* environment, line 2510: Called die
* The specific snippet of code:
* "${@}" || die "ebazel failed"
*
* If you need support, post the output of `emerge --info '=sci-libs/tensorflow-2.5.0::gentoo'`,
* the complete build log and the output of `emerge -pqv '=sci-libs/tensorflow-2.5.0::gentoo'`.
* The complete build log is located at '/var/tmp/portage/sci-libs/tensorflow-2.5.0/temp/build.log'.
* The ebuild environment file is located at '/var/tmp/portage/sci-libs/tensorflow-2.5.0/temp/environment'.
* Working directory: '/var/tmp/portage/sci-libs/tensorflow-2.5.0/work/tensorflow-2.5.0-python3_8'
* S: '/var/tmp/portage/sci-libs/tensorflow-2.5.0/work/tensorflow-2.5.0 |
And here is source of error:
| Code: | tensorflow/core/kernels/example_parsing_ops.cc: In member function ‘virtual void tensorflow::DecodeJSONExampleOp::Compute(tensorflow::OpKernelContext*)’:
tensorflow/core/kernels/example_parsing_ops.cc:1221:57: error: ‘class google::protobuf::util::status_internal::Status’ has no member named ‘error_message’; did you mean ‘error_message_’?
1221 | string(status.error_message())));
| ^~~~~~~~~~~~~
./tensorflow/core/framework/op_requires.h:45:46: note: in definition of macro ‘OP_REQUIRES’
45 | (CTX)->CtxFailure(__FILE__, __LINE__, (STATUS)); \
| ^~~~~~
tensorflow/core/kernels/example_parsing_ops.cc:1221:57: error: ‘std::string google::protobuf::util::status_internal::Status::error_message_’ is private within this context
1221 | string(status.error_message())));
| ^~~~~~~~~~~~~
./tensorflow/core/framework/op_requires.h:45:46: note: in definition of macro ‘OP_REQUIRES’
45 | (CTX)->CtxFailure(__FILE__, __LINE__, (STATUS)); \
|
|
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6768
|
| Posted: Sun Jul 18, 2021 11:26 am Post subject: |
|
|
| janbielanski wrote: | | Problem return after update dev-libs/protobuf-3.15.8 to dev-libs/protobuf-3.17.3 |
It it not a "return" of the problem but a completely unrelated problem.
It is rather obviously an incompatibility with tensorflow and the new protobuf library which needs to be fixed upstream.
As a hackish temporary fix, you can try to patch error_message() to error_message_() as suggested, but it is unlikely that this is the correct solution (or will even compile). As an even more hackish temporary fix, you can try to remove the whole function call (or replace it by some call to exit), and hope that this code (which is apparently related with error handling) will actually not be used.
The correct (non-hackish) solution requires deeper inspection of the code which should be done by upstream. |
|
| Back to top |
|
|
janbielanski
n00b
Joined: 06 Jul 2019
Posts: 32
Location: Krakow
|
| Posted: Mon Jul 19, 2021 12:30 pm Post subject: |
|
|
| I reported a bug via Gentoo bugzilla. |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22162
|
| Posted: Mon Jul 19, 2021 3:22 pm Post subject: |
|
|
| For the benefit of future readers, the reported bug is likely sci-libs/tensorflow-2.5.0 - compilation error, based on the text in the body of the bug. (The bug title could use a bit of work.) |
|
| Back to top |
|
|
mr-simon
Guru
Joined: 22 Nov 2002
Posts: 367
Location: Leamington Spa, Warks, UK
|
| Posted: Wed Jul 21, 2021 12:58 pm Post subject: |
|
|
I also ran into this, and downgrading protobuf to 3.15.8 fixed the problem, so I've pinned my version to that until there's a fix.
_________________
"Pokey, are you drunk on love?"
"Yes. Also whiskey. But mostly love... and whiskey." |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
