Keras 2.11 build failure

[smilr]

Hello,

Today I ran into an issue updating my gentoo installation. I have both Keras and Tensorflow installed, however updating from Keras 2.10 to Keras 2.11 failed with the following:

Code: Select all

[21 / 24] Executing genrule //keras/api:keras_python_api_gen_compat_v1; 1s local ... (3 actions running)
ERROR: /var/tmp/portage/sci-libs/keras-2.11.0/work/keras-2.11.0-python3_10/keras/api/BUILD:173:19: Executing genrule //keras/api:keras_python_api_gen_compat_v1 failed: (Exit 1): bash failed: error executing command 
  (cd /var/tmp/portage/sci-libs/keras-2.11.0/work/keras-2.11.0-python3_10-bazel-base/execroot/org_keras && \
  exec env - \
    HOME=/var/tmp/portage/sci-libs/keras-2.11.0/homedir \
    PATH=/var/tmp/portage/sci-libs/keras-2.11.0/temp/python3.10/bin:/usr/lib/portage/python3.10/ebuild-helpers/xattr:/usr/lib/portage/python3.10/ebuild-helpers:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/lib/llvm/14/bin:/opt/cuda/bin \
    TF2_BEHAVIOR=1 \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; bazel-out/k8-opt-exec-2B5CBBC6/bin/keras/api/create_keras_api_1_keras_python_api_gen_compat_v1  --apidir=bazel-out/k8-opt/bin/keras/api_v1/
 --apiname=keras --apiversion=1   --packages=keras,keras.activations,keras.applications.convnext,keras.applications.densenet,keras.applications.efficientnet,keras.applications.efficientnet_v2,keras.applications.imagenet_utils,keras.applications.inception_resnet_v2,keras.applications.inception_v3,
<<Removed WAY too many packages on the same line that made the post preview look HORRIBLE>>
bazel-out/k8-opt/bin/keras/api/_v1/keras/wrappers/__init__.py bazel-out/k8-opt/bin/keras/api/_v1/keras/wrappers/scikit_learn/__init__.py')
# Configuration: 5f5ff3436983feae30214b72f08aee4bd43515c7924f344d71bf6e2bac85336c
# Execution platform: @local_config_platform//:host
Traceback (most recent call last):
  File "/var/tmp/portage/sci-libs/keras-2.11.0/work/keras-2.11.0-python3_10-bazel-base/execroot/org_keras/bazel-out/k8-opt-exec-2B5CBBC6/bin/keras/api/create_keras_api_1_keras_python_api_gen_compat_v1.runfiles/org_keras/keras/api/create_python_api_wrapper.py", line 26, in <module>
    import keras  # noqa: F401
  File "/var/tmp/portage/sci-libs/keras-2.11.0/work/keras-2.11.0-python3_10-bazel-base/execroot/org_keras/bazel-out/k8-opt-exec-2B5CBBC6/bin/keras/api/create_keras_api_1_keras_python_api_gen_compat_v1.runfiles/org_keras/keras/__init__.py", line 21, in <module>
    from keras import models
  File "/var/tmp/portage/sci-libs/keras-2.11.0/work/keras-2.11.0-python3_10-bazel-base/execroot/org_keras/bazel-out/k8-opt-exec-2B5CBBC6/bin/keras/api/create_keras_api_1_keras_python_api_gen_compat_v1.runfiles/org_keras/keras/models/__init__.py", line 18, in <module>
    from keras.engine.functional import Functional
  File "/var/tmp/portage/sci-libs/keras-2.11.0/work/keras-2.11.0-python3_10-bazel-base/execroot/org_keras/bazel-out/k8-opt-exec-2B5CBBC6/bin/keras/api/create_keras_api_1_keras_python_api_gen_compat_v1.runfiles/org_keras/keras/engine/functional.py", line 34, in <module>
    from keras.engine import training as training_lib
  File "/var/tmp/portage/sci-libs/keras-2.11.0/work/keras-2.11.0-python3_10-bazel-base/execroot/org_keras/bazel-out/k8-opt-exec-2B5CBBC6/bin/keras/api/create_keras_api_1_keras_python_api_gen_compat_v1.runfiles/org_keras/keras/engine/training.py", line 45, in <module>
    from keras.saving.legacy import save
  File "/var/tmp/portage/sci-libs/keras-2.11.0/work/keras-2.11.0-python3_10-bazel-base/execroot/org_keras/bazel-out/k8-opt-exec-2B5CBBC6/bin/keras/api/create_keras_api_1_keras_python_api_gen_compat_v1.runfiles/org_keras/keras/saving/legacy/save.py", line 24, in <module>
    from keras.saving.legacy.saved_model import load_context
  File "/var/tmp/portage/sci-libs/keras-2.11.0/work/keras-2.11.0-python3_10-bazel-base/execroot/org_keras/bazel-out/k8-opt-exec-2B5CBBC6/bin/keras/api/create_keras_api_1_keras_python_api_gen_compat_v1.runfiles/org_keras/keras/saving/legacy/saved_model/load_context.py", line 68, in <module>
    tf.__internal__.register_load_context_function(in_load_context)
AttributeError: module 'tensorflow.compat.v2.__internal__' has no attribute 'register_load_context_function'. Did you mean: 'register_call_context_function'?

The ebuild for Keras 2.11 lists the following dependencies:

Code: Select all

RDEPEND="
	>=dev-libs/protobuf-3.13.0:=
	dev-python/absl-py[${PYTHON_USEDEP}]
	dev-python/h5py[${PYTHON_USEDEP}]
	dev-python/numpy[${PYTHON_USEDEP}]
	dev-python/pandas[${PYTHON_USEDEP}]
	dev-python/pillow[${PYTHON_USEDEP}]
	>=dev-python/protobuf-python-3.13.0[${PYTHON_USEDEP}]
	dev-python/six[${PYTHON_USEDEP}]
	>=sci-libs/keras-applications-1.0.8[${PYTHON_USEDEP}]
	>=sci-libs/keras-preprocessing-1.1.2[${PYTHON_USEDEP}]
	>=sci-libs/tensorflow-2.10[python,${PYTHON_USEDEP}]"
DEPEND="${RDEPEND}
	dev-python/setuptools[${PYTHON_USEDEP}]"
BDEPEND="
	app-arch/unzip
	>=dev-libs/protobuf-3.13.0
	dev-java/java-config
	>=dev-util/bazel-4.2.2"

Should Tensorflow be added as a build dependency for Keras? Or is that superceded by it being a runtime dependency already?
Either way it looks like Keras 2.11 requires Tensorflow 2.11.

In my case, emerge did schedule Tensorflow to update to 2.11, but *after* Keras was scheduled to update.

I am updating Tensorflow separately to 2.11 now, and hopefully I will be able to report back that doing so will allow Keras 2.11 to build.

[smilr]

Success!

Keras updated to 2.11 without issue after manually updating Tensorflow to 2.11.

[Hu]

The ebuild for Keras 2.11 lists the following dependencies:

Code: Select all

RDEPEND="
	>=sci-libs/tensorflow-2.10[python,${PYTHON_USEDEP}]"
DEPEND="${RDEPEND}
	dev-python/setuptools[${PYTHON_USEDEP}]"

Should Tensorflow be added as a build dependency for Keras? Or is that superceded by it being a runtime dependency already?

Neither. This ebuild's DEPEND explicitly includes RDEPEND, then adds more on to it, so as written, it has both a build-time and runtime dependency on >=tensorflow-2.10. An ebuild that did not have $RDEPEND in DEPEND would not automatically have a build-time dependency on its runtime dependencies.

Either way it looks like Keras 2.11 requires Tensorflow 2.11.

In that case, please [bug=]file a bug[/bug] to have the ebuild changed to request >=sci-libs/tensorflow-2.11 so that Portage will enforce a proper build order.