gentoo-sources-5.1.0 ebuild question

Message

Tony0945 · Post by **Tony0945** » Sun May 12, 2019 1:34 pm

kajzer wrote:IMHO that USE flag should be removed and the additional CPU optimizations patch should be included with the rest of the patches.

Absolutely! Would it help to file an improvement bug? Against gentoo-sources generally?

Post by **NeddySeagoon** » Sun May 12, 2019 4:18 pm

Tony0945,

File the bug against the upstream kernel.
There is probably a reason why its not been adopted. Like it increases the test burden on kernel devs.

wrc1944 · Post by **wrc1944** » Sun May 12, 2019 4:47 pm

Well, it's now in the 5.1.1 ebuild (but NOT in the 5.1.0 ebuild),

Code: Select all

# Copyright 1999-2019 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2

EAPI="6"
ETYPE="sources"
K_WANT_GENPATCHES="base extras"
K_GENPATCHES_VER="2"

inherit kernel-2
detect_version
detect_arch

KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~x86"
HOMEPAGE="https://dev.gentoo.org/~mpagano/genpatches"
IUSE="experimental"

DESCRIPTION="Full sources including the Gentoo patchset for the ${KV_MAJOR}.${KV_MINOR} kernel tree"
SRC_URI="${KERNEL_URI} ${GENPATCHES_URI} ${ARCH_URI}"

pkg_postinst() {
	kernel-2_pkg_postinst
	einfo "For more info on this patchset, and how to report problems, see:"
	einfo "${HOMEPAGE}"
}

pkg_postrm() {
	kernel-2_pkg_postrm
}

and, I get:

Code: Select all

amd64 ~ # emerge gentoo-sources -pv

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] sys-kernel/gentoo-sources-5.1.1:5.1.1::gentoo  USE="experimental -build -symlink" 103,755 KiB

So I thought it was good to go, but I still get:

Code: Select all

2019-05-12 12:28:34 (2.06 MB/s) - ‘/usr/portage/distfiles/linux-5.1.tar.xz.__download__’ saved [106244476/106244476]

 * linux-5.1.tar.xz BLAKE2B SHA512 size ;-) ...                                                                                             [ ok ]
 * genpatches-5.1-2.base.tar.xz BLAKE2B SHA512 size ;-) ...                                                                                 [ ok ]
 * genpatches-5.1-2.extras.tar.xz BLAKE2B SHA512 size ;-) ...                                                                               [ ok ]
>>> Preparing to unpack ...
>>> Unpacking source...
>>> Unpacking linux-5.1.tar.xz to /var/tmp/portage/sys-kernel/gentoo-sources-5.1.1/work
>>> Unpacking genpatches-5.1-2.base.tar.xz to /var/tmp/portage/sys-kernel/gentoo-sources-5.1.1/work/patches
>>> Unpacking genpatches-5.1-2.extras.tar.xz to /var/tmp/portage/sys-kernel/gentoo-sources-5.1.1/work/patches
 * Excluding Patch #5010_enable-additional-cpu-optimizations-for-gcc.patch ...                                                              [ ok ]
 * Excluding Patch #5010_enable-additional-cpu-optimizations-for-gcc-4.9.patch ...                                                          [ ok ]
 * Excluding Patch #5010_enable-additional-cpu-optimizations-for-gcc.patch ...                                                              [ ok ]
 * Excluding Patch #5010_enable-additional-cpu-optimizations-for-gcc-4.9.patch ...                                                          [ ok ]
 * Applying 1000_linux-5.1.1.patch (-p1) ...                                                                                                [ ok ]
 * Applying 1500_XATTR_USER_PREFIX.patch (-p1) ...                                                                                          [ ok ]
 * Applying 1510_fs-enable-link-security-restrictions-by-default.patch (-p1) ...                                                            [ ok ]
 * Applying 2500_usb-storage-Disable-UAS-on-JMicron-SATA-enclosure.patch (-p1) ...                                                          [ ok ]
 * Applying 2600_enable-key-swapping-for-apple-mac.patch (-p1) ...                                                                          [ ok ]
 * Applying 4567_distro-Gentoo-Kconfig.patch (-p1) ...                                                                                      [ ok ]
>>> Source unpacked in /var/tmp/portage/sys-kernel/gentoo-sources-5.1.1/work
>>> Preparing source in /var/tmp/portage/sys-kernel/gentoo-sources-5.1.1/work/linux-5.1.1-gentoo ...
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/sys-kernel/gentoo-sources-5.1.1/work/linux-5.1.1-gentoo ...
>>> Source configured.
>>> Compiling source in /var/tmp/portage/sys-kernel/gentoo-sources-5.1.1/work/linux-5.1.1-gentoo ...
>>> Source compiled.
>>> Test phase [not enabled]: sys-kernel/gentoo-sources-5.1.1

>>> Install gentoo-sources-5.1.1 into /var/tmp/portage/sys-kernel/gentoo-sources-5.1.1/image/ category sys-kernel
>>> Copying sources ...
>>> Completed installing gentoo-sources-5.1.1 into /var/tmp/portage/sys-kernel/gentoo-sources-5.1.1/image/

 * Final size of build directory:      4 KiB
 * Final size of installed tree:  965256 KiB (942.6 MiB)

Checked on https://dev.gentoo.org/~mpagano/genpatc ... 5.1-1.html and the 5.1.1 expermental.tar.xz patch still isn't on the server.

Maybe soon. If not, in a day or two I'll try patching 5.1.1 with the last 5011 gen-opts patch, which does apply cleanly to 5.1.0, and apparently works normally.

Anon-E-moose · Post by **Anon-E-moose** » Sun May 12, 2019 5:08 pm

5010/5011 (the last ones provided) should patch against newer kernels, with perhaps some fuzz factor involved.

They just basically add optimizations (for other cpus) against a few files.

I've been using zen kernels for years now (even after gentoo quit messing with them) and I think they just use their mods.

wrc1944 · Post by **wrc1944** » Sun May 12, 2019 7:15 pm

Hmmm... IIRC, those "zen" kernels of past years were a custom so-called "performance" kernel just using zen as a fancy name, and has nothing to do with the gcc kernel option "zen, zen1" which is short for the AMD Ryzen family of processors. At least that was my understanding- could be wrong.

On the other hand, did the old zen patch set evolve into the zen-sources package now found in portage, and did it actually new adopt code for the current Ryzen family of cpus?

Anon-E-moose · Post by **Anon-E-moose** » Sun May 12, 2019 7:22 pm

The zen kernels allow lots of performance tuning, but one of the things they've done for a long time, is allow cpu optimization of the code, up to and including the zen processors, coffee lake processors, etc. My guess is that gentoo sources borrows the code for the cpu optimizations. It's not that much in the scheme of things.

Code: Select all

+config MZEN
+   bool "AMD Zen"
+   ---help---
+     Select this for AMD Family 17h Zen processors.
+
+     Enables -march=znver1
...
+        cflags-$(CONFIG_MZEN) += $(call cc-option,-march=znver1)

The cpu opts just basically pass on march and sometime mtune flags.

anyway, my original point was that the ones from 4.19/4.20/5.0 should work on 5.1 (with possibly fuzz)

wrc1944 · Post by **wrc1944** » Sun May 12, 2019 8:30 pm

Thanks much Anon-E-moose,
Interesting info. Wasn't aware that gentoo-sources might be borrowing the code for the cpu optimizations. I have used the 5011 patch on gentoo-sources-5.1.0 with success- applied cleanly.

Are the zen-sources in portage expected to move to more current kernels? Latest ebuilds are from April 2018.

Anon-E-moose · Post by **Anon-E-moose** » Sun May 12, 2019 8:48 pm

I doubt that gentoo is going to keep up with zen kernel.

I just use the website keep a git copy on my system https://github.com/zen-kernel/zen-kernel

Note: with zen kernel, they do keep up with the latest (stable) kernels, up to a certain point.
They basically just make sure that the different parts they add are up to date in whatever kernel version they have.
But they don't follow the kernels completely, right now it's 4.19 - 5.1, which is why I keep git up to date, I can pull old kernels (if I had too).

Edit to add: I don't use the ebuilds anyway. I do it the old fashioned way, go to the src directory and do make bzImage modules modules_install install, or some part of the make depending on what I'm doing. I add an entry for a kernel source to packages.provided to satisfy portage and it's all good.

wrc1944 · Post by **wrc1944** » Sun May 12, 2019 11:57 pm

Patched 5.1.1 with the 5011 patch- applied cleanly:

Code: Select all

amd64 ~ # cd /usr/src/linux-5.1.1-gentoo
amd64 /usr/src/linux-5.1.1-gentoo # patch -p1 < 5011_enable-cpu-optimizations-for-gcc8.patch
patching file arch/x86/Makefile_32.cpu
patching file arch/x86/Kconfig.cpu
patching file arch/x86/Makefile
patching file arch/x86/include/asm/module.h

gcc-opts were then in my make xconfig window listings.
One oddity, not in the patch or kernel compile, but in make xconfig. The Kernel compiled normally.

After you load your config file into your kernel source from the main window (I have all mine in /usr/src and get there in a drop-down menu dialog box), make edits and/add/remove settings as needed, you then select save (or save as) in the main make xconfig window and for years it has auto saved the new .config file directly to your new kernel source directory, correctly.

Now, since 5.1.0, it doesn't do that, and now forces you to rename the file to .config and select the proper location in a multi-level directory search. This use to be automatic, but now it's very inconvenient, and defeating the main purpose of make xconfig.

Is it now this way on purpose, or possibly a qt/kde version change that broke the previous behavior of make xconfig? Sorry if this last point is off topic- will edit out and open a different thread if needed.

wrc1944 · Post by **wrc1944** » Fri May 17, 2019 11:56 pm

The gcc-opts patch (5011) is now back in gentoo-sources 5.1.3.

Thanks very much to mpagano!

Goverp · Post by **Goverp** » Sat May 18, 2019 10:04 am

Out of interest, anyone know if the old KCFLAGS="-march=native" technique would have worked?

wrc1944 · Post by **wrc1944** » Sat May 18, 2019 1:34 pm

I think you're talking about what's discussed here. https://stackoverflow.com/questions/404 ... ompilation

Back just after kernel 3.0.0 was released (or even before that) I tried the makefile edits and/or a few other files on my amd k6 systems, looking for any performance improvement possible.

Running Gentoo back then was a really good way to develop extreme patience. As I recall, kernel source editing was kind of tricky, and results weren't exactly reliable.

As hardware improved, and I learned more I decided it wasn't worth the time and effort as with each kernel version you never felt confident what worked before would
continue to be effective. Sometimes it worked, or at least I thought it did. In those days, what we have now would have seemed like science fiction or even unimaginable.

Goverp · Post by **Goverp** » Sat May 18, 2019 7:39 pm

That's right, but AFAIR a few years ago they "fixed" the kernel build so it no longer worked - possibly ordering the parameters so that the kernel setting overrode the KCFLAG settings.

Goverp · Post by **Goverp** » Sun May 19, 2019 12:29 pm

I did a test: reinstall gentoo-sources without the experimental patches, change the kernel configuration to specify Opteron/Athlon64/Hammer/K8 processor type (right for me), then

Code: Select all

export KCLAGS='-march=native'
make
make install

extract the kernel, disassemble the results, and compare the output with the same for a kernel compiled with the experimental patches and Native processor type.

The results were identical apart from timestamp. so setting KCFLAGS still works.

This also answered a niggling question, that whether configuring processor type for Opteron actually changed what was compiled (i.e.conditional source code) rather than how it was compiled (i.e. gcc parameters).
The answer appears to be No.

Anon-E-moose · Post by **Anon-E-moose** » Sun May 19, 2019 12:40 pm

Code: Select all

# Add any arch overrides and user supplied CPPFLAGS, AFLAGS and CFLAGS as the
# last assignments
KBUILD_CPPFLAGS += $(ARCH_CPPFLAGS) $(KCPPFLAGS)
KBUILD_AFLAGS   += $(ARCH_AFLAGS)   $(KAFLAGS)
KBUILD_CFLAGS   += $(ARCH_CFLAGS)   $(KCFLAGS)