Meltdown/Spectre: Unauthorized Disclosure of Kernel Memory

[transpetaflops]

portage does not care which compiler you use, i.e. these variables just refer to your currently configured compiler.

If gcc and clang are using different parameters for the mitigations, do we just add them all to CFLAGS and the currently used compiler will just drop the parameters it doesn't understand or how do we handle this?

[mv]

If gcc and clang are using different parameters for the mitigations, do we just add them all to CFLAGS and the currently used compiler will just drop the parameters it doesn't understand or how do we handle this?

The compiler might spit errors/warnings on unknown flags. If you use portage-bashrc-mv, it will filter the flags before they reach the compiler.

[mv]

world is now rebuilt with the new flags on x86 and am64.

On amd64, some standard programs (zsh, tmux, and xterm) sometimes segfaulted in glibc while compiling, but this might be due to a mixture of already loaded and freshly compiled libs in different formats.

After a reboot on amd64, X did not start: It seems that xorg-server and x11-drivers/* (at least xf86-input-libinput) do not like -fno-plt.
After recompiling these programs with -fno-plt filtered, X starts.
So far, everything appears to be stable on amd64.

x86 successfully recompiled, but it is not tested yet.

[ChrisJumper]

world is now rebuilt with the new flags on x86 and am64.

Which Flags did you use?

Code: Select all

-O2 -fno-plt -mretpoline

Code: Select all

O2 -fno-plt -mindirect-branch=thunk -mfunction-return=thunk

And why no-plt in the first place? Did it affect the speculative execution/branch prediction in a Way that make an attack more difficult?

Gloss about plt:
Position independent Code (PIC)
plt (process linkage table)

Some Code seems to need the plt, especially if you have position dependent code (non-PIC), in combination with dynamically linked/use shared libraries. Because on position dependent code the compiler have no chance to forecast actual used address for functions, later at runtime. That depend on the used shared libraries, so the compiler just use an ordinary call instruction.
Now its the Linkers job when it recognize a relocation of a not locally used symbol to generate a plt entry that load the Address from the Global Offset Table and create a jump to it.
So with the plt you got a function call working on dynamically linked libraries, without a need to modify the position dependent Code (non-PIC).

I suppose those packages who dislike no-plt needs to be static linked? Did not try it by myself just a suggestion.

[mv]

Which Flags did you use?

A lot of flags, but new were only

Code: Select all

-fno-plt -mindirect-branch=thunk -mfunction-return=thunk

For the records: My other “standard” flags are:

Code: Select all

CPPFLAGS='-DNDEBUG -DNO_DEBUG'
CXXFLAGS='-march=native -O2 -fno-ident -pipe -Wl,--hash-style=gnu -Wl,--sort-common -Wl,-O9 -Wl,--enable-new-dtags -Wl,-z,now -Wl,-z,relro -Wl,-z,noexecstack -Wl,-z,combreloc -Wl,--relax -Wl,--as-needed -Wl,--build-id=none -fstack-protector-strong -fstack-check=specific -fomit-frame-pointer -fno-common -g0 -ffast-math -fmerge-all-constants -ftree-partial-pre -fno-unwind-tables -fno-asynchronous-unwind-tables -fvect-cost-model -fgcse-sm -fgcse-las -fweb -finline-functions -fgcse-after-reload -fpredictive-commoning -fdirectives-only -ftree-switch-conversion -fira-loop-pressure -ftree-loop-distribution -ftree-loop-distribute-patterns -ftree-loop-im -fivopts -fdiagnostics-color=always -freorder-functions -fdevirtualize-speculatively -fno-semantic-interposition -frename-registers -flto -fuse-linker-plugin -flto-partition=none -flto-odr-type-merging -fvisibility-inlines-hidden -fno-enforce-eh-specs -fnothrow-opt -D_GLIBCXX_ASSERTIONS'

(and CFLAGS, LDFLAGS, FFLAGS, FCFLAGS, F77FLAGS a certain subset of these), and I omitted some processor-specific flags in this list (which are contained in -march=native if the latter is not filtered).
However, do not consider this full list as a general recommendation: Some are quite risky, and I filter a lot of them for specific packages (see here for the exceptions)

And why no-plt in the first place? Did it affect the speculative execution/branch prediction in a Way that make an attack more difficult?

That's what I suppose. Of course, this cannot be really checked except by attempting to write an exploit...

if you have position dependent code (non-PIC), in combination with dynamically linked/use shared libraries

Nowadays, libraries with non-PIC practically don't exist. (Moreover, I use -pie -fPIE -Wl,-z,now -Wl,-z,relro since years, so even position-dependent non-libraries are rare on my systems.)

I suppose those packages who dislike no-plt needs to be static linked?

Essentially, this is xorg-server (and the X drivers), and these cannot be statically linked: The reason why they do not like this flag is that (according to my understanding) they do not use ld.so directly as other packages but instead some hand-crafted X-only runtime linker wrapper which has less/different features. To my knowledge this is one of the reasons why wayland was born: to get rid of such ancient bloat which really blocks progress.
My “solution” is to avoid -fno-plt for these few packages (I had done this earlier with -Wl,-z,now, of course).

Edit: Add other flags, refer to exceptions, fix typos.

[mv]

x86 successfully recompiled, but it is not tested yet.

A quick test (booting, running X, testing a few programs) showed so far no problems, either.

For neither system I could observe an obvious slowdown for my usual tasks. Of course, I expect that this might be different for certain long-running calculations, but for standard development (compiling etc, including the emerge -e world itself), I did not recognize an observable slowdown.

[eccerr0r]

Is the only way to get x86 KPTI from a git repo or is there a released version now?

Have anyone seen of a live exploit now? I'd love to be wrong but still think a general purpose exploit is hard to write especially with KASLR.

On amd64, some standard programs (zsh, tmux, and xterm) sometimes segfaulted in glibc while compiling, but this might be due to a mixture of already loaded and freshly compiled libs in different formats.

Crap, more non-deterministic behavior due to ASLR? :-( One would hope things are deterministic.

[mv]

Crap, more non-deterministic behavior due to ASLR? :-( One would hope things are deterministic.

The crashes occurred only while working on the system during the world rebuild: After the reboot, I had no such crashes anymore (so far). So it seems that my conjecture is true that these crashes were caused by using libraries in mixed versions (partially through programs loaded before the rebuild and partially after the rebuild with the new flags -fno-plt ...). I do not think that ASLR has anything to do with this.

[eccerr0r]

Did you have to do anything to get beyond the crash or simply restarting will succeed (regardless if incorrect versions are in play)? The latter would imply non-determinism. If you mean adding -f no-pit will workaround the crash, then that's okay...

[transpetaflops]

Have anyone seen of a live exploit now? I'd love to be wrong but still think a general purpose exploit is hard to write especially with KASLR.

Symantec has a Trojan listed referencing CVE-2017-5754 at least or it may just be a general info. I'm not certain.
https://www.symantec.com/security_respo ... 15-0609-99

[eccerr0r]

Symantec has a Trojan listed referencing CVE-2017-5754 at least or it may just be a general info. I'm not certain.
https://www.symantec.com/security_respo ... 15-0609-99

From the wording it seems like it's just a heuristic added to antivirus, but no actual code in the wild. Theoretically it should trigger off of the PoC code.

Also note "Risk Level 1: Very Low" which somewhat fits with my guess that it is hard to exploit but easy to demonstrate...

[mv]

simply restarting will succeed

Rebooting succeeded.

[eccerr0r]

simply restarting will succeed

Rebooting succeeded.

Err. No, I meant when emerge/gcc fails, you had to do something to retry it, whether you did nothing and rerun emerge, or added an option to a config file and emerge. If you did nothing and restarted the emerge and it succeeded the second time around - this would be a determinism problem that would be alarming.

[mv]

Err. No, I meant when emerge/gcc fails

It was not emerge/gcc that segfaulted, but other programs I was using during the emerge (not related to the emerge): xterm, tmux, zsh, once also firefox crashed; usually when they should display something. The corresponding X session (and some or even all of these programs) had been open during the whole emerge world process.

[eccerr0r]

Ah, thanks for clearing that up. Something like that is quite expected indeed.

[mike155]

Intel just released a microcode revision guide, which contains detailed list of currently planned microcode updates (MCUs).

The list contains version numbers of
- pre-mitigation MCUs
- stopped MCUs
- new (planned) productions MCUs

I found the link in this message.

[Luke-Jr]

Edit: Another surprising observation: -mindirect-branch=thunk without -O2 (or better) will not protect you...

Why not? What part of -O2 is important? (-O2 contains optimizations that make debugging annoying...)

[mv]

Edit: Another surprising observation: -mindirect-branch=thunk without -O2 (or better) will not protect you...

Why not? What part of -O2 is important?

I did not try the switches implied by -O2 individually, but -O1 alone was not enough.
My test is very trivial: I have just compiled the spectre test with -mindirect-branch=thunk and with/without -O2 ...

O2 contains optimizations that make debugging annoying.

Why would somebody want to enable spectre mitigation patches in program which is just compiled for being debugged?
(Also, in my experience, debugging is not really harder with -O2. But then again I usually find the bugs by reading the code carefully instead of running a debugger, unless it is a code generation or compiler optimization bug in which case I would have to use -O2 or better anyway...)

[Luke-Jr]

Edit: Another surprising observation: -mindirect-branch=thunk without -O2 (or better) will not protect you...

Why not? What part of -O2 is important?

I did not try the switches implied by -O2 individually, but -O1 alone was not enough.
My test is very trivial: I have just compiled the spectre test with -mindirect-branch=thunk and with/without -O2 ...

AFAICT, that test is just for Spectre variant "1a" aka variant 4, which is NOT a CPU bug, but merely a bug in sandboxing frameworks.

Anyhow, strangely enough... turning on every -O2 option by hand does not cause that test to fail (although I confirmed -O2 does)!
So -O2 is doing something undocumented and that doesn't affect the output from -Q --help=optimizers,warnings,params,common,undocumented :/

The minimal I could get the test to fail with was: -O1 -fgcse -finline-small-functions

O2 contains optimizations that make debugging annoying.

Why would somebody want to enable spectre mitigation patches in program which is just compiled for being debugged?

I prefer that if a program crashes, I can immediately debug what happened, so I try to build everything debuggable.

BTW, a number of ebuilds ignore the CFLAGS, so I'm modifying my GCC's defaults instead.

Is -mindirect-branch-register completely unnecessary? (Why was it backported?)

[mv]

So -O2 is doing something undocumented

Not really. It is documented that it implies certain switches, not that it does nothing more than to enable these switches. I suppose that a whole optimization phase (or at least a considerable part of it) is missing without -O2.

I prefer that if a program crashes, I can immediately debug what happened, so I try to build everything debuggable.

For backtraces and such things, optimizations practically do not hurt: If one roughly knows where it crashed, one can guess whether it is a buffer overflow or a null pointer assignment or something else. There is no need to follow every single instruction step by step in such a case, in particular since a bug causing a crash usually happened somewhere else, anyway (wrong arguments to the crashing function or of other data, perhaps caused by a buffer overflow in a completely different place).

BTW, a number of ebuilds ignore the CFLAGS, so I'm modifying my GCC's defaults instead.

How do you do this? I never found a convenient way for it: AFAIK, there is not a file which is read on startup, but it seems to be compiled in hard, i.e. you really have to switch binaries?

Is -mindirect-branch-register completely unnecessary? (Why was it backported?)

I do not really understand the purpose of this switch. My guess is that it just modifies a detail in the implementation of -mindirect-branch (and of -mfunction-return?) which is only important in connection with =extern. In particular, my guess is that the "extern" part provided by the kernel will not work without this switch and that with =thunk it only influences the retpoline runtime (if at all). But this is only a guess, after I received no reply on my corresponding question. If somebody knows more, please tell...

[Luke-Jr]

So -O2 is doing something undocumented

Not really. It is documented that it implies certain switches, not that it does nothing more than to enable these switches.

Not documented is the same as undocumented.

BTW, a number of ebuilds ignore the CFLAGS, so I'm modifying my GCC's defaults instead.

How do you do this? I never found a convenient way for it: AFAIK, there is not a file which is read on startup, but it seems to be compiled in hard, i.e. you really have to switch binaries?

I throw a patch file in /etc/portage/patches/sys-devel/gcc/

Still working on my revised one, however - it seems parts of libgcc (crtbegin/end?) are built with -fno-inline, which breaks the retpoline (and in my patch, throws an error at compile-time).
Fetching the GCC git overnight to see if I can identify the reason it's build with -fno-inline and whether it's safe to remove that...

Is -mindirect-branch-register completely unnecessary? (Why was it backported?)

I do not really understand the purpose of this switch. My guess is that it just modifies a detail in the implementation of -mindirect-branch (and of -mfunction-return?) which is only important in connection with =extern. In particular, my guess is that the "extern" part provided by the kernel will not work without this switch and that with =thunk it only influences the retpoline runtime (if at all). But this is only a guess, after I received no reply on my corresponding question. If somebody knows more, please tell...

IIRC the kernel doesn't use the switch either.

[mv]

So -O2 is doing something undocumented

Not really. It is documented that it implies certain switches, not that it does nothing more than to enable these switches.

Not documented is the same as undocumented.

I think there is a misunderstanding of the double negation I used.

I throw a patch file in /etc/portage/patches/sys-devel/gcc/

That's the inconvenient way I meant. So it is not possible to switch back easily (without implementing anti-options as well). For instance, while this might be the right approach for spectre in user space it calls for trouble for e.g. kernel compilation or kernel modules.

IIRC the kernel doesn't use the switch either.

Code: Select all

$ grep -R indirect-branch-register .
./arch/x86/Makefile:    RETPOLINE_CFLAGS += $(call cc-option,-mindirect-branch=thunk-extern -mindirect-branch-register)

[mv]

The segfaults have occurred again.

They occurred while (and probably only while) I was re-compiling chromium. I had used jumbo-build because there was a speedup (I had 8GB RAM and 20 GB Swap); when disabling jumbo-build there was still a crash, but only one.

So it seems that the crashes are perhaps related with the kernel OOM-killer. Can it be that the OOM-killer does not log but only causes segfaults? Somehow I have doubts that this is related to retpoline. Here are 2 typical messages:

13:56:35 03.02.18 [kernel] zsh[8462]: segfault at 0 ip ... sp ... error 4 in libc-2.25.so[...]
13:58:48 03.02.18 [kernel] xterm[25816]: segfault at 18 ip ... sp ... error 4 in libc-2.25.so[...]

Also a daemon written in perl segfaulted during chrome compilation while it did nothing else than listening to a socket...

[Spargeltarzan]

Hi,

I am not seeing the forest with all the trees any more. With kernel 4.14.18, microcode-update, gcc7.3 and binutils-2.30 the speed47 spectre-meltdown-checker reports "not vulnerable" for all three variants.

The output of /sys/devices/system/cpu/vulnerabilities/* is:

Code: Select all

cat /sys/devices/system/cpu/vulnerabilities/*  
Mitigation: PTI
Mitigation: __user pointer sanitization
Mitigation: Full generic retpoline

What do we win when modifying the CFLAGS? And what should we add, only "-mindirect-branch=thunk"? some posts before there was a discussion about it only when no microcode update is used, maybe someone can summarize us what we should or could do here...

Thank you!

[Luke-Jr]

So it is not possible to switch back easily (without implementing anti-options as well).

There are already anti-options. It's just a matter of defaults and errors to prevent accidentally breaking the retpoline.