Meltdown/Spectre: Unauthorized Disclosure of Kernel Memory

[alex-kas]

+1 to Kajzer,
How binary packages can be controlled?
To NeddySeagoon:
I believe too few species in this world and trust ..., sometimes do not trust even to myself ...
Compiling kernel with gcc 7.3.0 produces to me on kernel-4.14.15:

Code: Select all

cat /sys/devices/system/cpu/vulnerabilities/* 
Mitigation: PTI
Vulnerable
Mitigation: Full generic retpoline

And I for sure have not recompiled anything else yet.

I can imagine that generically any kind of a kernel or a hypervisor may do a better job having retpoline protection in it. I thus can imagine that kernel is what awaits for retpoline to protect us in the userland. On the same ground firefox maybe can become safer for running javascripts if the firefox itself has some setting for -mindirect-branch during the emerge ... The same for virtualbox and similar stuff.
The last proposition is however a conjecture. What do you guys think about all of that?

[PrSo]

I agree with kajzer.
Previously, at time when I wrote post about compiling everything (@world) with retpoline there was a discussion among kernel people how to resolve mitigation for Spectre.

Since AMD's KVM and Intel's VT are vulnerable to Spectre v2 counter action should be done by authors of VM's, ie. Qemu:
https://www.qemu.org/2018/01/04/spectre/
With mitigation applied to kernel host machines should be safe.

Same regards to web browsers. Firefox applied own patches, and so Chrome. Qtwebengine would have mitigation from version 5.9.4 applied:
https://codereview.qt-project.org/#/c/216729/

https://bugreports.qt.io/browse/QTBUG-65561

To conclude IMHO kernel should be enough but also needed to be supported by patches to critical applications.

[tholin]

Can anyone explain me why should someone think about recompiling something more than a kernel??? I mean to defend from Specter. I was thinking that having kernel compiled with the retpoline switches I kinda guarantee this kernel will help me protecting my box. Am I wrong?

Compiling the kernel with retpolines will protect the kernel but only the kernel. Applications could still read each others memory but not the kernels memory. Recompiling everything will protect everything (except the firmware so you still need the microcode fix).

Also, as tholin wrote in the previous post, devs try or at least think to implement ELF changes which would be used by the kernel in order to speedup things IF the program with such an ELF has a bit indicating the retpoline protection switches were used. Is it a joke? I will compile my binary my way and then put whatever bit you want And the kernel would blindly think it is a wise and proper app???

The new bit would be use to tell the kernel that a program is protected by itself (with retpolines) so that the kernel doesn't have to protect it (using the new microcode functionality). You could compile a program without retpolines and set the bit. Then your program could be attacked by other program but in that case you have yourself to blame. The bit is only used to say a program i immune to attack, it says nothing about how trustworthy a program is.

So, back to the question: what is the background for aiming to the whole box re-compilation? And, if yes, how to do the same with external/binary packages??? And if yes, recompile and yes, no control of binary packages then how to protect if these are malicious intruders???

Binary packages without retpolines would have to be protected my the kernel using using the new microcode functionality. This has nothing to do with binary packages being evil.

[mv]

There are a lot of different things mixed here:

The point of spectre v2 is that a malevolent program can steal "any" data from every program it can access (e.g. by calling) which is not compiled with retpoline.

Since the kernel can be called by any program and can access any data, this is the most critical part: If this is not protected against stealing data, every other protection makes no sense.

But you will of course want to protect (i.e. recompile with retpoline) as well all other programs which deal with (or can be used to access) sensible data: All login/encryption/mail/... software

counter action should be done by authors of VM's, ie. Qemu [...]
With mitigation applied to kernel host machines should be safe.

If VM/Qemu/... itself uses retpolines in a proper way (and has no other security hole) the host machines are "safe" in the sense that one virtual machine cannot steal data from another one. But inside one virtual machine programs can happily steal data from each other unless these programs protect themselves against data theft by using retpoline.

Same regards to web browsers. Firefox applied own patches, and so Chrome

This is again a different story: Browsers attempt to make it harder to steal data by restricting the possibilities of browsers. So with a "protected" browser it is not so simple for a malevolent website (even if you access it with activated javascript) to read all data it wants from all of your programs which still lack repoline support. However, "not so simple" really means what it says; it is not impossible but only some additional obstacles have been built (e.g. by giving the website less accurate timers).

[PrSo]

With the assumption that every program should be treated as "untrusted" with a malicious code inside? Fair enough.

But even author of retpoline patches says:

Binaries with shared linkage
While our initial focus has been the protection of operating system and hypervisor-type targets, there are classes of user application for which this coverage is valuable. In these cases, it should be called out that shared linkage and runtimes will lead to frequent additional interactions with indirect branches. Ubiquitous examples include the Program Link Table (PLT) and dynamically loaded standard libraries. We will be publishing additional optimization notes and techniques for these cases.

source:
https://support.google.com/faqs/answer/7625886

As I my understanding is correct Spectre is cpu architectural design flaw, so without "in silicon" change it is impossible to eliminate the threat, and all those mitigations should only minimize the probability of stealing other program memory.

In that case I also wonder if there would be any problems with compile everything with retpoline, similar like with LTO and graphite.

[NeddySeagoon]

PrSo,

The software fix reduces or eliminates the threat by reducing or eliminating the use of the flawed silicon.
This comes at a performance cost.

Silicon changes will be required so that the retpoline patches are no longer required.
Once revised silicon is available, the retpoline patches will need to be dropped or the lost performance will not be regained.

[mv]

Once revised silicon is available, the retpoline patches will need to be dropped or the lost performance will not be regained.

The lost performance can never be regained: The whole prediction technique is vulnerable.
If newer processor will be faster then only because of other innovations, but not because of "regaining" the loss...
If I understand correctly, the current retpoline technique does not add additional overhead but just exactly prevents prediction.

[NeddySeagoon]

mv,

When you prevent prediction, you lose the performance that correct prediction would have gained.
That's a net loss to CPU throughput.

The lost performance can never be regained: The whole prediction technique is vulnerable.

That may be an implementation detail. I don't know enough about the implementation of branch prediction to know if it can be made safe.
There will be a lot of effort going into making it work safely.

[Pearlseattle]

Can anyone explain me why should someone think about recompiling something more than a kernel??? I mean to defend from Specter. I was thinking that having kernel compiled with the retpoline switches I kinda guarantee this kernel will help me protecting my box. Am I wrong?

Compiling the kernel with retpolines will protect the kernel but only the kernel. Applications could still read each others memory but not the kernels memory. Recompiling everything will protect everything (except the firmware so you still need the microcode fix).

Also, as tholin wrote in the previous post, devs try or at least think to implement ELF changes which would be used by the kernel in order to speedup things IF the program with such an ELF has a bit indicating the retpoline protection switches were used. Is it a joke? I will compile my binary my way and then put whatever bit you want And the kernel would blindly think it is a wise and proper app???

The new bit would be use to tell the kernel that a program is protected by itself (with retpolines) so that the kernel doesn't have to protect it (using the new microcode functionality). You could compile a program without retpolines and set the bit. Then your program could be attacked by other program but in that case you have yourself to blame. The bit is only used to say a program i immune to attack, it says nothing about how trustworthy a program is.

So, back to the question: what is the background for aiming to the whole box re-compilation? And, if yes, how to do the same with external/binary packages??? And if yes, recompile and yes, no control of binary packages then how to protect if these are malicious intruders???

Binary packages without retpolines would have to be protected my the kernel using using the new microcode functionality. This has nothing to do with binary packages being evil.

Well, I'm following this whole story since Jan 3 and everything has been very exciting and dynamic and etc... but to sum it up, especially regarding the fixes, I admit that I did not quite understand anything, hehe... .
Therefore, the questions posted by alex-kas kind of mirror my own ones.
I think that I do understand quite well the part involving the kernel being protected (by using retpolines, Kpti, whatever... AND the firware of case of Intel CPUs) => the kernel is protected no-matter-what, and that's it, right?!

But I still do not understand basically "everything else":
1) The new bit would be use to tell the kernel that a program is protected by itself (with retpolines) so that the kernel doesn't have to protect it (using the new microcode functionality)
The kernel would therefore "speculate" (hehe) that the program hasn't been corrupted? Meaning that if I'm a great cracker and I find out that I can (re/over)write program X on my target's server I'll just flip the bit and I'll have access to everything (not related to the kernel)?

2) Binary packages without retpolines would have to be protected my the kernel using using the new microcode functionality. This has nothing to do with binary packages being evil.
Basically same thing as #1, but let's say that somebody flips the retpoline-bit in a binary package (being nVidia drivers, Teamviewer, Chrome, etc...) => then just because of "THAT ONE PACKAGE" the whole environment would potentially be back to stoneage (if anyone uses that SW to find a flaw in the environment being OS or other apps), or not?

1+2)
Ok, summarized, I'm basically asking the following:
in a best-case-scenario, are we back to stoneage every-single-time that an attacker manages to introduce and execute on a host ANY ((pre)compiled) code that has the retpoline-flag switched on (as fake) which in turn woud exploit the meltdown/spectre analysis to gather critical info about the running kernel/programs (to then of course access through "normal" app and/or kernel bugs to then gain control of the target device)?

Thaaaaaaks a lot!


Edit: added some details 3 minutes later

[mv]

I'll just flip the bit and I'll have access to everything (not related to the kernel)?

No, the opposite: If you flip the bit, that program (and everything it can access) can potentially be accessed "from the outside". So if it is a program which is run with root permissions, everything can potentially be accessed "from the outside". But if in attacker can manage to flip the bit in a program which is run with root permissions, he can do everything else with this program, anyway, and in particular directly access and send the data where he wants to.

[mv]

When you prevent prediction, you lose the performance that correct prediction would have gained.
[...]
That may be an implementation detail.

It is not: If you don't lose time in one case but lose it in another, this can always be measured and used to understand which branch you have taken.
The only "solution" is that you do not loose time in any case; but this means simply higher internal parallelism and not prediction.

[mv]

The released binuitils-2.30 does not seem to know -z,retpolineplt:

Code: Select all

$ grep -Ri retpo binutils-2.30 || echo :\(
:(

Edit: I guess the newest gold linker from llvm does.

[lagalopex]

The released binuitils-2.30 does not seem to know -z,retpolineplt

Maybe have a look at their ml.
tl;dr:

I expect that compiler support for indirect branch rewriting together with -fno-plt is sufficient. There is simply no need to put any code into binutils at this point.

[The Main Man]

With gcc-7.3.0 :

Code: Select all

$ cat /sys/devices/system/cpu/vulnerabilities/*                                                                                                                                                         
Mitigation: PTI
Vulnerable
Mitigation: Full generic retpoline

[gengreen]

I applied the retpoline patch on a kernel 4.9.74 (minipli) and have rebuild my entire system with gcc 7.3.0 p1.0 on my glibc system.

Unlike the most recent kernel, I do not have

Code: Select all

/sys/devices/system/cpu/vulnerabilities/

to give me some information about the effect of retpoline on my system, but the performance did decrease badly, especially with qemu.

I'm currently rebuilding my musl system with gcc 7.3.0 and a retpoline kernel too, I except it will be more or less the same in loss of performance.

[Elleni]

That sounds to me like CONFIG_PAGE_TABLE_ISOLATION should be enabled for AMD processors. Or at least not setting it with the knowledge of leaving the vulnerability exposed.

So it seems that even if you compile kernel with CONFIG_PAGE_TABLE_ISOLATION=Y PTI is auto-disabled on AMD cpu anyway.

But the underlying issue is still whether or not AMD should have it enabled. From the prior information, the answer appears to be yes.

To enable the functionality, I had to enable the kernel option AND enable it on the kernel command line with "pti=on". After that (and only after that):

Code: Select all

 dmesg |grep -i isol
[    0.000000] Kernel/User page tables isolation: force enabled on command line.
[    0.000000] Kernel/User page tables isolation: enabled

(I got the idea from Naib's post on page 5 of this thread which referenced "pti=off". Thanks Naib!)

I think that you are playing here the advocatus diaboli role.

With the knowledge that the test case provided on wiki page was performed in 2013, and should be mitigated by KAISER (now PTI) I personally think that AMD statement to which you got link in mike155 post is still in power, of course with the assumption that AMD is aware of that vulnerability.

AMD processors are not subject to the types of attacks that the kernel page table isolation feature protects against.

I know that this could be some kind of uncomfortable situation but there is nothing more we can do for now than to trust AMD with that. Maybe someone will write PoC on that case in the near future proofing that AMD was duly diligent.

If you think different on that subject please feel free to contact AMD an ask them to resolve your possible concerns.

Mhm, now what is officially recommended on amd ryzen boxes ? Ehable CONFIG_PAGE_TABLE_ISOLATION=Y PTI and as its autodisabled by default enable it on the kernel command line with "pti=on" ? Or is this not required ?

Can anyone say something on amd ryzen really not being affected or may it be advisable to enable autodisabled PTI even on amd ryzen?

[mv]

The new gcc flags cause some problems on x86 (i.e. 32 bit):
gcc-7.3.0 and rust won't compile with -fno-plt (segfault), and firefox won't compile if any of -mindirect-branch=thunk or -mfunction-return=thunk is used.

There might be more (perhaps also for 64 bit...): This git repository will probably be updated regularly.

Edit: Another surprising observation: -mindirect-branch=thunk without -O2 (or better) will not protect you...

[kernelOfTruth]

The new gcc flags cause some problems on x86 (i.e. 32 bit):
gcc-7.3.0 and rust won't compile with -fno-plt (segfault), and firefox won't compile if any of -mindirect-branch=thunk or -mfunction-return=thunk is used.

There might be more (perhaps also for 64 bit...): This git repository will probably be updated regularly.

Edit: Another surprising observation: -mindirect-branch=thunk without -O2 (or better) will not protect you...

*Subscribing*

good to know !


Exactly what I was attempting to do: compile firefox

Thanks for saving me (and probably others) some time with troubleshooting to get firefox to compile, mv


I'll wait and see what will be done to get firefox to work with those retpoline flags ...


4.14.16 with gcc 7.3:

cat /sys/devices/system/cpu/vulnerabilities/*
Mitigation: PTI
Vulnerable
Mitigation: Full generic retpoline

[Naib]

Code: Select all

uname -a; for f in /sys/devices/system/cpu/vulnerabilities/*; do echo "${f##*/}:$(cat $f)" ;done
Linux fluidmotion 4.15.0-gentoo #1 SMP PREEMPT Mon Jan 29 23:30:55 GMT 2018 x86_64 AMD Ryzen 5 1600 Six-Core Processor AuthenticAMD GNU/Linux
meltdown:Not affected
spectre_v1:Vulnerable
spectre_v2:Mitigation: Full AMD retpoline

just need to sort out v1

[pjp]

Can anyone say something on amd ryzen really not being affected or may it be advisable to enable autodisabled PTI even on amd ryzen?

I can only add another reference of someone else also expressing interest. Until I learn otherwise, I'm considering it a desirable security setting.

I wish to present something else for consideration. The KAISER patchset was originally intended to harden the implementation of KASLR in the Linux kernel [1] [2]. It was hastily re-purposed to address Meltdown, and re-branded as KPTI in the process. Later, Thomas Lendacky submitted a patch that prevents KPTI from being activated by default for AMD processors - a patch that gentoo-sources is now carrying. AMD's pretext is that their processors are not affected by Meltdown.

My concern over this situation is that it may put AMD processors at a disadvantage in lieu of the security issue that KAISER was originally intended to protect against. That is, KASLR may be unnecessarily weakened on AMD processors, by default. Indeed, the "Practical Timing Side Channel Attacks Against Kernel Space ASLR" whitepaper [3] specifically tested their attacks on AMD processors, which were found to be affected.

Assuming that I'm correct, AMD users who enable both CONFIG_RANDOMIZE_BASE and CONFIG_PAGE_TABLE_ISOLATION will need to explicitly pass "pti=on" as a kernel option in order to harden KASLR, whereas Intel users will not. I realise that this is a less pressing concern then attending to Meltdown, but it struck me as being worthy of mention.

[Hossie]

So what compilation options for userspace is recommended now?

Code: Select all

-mindirect-branch=thunk -mfunction-return=thunk

?

[mv]

So what compilation options for userspace is recommended now?

I would guess

Code: Select all

-O2 -fno-plt -mindirect-branch=thunk -mfunction-return=thunk

for gcc. And with the next clang, I guess either

Code: Select all

-O2 -fno-plt -mretpoline

or (if this should be supported by the llvmgold plugin):

Code: Select all

-O2 -Wl,-z,repolineplt -mrepoline

[transpetaflops]

And with the next clang, I guess either

Code: Select all

-O2 -fno-plt -mretpoline

or (if this should be supported by the llvmgold plugin):

Code: Select all

-O2 -Wl,-z,repolineplt -mrepoline

Just out of curiosity. Do I define CFLAGS/CXXFLAGS specifically for clang somewhere in make.conf? I don't remember seeing it in the handbook.

[mv]

Do I define CFLAGS/CXXFLAGS specifically for clang somewhere in make.conf?

portage does not care which compiler you use, i.e. these variables just refer to your currently configured compiler.
If you want to switch compilers on a per-package bases, you might want to use app-portage/portage-bashrc-mv (from the mv overlay) which a few days ago has been updated to filter by default the correspondingly unknown new flags for gcc/clang.

[lagalopex]

I would guess

Code: Select all

-O2 -fno-plt -mindirect-branch=thunk -mfunction-return=thunk

for gcc. And with the next clang, I guess either

Code: Select all

-O2 -fno-plt -mretpoline

or (if this should be supported by the llvmgold plugin):

Code: Select all

-O2 -Wl,-z,repolineplt -mrepoline

That should be described in wiki (Project:Security/Vulnerabilities/Meltdown and Spectre) as well?
Pushing a news item would be nice as well, right?