Meltdown/Spectre: Unauthorized Disclosure of Kernel Memory

[blopsalot]

see below.

[Tony0945]

The designers seemingly never contemplated that abusing microarchitectural quirks was a viable way of attacking the system.

Reminds me of the phone phreaker "Captain Crunch" back in the day. The story is that he was eating breakfast and the box of Captain Crunch cereal had a toy whistle as a prize (cereal used to have prizes to entice kids to bug their moms to buy it). He blew the whistle into a phone and found it opened an unregistered trunk line. Now who would think to do that?

AT&T (then a monopoly) paid the cereal company (Kellog?) to redesign the whistle and bought up all the existing boxes, many millions of dollars.

[krinn]

The designers seemingly never contemplated that abusing microarchitectural quirks was a viable way of attacking the system.

Reminds me of the phone phreaker "Captain Crunch" back in the day. The story is that he was eating breakfast and the box of Captain Crunch cereal had a toy whistle as a prize (cereal used to have prizes to entice kids to bug their moms to buy it). He blew the whistle into a phone and found it opened an unregistered trunk line. Now who would think to do that?

AT&T (then a monopoly) paid the cereal company (Kellog?) to redesign the whistle and bought up all the existing boxes, many millions of dollars.

Money is the key in our world, to a point, that even if designers see a problem and report it: the issue will be balance against money gain ; and if product expected sells > money invest to fix the already known issue -> sell it.
Look at wolswagen, they aren't doing what they can to not pay any fee for the cheating program in their engines: they are doing what they can to pay the fee they have estimate they will be force to pay + estimate cost to fix cars affected. And that money is already in some account (doing even more money while they were waiting someone to caught them).

Because even AT&T appears like looser paying millions dollars to Kellog for a whistle, they were able to spend that much money because their pockets were full.
That's politic of systemd: who gives a fuck if systemd will be good or bad at Redhat? Redhat just need everyone to depends on them, systemd could be swap with anything they wish once your systems are tied to Redhat and they own all the market share. They don't need everyone using systemd, they need everyone linux to be Redhat, and if systemd is a failure, they will fix it or create a new product ; but you will be force to use that new product.

And if users/customers are force to pay for the new product, oh dude, you're a terrible financial genius!
People have pay your dirty product and will happily or not pay again to have functional one

Everyone point Intel, but the security issue is in all "modern" cpu, which will lead to "our products sucks, but not just us, and even AMD could claim meltdown is not an issue, spectre1 still hit them like us, and the cpu are not truly fixable, here are new cpu that aren't affected at all: no PTI, no hack, no microcode need: just some little more money from you buying them".
Because old product sells are expect to get down, little investors are scared and Intel is loosing them, but big investors that could afford to wait, are expecting Intel to get a new cpu all fixed (and this one out faster than concurrent because Intel is sleeping on a big money bed and could afford to spent some to faster research), and once that product is out: huge money coming in, what cpu will you buy if all cpu are "defective" but not the new cpu Intel has just release?
The best thing big investors in Intel or AMD company should do is investing few bucks in building spectre1 viruses that could be release once fixed design cpu are out to even get more crash return faster...

Face it guys, we do whatever we can to fix that: using PTI, waiting like crazy patches for gcc... But first they only "mitigate" the issue and not fix it, and second, if i were a "bad guy" doing virus, i would be coding a spectre1 one...
To me the future is all clear: spectre1 virus everywhere, and fixed cpu out top sells. If we trust the guys that found the issues, first fixed cpu out will take some time, but in mean time i think cpu maker will be able to get out "crappy" fixed cpus: cpu not affect by that because they lack speculation or cache or whatever, so running crappy but 100% safe ; which will be of value for many : think, do you prefer a kick ass cpu running fast for your server but everyone could hack them, or a slower one, but totally safe?
Which is even better for their pockets: you reuse a design you know but cut it to avoid the bugs, you have sold millions of affected cpu, you will sold millions of crappy fixed ones, and will later then sold millions of brand new one that run fast and are secure.
For one task your customer might brought 3 cpu (the "old" one, the crappy/slow fixed one, and the fixed fast one) instead of just one, awesome!
Even if customer will wait for fixed cpu (or no crappy fixed one appears in between), the second market will be killed for a while, and that second hand cpu is worth 0 for cpu maker, they have sold the unit, and anyone reselling it later will gave 0 money in their pocket, but if all second hand cpu are flaw, the second hand market will be fucked for some time, meaning sells of cpu will only be new cpu ones for some time, and on those sells, they do take the money.
Even Apple good plan to slow down cpu of older products is no more need: no need to slow down the iphone X, just build an iphone X+1 with a fixed cpu and users will rush to it.
To me that spectre/meldown is a total kick ass news for cpu maker, and good days are coming for them

[eccerr0r]

Hooray more planned/forced obsolescence! *sigh*

[1clue]

The designers seemingly never contemplated that abusing microarchitectural quirks was a viable way of attacking the system.

Reminds me of the phone phreaker "Captain Crunch" back in the day. The story is that he was eating breakfast and the box of Captain Crunch cereal had a toy whistle as a prize (cereal used to have prizes to entice kids to bug their moms to buy it). He blew the whistle into a phone and found it opened an unregistered trunk line. Now who would think to do that?

AT&T (then a monopoly) paid the cereal company (Kellog?) to redesign the whistle and bought up all the existing boxes, many millions of dollars.

How would you know, without prior knowledge? Assuming you opened the trunk line, how could you tell? What could you do with it, unless you had all the gear sitting there to take advantage?

[Aiken]

Exactly.

The PoC are earlier on this thread but need to be modified to run on the older chips. I was able determine that my Core2 Duo and Core2 Quad are affected by at least Spectre, but the PoC run so poorly that it was successful 15% of the time.

However if it was able to get it 15% of the time, it means that a determined hacker is still able to get the information, just takes longer.

The Am-I-affected-by-Meltdown checker which seems to do an actual check instead of looking for the pti flag shows my core2 duo as affected by metldown. The only machine I have not been able to get a positive result on is the p4 and that may be a timing issue like getting the spectre poc working on older processors. I have 1 c2d with 4.14.15 and another with 4.9.78. If I cat /sys/devices/system/cpu/vulnerabilities/* both are now showing

Mitigation: PTI
Vulnerable
Vulnerable: Minimal generic ASM retpoline

If I boot with pti=pff they both show

Vulnerable
Vulnerable
Vulnerable: Minimal generic ASM retpolin

[mike155]

GCC 7.3 was released a couple of hours ago:

The GNU Compiler Collection version 7.3 has been released.

GCC 7.3 is a bug-fix release from the GCC 7 branch
containing important fixes for regressions and serious bugs in
GCC 7.2 with more than 99 bugs fixed since the previous release.

This release includes code generation options to mitigate
Spectre Variant 2 (CVE 2017-5715) for the x86 and powerpc targets.

[...]

See: https://gcc.gnu.org/ml/gcc-announce/2018/msg00000.html

[blopsalot]

see below

[rburcham]

Should we expect to see the gcc and binutils updates in portage as ~arch here pretty soon?

[Zentoo]

The designers seemingly never contemplated that abusing microarchitectural quirks was a viable way of attacking the system.

Reminds me of the phone phreaker "Captain Crunch" back in the day. The story is that he was eating breakfast and the box of Captain Crunch cereal had a toy whistle as a prize (cereal used to have prizes to entice kids to bug their moms to buy it). He blew the whistle into a phone and found it opened an unregistered trunk line. Now who would think to do that?

AT&T (then a monopoly) paid the cereal company (Kellog?) to redesign the whistle and bought up all the existing boxes, many millions of dollars.

Money is the key in our world, to a point, that even if designers see a problem and report it: the issue will be balance against money gain ; and if product expected sells > money invest to fix the already known issue -> sell it.
Look at wolswagen, they aren't doing what they can to not pay any fee for the cheating program in their engines: they are doing what they can to pay the fee they have estimate they will be force to pay + estimate cost to fix cars affected. And that money is already in some account (doing even more money while they were waiting someone to caught them).

Because even AT&T appears like looser paying millions dollars to Kellog for a whistle, they were able to spend that much money because their pockets were full.
That's politic of systemd: who gives a fuck if systemd will be good or bad at Redhat? Redhat just need everyone to depends on them, systemd could be swap with anything they wish once your systems are tied to Redhat and they own all the market share. They don't need everyone using systemd, they need everyone linux to be Redhat, and if systemd is a failure, they will fix it or create a new product ; but you will be force to use that new product.

And if users/customers are force to pay for the new product, oh dude, you're a terrible financial genius!
People have pay your dirty product and will happily or not pay again to have functional one

Everyone point Intel, but the security issue is in all "modern" cpu, which will lead to "our products sucks, but not just us, and even AMD could claim meltdown is not an issue, spectre1 still hit them like us, and the cpu are not truly fixable, here are new cpu that aren't affected at all: no PTI, no hack, no microcode need: just some little more money from you buying them".
Because old product sells are expect to get down, little investors are scared and Intel is loosing them, but big investors that could afford to wait, are expecting Intel to get a new cpu all fixed (and this one out faster than concurrent because Intel is sleeping on a big money bed and could afford to spent some to faster research), and once that product is out: huge money coming in, what cpu will you buy if all cpu are "defective" but not the new cpu Intel has just release?
The best thing big investors in Intel or AMD company should do is investing few bucks in building spectre1 viruses that could be release once fixed design cpu are out to even get more crash return faster...

Face it guys, we do whatever we can to fix that: using PTI, waiting like crazy patches for gcc... But first they only "mitigate" the issue and not fix it, and second, if i were a "bad guy" doing virus, i would be coding a spectre1 one...
To me the future is all clear: spectre1 virus everywhere, and fixed cpu out top sells. If we trust the guys that found the issues, first fixed cpu out will take some time, but in mean time i think cpu maker will be able to get out "crappy" fixed cpus: cpu not affect by that because they lack speculation or cache or whatever, so running crappy but 100% safe ; which will be of value for many : think, do you prefer a kick ass cpu running fast for your server but everyone could hack them, or a slower one, but totally safe?
Which is even better for their pockets: you reuse a design you know but cut it to avoid the bugs, you have sold millions of affected cpu, you will sold millions of crappy fixed ones, and will later then sold millions of brand new one that run fast and are secure.
For one task your customer might brought 3 cpu (the "old" one, the crappy/slow fixed one, and the fixed fast one) instead of just one, awesome!
Even if customer will wait for fixed cpu (or no crappy fixed one appears in between), the second market will be killed for a while, and that second hand cpu is worth 0 for cpu maker, they have sold the unit, and anyone reselling it later will gave 0 money in their pocket, but if all second hand cpu are flaw, the second hand market will be fucked for some time, meaning sells of cpu will only be new cpu ones for some time, and on those sells, they do take the money.
Even Apple good plan to slow down cpu of older products is no more need: no need to slow down the iphone X, just build an iphone X+1 with a fixed cpu and users will rush to it.
To me that spectre/meldown is a total kick ass news for cpu maker, and good days are coming for them

Hey dude ! You make my day ! It's accuratelly that I think... Big money pocket always find a way to make their pockets bigger !
Even after that everyone knows that Intel CEO have sold all his company shares on the stock market before the Meltdown NDA and even after Intel have just diluated their responsabilities about Meltdown by creating the confusion with Spectre problem (telling to the world that they was not alone to be affected: other cpu sellers like AMD, IBM, ARM are impacted too), no one take care about how goes the business behind....
To sum up, Meltdown is an Intel flaw while Spectre is a a flaw by design (Tomasulo algorithm) implemented in almost all modern processors.
Everyone are thinking now it's the same problem but it's not. Really good communication strategy from Intel.
And for sure we will all of us buy a fixed cpu the day we can !

The most interesting part is coming: how they could sell us new fixed cpu with less performance since I can't see how they can create more powerfull CPU easily.
Adding more CPU cache could be the key but actual problems are related to CPU cache so it's a no go. And since they will hit the lowest thickness manufacturing process, there is no so much room to go ahead on performance. The only way that I can see is parallelism (so more core) because AMD force them to open this path with their last architecture....
So wait and see.... but stay aware in which pocket your money will go !

[blopsalot]

updated

[mv]

add "-mindirect-branch=thunk" to your CFLAGS

I am again somewhat confused: The kernel adds -mindirect-branch=thunk-extern. Is this meant only for the kernel or is it better/worse?
Also AFAIK the kernel uses -mindirect-branch-register. Sounds like it might be quicker. Shouldn't one activate it?
And why does nobody seem to use -mfunction-return? Is it not necessary? (And if it is useful: Is the value thunk-inline or thunk-extern preferrable?)
Moreover, what is the connection of all these flags with new binutils? Won't they work without it?

[blopsalot]

i am still trying to catch up too, so please correct any mistakes i make. my understanding is that the mindirect-branch can go different ways, using updated microcode or if not available, comparable assembly. the kernel is already configured to set compiler options appropriately automatically, it is my understanding only mindirect-branch=thunk is needed universally, but i might be wrong. binutils must be updated so it knows what to do with new code, I can't find mailing list post atm.

edit:

Add -mindirect-branch= option to convert indirect call and jump to call
and return thunks. The default is 'keep', which keeps indirect call and
jump unmodified. 'thunk' converts indirect call and jump to call and
return thunk. 'thunk-inline' converts indirect call and jump to inlined
call and return thunk. 'thunk-extern' converts indirect call and jump to
external call and return thunk provided in a separate object file. You
can control this behavior for a specific function by using the function
attribute indirect_branch.

https://sourceware.org/ml/binutils/2018 ... 00030.html

[mv]

https://sourceware.org/ml/binutils/2018 ... 00030.html

This looks like it is a mitigation technique for libraries which is independent of the gcc compiler options. But I might be wrong.

[mike155]

add "-mindirect-branch=thunk" to your CFLAGS

I got the result I posted here ('Mitigation: Full generic retpoline') with vanilla GCC 7.3.0RC1 and vanilla Linux kernel 4.14.14, but WITHOUT adding anything to CFLAGS and WITHOUT updating binutils.

The Linux kernel build system knows which options it has to pass to GCC. Look at /usr/src/linux/arch/x86/Makefile, line 239:

Code: Select all

# Avoid indirect branches in kernel to deal with Spectre
ifdef CONFIG_RETPOLINE
    RETPOLINE_CFLAGS += $(call cc-option,-mindirect-branch=thunk-extern -mindirect-branch-register)
    ifneq ($(RETPOLINE_CFLAGS),)
        KBUILD_CFLAGS += $(RETPOLINE_CFLAGS) -DRETPOLINE
    endif
endif

[blopsalot]

https://sourceware.org/ml/binutils/2018 ... 00030.html

This looks like it is a mitigation technique for libraries which is independent of the gcc compiler options. But I might be wrong.

i guess my edit implied they were related? no i was just seeing if it worked in as many places as possible. i ended up editing the eclass.

my point was binutils needs updated as well if you want to harden your toolchain.

add "-mindirect-branch=thunk" to your CFLAGS

I got the result I posted here ('Mitigation: Full generic retpoline') with vanilla GCC 7.3.0RC1 and vanilla Linux kernel 4.14.14, but WITHOUT adding anything to CFLAGS and WITHOUT updating binutils.

The Linux kernel build system knows which options it has to pass to GCC. Look at /usr/src/linux/arch/x86/Makefile, line 239:

Code: Select all

# Avoid indirect branches in kernel to deal with Spectre
ifdef CONFIG_RETPOLINE
    RETPOLINE_CFLAGS += $(call cc-option,-mindirect-branch=thunk-extern -mindirect-branch-register)
    ifneq ($(RETPOLINE_CFLAGS),)
        KBUILD_CFLAGS += $(RETPOLINE_CFLAGS) -DRETPOLINE
    endif
endif

yeah i said that lol

edit:
the conclusion I have come to is that you use -mindirect-branch=thunk-extern -mindirect-branch-register if your processor got a microcode update. -mindirect-branch=thunk if not. -mindirect-branch=thunk is safe(ish)(my world sets are not that big) to apply globally, ignoring performance impact

heres what i was referring to:

What you should know?
> --------------------------------
>
> * Techniques such as PGO and LTO dramatically reduce the impact of hot
> indirect calls (by speculatively promoting them to direct calls). If you
> need to deploy these techniques in C++ applications, we *strongly* recommend
> that you ensure all hot call targets are statically linked (avoiding PLT
> indirection) and use both PGO and LTO. Well tuned servers using all of these
> techniques saw 5% - 10% overhead from the use of the full retpoline
> mitigation (including compiler support).
>
> * Binutils tools like readelf and objdump will not disassemble the PLT
> section accurately as they assume that the PLT entry size is 16 bytes. A
> patch to fix this is in progress.

[mv]

Full generic retpoline') with vanilla GCC 7.3.0RC1 and vanilla Linux kernel 4.14.14

This might be true for the kernel, but my question is more related to mitigating spectre for other applications.

[mv]

the conclusion I have come to is that you use -mindirect-branch=thunk-extern -mindirect-branch-register if your processor got a microcode update.

The description does not indicate this. From the description of -mindirect-branch it sounds more like thunk-extern is appropriate for the kernel while thunk is appropriate for "ordinary" linking (as in userspace). But actually the description is so vague that it is impossible to understand what is meant. That's why I asked here...

[blopsalot]

the conclusion I have come to is that you use -mindirect-branch=thunk-extern -mindirect-branch-register if your processor got a microcode update.

The description does not indicate this. From the description of -mindirect-branch it sounds more like thunk-extern is appropriate for the kernel while thunk is appropriate for "ordinary" linking (as in userspace). But actually the description is so vague that it is impossible to understand what is meant. That's why I asked here...

at least llvm made it easy to understand lol.

edits:
tldr:
thunk is the original google implementation, but after realizing it didn't work in kernel --- variant 1
https://support.google.com/faqs/answer/7625886

they needed another version: thunk-extern --- variant 2
https://lwn.net/Articles/742756/

thunk-inline is a gcc rendition to inline the code in their asm, IT IS compatible with -mcmodel=large.
https://patchwork.ozlabs.org/patch/856627/
https://github.com/gentoo/gentoo/commit ... 88822ebba2

binutils better explanation on PLT speculative execution barriers --- another variant 2 (sorry for confusion, amd64 was NOT patched besides readelf and objdump, so looks like it only contains variant 2 "fix" for powerpc and powerpc64
https://sourceware.org/ml/binutils/2018 ... 00290.html
https://sourceware.org/ml/binutils/2018 ... 00134.html

-mindirect-branch=thunk is the right choice to enable globally. kernel will use what its supposed to automagically.

the processors are broke, retpoline is duck tape. lol

[mv]

So let me summarize my current understanding.
Every claim in this list is to be understood as a question whether I understood correctly...

1. thunk-extern is needed only for the kernel or kernel modules; everything else should use thunk.
2. Only if -mcmodel=large is needed one should fallback to thunk-internal because it is the slowest or most memory-expensive variant.
3. -mindirect-branch=... is needed for practically every x86/amd64 processor. On llvm one should use -mretpoline instead
4. -mfunction-return=... is needed only on Intel I3-6???* and newer processors
5. llvm currently does not have the additional protection against the return prediction variant of spectre provided by -mfunction-return=...
6. -Wl,-z,retpolineplt is not needed at all on amd64/x86 architecture
7. -mindirect-branch-register is either
   (a) an additional protection for some processors for which certain register jumps can be exploited (i.e. it uses the -mindirect-branch=... mechanism also in these cases) or
   (b) it is a different implementation of -mindirect-branch=... which uses a register which might be slower or faster than the original implementation.
8. The microcode update is something independent which is not needed for any of the flags: The retpoline mechanism is superior to using an extra command which was intel's oiriginal suggestion

In particular, I would be interested in case (a) of -mindirect-branch-register for which processors this is needed and whether it is provided by llvm.
Also, I am curious why the kernel is not using -mfunction-return=thunk-extern, at least optionally.

[blopsalot]

1. I have only tested thunk, it seems stable so far on amd64 glibc and uclibc.
2. That is the question.
3. I would assume all processors with similar branch prediction are even though it will not be admitted until proven. powerpc and powerpc64 for sure. arm has admitted some chips are.
4. i have not messed with it yet, it looks like it though. Theres also -mindirect-branch-loop
https://gcc.gnu.org/ml/gcc-patches/2018 ... 00423.html
https://raw.githubusercontent.com/gcc-m ... 386/i386.c

Code: Select all

else
	cfun->machine->function_return_type = ix86_function_return;

      /* -mcmodel=large is not compatible with -mfunction-return=thunk
	 nor -mfunction-return=thunk-extern.  */
      if ((ix86_cmodel == CM_LARGE || ix86_cmodel == CM_LARGE_PIC)
	  && ((cfun->machine->function_return_type
	       == indirect_branch_thunk_extern)
	      || (cfun->machine->function_return_type
		  == indirect_branch_thunk)))
	error ("%<-mfunction-return=%s%> and %<-mcmodel=large%> are not "
	       "compatible",
	       ((cfun->machine->function_return_type
		 == indirect_branch_thunk_extern)
		? "thunk-extern" : "thunk"));
    }

5. I mislabeled as variant 1. variant 1 is the inherent hardware flaw
6. The patch was withheld for now to those targets, it is however the only variant 2 ppc and ppc64 fix now that I know of and is in 2.30. They are rapidly stabilizing 2.30
7. it's one of the tricks the kernel does
https://patchwork.ozlabs.org/patch/859912/
8. My testing machine is a westmere xeon, I will test more on that soon. I would think you use -mfunction-return -mindirect-branch-loop based on the naming "cfun->machine->function_return_type = ix86_function_return;" from above, maybe someone else knows?

They are both kernel specific as far as I know, but I am sure there will be other uses, cannot remember details atm, but it has to do with the protected memory. Not sure about llvm kernel compiling, I would think it's gcc only for now, but they do not take long. This is a good write up though.
https://reviews.llvm.org/rLLD323155

[tholin]

The microcode update is something independent which is not needed for any of the flags: The retpoline mechanism is superior to using an extra command which was intel's oiriginal suggestion

These compiler switches are only related retpolines but my understanding is that even if the entire system is compiled with retpolines the microcode fix is still needed. The kernel need some way to flush the branch predictor before calls to UEFI runtime services because the UEFI is not compiled with retpolines. Transitions to system management mode also needs to be protected and the new microcode do that automatically without operating system involvement.

An idea I saw somewhere was to add a bit the ELF headers so programs can tell the kernel they have been compiled with retpolines. That way the kernel don't have flush the branch predictor when context switching to such a program. The kernel and toolchain people are still weighing their options.

[alex-kas]

Hi folks,
I'm totally confused, my brain is exploding.

Can anyone explain me why should someone think about recompiling something more than a kernel??? I mean to defend from Specter. I was thinking that having kernel compiled with the retpoline switches I kinda guarantee this kernel will help me protecting my box. Am I wrong?
Reading previous posts in this and other threads I got that people want other stuff to be recompiled. Why? Do you want to say that even if the kernel is done right some program may still penetrate inside because this program itself has not been retpoline-d? But than what the point in all of that? I think I'm kinda sure that standard packages are not intended to steal my data. I want to be protected from something of the totally third-party staff, which I had no control at all during the stage of its construction/compiling/linking/etc. Isn't this the point of defense???

Also, as tholin wrote in the previous post, devs try or at least think to implement ELF changes which would be used by the kernel in order to speedup things IF the program with such an ELF has a bit indicating the retpoline protection switches were used. Is it a joke? I will compile my binary my way and then put whatever bit you want And the kernel would blindly think it is a wise and proper app???

So, back to the question: what is the background for aiming to the whole box re-compilation? And, if yes, how to do the same with external/binary packages??? And if yes, recompile and yes, no control of binary packages then how to protect if these are malicious intruders???

Being driven nuts

[NeddySeagoon]

alex-kas,

Do you believe the kernel ?

Code: Select all

cat /sys/devices/system/cpu/vulnerabilities/*
Not affected
Vulnerable
Vulnerable: Minimal AMD ASM retpoline

Thats on

Code: Select all

uname -a
Linux NeddySeagoon_Static 4.15.0-rc8 #1 SMP PREEMPT Mon Jan 15 15:49:15 GMT 2018 x86_64 AMD Phenom(tm) II X6 1090T Processor AuthenticAMD GNU/Linux

built with gcc-7.2.0-r1.
I'll test with gcc-7.3 later.

[The Main Man]

But that doesn't make sense (the need to compile everything with repto and only then you're safe), ok here in gentoo we do that from time to time for various reasons, so no big deal, but how would that work in binary distributions, what about packages coming from ppa's or other non-distro sources, not to mention other OS's like Windows, how's that gonna work ...