Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[SOLVED]hardened gentoo on a SSD with lvm2 over dm-crypt
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
LoTeK
Apprentice
Apprentice


Joined: 26 Jul 2012
Posts: 270

PostPosted: Thu Dec 20, 2012 11:31 am    Post subject: [SOLVED]hardened gentoo on a SSD with lvm2 over dm-crypt Reply with quote

hi,
I want to install the most secure setup that I could find:
hardened gentoo (PaX with RSBAC) on a fully encrypted SSD with the unencrypted boot information on a USB-stick and LVM over the encrypted disk.

First question:
is this even possible?

Second question:
On http://en.gentoo-wiki.com/wiki/DM-Crypt_with_LUKS I've read:
Quote:

Note: using an SSD to store confidential data needs more consideration because of the way that an SSD relocates data in order to provide "wear levelling". Should an attacker have unfettered physical access to the drive, they will be able to read data that the operating system appeared to have overwritten by bypassing the flash controller. Even if you filled the entire disk, there's a large amount of "hidden" space for wear levelling that will contain previously written and now discarded data. You should instead consider using an SSD that does its own encryption.

but no information how to do that.

third question:
I've heard that skype and hardened project features don't work or don't work easy together, is this true? and are there big performance issues with this setup? my machines are pretty new (T420s), 15 seconds boot time without optimization (standart amd64 desktop profile, nothing encrypted), but too much lack of performance wouldn't justify my interest in a fortress like system.

btw, only because I'm paranoid doesn't mean that "they" aren't following me.. :D


Last edited by LoTeK on Fri Jan 25, 2013 7:41 pm; edited 1 time in total
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2458
Location: Germany

PostPosted: Thu Dec 20, 2012 12:36 pm    Post subject: Reply with quote

I can't help you with hardened Gentoo.

But I have fully encrypted disks (doesn't really matter if SSD or HDD), boot on USB (with encrypted keyfiles - attacker needs both the stick and my password to get at the HDDs), and root on LVM on LUKS on mdadm.

So yes, I'd say it's possible, under the assumption that "hardened Gentoo" doesn't mean "broken Gentoo" ;)

I've done it with a custom Initramfs http://en.gentoo-wiki.com/wiki/Initramfs

You should not have to worry about SSD wear leveling if you never put unencrypted data on it. So it gets relocated, so what? It doesn't make decryption any easier.

Quote:
and are there big performance issues with this setup?


If your CPU does not come with AES-NI support, or if you're not using AES for encryption with such a CPU, you will lose the speed of your SSD to the encryption. It's just not that fast (can't even utilize full HDD speeds).
Back to top
View user's profile Send private message
LoTeK
Apprentice
Apprentice


Joined: 26 Jul 2012
Posts: 270

PostPosted: Thu Dec 20, 2012 1:17 pm    Post subject: Reply with quote

Quote:
But I have fully encrypted disks (doesn't really matter if SSD or HDD), boot on USB (with encrypted keyfiles - attacker needs both the stick and my password to get at the HDDs), and root on LVM on LUKS on mdadm.

Cool.. but I thought the "boot stuff" have to be unencrypted?! So you boot up and have to enter the password for the encrypted keyfiles before anything else, or how does this work?
do this setup need more experience?

do you use TRIM? I've read on this page http://wiki.ubuntuusers.de/SSD/Verschl%C3%BCsselung that if one use TRIM one doesn't have to fill the SSD with random numbers first.

Quote:
So yes, I'd say it's possible, under the assumption that "hardened Gentoo" doesn't mean "broken Gentoo" ;)

So I sense that you aren't very excited about the hardened project, if not, why?

Quote:
You should not have to worry about SSD wear leveling if you never put unencrypted data on it. So it gets relocated, so what? It doesn't make decryption any easier.

Ok, but when I'm logged in my system the disk is unencrypted and if I store some data then it is unencrypted first, isn't it?
Back to top
View user's profile Send private message
Veldrin
Veteran
Veteran


Joined: 27 Jul 2004
Posts: 1942
Location: Zurich, Switzerland

PostPosted: Thu Dec 20, 2012 2:40 pm    Post subject: Reply with quote

LoTeK wrote:

Quote:
So yes, I'd say it's possible, under the assumption that "hardened Gentoo" doesn't mean "broken Gentoo" ;)

So I sense that you aren't very excited about the hardened project, if not, why?

I guess hardening has a tendency of breaking things.

I do run hardened gentoo on desktop and notebook, so it is perfectly possible.
But if you need proprietary application or worse kernel modules (vmware, graphic drivers et all) you may have to switch to a normal kernel and thus have a lower level of hardening.

Quote:
Quote:
But I have fully encrypted disks (doesn't really matter if SSD or HDD), boot on USB (with encrypted keyfiles - attacker needs both the stick and my password to get at the HDDs), and root on LVM on LUKS on mdadm.


Cool.. but I thought the "boot stuff" have to be unencrypted?! So you boot up and have to enter the password for the encrypted keyfiles before anything else, or how does this work?
do this setup need more experience?
You still need some part of the that is not-encrypted to boot - if that is a separate boot partition or some kind of usb stick is up to you.

frostschutz mentioned something else: there are 3 possibility to store passphrases: not at all (i.e they need to be entered as password), in a key file (which in case of the root partition defeats the purpose of encrypting imo), and in an encrypted key file.
The last option still forces you to enter a password (to decrypt the key), but it allows long (and therefore 'stronger) password/passphrases.

genkernel allows the use of encrypted key files, but they use a slightly different method of decrypting the luks/dm-crypt container than dmcrypt used in openrc. If you put everything into one large lvm, that you should have to worry, if you need some additional partitions decrypted (which are not handled by the initrd) then you need to be careful.

Quote:
third question:
I've heard that skype and hardened project features don't work or don't work easy together, is this true? and are there big performance issues with this setup? my machines are pretty new (T420s), 15 seconds boot time without optimization (standart amd64 desktop profile, nothing encrypted), but too much lack of performance wouldn't justify my interest in a fortress like system.

I haven't look in detail at skype, but the latest version 4.x seems to work with a hardened kernel.

The performance impact comes mainly from encrypting/decrypting the data during read and write. I never compared a plain ssd to a encrypted on, but imo the performance impact is not that big.

just my .02$
V.

PS. if some parts are unclear, feel free to ask back.
_________________
read the portage output!
If my answer is too concise, ask for an explanation.
Back to top
View user's profile Send private message
LoTeK
Apprentice
Apprentice


Joined: 26 Jul 2012
Posts: 270

PostPosted: Thu Dec 20, 2012 7:26 pm    Post subject: Reply with quote

Quote:
But if you need proprietary application or worse kernel modules (vmware, graphic drivers et all) you may have to switch to a normal kernel and thus have a lower level of hardening.

is it possible to have a hardend and a normal kernel on the same system? and to switch profiles?

Quote:
genkernel allows the use of encrypted key files, but they use a slightly different method of decrypting the luks/dm-crypt container than dmcrypt used in openrc. If you put everything into one large lvm, that you should have to worry, if you need some additional partitions decrypted (which are not handled by the initrd) then you need to be careful.

ok, so I'll go with the option that don't stores passwords... I want to put everything in one large lvm (the SSD is only 160 GB), is there a specific reason why one could need decrypted partitions?
Back to top
View user's profile Send private message
LoTeK
Apprentice
Apprentice


Joined: 26 Jul 2012
Posts: 270

PostPosted: Thu Dec 20, 2012 11:17 pm    Post subject: Reply with quote

well, I've tried it and failed of course :)
after following:
http://en.gentoo-wiki.com/wiki/Booting_encrypted_system_from_USB_stick#Setting_up_LVM
http://en.gentoo-wiki.com/wiki/DM-Crypt_with_LUKS
http://en.gentoo-wiki.com/wiki/LVM#Installing_Gentoo_on_LVM2
http://en.gentoo-wiki.com/wiki/Root_filesystem_over_LVM2,_DM-Crypt_and_RAID#DM-Crypt
and of course the installation documentations, after rebooting (with the usb-stick inside and boot from usb as first device) I've only seen a blank screen.

My guess is that my grubl.conf is wrong, because I didn't found a sample configuration. Here is my /boot/grub/grub.conf and /etc/fstab:

http://bpaste.net/show/65679
http://bpaste.net/show/65680

if I boot with my life usb-stick I can open the encrypted disk with
Code:
cryptsetup luksOpen /dev/sda luks
vgscan
vgscan - a y
Back to top
View user's profile Send private message
eccerr0r
Advocate
Advocate


Joined: 01 Jul 2004
Posts: 4011
Location: USA

PostPosted: Fri Dec 21, 2012 4:50 am    Post subject: Reply with quote

As for SSD's with their "own encryption" there are certain SSDs out there that will encrypt the data along with lock the drive with the ATA password. This is different than many mechanical disks that simply lock the ATA interface when it's "protected" which can be defeated by overwriting the password/replacing the nonvolatile memory that contains it. The only consumer SSD that I know of off the top of my head that has AES encryption is the Intel SSD520 series, but if that has encryption, likely many other Sandforce SSDs will as well. The advantage of this hopefully is the whole disk including metadata is encrypted, blocks can't be discerned as old or new, and might well be considered secure erased when put into the spare block/wear levelling pool. A layered software full disk encryption, it's possible to query the controller which blocks are in the deleted/spare pool and look for "interesting data."

http://www.intel.com/content/www/us/en/solid-state-drives/ssd-520-aes-tech-brief.html

It looks like they have gone to 256-bit AES too... But appears that not all of them have it.

And I still don't trust it yet :D
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed to be advocating?
Back to top
View user's profile Send private message
LoTeK
Apprentice
Apprentice


Joined: 26 Jul 2012
Posts: 270

PostPosted: Fri Dec 21, 2012 4:28 pm    Post subject: Reply with quote

yesterday I forgot to put "luks" in:
Code:
genkernel --lvm --luks --install initramfs

could this be the problem?
now when I'm chrooting with my minimal-usb-stick and try to mount the usb-stick with the /boot partition after mounting the following error message appears:
Code:
ls /boot:
ls: reading directory /boot/: Input/output error
Back to top
View user's profile Send private message
LoTeK
Apprentice
Apprentice


Joined: 26 Jul 2012
Posts: 270

PostPosted: Fri Dec 21, 2012 5:24 pm    Post subject: Reply with quote

ok, one step further:
the input/output error I've solved with fsck and then recreating the partition and the filesystem.
then I've run
Code:
gernkernel --lvm --luks --install initramfs

then I've followed closely this section from http://en.gentoo-wiki.com/wiki/Booting_encrypted_system_from_USB_stick#Setting_up_LVM
Quote:
Mount the boot partition.
mount LABEL=boot_stick /mnt/gentoo/boot
Create a boot -> . symlink
ln -s . /mnt/gentoo/boot/boot
Run grub-install.
grub-install --root-directory=/mnt/gentoo/boot /dev/sdu
Create a menu.lst -> grub.conf symlink
ln -s grub.conf /mnt/gentoo/boot/grub/menu.lst
Of course, you have to create the grub.conf itself for grub to know what it is supposed to boot. The following example is sufficient for booting a kernel with integrated ram disk, like the one you compiled earlier.
File: /mnt/gentoo/boot/grub.conf

timeout 30
default 0

title=Gentoo Linux (2.6.25.4)
root (hd0,0)
kernel /bzImage-2.6.25.4

Don't forget to copy the kernel image and System.map onto the USB key.
cp System.map /boot/System.map-2.6.25.4 cp arch/x86_64/boot/bzImage /boot/bzImage-2.6.25.4

now after rebooting a "command - line boot terminal" appears (is this busybox?) with the text:
Code:
GNU GRUB version 0.97 (630K lower / 523264K upper memory)

 [ Minimal BASH-like line editing is supported. For the first word, TAB lists possible command completions. Anywhere else TAB lists the possible completions of a device/filename. ]

then I type:
Code:
kernel /bzImage-3.6.11
boot /bzImage-3.6.11

then the system boots but then it hangs and I didn't asked for the password for the encrypted SSD.
Back to top
View user's profile Send private message
LoTeK
Apprentice
Apprentice


Joined: 26 Jul 2012
Posts: 270

PostPosted: Fri Dec 21, 2012 6:43 pm    Post subject: Reply with quote

I'm playing since 2 hours with initramfs and still the same hang when booting (without asked to prompt my password)
I've tried it with this pages:
http://whitehathouston.com/documentation/gentoo/initramfs_howto.htm
http://en.gentoo-wiki.com/wiki/Initramfs#Bootchart
Now I don't know what to do anymore and I'm going to get liquored up....

Here is my init: (/usr/src/initramfs/init)
http://bpaste.net/show/65829

/boot/grub.conf:
http://bpaste.net/show/65830

output of ls -la /boot:
http://bpaste.net/show/65831
when booting up and entering the grub from the usb-stick I did several different things like:
Code:
kernel /bzImage-3.6.11
initrd /initramfs
boot /bzImage-3.6.11


with always the same hang...
plz say if you need more information and if you have any suggestions...
Back to top
View user's profile Send private message
Veldrin
Veteran
Veteran


Joined: 27 Jul 2004
Posts: 1942
Location: Zurich, Switzerland

PostPosted: Sat Dec 22, 2012 9:44 pm    Post subject: Reply with quote

you need some additional parameters to the kernel line to get this working:
the following works for a plain installation, but might needs some tweaking for a lvm setup.

Code:
kernel / initramfs-genkernel-x86_64-3.6.11-gentoo ro crypt_root=/dev/sda3 real_root=/dev/mapper/root
initrd /kernel-3.6.11-gentoo

in essence you need to tell the kernel, where to look for the encrypted root partition (crypt_root=/dev/sda3, adapt to your system), then tell the kernel, where the decrypted device is located (real_root=/dev/mapper/root - this is the configuration if genkernel is used!)

which method are you using? dmcrypt or luks?

I am not using LVM, but this might help in the setup - if I understood correctly, you only need to add dolvm to the kernel line.


V.
_________________
read the portage output!
If my answer is too concise, ask for an explanation.
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2458
Location: Germany

PostPosted: Sat Dec 22, 2012 10:34 pm    Post subject: Reply with quote

Your init script seems to be missing the #!/bin/busybox sh in the first line unless you forgot to post that. Once the init runs it should not hang but give you a rescue shell if something fails...
Back to top
View user's profile Send private message
LoTeK
Apprentice
Apprentice


Joined: 26 Jul 2012
Posts: 270

PostPosted: Sun Dec 23, 2012 2:45 pm    Post subject: Reply with quote

@ Veldrin
I think I'm using dm-crypt with luks :oops:

shouldn't it be:
Code:
kernel /kernel-3.6.11-gentoo    ro crypt_root=/dev/sda3 real_root=/dev/mapper/root
initrd /initramfs-3.6.11-gentoo


since I have only one lvm-partition I have only /dev/sda
with 7 lvm's:
/dev/lvm/root
/dev/lvm/usr and so on

with this setup during the boot-process appears:
could not find the root block device in .

again I'm not ask to prompt my password
@ frostschutz
yes, I forgot to add "#!/bin/busybox" but either with it it doesn't work..

I've read that one shouldn't do the root partition in the LVM-stuff, could this be the problem? On http://en.gentoo-wiki.com/wiki/Booting_encrypted_system_from_USB_stick#Setting_up_LVM it's described to put everything in the LVM.
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2458
Location: Germany

PostPosted: Sun Dec 23, 2012 3:58 pm    Post subject: Reply with quote

Root on LVM works fine. However if you have a separate /usr partition, you have to mount that too.

But if you don't even get a rescue shell, something is wrong.

As for the kernel parameters, those are only required for genkernel initramfs. In your own initramfs, the root etc. parameters do not matter, unless you write code to evaluate those parameters in your init script.

http://en.gentoo-wiki.com/wiki/Booting_encrypted_system_from_USB_stick is a guide I wrote before the Gentoo Wiki died. It was transferred into the new wiki by someone else. Can't vouch for its quality - it's old.

http://en.gentoo-wiki.com/wiki/Initramfs is what I wrote in its stead and it works fine unless you miss something (like, modules - anything required to boot should be builtin unless you want module files in the initramfs).

If you're still following that Initramfs wiki page, can you upload your initramfs and kernel image somewhere. I'll have a look at it.
Back to top
View user's profile Send private message
LoTeK
Apprentice
Apprentice


Joined: 26 Jul 2012
Posts: 270

PostPosted: Mon Dec 24, 2012 1:23 pm    Post subject: Reply with quote

I've reinstalled the whole system because I thought I should move the root on a separate partition. Then I've read your post and put everything in one LVM-partition, so now I have /dev/sda encrypted with LVM on top:
Code:
/dev/mapper/vg-root
/dev/mapper/vg-usr
/dev/mapper/vg-var
/dev/mapper/vg-tmp
/dev/mapper/vg-home
/dev/mapper/vg-opt
/dev/mapper/vg-swap

I've followed your new initramfs page (the old one only for the boot-stick), now it's a bit different, I'm still not asked to prompt my password and during the boot-process the line : "something went wrong, dropping you in a rescue-shell" appears, but I'm not dropped into it.

then "/dev/sda doesn't exists or access denied" appears, so I guess the problem is the encryption?! First I forgot to copy /dev/{random, urandom}, then the line "/dev/random not found" appears, but then I've copied both and it's still not working...

my new /usr/src/initramfs/init:
http://bpaste.net/show/66283

/boot/grub/grub.conf:
http://bpaste.net/show/66284

/usr/src/linux/.config:
http://bpaste.net/show/66285
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2458
Location: Germany

PostPosted: Mon Dec 24, 2012 2:01 pm    Post subject: Reply with quote

Make sure that the standard folders (/bin /sbin etc.) exist in your initramfs image even if they're empty. Also make sure you have the standard device nodes. Add devtmpfs for good measure.

http://en.gentoo-wiki.com/wiki/Initramfs#Device_Nodes
http://en.gentoo-wiki.com/wiki/Initramfs#devtmpfs

Unless I missed something in your Kernel config, making sure the device nodes are present should get the rescue shell working, and once that works, you have something to work with in the Initramfs itself to figure out why the rest is not working as it should. E.g. in the rescue shell you could cat /proc/partitions to see if the kernel sees your HDD at all, otherwise there'll be a driver missing or modular when it should be builtin. Your config looks like it should support AHCI SATA disks though, if that's what you have.
Back to top
View user's profile Send private message
LoTeK
Apprentice
Apprentice


Joined: 26 Jul 2012
Posts: 270

PostPosted: Mon Dec 24, 2012 2:14 pm    Post subject: Reply with quote

oh guys, can you believe it?! I've done it :)
I don't know if it didn't work before, because I didn't copied the standard device nodes (now I've done it). I just entered the password during boot time and then it worked, strange...
I was asked to prompt it (maybe even before), but the process continued ( ca 15 lines ), then it stopped, then I entered the password and then I could log in. Is there a way to get a "nicer" prompt for the encryption password?
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2458
Location: Germany

PostPosted: Mon Dec 24, 2012 5:09 pm    Post subject: Reply with quote

I believe Ubuntu uses a very nice prompt (framebuffer graphics and everything).

But I've never done it myself. I don't care for graphics in the boot process, I prefer to see messages.

What you can do, in case you have problems with scrolling kernel messages (such as USB detection or whatever) during prompt, you can quiet the kernel messages:

Code:

# quiet kernel
echo 0 > /proc/sys/kernel/printk

# do something in silence
cryptsetup foo bar

# clean up
echo 1 > /proc/sys/kernel/printk
Back to top
View user's profile Send private message
LoTeK
Apprentice
Apprentice


Joined: 26 Jul 2012
Posts: 270

PostPosted: Mon Dec 24, 2012 5:33 pm    Post subject: Reply with quote

no I want to see messages too, but I mean I've not been ask to prompt my password, I just enter it during the boot process, during messages are appearing...I want that messages are appearing, then I should asked to prompt the password, after that, messages should appearing again...
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2458
Location: Germany

PostPosted: Mon Dec 24, 2012 9:51 pm    Post subject: Reply with quote

cryptsetup should display a "Enter passphrase for /dev/dubdedoo: " when it asks you for the password. This works fine for me.

It's only possible that the kernel detects some hardware with delay, and while that happens it prints messages after cryptsetup already printed its prompt. So you don't see the prompt anymore because it scrolled off the screen. And to prevent that from happening you can disable kernel printk while you type your password.

If you don't see a prompt whatsoever but can enter the password anyway, then I'm not sure what's happening.
Back to top
View user's profile Send private message
eccerr0r
Advocate
Advocate


Joined: 01 Jul 2004
Posts: 4011
Location: USA

PostPosted: Tue Dec 25, 2012 12:30 am    Post subject: Reply with quote

The prompt is probably there, but the asynchronous diagnostic output (while it scans usb, scsi, etc.) is probably scrolling past... A cheap way out is to add a delay and wait a few seconds for things to settle, then prompt for the password?
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed to be advocating?
Back to top
View user's profile Send private message
LoTeK
Apprentice
Apprentice


Joined: 26 Jul 2012
Posts: 270

PostPosted: Wed Dec 26, 2012 3:09 pm    Post subject: Reply with quote

yes the prompt is there and it's scrolled off the screen.
Quote:
# quiet kernel
echo 0 > /proc/sys/kernel/printk

# do something in silence
cryptsetup foo bar

# clean up
echo 1 > /proc/sys/kernel/printk
I've added those lines to my initramfs, but nothing is different.
@eccerr0r, how can I add a delay?
Is it possible to "tell" the kernel what he should do when?
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2458
Location: Germany

PostPosted: Wed Dec 26, 2012 5:58 pm    Post subject: Reply with quote

You were supposed to replace cryptsetup foo bar with your own cryptsetup line - i.e. just add the printk stuff before/after your prompt command. did you do that? :P

You can add a delay with sleep, "sleep 5" would delay 5 seconds
Back to top
View user's profile Send private message
LoTeK
Apprentice
Apprentice


Joined: 26 Jul 2012
Posts: 270

PostPosted: Wed Dec 26, 2012 6:25 pm    Post subject: Reply with quote

Quote:
You were supposed to replace cryptsetup foo bar with your own cryptsetup line - i.e. just add the printk stuff before/after your prompt command. did you do that? :P
haha, yes of course, I don't like the "copy-paste mentality" :)
ok, I'll try it with sleep...
and just to know for sure, in your case the messages "wait" until you entered your password or they also go on and you don't care (because it's not that important :) )
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2458
Location: Germany

PostPosted: Wed Dec 26, 2012 7:45 pm    Post subject: Reply with quote

Any messages occuring during the printk are simply not shown. It should work (if proc is mounted and all). So while I have the prompt the kernel is quiet. (But the messagess show up in dmesg/syslog later).
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum