Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Encrypt entire disk on a running machine
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
sgao
Tux's lil' helper
Tux's lil' helper


Joined: 22 Apr 2006
Posts: 149

PostPosted: Tue Sep 15, 2015 3:43 pm    Post subject: Encrypt entire disk on a running machine Reply with quote

Hello,

Anyone has experience with adding entire disk encryption after the machine is already installed and running?

Thanks.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7051
Location: almost Mile High in the USA

PostPosted: Tue Sep 15, 2015 6:05 pm    Post subject: Reply with quote

None such luck on linux, unless you installed on a filesystem that supports in-place encryption (btrfs? maybe? someday?) you'll have to backup, wipe, setup encryption, and restore...
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Roman_Gruber
Advocate
Advocate


Joined: 03 Oct 2006
Posts: 3806
Location: Austro Bavaria

PostPosted: Wed Sep 16, 2015 4:39 pm    Post subject: Reply with quote

usually

build a setup with luks, i prefer luks + lvm

create lvm container (needs free space on the harddrive)
create luks

i forgot how i moved my installation, but i can say that this box here was before 2012 without luks / lvm.

one way is a stage 4 tarball,

and i use / used ext 3 => ext4 in my hole linux life, the few years before i used reiserfs and that was totally buggy...

it could be that you need to recreate some special folders when you copy over the files, but these are nearly the same as doing a fresh install and creating /dev and such (well what i remember vague, please refer to guides for moving installations ...)

yes it is possible
The best way is to switch harddrives in the process as you need to copy / anway ...
I never had harddrive failues because i regularly kick out those hdd after 2-3 years max.


---
it all depends on your setup.

With lvm you could create snapshots and move your installation. With prerequistes ...

I suggest you get a new harddrive or ssd and set it up that way, harddrives hsould be changed anyway regularly.

when you only have / on an ordinary file system you will need probably a sysremrescue-cd and an empty partition which is a bit bigger as / to move your existing installation.
Back to top
View user's profile Send private message
davidm
Guru
Guru


Joined: 26 Apr 2009
Posts: 557
Location: US

PostPosted: Wed Sep 16, 2015 6:35 pm    Post subject: Reply with quote

eccerr0r wrote:
None such luck on linux, unless you installed on a filesystem that supports in-place encryption (btrfs? maybe? someday?) you'll have to backup, wipe, setup encryption, and restore...


No, unfortunately btrfs hasn't got the native or on-the-fly encryption options yet. Though some do it with LUKS and LVM I believe (with some cost to performance and possibly redundancy depending on how it is done). I've been thinking about doing it myself when I next buy a couple more 2TB disks.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7051
Location: almost Mile High in the USA

PostPosted: Wed Sep 16, 2015 7:00 pm    Post subject: Reply with quote

davidm wrote:
eccerr0r wrote:
None such luck on linux, unless you installed on a filesystem that supports in-place encryption (btrfs? maybe? someday?) you'll have to backup, wipe, setup encryption, and restore...


No, unfortunately btrfs hasn't got the native or on-the-fly encryption options yet. Though some do it with LUKS and LVM I believe (with some cost to performance and possibly redundancy depending on how it is done). I've been thinking about doing it myself when I next buy a couple more 2TB disks.


LUKS / LVM does not do in-place encryption either like what some windows full disk encryption software can do (like McAfee full disk encryption - you can even use your computer while it encrypts your NTFS partition in the background without reformatting, like what the OP seems to want). Heck there has been no in-place online defragmenter for Linux where it's been around for years on Windows, mainly because Linux users expect people to do it the old fashioned way: use another disk/partition and copy back and forth, which is what all the options I see for encryption here are, too.

Having gparted was a miracle in itself. Alas you still can't use the partitions while it's resizing them.

The best hope is if a filesystem that supports (or eventually supports - unfortunately it's not ready now) encryption natively was chosen. But for now, the only option is to wipe and reinstall.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
szatox
Veteran
Veteran


Joined: 27 Aug 2013
Posts: 1717

PostPosted: Wed Sep 16, 2015 8:37 pm    Post subject: Reply with quote

Quote:
Heck there has been no in-place online defragmenter for Linux where it's been around for years on Windows
There is no such thing as "in place defragmenter", the whole process is all about moving scattered parts. Shake seems to be as close as possible though, for those rare cases when you really need it.

Regarding switching encryption on and off, you can definitely do that if you have some extra space. Start with rebuilding kernel (and initramfs, if you want to have encrypted root. You will need separate /boot too), it must be able to use container. Copy base system, make sure you can boot it (e.g. unplug primary drive). Once you know it works, rsync the rest.
I have moved from a single drive to raid in similar way.

And about doing it in place, you have just made me want to play a bit with device mapper. Too bad it will have to wait until I'm done with some other stuff :]
Back to top
View user's profile Send private message
papahuhn
l33t
l33t


Joined: 06 Sep 2004
Posts: 623

PostPosted: Wed Sep 16, 2015 9:29 pm    Post subject: Reply with quote

It is quite simple to encrypt an unmounted device in-place if there is no block shifting involved, e.g. plain dm-crypt or luks with detached header and payload-align 0.
_________________
Death by snoo-snoo!
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7051
Location: almost Mile High in the USA

PostPosted: Wed Sep 16, 2015 10:46 pm    Post subject: Reply with quote

Maybe the best term is "online defragmenter" or "online disk encrypter" rather than "in-place" where you can use the same, utilized disk as temporary space (and also able to use the disk while the operation is in progress). Neither of which exist in Linux but do for Windows. About the best thing I've heard of for Linux is the md reshape - I'm surprised this even exists, kudos for the MD team who added this functionality, as a progress indicator and scratch space is needed. Making sure you don't corrupt data while doing two things at once is the trick here.

All defragmentation and full disk encryption for Linux involves unmounting some volume/partition, implying there needs to be at minimum another disk/volume that can temporarily hold all of the data, and a quiescent period is needed to ensure that you don't forget to copy/encrypt newly created data. Sure if you want to rsync the data later, but that timing hole exists.

If either tool exists today (defrag or full disk encrypt a utilized disk without needing a separate partition) I'd be surprised as I haven't heard of them.

(and those who think Linux never gets fragmented files have never bittorrented with on the fly allocation. If you have double the space of the file being downloaded to save the blocks as they download to reassemble later, then this doesn't count - this is basically the same as offline defragmentation in userland except one would hope you don't try to defragment while blocks are still being downloaded...)

And well, of course after searching for it, one finds: http://www.johannes-bauer.com/linux/luksipc/
This is the solution that is needed, alas it still requires downtime and a separate partition that includes this software if the desired volume to be encrypted is root.
Back to top
View user's profile Send private message
papahuhn
l33t
l33t


Joined: 06 Sep 2004
Posts: 623

PostPosted: Thu Sep 17, 2015 7:38 am    Post subject: Reply with quote

eccerr0r wrote:
All defragmentation and full disk encryption for Linux involves unmounting some volume/partition, implying there needs to be at minimum another disk/volume that can temporarily hold all of the data, and a quiescent period is needed to ensure that you don't forget to copy/encrypt newly created data. Sure if you want to rsync the data later, but that timing hole exists.


Unmounting is true, but the implication is not. If one is willing to use LUKS with a detached header (which can be a benefit for certain scenarios), it is sufficient to use "dd if=/dev/unenc_dev of=/dev/enc_dev_onTopOf_unenc_dev". Btw. are you sure, that unmounting is not needed on Windows? What I think to know of TrueCrypt under Windows is, that you need to reboot first, and then TrueCrypt can actually begin to encrypt the system partition.
_________________
Death by snoo-snoo!
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7051
Location: almost Mile High in the USA

PostPosted: Thu Sep 17, 2015 1:10 pm    Post subject: Reply with quote

Try doing the same in Linux... Even if you have to reboot once
No equivalent tool exists, you get a copy hole in Linux, but Windows is seamless.
If anyone wants to write such a tool the opportunity is here:

Write a program for linux that will encrypt a hard drive without needing another hard disk or partition even if your disk is almost full, with minimal downtime and minimal if any loss when random writes occur to the source medium during encryption.

This is what the OP seems to want.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
HeissFuss
Guru
Guru


Joined: 11 Jan 2005
Posts: 413

PostPosted: Sun Oct 18, 2015 9:45 pm    Post subject: Reply with quote

papahuhn wrote:
eccerr0r wrote:
All defragmentation and full disk encryption for Linux involves unmounting some volume/partition, implying there needs to be at minimum another disk/volume that can temporarily hold all of the data, and a quiescent period is needed to ensure that you don't forget to copy/encrypt newly created data. Sure if you want to rsync the data later, but that timing hole exists.


Unmounting is true, but the implication is not. If one is willing to use LUKS with a detached header (which can be a benefit for certain scenarios), it is sufficient to use "dd if=/dev/unenc_dev of=/dev/enc_dev_onTopOf_unenc_dev". Btw. are you sure, that unmounting is not needed on Windows? What I think to know of TrueCrypt under Windows is, that you need to reboot first, and then TrueCrypt can actually begin to encrypt the system partition.


There is also a new experimental tool that's included with cryptsetup called cryptsetup-reencrypt (enabled by adding reencrypt use flag to cryptsetup.) Contrary to the name it's not just for reencryption, it also allows luks encryption of an unencrypted offline device and is probably safer than running dd (similar to md reshape, if you put log and temp files from the operations in a persistent location it can be resumed.) It would still be a very good idea to make a backup before doing this though. And you still need to set up grub to allow booting encrypted.


In relation to the online defragmentation, there are some options:
ext4 (e4defrag) - Not much better than shake as far as how it works though
xfs (xfs_fsr) - Been around forever.
btrfs (btrfs filesystem defragment) - as a bonus you can use it to convert uncompressed files to compressed

Yes, on ext4/xfs you won't be able to defragment a file that is larger than your freespace. That's true for ntfs as well since they're all defragging by allocating a new set of extents for the file and switching over.

And of course there is almost no point in defragmenting if using flash based storage. The only exception would be if the filesystem benefits from having less overhead for tracking file data locality, which only really matters on older filesystems such as ext3/fat (see below.)

A couple of notes about filesystems/fragmentation in general:
btrfs/zfs by their CoW nature will become heavily fragmented over time.
ext3/ext3/fat are inode based for file data allocation. data inodes are used to reference additional file allocations, so actually looking up a full file's data location may trigger many physical reads, so fragmentation on these filesystems is much more expensive than on extent based ones. This also is true of ext3 filesystem data that existed prior to converting an existing filesystem from ext3->ext4.
ext4/xfs/ntfs are extent based, so file data location on disk is in extent tables, not stored in the file data itself. Each extent is a contigous block of disk. Defragmentation on these filesystems involves reallocating the file in as few extents as possible. In addition, both ext4 and xfs have delayed/preallocation, so if you are continually extending a file it will likely wind up mostly contiguous. The exception would be sparse files (such as torrents) which are not created in sequential order. Most torrent clients do have options for file preallocation though, which is often combined with fallocate to quickly reserve the full space of the file on the filesystem rather than starting with a sparse file.

Wikipedia has a filesystem allocation comparison if you're interested.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum