Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Gentoo on BTRFS-on-LUKS
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1752
Location: PB, Germany

PostPosted: Fri Jul 14, 2017 6:54 am    Post subject: Gentoo on BTRFS-on-LUKS Reply with quote

BTRFS was the best change I did in the last 2 or 3 years

Quote from my discussion on #gentoo with someone that asked to get convinced to use btrfs...
Quote:
Yes, I had exactly one crash with btrfs under heavy testing. Yes, that could also break other filesystems but not that bad and unrecoverable like the crashed btrfs: I had 1000+ snapshots, I was deleting all snapshots, the fs was filled up to 99%, I was halting the system, and as it was freezing I did a hard shutdown via Sysreq, as this did not help I switched off power.

But now to the good parts: btrfs changed my life of backups. I have a "time machine" as Apple calls it, I can snapshot everything everyday using btrbk, without much disk space. I send those snapshots to a remote btrfs, without much IO, very fast.
I have rotating btrfs snapshots of all subvolumes for root, home, data.. I have portage creating snapshots for every package INSTALL phase, that means packages breaking the system I have a snapshot for the time before installation. I can boot into different root snapshots, which I never do as I can also chroot there, quickpkg, and use the tarball to repair my running system.

I have only one big LUKS and use btrfs-on-luks, no need for LVM or different LUKS and partitions. Having no partitions but only subs does scale very well. I have btrfs compression enabled everywhere, zlib on slow hdd, and lzo on fast hdd or ssd.
btrbk runs by portage triggers and by daily cronjob.
It's backgrounded and transparent. I don't think about snapshots anymore, I just have them when I need them.


Here is my setup:

Code:
# eix -Ic btr
[I] app-backup/btrbk (0.25.0@21.03.2017): Tool for creating snapshots and remote backups of btrfs subvolumes
[I] sys-fs/btrfs-progs (4.11@30.05.2017): Btrfs filesystem utilities

This is sys-kernel/liquorix-sources:
Code:
# uname -a
Linux moralin71 4.11.8-zen #3 ZEN SMP PREEMPT Mon Jul 3 14:54:43 CEST 2017 x86_64 Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz GenuineIntel GNU/Linux

Code:
# grep btrfs /etc/fstab
LABEL=gentoo                    /                       btrfs   defaults,noatime,nodiratime,compress=lzo,autodefrag,subvol=root,commit=100      0 1
LABEL=gentoo                    /mnt/btrfs-top-lvl      btrfs   defaults,noatime,nodiratime,compress=lzo,autodefrag,subvol=/                    0 1
LABEL=gentoo                    /home                   btrfs   defaults,noatime,compress=lzo,autodefrag,subvol=home                            0 1
LABEL=gentoo                    /usr/portage            btrfs   defaults,noatime,nodiratime,compress=lzo,autodefrag,subvol=portage              0 1
LABEL=gentoo                    /var/cache              btrfs   defaults,noatime,nodiratime,compress=lzo,autodefrag,subvol=var_cache            0 1
LABEL=gentoo                    /.snapshots             btrfs   defaults,noatime,nodiratime,compress=lzo,autodefrag,subvol=snapshots            0 1
LABEL=gentoo                    /mnt/data               btrfs   defaults,noatime,nodiratime,compress=lzo,autodefrag,subvol=data                 0 1

Code:
# grep btrfs /etc/autofs/auto.usb
data        -fstype=btrfs,defaults,users,noauto,noatime,nodiratime,compress=zlib,autodefrag     LABEL=usb_data

All further LUKS devices are opened by a key stored on the encrypted root fs:
Code:
# grep -v "^#" /etc/conf.d/dmcrypt
dmcrypt_key_timeout=1
dmcrypt_retries=5

swap=swap_crypt_1
source='PARTUUID=5e974344-05'
options='--cipher aes-xts-plain64 --key-size 512 --key-file /dev/urandom'
pre_mount='mkswap -f ${dev} -L swap_crypt_1'

target=ssd_crypt
source='/dev/disk/by-id/ata-SAMSUNG_MZ7PA256HMDR-010H1_S0NVNEAB900378-part6'
key='/etc/secret/key'

target=usbdata_crypt
source='/dev/disk/by-id/usb-ST375064_0A_D-0:0'
key='/etc/secret/key'

Code:
 # grep -v "^#" /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="keymap=de splash=silent,theme:livecd-2007.0 console=tty1 quiet crypt_root=UUID=e606e66c-419f-4677-a2fe-872290730e6f root=LABEL=gentoo dobtrfs"

Code:
# cat /etc/portage/bashrc
# (...)
if [[ ${EBUILD_PHASE} == "preinst" || ${EBUILD_PHASE} == "prerm" ]]; then
    einfo "Creating btrfs snapshot..."
    btrbk -c /etc/btrbk/btrbk_portage.conf run |grep "^\(===\|+++\|---\|\*\*\*\|>>>\)"
fi

As portage is running a different retention policy, I have 2 different btrbk configurations:
Code:
# cat /etc/btrbk/btrbk.conf
transaction_log         /var/log/btrbk.log
timestamp_format        long-iso

archive_preserve_min    latest
archive_preserve        12m 10y

volume /mnt/btrfs-top-lvl/
    snapshot_create  onchange

    subvolume root
        group root
        snapshot_dir            snapshots/root
        snapshot_preserve_min   5d
        snapshot_preserve       2d 1w 1m
        target send-receive     /mnt/usb/data/snapshots/root
            target_preserve_min     latest
            target_preserve         1d 4w 2m 1y
   
    subvolume home
        group home
        snapshot_dir            snapshots/home
        snapshot_preserve_min   3d
        snapshot_preserve       4d 4w 2m 1y
        target send-receive     /mnt/usb/data/snapshots/home
            target_preserve_min     3d
            target_preserve         6d 5w 4m 3y
       
    subvolume data
        group home
        snapshot_dir            snapshots/data
        snapshot_preserve_min   latest
        snapshot_preserve       no
        target send-receive     /mnt/usb/data/snapshots/data
            target_preserve_min     latest
            target_preserve         2m

Code:
# cat /etc/btrbk/btrbk_portage.conf
transaction_log         /var/log/btrbk.log
timestamp_format        long-iso

archive_preserve_min    latest
archive_preserve        12m 10y

volume /mnt/btrfs-top-lvl/
    snapshot_create  onchange

    subvolume root
        group root
        snapshot_dir            snapshots/root
        snapshot_preserve_min   5d
        snapshot_preserve       2d 1w 1m

Beside portage triggers, I'm running btrbk either manually or as daily cronjob. As cron daemon I'm using sys-process/fcron-3.2.1-r1 for load and uptime related scheduling. There I do the btrbk backups
Code:
# fcrontab -u root -l
2017-07-14 08:38:11  INFO listing root's fcrontab
# vim: set shiftwidth=8 softtabstop=8 noexpandtab: # vim-modline, don't delete

!mail(false)
#--------------------------------------------------------------------------------------
# ENTRIES BASED ON ELAPSED SYSTEM UP TIME
#@options,FirstExec,LoadAvg1m,5m,15m,timeout                            eq.     command

# HEAVY
%daily,first(10),lavgand,lavg(4,4,4),noticenotrun,nice(15)      * *     ionice -c 3 /usr/sbin/btrbk -c /etc/btrbk/btrbk.conf run && btrfs subvolume delete -C /.snapshots/data/*

@first(6h6),lavgand,lavg(4,4,4),noticenotrun,nice(15)           2w      /sbin/btrfs scrub start -c3 -B -d /
@first(7h7),lavgand,lavg(4,4,4),noticenotrun,nice(15)           2w      /sbin/btrfs scrub start -c3 -B -d /mnt/usb/data

@first(8h8),lavgand,lavg(4,4,4),noticenotrun,nice(15)           1w      ionice -c 3 schedtool -D -e duperemove --hashfile=/root/.duperemove/hashfile -dr /home/mb/

# Light
@first(20),lavg1(4),noticenotrun,nice(15)                       1d      ionice -c 3 /sbin/fstrim -av

And emulating the vixie cron behaviour:
Code:
# fcrontab -u systab -l
2017-07-14 08:41:29  INFO listing systab's fcrontab
# vim: set shiftwidth=8 softtabstop=8 noexpandtab: # vim-modline, don't delete

!mail(false)
#--------------------------------------------------------------------------------------
# *LY KEYWORDS                  min     hours   days    command

%hourly,first(5),lavg1(4),until(30),nolog                       *       run-parts /etc/cron.hourly
%daily,first(1h15),lavgand,lavg(4,4,4),until(6h)                * *     run-parts /etc/cron.daily
%weekly,first(2h30),lavgand,lavg(4,4,4),until(3d)               * *     run-parts /etc/cron.weekly
%monthly,first(3h45),lavgand,lavg(4,4,4),until(5d)              * * *   run-parts /etc/cron.monthly


If you wonder about the job doing btrfs subvolume delete -C /.snapshots/data/*:
I had issues with btrfs used as location for VirtualBox machines. Especially the linux guests were running fine but windows guests got bluescreens, probably due to IO timeouts which is very weird... Anyway I like to avoid snapshots on those subvolumes reducing the fragmentation. However I wanted to have a snapshot on the remote btrfs only, as backup. For the running system, having a snapshots leads to fragmentation as soon as changes are happening. As btrbk does not provide a method to create a read-only snapshot, send that and drop it locally, I delete the local ones myself after every btrbk run. The feature request for btrbk was filed as https://github.com/digint/btrbk/issues/151.
Code:
 # tree -L 2 /.snapshots/
/.snapshots/
├── data
│   └── data.20170714T063834+0200
├── home
│   ├── home.20170103
│   ├── home.20170502T073537+0200
│   ├── home.20170607T071118+0200
│   ├── home.20170612T071617+0200
│   ├── home.20170620T063932+0200
│   ├── home.20170626T071700+0200
│   ├── home.20170703T070612+0200
│   ├── home.20170710T085342+0200
│   ├── home.20170711T065304+0200
│   ├── home.20170712T071711+0200
│   ├── home.20170713T064816+0200
│   └── home.20170714T063834+0200
└── root
    ├── root.20170607T071118+0200
    ├── root.20170703T070612+0200
    ├── root.20170710T085342+0200
    ├── root.20170710T093949+0200
    ├── root.20170710T094132+0200
    ├── root.20170710T094627+0200
    ├── root.20170710T094729+0200
# (..) and ~100 more

_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770
Back to top
View user's profile Send private message
Zucca
Moderator
Moderator


Joined: 14 Jun 2007
Posts: 3309
Location: Rasi, Finland

PostPosted: Sat Jul 22, 2017 11:26 am    Post subject: Reply with quote

Hi.
I can also confirm that btrfs has been great (for me) and I've never lost any data because it.
When I had some problems with my new MB and I was trying to figure out where the problem was, I accidentaly pulled out one of my SSDs while the system was running. I meant to first remove it from the btrfs pool but forgot to do that. All data was intact and I could easily insert the cold"pulled" SSD back to the btrfs pool. After scrubbing and balancing the data, everything was like before. Also many sudden systemd freezings and power outakes - btrfs has managed to repair itself.
It's a good choice for easy multi device storage.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
Quote:
I am NaN! I am a man!
Back to top
View user's profile Send private message
kakarot
n00b
n00b


Joined: 02 Feb 2004
Posts: 10

PostPosted: Fri Dec 29, 2017 9:32 pm    Post subject: Reply with quote

Dear Massimo,
Could you please provide a step by step guide?
Thank you a priori.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum