"keep directories in sync" or "raid alternatives"

Message

Jogie214 · Post by **Jogie214** » Tue Apr 10, 2012 11:00 am

Hi,

I used a raid1/LVM array for quite some time, and I'm happy with the setup. Now I got my hands on a nice SSD, and I would like to migrate my /usr directory to the SSD, BUT I also would like to keep an up-to-date copy of /usr on the raid array.
Now the main question, what's the best approach to this? I know rsync can keep local directories in sync, I also stumbled upon unison, but I'm not certain if there isn't a more elegant solution.
So any ideas/suggestions?

Thanks,
Sebastian

gorkypl · Post by **gorkypl** » Tue Apr 10, 2012 1:20 pm

For periodic copies of whole directory running rsync from cron is fine, but you will probably need to force it also on shutdown, to prevent partial syncs.
You can also just add SSD to RAID1 array and set it as a preferred read/write drive (HDDs will be sync'd in background).

Jogie214 · Post by **Jogie214** » Tue Apr 10, 2012 2:02 pm

Hi,

and thanks for your answer.
I've read that SSD's are not really the best choice for a raid using linux, since the raid array would leave no empty blocks on the device, also some metadata would wear down the cells rather fast. I haven't checked whether it's the truth, since the drive is too small, both HDDs in the raid array are 1GB in size (the SDD has "only" 64GB), the space inside the array is split into different volume groups. I'm not sure whether it would be possible or a good idea to build a raid from a LVM volume inside a raid array with an SDD.
I was hoping for some kind of monitoring daemon (woken by e.g. inotify), that syncs the device as needed.

Greets,
Sebastian

gorkypl · Post by **gorkypl** » Tue Apr 10, 2012 2:47 pm

Jogie214 wrote: I've read that SSD's are not really the best choice for a raid using linux, since the raid array would leave no empty blocks on the device, also some metadata would wear down the cells rather fast. I haven't checked whether it's the truth

This is no longer true for the very latest kernels. You might want to do google search for 'linux raid trim support'
http://lwn.net/Articles/486842/

Jogie214 wrote: the space inside the array is split into different volume groups. I'm not sure whether it would be possible or a good idea to build a raid from a LVM volume inside a raid array with an SDD.

Ah, I understand now. I thought you have /usr on a separate RAID1 partition.
You might want to search for lvm mirroring and check if it supports TRIM (I suspect it does) - it should be able to do exactly what you want.
Unfortunately I have no personal experience with this kind of setup so I cannot help you further... Good luck anyway!

Jogie214 · Post by **Jogie214** » Tue Apr 10, 2012 6:54 pm

Hi,

thanks for pointing out that the trim issue is fixed, I'll check if there is a kernel version around with this patch applied.
LVM mirroring (I've to admit, I forgot about it) sound also very promising, but as far as I know, I need three devices for that, but I guess I'll do a few google searches and be wiser afterwards.
As soon as I've found something defined, I'll let you know.

Greets,
Sebastian

djdunn · Post by **djdunn** » Wed Apr 11, 2012 6:53 am

if you don't want to make it raid why not just use Tar or some other backup software, many are quite eloquent

and you need 3 physical volumes for lvm mirroring one for the data one for the mirror and one for the log, there's a way you can use less physical volumes with mirror core log, however every time you reboot you have to rebuild the mirror cause there's no log.

Jogie214 · Post by **Jogie214** » Thu May 03, 2012 4:41 pm

Hi,

I promised to give some feedback as soon as I've implemented some form of sync option. It's not automatic, but it works quite flawlessly so far, I use a cron job to run the script every 24h.
I decided to give rsync a try, the script I use is enclosed in this post see below. LVM mirror is no option, since I don't have a third device.
Everything is running smooth so far.

Code: Select all

#!/bin/sh
cmd="rsync -vurlpog --delete"
for dir in /boot /proc /sys /dev /tmp /home /usr/portage /usr/distfiles /opt /alt-root; do
        cmd="$cmd --exclude=$dir "
done
cmd="$cmd / /alt-root"
eval $cmd
[ -x /alt-root/boot ] ; mkdir /alt-root/boot
[ -x /alt-root/proc ] ; mkdir /alt-root/proc
[ -x /alt-root/sys ] ; mkdir /alt-root/sys
[ -x /alt-root/dev ] ; mkdir /alt-root/dev
[ -x /alt-root/temp ] ; mkdir /alt-root/temp
[ -x /alt-root/home ] ; mkdir /alt-root/home
[ -x /alt-root/usr/portage ] ; mkdir /alt-root/usr/portage
[ -x /alt-root/usr/distfiles ] ; mkdir /alt-root/usr/distfiles
[ -x /alt-root/opt ] ; mkdir /alt-root/opt
[ -x /alt-root/alt-root ] ; mkdir /alt-root/alt-root

The last part of the script is just implemented if at some time in the future I'll use it on a freshly formatted hard drive.
The backup root dir is mounted at /alt-root. The dirs are excluded since there are on partitions on their own.

Greets,
Sebastian