View previous topic :: View next topic |
Author |
Message |
slick Bodhisattva
Joined: 20 Apr 2003 Posts: 3495
|
Posted: Thu Jun 09, 2005 10:32 am Post subject: RAID over NFS |
|
|
This is a HowTo to create a RAID over NFS
(this post in german)
It's not for common use, it's very experimental!
We have a server who mount the RAID and (in this case 3) clients who be part of the RAID.
Install nfs-utils on server and clients.
create a shared directory and start the nfs-server on every client (example for trusted network)
Code: | mkdir /share
echo "/share 192.168.0.0/24(rw,async,no_root_squash)" >> /etc/exports
/etc/init.d/nfs start |
On the server install raidtools (and prepare the kernel to use RAIDs)
now create mountpoints on the server and mount the shares from the clients
Code: | mkdir /mnt/client1
mkdir /mnt/client2
mkdir /mnt/client3
/etc/init.d/nfs start
mount host1:/share /mnt/client1
mount host2:/share /mnt/client2
mount host3:/share /mnt/client3 |
now create "container-files" on the clients (in this case for RAID5, all with the same size)
Code: | dd if=/dev/zero of=/mnt/client1/container bs=1M count=500
dd if=/dev/zero of=/mnt/client2/container bs=1M count=500
dd if=/dev/zero of=/mnt/client3/container bs=1M count=500 |
now create a loop-device for every container-file
Code: | losetup /dev/loop1 /mnt/client1/container
losetup /dev/loop2 /mnt/client2/container
losetup /dev/loop3 /mnt/client3/container |
create a /etc/raidtab for the RAID (example for RAID5)
Code: | raiddev /dev/md0
raid-level 5
nr-raid-disks 3
nr-spare-disks 0
persistent-superblock 1
parity-algorithm left-symmetric
chunk-size 128
device /dev/loop1
raid-disk 0
device /dev/loop2
raid-disk 1
device /dev/loop3
raid-disk 2 |
create the RAID (and watch until is finish)
Code: | mkraid /dev/md0
watch cat /proc/mdstat |
create a filesystem and mount it
Code: | mke2fs /dev/md0
mount /dev/md0 /mountpoint |
Finish
------
small Performance-Test (RAID5, 3 Clients, 100MB LAN)
Code: | # tiotest
Tiotest results for 4 concurrent io threads:
,----------------------------------------------------------------------.
| Item | Time | Rate | Usr CPU | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write 40 MBs | 6.3 s | 6.308 MB/s | 0.2 % | 13.7 % |
| Random Write 16 MBs | 6.0 s | 2.602 MB/s | 0.1 % | 4.4 % |
| Read 40 MBs | 0.1 s | 268.125 MB/s | 6.7 % | 92.5 % |
| Random Read 16 MBs | 0.1 s | 244.500 MB/s | 6.3 % | 89.2 % |
`----------------------------------------------------------------------'
Tiotest latency results:
,-------------------------------------------------------------------------.
| Item | Average latency | Maximum latency | % >2 sec | % >10 sec |
+--------------+-----------------+-----------------+----------+-----------+
| Write | 0.127 ms | 327.244 ms | 0.00000 | 0.00000 |
| Random Write | 0.107 ms | 66.123 ms | 0.00000 | 0.00000 |
| Read | 0.014 ms | 0.795 ms | 0.00000 | 0.00000 |
| Random Read | 0.014 ms | 1.582 ms | 0.00000 | 0.00000 |
|--------------+-----------------+-----------------+----------+-----------|
| Total | 0.068 ms | 327.244 ms | 0.00000 | 0.00000 |
`--------------+-----------------+-----------------+----------+-----------' |
Is one client goes down all prozesses who read or write on the RAID will "freeze" ... |
|
Back to top |
|
|
rojaro l33t
Joined: 06 May 2002 Posts: 732
|
Posted: Thu Jun 09, 2005 3:20 pm Post subject: |
|
|
Crazy Idea, but this might be kind of usefull when used with RAID1 .. this would provide a pretty fault tollerant online Backup ... if it just wouldnt be so slow ... _________________ A mathematician is a machine for turning coffee into theorems. ~ Alfred Renyi (*1921 - †1970) |
|
Back to top |
|
|
slick Bodhisattva
Joined: 20 Apr 2003 Posts: 3495
|
Posted: Thu Jun 09, 2005 3:31 pm Post subject: |
|
|
sure, RAID5 was only an example, but the problem is that the RAID does not known if a client offline. He "only" knows that /dev/loopX ist present (or not), not that the file behind /dev/loopX is not available. In this moment the RAID does only "freeze" and do not work like with normal disks. |
|
Back to top |
|
|
Forse Apprentice
Joined: 26 Dec 2002 Posts: 260 Location: /dev/random
|
Posted: Fri Jun 17, 2005 11:13 am Post subject: |
|
|
I agree about crazy idea, but since gigabit nics and switches/hubs are cheap nowdays...I find this to be useful idea. _________________ [ My sites ]: UnixTutorials : AniFIND : AnimeYume |
|
Back to top |
|
|
-Craig- Guru
Joined: 03 Jun 2004 Posts: 333
|
Posted: Mon Jun 19, 2006 11:40 pm Post subject: |
|
|
Couldn't you ping all the computers periodically and remove /dev/loopX if the PC serving that is not available anymore?!
BTW: It's a crazy idea |
|
Back to top |
|
|
slick Bodhisattva
Joined: 20 Apr 2003 Posts: 3495
|
Posted: Tue Jun 20, 2006 5:48 am Post subject: |
|
|
I think you must remove the /dev/loopX before the PC goes offline, otherwise it is inpossible to remove it because the raid crash. But I didnt try this in every case. Try it and report us.
If you looking for an filesystem like raid0 over LAN try MapFS, I read his can do like this. But I dont know how stable it is or details about it.
Quote: | MapFS implements a Linux filesystem which utilizes copy-on-write functionality and existing Linux filesystems to allow component filesystems (or portions thereof) to be combined into a single virtual filesystem that appears to be fully writable. |
|
|
Back to top |
|
|
NewBlackDak Guru
Joined: 02 Nov 2003 Posts: 512 Location: Utah County, UT
|
Posted: Fri Jun 23, 2006 4:34 am Post subject: |
|
|
Wouldn't this work better with ATA-over-ethernet?
If you could get rid of the loopbacks, and have a couple setup as online spares then loosing a network node "in theory" should let you work along without missing anything. _________________ Gentoo systems.
X2 4200+@2.6 - Athy
X2 3600+ - Myth
UltraSparc5 440 - sparcy |
|
Back to top |
|
|
den_RDC Apprentice
Joined: 25 Aug 2002 Posts: 166 Location: beercountry, Belgium;)
|
Posted: Fri Jun 23, 2006 1:23 pm Post subject: |
|
|
Well, EVMS has cluster/network features etc, so i think this could be setup way easier with EVMS, probably with a slightly better redundancy ( if the raid array freezes if 1 of the disk fails, you actually increase your risk of losing access, so the R in raid isn't really applicable anymore ) _________________ Fan of the "Survivor Warriors of the Evil Empire of Bloody Destruction and Bloody Darkness" |
|
Back to top |
|
|
Mad Merlin Veteran
Joined: 09 May 2005 Posts: 1155
|
|
Back to top |
|
|
adenum n00b
Joined: 30 Oct 2009 Posts: 1
|
Posted: Fri Oct 30, 2009 3:45 pm Post subject: what a great idea |
|
|
Hi every body
this night I ve been thinking of RAID over NFS
I think we should take a raid 0 over NFS
the perfs might be as good as a miror during rebuild
-no test for the moment-
imagine a free service
I gave you back 1 Go totally secured space for free
you gave me by sharing 20 Go to make the service free
otherwise you rent it for 1 $ / year
consider the level of security for grappes made ith 10 disks
the first is your local one
the 9 others are balanced with the 8 bits / byte and a parity bit then 9 + 1 disks where your datas are balanced
in level 2 we need 21 disks
in level 12, 22 527 shares it is possible
your data is secured
1 0
10 1
21 2
43 3
87 4
175 5
351 6
703 7
1 407 8
2 815 9
5 631 10
11 263 11
22 527 12
45 055 13
90 111 14
180 223 15
360 447 16
720 895 17
1 441 791 18
2 883 583 19
5 767 167 20
11 534 335 21
23 068 671 22
46 137 343 23
92 274 687 24
184 549 375 25
369 098 751 26
738 197 503 27
1 476 395 007 28
2 952 790 015 29
5 905 580 031 30
imagine a bussiness plan if we offer up to level 2 for free
and make pay the others level
we could be rich ...
Who wants to participate to that crazy idea ?
let me know |
|
Back to top |
|
|
stobbsm Guru
Joined: 23 May 2004 Posts: 452
|
Posted: Sat Nov 07, 2009 1:49 am Post subject: |
|
|
That may be just crazy enough to work out....
I'll run some VM tests and see what comes back...at least as far as stability is concerned. _________________ Sysadmin of Ubuntu systems and servers....
Although my own server is gentoo.... |
|
Back to top |
|
|
John R. Graham Administrator
Joined: 08 Mar 2005 Posts: 10589 Location: Somewhere over Atlanta, Georgia
|
Posted: Sat Nov 07, 2009 2:52 am Post subject: |
|
|
Tahoe Filesystem is probably a much better way to go.
- John _________________ I can confirm that I have received between 0 and 499 National Security Letters. |
|
Back to top |
|
|
thejbo n00b
Joined: 15 Jan 2008 Posts: 4 Location: Florida, USA
|
Posted: Thu Nov 12, 2009 9:48 pm Post subject: |
|
|
GlusterFS is probably a better solution.
It's a great way to get a distributed, fault-tolerant file system without the expense of a SAN.
The latest version has recently been added to Portage:
http://gentoo-portage.com/sys-cluster/glusterfs |
|
Back to top |
|
|
richard.scott Veteran
Joined: 19 May 2003 Posts: 1497 Location: Oxfordshire, UK
|
Posted: Thu Nov 12, 2009 10:37 pm Post subject: |
|
|
NewBlackDak wrote: | Wouldn't this work better with ATA-over-ethernet? |
Yes, ATAoE works much better... it even notices when a node goes down and doesn't just hang like this solution |
|
Back to top |
|
|
|