Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
RAID over NFS
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
slick
Bodhisattva
Bodhisattva


Joined: 20 Apr 2003
Posts: 3495

PostPosted: Thu Jun 09, 2005 10:32 am    Post subject: RAID over NFS Reply with quote

This is a HowTo to create a RAID over NFS
(this post in german)

:!: It's not for common use, it's very experimental!

We have a server who mount the RAID and (in this case 3) clients who be part of the RAID.

Install nfs-utils on server and clients.
Code:
emerge nfs-utils

create a shared directory and start the nfs-server on every client (example for trusted network)
Code:
mkdir /share
echo "/share 192.168.0.0/24(rw,async,no_root_squash)" >> /etc/exports
/etc/init.d/nfs start

On the server install raidtools (and prepare the kernel to use RAIDs)
Code:
emerge raidtools

now create mountpoints on the server and mount the shares from the clients
Code:
mkdir /mnt/client1
mkdir /mnt/client2
mkdir /mnt/client3
/etc/init.d/nfs start
mount host1:/share /mnt/client1
mount host2:/share /mnt/client2
mount host3:/share /mnt/client3

now create "container-files" on the clients (in this case for RAID5, all with the same size)
Code:
dd if=/dev/zero of=/mnt/client1/container bs=1M count=500
dd if=/dev/zero of=/mnt/client2/container bs=1M count=500
dd if=/dev/zero of=/mnt/client3/container bs=1M count=500

now create a loop-device for every container-file
Code:
losetup /dev/loop1 /mnt/client1/container
losetup /dev/loop2 /mnt/client2/container
losetup /dev/loop3 /mnt/client3/container


create a /etc/raidtab for the RAID (example for RAID5)
Code:
raiddev /dev/md0
    raid-level                  5
    nr-raid-disks               3
    nr-spare-disks              0
    persistent-superblock       1
    parity-algorithm            left-symmetric
    chunk-size                  128
    device                      /dev/loop1
    raid-disk                   0
    device                      /dev/loop2
    raid-disk                   1
    device                      /dev/loop3
    raid-disk                   2

create the RAID (and watch until is finish)
Code:
mkraid /dev/md0
watch cat /proc/mdstat

create a filesystem and mount it
Code:
mke2fs /dev/md0
mount /dev/md0 /mountpoint


Finish

------

small Performance-Test (RAID5, 3 Clients, 100MB LAN)
Code:
# tiotest
Tiotest results for 4 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write          40 MBs |    6.3 s |   6.308 MB/s |   0.2 %  |  13.7 % |
| Random Write   16 MBs |    6.0 s |   2.602 MB/s |   0.1 %  |   4.4 % |
| Read           40 MBs |    0.1 s | 268.125 MB/s |   6.7 %  |  92.5 % |
| Random Read    16 MBs |    0.1 s | 244.500 MB/s |   6.3 %  |  89.2 % |
`----------------------------------------------------------------------'
Tiotest latency results:
,-------------------------------------------------------------------------.
| Item         | Average latency | Maximum latency | % >2 sec | % >10 sec |
+--------------+-----------------+-----------------+----------+-----------+
| Write        |        0.127 ms |      327.244 ms |  0.00000 |   0.00000 |
| Random Write |        0.107 ms |       66.123 ms |  0.00000 |   0.00000 |
| Read         |        0.014 ms |        0.795 ms |  0.00000 |   0.00000 |
| Random Read  |        0.014 ms |        1.582 ms |  0.00000 |   0.00000 |
|--------------+-----------------+-----------------+----------+-----------|
| Total        |        0.068 ms |      327.244 ms |  0.00000 |   0.00000 |
`--------------+-----------------+-----------------+----------+-----------'


Is one client goes down all prozesses who read or write on the RAID will "freeze" ...
Back to top
View user's profile Send private message
rojaro
l33t
l33t


Joined: 06 May 2002
Posts: 732

PostPosted: Thu Jun 09, 2005 3:20 pm    Post subject: Reply with quote

Crazy Idea, but this might be kind of usefull when used with RAID1 .. this would provide a pretty fault tollerant online Backup ... if it just wouldnt be so slow ...
_________________
A mathematician is a machine for turning coffee into theorems. ~ Alfred Renyi (*1921 - †1970)
Back to top
View user's profile Send private message
slick
Bodhisattva
Bodhisattva


Joined: 20 Apr 2003
Posts: 3495

PostPosted: Thu Jun 09, 2005 3:31 pm    Post subject: Reply with quote

sure, RAID5 was only an example, but the problem is that the RAID does not known if a client offline. He "only" knows that /dev/loopX ist present (or not), not that the file behind /dev/loopX is not available. In this moment the RAID does only "freeze" and do not work like with normal disks.
Back to top
View user's profile Send private message
Forse
Apprentice
Apprentice


Joined: 26 Dec 2002
Posts: 260
Location: /dev/random

PostPosted: Fri Jun 17, 2005 11:13 am    Post subject: Reply with quote

I agree about crazy idea, but since gigabit nics and switches/hubs are cheap nowdays...I find this to be useful idea.
_________________
[ My sites ]: UnixTutorials : AniFIND : AnimeYume
Back to top
View user's profile Send private message
-Craig-
Guru
Guru


Joined: 03 Jun 2004
Posts: 333

PostPosted: Mon Jun 19, 2006 11:40 pm    Post subject: Reply with quote

Couldn't you ping all the computers periodically and remove /dev/loopX if the PC serving that is not available anymore?!

BTW: It's a crazy idea ;)
Back to top
View user's profile Send private message
slick
Bodhisattva
Bodhisattva


Joined: 20 Apr 2003
Posts: 3495

PostPosted: Tue Jun 20, 2006 5:48 am    Post subject: Reply with quote

I think you must remove the /dev/loopX before the PC goes offline, otherwise it is inpossible to remove it because the raid crash. But I didnt try this in every case. Try it and report us.

If you looking for an filesystem like raid0 over LAN try MapFS, I read his can do like this. But I dont know how stable it is or details about it.

Quote:
MapFS implements a Linux filesystem which utilizes copy-on-write functionality and existing Linux filesystems to allow component filesystems (or portions thereof) to be combined into a single virtual filesystem that appears to be fully writable.
Back to top
View user's profile Send private message
NewBlackDak
Guru
Guru


Joined: 02 Nov 2003
Posts: 512
Location: Utah County, UT

PostPosted: Fri Jun 23, 2006 4:34 am    Post subject: Reply with quote

Wouldn't this work better with ATA-over-ethernet?

If you could get rid of the loopbacks, and have a couple setup as online spares then loosing a network node "in theory" should let you work along without missing anything.
_________________
Gentoo systems.
X2 4200+@2.6 - Athy
X2 3600+ - Myth
UltraSparc5 440 - sparcy
Back to top
View user's profile Send private message
den_RDC
Apprentice
Apprentice


Joined: 25 Aug 2002
Posts: 166
Location: beercountry, Belgium;)

PostPosted: Fri Jun 23, 2006 1:23 pm    Post subject: Reply with quote

Well, EVMS has cluster/network features etc, so i think this could be setup way easier with EVMS, probably with a slightly better redundancy ( if the raid array freezes if 1 of the disk fails, you actually increase your risk of losing access, so the R in raid isn't really applicable anymore )
_________________
Fan of the "Survivor Warriors of the Evil Empire of Bloody Destruction and Bloody Darkness"
Back to top
View user's profile Send private message
Mad Merlin
Veteran
Veteran


Joined: 09 May 2005
Posts: 1155

PostPosted: Sat Jul 01, 2006 1:27 am    Post subject: Reply with quote

Cool idea, I'm just playing with this now, except using sshfs rather than nfs, works quite well.
_________________
Game! - Where the stick is mightier than the sword!
Back to top
View user's profile Send private message
adenum
n00b
n00b


Joined: 30 Oct 2009
Posts: 1

PostPosted: Fri Oct 30, 2009 3:45 pm    Post subject: what a great idea Reply with quote

:D

Hi every body
this night I ve been thinking of RAID over NFS

I think we should take a raid 0 over NFS
the perfs might be as good as a miror during rebuild
-no test for the moment-

imagine a free service

I gave you back 1 Go totally secured space for free
you gave me by sharing 20 Go to make the service free
otherwise you rent it for 1 $ / year


consider the level of security for grappes made ith 10 disks
the first is your local one
the 9 others are balanced with the 8 bits / byte and a parity bit then 9 + 1 disks where your datas are balanced
in level 2 we need 21 disks
in level 12, 22 527 shares it is possible
your data is secured

1 0
10 1
21 2
43 3
87 4
175 5
351 6
703 7
1 407 8
2 815 9
5 631 10
11 263 11
22 527 12
45 055 13
90 111 14
180 223 15
360 447 16
720 895 17
1 441 791 18
2 883 583 19
5 767 167 20
11 534 335 21
23 068 671 22
46 137 343 23
92 274 687 24
184 549 375 25
369 098 751 26
738 197 503 27
1 476 395 007 28
2 952 790 015 29
5 905 580 031 30

imagine a bussiness plan if we offer up to level 2 for free
and make pay the others level

we could be rich ...

Who wants to participate to that crazy idea ?


let me know
Back to top
View user's profile Send private message
stobbsm
Guru
Guru


Joined: 23 May 2004
Posts: 452

PostPosted: Sat Nov 07, 2009 1:49 am    Post subject: Reply with quote

That may be just crazy enough to work out....

I'll run some VM tests and see what comes back...at least as far as stability is concerned.
_________________
Sysadmin of Ubuntu systems and servers....
Although my own server is gentoo....
Back to top
View user's profile Send private message
John R. Graham
Administrator
Administrator


Joined: 08 Mar 2005
Posts: 10587
Location: Somewhere over Atlanta, Georgia

PostPosted: Sat Nov 07, 2009 2:52 am    Post subject: Reply with quote

Tahoe Filesystem is probably a much better way to go.

- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters.
Back to top
View user's profile Send private message
thejbo
n00b
n00b


Joined: 15 Jan 2008
Posts: 4
Location: Florida, USA

PostPosted: Thu Nov 12, 2009 9:48 pm    Post subject: Reply with quote

GlusterFS is probably a better solution.

It's a great way to get a distributed, fault-tolerant file system without the expense of a SAN.

The latest version has recently been added to Portage:
http://gentoo-portage.com/sys-cluster/glusterfs
Back to top
View user's profile Send private message
richard.scott
Veteran
Veteran


Joined: 19 May 2003
Posts: 1497
Location: Oxfordshire, UK

PostPosted: Thu Nov 12, 2009 10:37 pm    Post subject: Reply with quote

NewBlackDak wrote:
Wouldn't this work better with ATA-over-ethernet?


Yes, ATAoE works much better... it even notices when a node goes down and doesn't just hang like this solution :-)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum