New Gentoo wiki entry for Ceph

vaxbrat · l33t Joined: 05 Oct 2005 Posts: 731 Location: DC Burbs

I just posted a new page on the wiki that shows how to do a Ceph rollout.

https://wiki.gentoo.org/wiki/Ceph

This gets the basic infrastructure up and running with mon servers and object store daemons. I'll get around to playing with RADOS block device and the metadata server necessary for building the Posix filesystem on top later.

vaxbrat · l33t Joined: 05 Oct 2005 Posts: 731 Location: DC Burbs

Just finished off the magic sauce that was necessary to spin up the mds.0 server and then get a Posix Filesystem mounted and shared via NFS. I'll probably play around with a VM using an RBD within a couple of days once I satisfy the nagging requirements of my real job.

vaxbrat · l33t Joined: 05 Oct 2005 Posts: 731 Location: DC Burbs

I haven't gotten around to playing with RBD yet, but I have been shoving a boatload of stuff into the object store. I added a couple of more osd servers and watched as my I/O rates proceeded to shoot up. The stress on NICs did show I had some problems with the builtin Realtek or the R8169 driver on the A10 APU's mobo. I guess I'll be dropping in a decent Intel card the next time I get by the local MicroCenter. At least I got some experience playing with OSDs while dropping that one out for now and adding the other two to the Cluster.

I didn't realize that the OSDs self-scrub their pagestores on a periodic basis. I guess I won't be worrying about doing a btrfs scrub anymore on the arrays, but I will be testing out btrfs balance on an osd in the near future.

I'm impressed with what I see so far. I kind of wished I had gone further the last time I started to play with ceph about a year ago, but I was struck trying to get the ceph-deploy script working and then got distracted.