Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
read-only filesystems
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
Atom2
Tux's lil' helper
Tux's lil' helper


Joined: 01 Aug 2011
Posts: 87

PostPosted: Sun Jun 30, 2013 10:09 pm    Post subject: read-only filesystems Reply with quote

Hallo Forum,
I am in the process of building up a XEN gentoo system. My ideas are currently as follows:

Dom0 with a hardened system - that's basically up and running. I can start PVs and they work.

Next steps would be:
A number of paravirtualized gentoo DomUs for various services like DNS, DHCP, squid, samba, nagios, SMTP, IMAP, etc.
Over and above this I'd plan to use FreeNAS as the storage system - clearly with a HVM and direct access to storage disks (PCI-passthrough).
For the firewall my idea currently is pfsense (FreeBSD), also with HVM and PCI-passthrough of a network card for the internet access.
Lastly a few win7 HVMs.
Logging from the paravirtualized DomUs on gentoo would probably use remote logging to the Dom0 syslog-ng daemon.

In order to minimize the administration of those PV gentoo DomUs my idea is to provide them with identical kernels and also share common files (e.g. binaries). From my understanding this would work provided the relevant filesystems can be mounted r/o.

The same principal of sharing a common system disk image might also be on the plate for the win7 virtual machines, but that's still a bit down the line at the moment.

So my question is: Which filesystems / directories can be mounted r/o without limiting operation of a gentoo system during normal operation.

Updates could work as follows: There are two sets of all those shared files / directories (production & test). Updates to newer versions could be done on the test set and then tested in a separate virtual machine. Once o.k., the changes could be rolled into the produciton set. After that only a restart of all virtuial machines would be required and all VMs would have the latest set of shared files once they are up again.

I'd think that the following directories would need to be separate and mounted read/write:
Code:
/ (incl. /lost+found)
/etc
/dev
/mnt
/proc
/run
/sys
/tmp
/var


On the other hand I think that the following directories could be shared and possible be r/o as well:
Code:
/bin
/boot - NOTE: because the kernel gets loaded from Dom0 /boot is empty anyways
/lib
/lib32
/lib64
/opt
/sbin
/usr


At the moment a lot of this is theory and I'd be very interested in feedback on my ideas. There might also be a fundamental flaw in my thought proceess and it's not going to work at all. Over and above this ideas for other / better approaches are also very much welcome.

Thanks Atom2
Back to top
View user's profile Send private message
vaxbrat
Guru
Guru


Joined: 05 Oct 2005
Posts: 594
Location: DC Burbs

PostPosted: Sun Jul 07, 2013 5:29 pm    Post subject: look into using aufs maybe Reply with quote

You might want to look into using aufs to do a union of a base filesystem as readonly and then a custom one for each guest that gets rw mount with whatever files it has/writes unique from other guests. Most of the time, the readonly one is also compressed (ie squashfs) since the original use case for this approach was for thumb drive or network boots. If you rummage around in the wiki, you will see the howto's for thin clients, etc doing this.
Back to top
View user's profile Send private message
Atom2
Tux's lil' helper
Tux's lil' helper


Joined: 01 Aug 2011
Posts: 87

PostPosted: Mon Jul 08, 2013 7:56 am    Post subject: Reply with quote

Hi vaxbrat,
that approach sounds interesting indeed. I am not sure yet how that all plays out with XEN, but I assume it requires an initramfs (which, if possible at all, I'd rather avoid).

In any case thanks for sharing your idea; I'll have a closer look, although ressources to read into the subject seem to be scarce. I only found one article specifically related to XEN and aufs here. If you happen to have a few more links available, I'd very much appreciate that.

Another approach that I came across when searching for shared storage under XEN is to use an LVM-backed block device and create a snapshot for every guest as briefly described here. That would do away with the requirement of an initramfs as far as I understand it and - given that I do already use LVM (albeit without snapshots so far) at the moment - might be an easier way forward. If I am not mistaken, the snapshot could be mounted rw - so no special treatment or ro mounting required - and the storage basically behaves like a cow filesystem ...

Any further ideas/suggestions are welcome as well as opinions on which of the two approaches above are preferrable.


Atom2
Back to top
View user's profile Send private message
vaxbrat
Guru
Guru


Joined: 05 Oct 2005
Posts: 594
Location: DC Burbs

PostPosted: Thu Jul 11, 2013 3:26 am    Post subject: maybe btrfs too Reply with quote

I've been getting more into using btrfs lately but haven't dived yet off the deep end by doing it as a root filesystem. That has snapshotting, subvolumes and can do compression on the fly. You should be able to avoid an initramfs using it by just specifying the volume label on the kernel line.
Back to top
View user's profile Send private message
Atom2
Tux's lil' helper
Tux's lil' helper


Joined: 01 Aug 2011
Posts: 87

PostPosted: Sun Jul 14, 2013 7:44 pm    Post subject: Reply with quote

vaxbrat,
after a lot of research and reading, I have decided to try the route of your initial idea with squashfs and aufs.

I have also managed to make some progress and have a pv gentoo guest up and running including an initramfs. There are however a few problems which need ironing out. The most pressing issue is that whenever I shutdown the domU, the two mounted filesystems that make up the union overlay filesystem with aufs refuse to be unmounted. This results in the r/w filesystem part (an LVM volume formatted as ext2 that serves as the overlay to the r/o squashfs) to get corrupted due to unclean shutdown and at every reboot there's a fsck required (which so far always only clears a few inodes). The r/o squashfs, which also refuses to unmount during shutdown, obviuosly doesn't have that problem due to its read-only nature.

I have no idea how I can overcome this as the unmount obviously thinks that these two filesystems are busy (but at the same time states, that fuser can't find any active process using it), and it certainly does not increase my confidence in this setup if I (potentially) face data loss at powerdown. I have also manually tried to unmount them from within the virtual machine, and that also doesn't work; also remounting with r/o access does not work. If I can's solve this, I think unfortunately that's a no-go.

The other issue, albeit much less pressurizing, but still very annoying, is that after the init process from my initramfs makes the switch to the real root and passes control to the standard boot-process of gentoo, no more boot messages show up on the virtual xen console. After a few seconds of silence, however the console login comes up and the system is fully usable. I assume there's someting not in order with device nodes, but haven't been able to track down what it actually is.

The other ideas with snapshots (whether LVm or btrfs) all have one drawback that I either don't understand correctly or make that approach much less appealing: Updates of the original data underneath the snapshot (e.g. in case of new releases of gentoo or additional packages) would not really work as easily as with squashfs. As far as I understand changes to the original filesystem before the snapshot would result in all old files to be propagated to the snapshot resulting in asubstantial increase in its size.

That would not be the case for squashfs and aufs and that's why so far I think aufs is the sexier option of the two. That's mainly because making updates is so much easier: You just need to create a new squashfs and you are ready to go.

That's my current understanding, but who knows, I might be wrong, so all inputs are very much appreciated ...

Thanks and regards Atom2
Back to top
View user's profile Send private message
vaxbrat
Guru
Guru


Joined: 05 Oct 2005
Posts: 594
Location: DC Burbs

PostPosted: Sun Jul 14, 2013 10:14 pm    Post subject: you're welcome Reply with quote

I looked at xen once a number of years ago when rhel was big on it in early 5.x versions. However I've been with kvm when I have the choice and am stuck with vmware when the windows guys at work stick me with that underneath.

Now that I've gotten a bit more warm and fuzzy with btrfs I've gotten around to deploying ceph on top.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum