Hypervisor recommendation for new home server

[keet]

I'd like to build a new home server to replace my Dell PowerEdge that is running VMWare E.S.X. I'm not sure that I want to continue using E.S.X., though; I'm interested in K.V.M., Xen, XenServer, or maybe something else if you recommend it. Ideally, I'd like to have G.P.U. passthrough so that I can give a Windows guest direct access to it and use it for gaming. Does anyone here have a recommendation for which hypervisor to use, particularly one that would work with G.P.U. passthrough? I'd also welcome recommendations for video cards. Thanks!

[audiodef]

I've had good success with Virtualbox. I'm not super-knowledgeable about its inner workings, but I'm happy with its performance using KVM.
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN

[1clue]

KVM. Or more appropriately, QEMU with kvm acceleration.

Assuming your hardware supports pass-through you should have no problems.

[1clue]

When running on a Linux host, KVM is the most native form of virtualization. QEMU is a broader virtualization scheme which doesn't necessarily require the guest to have the same hardware as the host, but more importantly it has some extras you might want.

Virtualbox, for me anyway, is a nonstarter if you're used to a full hyperviser like ESX. Mine tends to shut down the guest after an hour or two, and I still don't know why. On top of that, it's not actually freeware in all conditions. For business use you need to buy a license.

I tried to get into Xen, but after a brief struggle trying to figure it out I gave up. I admit to not having given it a fair shake, but it really doesn't appeal to me anyway. My experiences with ESXi have made me suspicious of "thin" hypervisors. If the hypervisor is broken you're pretty much screwed if that's the only hardware on the site you have access to. As well, when I was learning it my dom0 would have been my desktop box and I didn't want to mess with it.

KVM can be as thin as you want, or as thick as you want.

[keet]

[keet]

[1clue]

https://wiki.gentoo.org/wiki/QEMU
https://wiki.gentoo.org/wiki/QEMU/Linux_guest

Also consider that there are things you can do when you have more than one Gentoo system in the mix, whether or not one of those is the host. You can share the portage data directories, for example.

Not sure if you're familiar or if the lesser isolation works for your scenario, but you may also want to look at docker to augment your kvm host. For little stuff it's really slick.

[1clue]

Another thing:

I'm not sure how familiar you are with Linux based on your comments.

If you use RAID then you might want to consider using software RAID. The performance is not significantly worse if you're comparing against a cheaper RAID card, and with software RAID you don't need to worry about keeping a spare RAID card. Hardware RAID, you can lose your data if you don't have a compatible same-age RAID card handy. Software RAID can be handled by any Linux box with the appropriate software.

[1clue]

GPU passthrough appears to be possible (I had never heard of it before your thread) and there are lots of hits for it in Google. Including a warning about something called Valve Anti-Cheat possibly detecting that the guest is in a VM and banning you.

While I have some small bit of experience with KVM and QEMU (NOT an expert!) I have zero experience with gaming or GPU passthrough.

I've used ESXi though, and I have some observations that may be relevant and/or informative for someone moving to KVM/QEMU:

1. QEMU is the user-space virtualization engine. It can work with or without hardware acceleration which is supplied by KVM or Xen.
   
2. KVM is inside the Linux kernel, controlled by some user-space software and apps.
   
3. Unlike ESXi there are multiple features you'll either want or not want, and it takes some serious reading on your part to decide.
   
4. You could make a console-only hypervisor install very similar to ESX, with a very minimal command line and selected tools for managing hardware.
   
5. You could alternately install a full GUI workstation and have it host QEMU/KVM as well, and it would still have the same hypervisor, unlike VMware Workstation.
   
6. You can administer a QEMU box through virt-manager, which is a gui client that does not need to be on the host. You can also do everything from a command line.
   
7. There are other gui clients but I have no experience with them, virt-manager does what I want.
   
8. There are Open Source tools to do everything you need with respect to QEMU/KVM.
   
9. There are also commercial offerings which make it feasible to manage guests on multiple hosts and other enterprise-like things. They will show up in your search results.
   

[Spargeltarzan]

I like Xen for Linux Pvh Guests, but also want to start gpu passthrough and I am unsure aboutits performance difference and easy maintenance/setup
_________________
___________________
Regards

Spargeltarzan

Notebook: Lenovo YOGA 900-13ISK: Gentoo stable amd64, GNOME systemd, KVM/QEMU
Desktop-PC: Intel Core i7-4770K, 8GB Ram, AMD Radeon R9 280X, ZFS Storage, GNOME openrc, Dantrell, Xen

[1clue]

Also, my most recent KVM box is a supermicro atom-based board found here http://www.supermicro.com/products/motherboard/atom/x10/a1srm-ln7f-2758.cfm .

Warning: I'm a converted supermicro fanboy. I don't work there, I just buy their stuff. This post has a small bit of relevance for your situation and a whole lot of fanboy propaganda.

My rig has IPMI v2.0 support, which I highly recommend for a remotely administered box. It also has high quality everything. It's the first system I ever built which didn't have some piece of hardware I had counted on that didn't turn out to be junk, based on building systems since the mid 90s. 7x Intel nics, as opposed to Realtek or some other BS. It turns out that https://antsle.com markets a lightweight virtual machine host with almost exactly my board's specs. I'm converting mine to use kvm/qemu and docker (https://docker.com) for lightweight images.

I can literally do anything through IPMI except move it to another shelf or add/remove physical cables. I can turn it on/off, reset, get console or gui console access, whatever I want. I can control fan speeds.

Presumably other manufacturers have the same capability, but IMO if you're building something new then I say supermicro rocks and you should at least look at what they offer.

My rig isn't a gaming system, but SuperMicro offers the full range, from atom to i7 to 8-socket e7 hardware to ultra-compact server farm hardware with a huge variety of GPU arrangements.

[Spargeltarzan]

1Clue,


Would you help me in my thread "supermicro ZFS Workstation" in Kernel and Hardware too, if you have some experience? Would be great, many thanks in advance!!
_________________
___________________
Regards

Spargeltarzan

Notebook: Lenovo YOGA 900-13ISK: Gentoo stable amd64, GNOME systemd, KVM/QEMU
Desktop-PC: Intel Core i7-4770K, 8GB Ram, AMD Radeon R9 280X, ZFS Storage, GNOME openrc, Dantrell, Xen

[1clue]

Posted, but I need to remind you I'm not really an expert in any of this. I just type really fast.

[keet]

I would like to use RAID, more for parity and protecting against data loss than to gain performance. I thought that some of the newer filesystems implemented their own RAID-like features, as well.

In any case, I ordered the parts for my new home server, so I'll probably build it early next week. I'll probably start with K.V.M. and Qemu and just set it up without G.P.U. passthrough, since I don't really need it, and then see whether I can make it work.

[1clue]

RAID does not improve performance. It sacrifices performance in almost every case.
RAID is not a backup. It satisfies only a few cases of many problems that a backup prevents.

[keet]

[1clue]

[keet]

[1clue]

I'm not going to tell you how to set up your box. I'll only tell you my observations and recommendations and let you decide.

I've never worked with a raid5 array -- hardware or software -- that I didn't think should be twice as fast as it was. I don't know how much performance matters to you, but in one case we took down a production box with hardware raid5, with a higher quality controller, because it was too slow to get the job done. Even raid1 has to wait for the slowest drive, and a bit of overhead.

If you use a raid array, I recommend that you only use it for the data you insist must remain hot all the time. Put the OS on a non-raid ssd, or at least raid1. The more complicated the raid strategy the slower you're going to be.

[keet]

Thank you for the recommendation. I was (and still am) having trouble exactly how to accomplish. I want for all the data to be encrypted, as much as possible. I was reading this article for recommendations:

https://wiki.archlinux.org/index.php/Dm-crypt/Encrypting_an_entire_system

I would prefer to have one encrypted LUKS volumes with multiple LVM volumes inside it (inheriting the overall encryption). However, it sounds like that's impossible with multiple disks, unless one can encrypt a volume group, which I didn't see in that article (alternately, I could encrypt the volumes separately, but I'm not sure that I want to do that). However, it does mention LUKS on software RAID, and I assume that I could put LVM inside the LUKS container. I ran this command:

[keet]

Since some of your signatures mention Z.F.S... does anyone here know whether 0.7.5. includes native encryption? Also, it's marked as testing/unstable, but I'm guessing thati it 's actually reliable and just that every version is keyworded for Gentoo?

[1clue]

You want to encrypt a host filesystem which contains guests or you want to encrypt a volume on a guest?

IMO you need to define what it is you need to have encrypted and only encrypt that. I have never found a need to encrypt a filesystem, only individual files. So I don't have experience for what you're saying.

What I know is that encryption=slower and encrypting a filesystem on the host which will then contain guests will be slowest of all.

IMO it makes no sense to encrypt application binaries unless you need to hide the fact that your computer had any bytes at all written to the disk. The CIA may find that useful, but I can't see how anyone else would.

The way I understand it, encrypting a volume group should be possible.

In my limited understanding of encrypted filesystems, the encryption key should be in the possession of the user, not on the disk itself. It should be inserted when the system boots, then removed once the system is up to prevent theft. I may have that wrong, but it seems to me that if you have the key on the system then it boots automatically anyway, and you'd just as well not encrypt it in the first place.

If you're serious about this encryption thing you may want to consider getting hardware acceleration for encryption. For example, Intel's QuickAssist can be found in a couple different PCI cards and also embedded into some of the atom chips. c2x58 and c3x58 for example.

[szatox]

[keet]

1clue, it sounds to me like you are discouraging full-disk encryption. However, it doesn't seem that much more difficult simply to encrypt the whole thing. If I encrypt selectively, it just becomes more complicated. Full-disk encryption (apart from the bootloader) has been easy and reliable in my experience; I've never had any problems with it.

As szatox said, I'm going to keep the decryption key separate (written on a flash drive, or as a password...).

The part that complicates it for me is that I have multiple disks that I want to encrypt as one container. Making them into a RAID array seems to accomplish this as well as giving me some redundancy/protection against disk failure (which is certainly not to say that I'm going to eschew backups; I'll definitely still back it up periodically).

Thus, what I'm doing at the moment is a RAID 5 array (/dev/md0) using all three drives, which I'm encrypting using LUKS (/dev/mapper/encryptedmd0), and then LVM inside that encrypted container. That way (almost) everything is encrypted as one container and I have the benefits of RAID 5 and LVM.

[Hu]

To elaborate on szatox's comment regarding why encryption should be done on a broad basis:

• Filesystem or block device encryption can be transparent to the user applications, so it works automatically for programs without requiring dedicated support in each application.
• Some programs have a habit of writing to files other than just the ones the user specifically directs. For example, many editors keep a "recently opened files" list, and some also keep a "recently copied text snippets" list. If you care about encrypting a given file, you likely also care about encrypting any content that comes from that file, so you want the snippets from that file encrypted even when they are not in that file. If all places the editor writes its automatic files are backed by encrypted filesystems/block devices, then the snippets are protected even when separated from their original file.
• When using file-specific encryption, any temporary decrypted copies required for working with other tools (text editor, browser, word processor, etc.) need to be securely purged when the user is done. This may or may not be easy depending on where the temporaries are stored. Even when it is easy, it's an extra manual step.

This does not even get into the philosophical issues regarding deniability, metadata leaks, etc.

The only substantial upside I see to encrypted files is that it is easier to leave them secured while the system is otherwise operational. For example, even on a system with full-disk encryption, I might store my password database in a separately encrypted file, so that it's not trivially readable to all my programs whenever I'm logged in, but is instead open only when I choose to open it.