Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Hypervisor recommendation for new home server
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
keet
Guru
Guru


Joined: 09 Sep 2008
Posts: 527

PostPosted: Sun Dec 17, 2017 6:53 pm    Post subject: Hypervisor recommendation for new home server Reply with quote

I'd like to build a new home server to replace my Dell PowerEdge that is running VMWare E.S.X. I'm not sure that I want to continue using E.S.X., though; I'm interested in K.V.M., Xen, XenServer, or maybe something else if you recommend it. Ideally, I'd like to have G.P.U. passthrough so that I can give a Windows guest direct access to it and use it for gaming. Does anyone here have a recommendation for which hypervisor to use, particularly one that would work with G.P.U. passthrough? I'd also welcome recommendations for video cards. Thanks!
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6192
Location: /usr/lib64/lv2

PostPosted: Mon Dec 18, 2017 10:36 pm    Post subject: Reply with quote

I've had good success with Virtualbox. I'm not super-knowledgeable about its inner workings, but I'm happy with its performance using KVM.
_________________
Gentoo Studio in open beta. Feedback wanted.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2302

PostPosted: Mon Dec 18, 2017 10:51 pm    Post subject: Reply with quote

KVM. Or more appropriately, QEMU with kvm acceleration.

Assuming your hardware supports pass-through you should have no problems.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2302

PostPosted: Mon Dec 18, 2017 11:02 pm    Post subject: Reply with quote

When running on a Linux host, KVM is the most native form of virtualization. QEMU is a broader virtualization scheme which doesn't necessarily require the guest to have the same hardware as the host, but more importantly it has some extras you might want.

Virtualbox, for me anyway, is a nonstarter if you're used to a full hyperviser like ESX. Mine tends to shut down the guest after an hour or two, and I still don't know why. On top of that, it's not actually freeware in all conditions. For business use you need to buy a license.

I tried to get into Xen, but after a brief struggle trying to figure it out I gave up. I admit to not having given it a fair shake, but it really doesn't appeal to me anyway. My experiences with ESXi have made me suspicious of "thin" hypervisors. If the hypervisor is broken you're pretty much screwed if that's the only hardware on the site you have access to. As well, when I was learning it my dom0 would have been my desktop box and I didn't want to mess with it.

KVM can be as thin as you want, or as thick as you want.
Back to top
View user's profile Send private message
keet
Guru
Guru


Joined: 09 Sep 2008
Posts: 527

PostPosted: Mon Dec 18, 2017 11:16 pm    Post subject: Reply with quote

audiodef wrote:
I've had good success with Virtualbox. I'm not super-knowledgeable about its inner workings, but I'm happy with its performance using KVM.


Sorry, I should have clarified: I meant a type one/bare metal hypervisor. Thanks for your recommendation, though; VirtualBox is good when you're running it on top of a full O.S., for example, on a normal desktop computer.


Last edited by keet on Mon Dec 18, 2017 11:19 pm; edited 1 time in total
Back to top
View user's profile Send private message
keet
Guru
Guru


Joined: 09 Sep 2008
Posts: 527

PostPosted: Mon Dec 18, 2017 11:19 pm    Post subject: Reply with quote

1clue wrote:
When running on a Linux host, KVM is the most native form of virtualization... I tried to get into Xen, but after a brief struggle trying to figure it out I gave up... KVM can be as thin as you want, or as thick as you want.


Thank you. I also tried Xen a few years ago, but had tons of trouble trying to make V.G.A. passthrough work. E.S.X. was easy to set up, but it's also a (non-libre) licensed, proprietary product that doesn't offer full control (I think). K.V.M. (with Qemu) is the most likely choice, based on what I've read, so I'll probably give that a try first.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2302

PostPosted: Mon Dec 18, 2017 11:29 pm    Post subject: Reply with quote

https://wiki.gentoo.org/wiki/QEMU
https://wiki.gentoo.org/wiki/QEMU/Linux_guest

Also consider that there are things you can do when you have more than one Gentoo system in the mix, whether or not one of those is the host. You can share the portage data directories, for example.

Not sure if you're familiar or if the lesser isolation works for your scenario, but you may also want to look at docker to augment your kvm host. For little stuff it's really slick.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2302

PostPosted: Tue Dec 19, 2017 5:56 pm    Post subject: Reply with quote

Another thing:

I'm not sure how familiar you are with Linux based on your comments.

If you use RAID then you might want to consider using software RAID. The performance is not significantly worse if you're comparing against a cheaper RAID card, and with software RAID you don't need to worry about keeping a spare RAID card. Hardware RAID, you can lose your data if you don't have a compatible same-age RAID card handy. Software RAID can be handled by any Linux box with the appropriate software.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2302

PostPosted: Tue Dec 19, 2017 10:58 pm    Post subject: Reply with quote

GPU passthrough appears to be possible (I had never heard of it before your thread) and there are lots of hits for it in Google. Including a warning about something called Valve Anti-Cheat possibly detecting that the guest is in a VM and banning you.

While I have some small bit of experience with KVM and QEMU (NOT an expert!) I have zero experience with gaming or GPU passthrough.

I've used ESXi though, and I have some observations that may be relevant and/or informative for someone moving to KVM/QEMU:

  1. QEMU is the user-space virtualization engine. It can work with or without hardware acceleration which is supplied by KVM or Xen.
  2. KVM is inside the Linux kernel, controlled by some user-space software and apps.
  3. Unlike ESXi there are multiple features you'll either want or not want, and it takes some serious reading on your part to decide.
  4. You could make a console-only hypervisor install very similar to ESX, with a very minimal command line and selected tools for managing hardware.
  5. You could alternately install a full GUI workstation and have it host QEMU/KVM as well, and it would still have the same hypervisor, unlike VMware Workstation.
  6. You can administer a QEMU box through virt-manager, which is a gui client that does not need to be on the host. You can also do everything from a command line.
  7. There are other gui clients but I have no experience with them, virt-manager does what I want.
  8. There are Open Source tools to do everything you need with respect to QEMU/KVM.
  9. There are also commercial offerings which make it feasible to manage guests on multiple hosts and other enterprise-like things. They will show up in your search results.
Back to top
View user's profile Send private message
Spargeltarzan
Apprentice
Apprentice


Joined: 23 Jul 2017
Posts: 213

PostPosted: Tue Dec 19, 2017 11:26 pm    Post subject: Reply with quote

I like Xen for Linux Pvh Guests, but also want to start gpu passthrough and I am unsure aboutits performance difference and easy maintenance/setup
_________________
___________________
Regards

Spargeltarzan

Notebook: Lenovo YOGA 900-13ISK: Gentoo stable amd64, GNOME systemd, KVM/QEMU
Desktop-PC: Intel Core i7-4770K, 8GB Ram, AMD Radeon R9 280X, ZFS Storage, GNOME openrc, Dantrell, Xen
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2302

PostPosted: Tue Dec 19, 2017 11:26 pm    Post subject: Reply with quote

Also, my most recent KVM box is a supermicro atom-based board found here http://www.supermicro.com/products/motherboard/atom/x10/a1srm-ln7f-2758.cfm .

Warning: I'm a converted supermicro fanboy. I don't work there, I just buy their stuff. This post has a small bit of relevance for your situation and a whole lot of fanboy propaganda.

My rig has IPMI v2.0 support, which I highly recommend for a remotely administered box. It also has high quality everything. It's the first system I ever built which didn't have some piece of hardware I had counted on that didn't turn out to be junk, based on building systems since the mid 90s. 7x Intel nics, as opposed to Realtek or some other BS. It turns out that https://antsle.com markets a lightweight virtual machine host with almost exactly my board's specs. I'm converting mine to use kvm/qemu and docker (https://docker.com) for lightweight images.

I can literally do anything through IPMI except move it to another shelf or add/remove physical cables. I can turn it on/off, reset, get console or gui console access, whatever I want. I can control fan speeds.

Presumably other manufacturers have the same capability, but IMO if you're building something new then I say supermicro rocks and you should at least look at what they offer.

My rig isn't a gaming system, but SuperMicro offers the full range, from atom to i7 to 8-socket e7 hardware to ultra-compact server farm hardware with a huge variety of GPU arrangements.
Back to top
View user's profile Send private message
Spargeltarzan
Apprentice
Apprentice


Joined: 23 Jul 2017
Posts: 213

PostPosted: Tue Dec 19, 2017 11:35 pm    Post subject: Reply with quote

1Clue,


Would you help me in my thread "supermicro ZFS Workstation" in Kernel and Hardware too, if you have some experience? Would be great, many thanks in advance!!
_________________
___________________
Regards

Spargeltarzan

Notebook: Lenovo YOGA 900-13ISK: Gentoo stable amd64, GNOME systemd, KVM/QEMU
Desktop-PC: Intel Core i7-4770K, 8GB Ram, AMD Radeon R9 280X, ZFS Storage, GNOME openrc, Dantrell, Xen
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2302

PostPosted: Wed Dec 20, 2017 3:25 am    Post subject: Reply with quote

Posted, but I need to remind you I'm not really an expert in any of this. I just type really fast. :)
Back to top
View user's profile Send private message
keet
Guru
Guru


Joined: 09 Sep 2008
Posts: 527

PostPosted: Wed Dec 20, 2017 4:20 am    Post subject: Reply with quote

I would like to use RAID, more for parity and protecting against data loss than to gain performance. I thought that some of the newer filesystems implemented their own RAID-like features, as well.

In any case, I ordered the parts for my new home server, so I'll probably build it early next week. I'll probably start with K.V.M. and Qemu and just set it up without G.P.U. passthrough, since I don't really need it, and then see whether I can make it work.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2302

PostPosted: Wed Dec 20, 2017 4:46 am    Post subject: Reply with quote

RAID does not improve performance. It sacrifices performance in almost every case.
RAID is not a backup. It satisfies only a few cases of many problems that a backup prevents.
Back to top
View user's profile Send private message
keet
Guru
Guru


Joined: 09 Sep 2008
Posts: 527

PostPosted: Thu Dec 21, 2017 2:13 pm    Post subject: Reply with quote

1clue wrote:
... IMO if you're building something new then I say supermicro rocks and you should at least look at what they offer...


Thank you for the recommendation. I've always built my own desktops. I did check their site, and while it looked quite good, I decided to choose my own parts and built it myself. Also, in that case, it's an educational opportunity for my children. :)
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2302

PostPosted: Thu Dec 21, 2017 2:37 pm    Post subject: Reply with quote

keet wrote:
1clue wrote:
... IMO if you're building something new then I say supermicro rocks and you should at least look at what they offer...


Thank you for the recommendation. I've always built my own desktops. I did check their site, and while it looked quite good, I decided to choose my own parts and built it myself. Also, in that case, it's an educational opportunity for my children. :)


Both the supermicro boxes I have contact with were built from the ground up. You can get motherboards and populate them yourself. Start with good hardware you'll get a better machine.
Back to top
View user's profile Send private message
keet
Guru
Guru


Joined: 09 Sep 2008
Posts: 527

PostPosted: Fri Dec 22, 2017 2:54 am    Post subject: Reply with quote

1clue wrote:
RAID is not a backup. It satisfies only a few cases of many problems that a backup prevents.


True, I would also have separate backups. Still, it might be worth using R.A.I.D. 5 for at least some protection, probably in an L.V.M. inside an encrypted L.U.K.S. container:

https://wiki.gentoo.org/wiki/LVM#Striping_with_parity_.28RAID4_and_RAID5.29
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2302

PostPosted: Fri Dec 22, 2017 6:59 pm    Post subject: Reply with quote

I'm not going to tell you how to set up your box. I'll only tell you my observations and recommendations and let you decide.

I've never worked with a raid5 array -- hardware or software -- that I didn't think should be twice as fast as it was. I don't know how much performance matters to you, but in one case we took down a production box with hardware raid5, with a higher quality controller, because it was too slow to get the job done. Even raid1 has to wait for the slowest drive, and a bit of overhead.

If you use a raid array, I recommend that you only use it for the data you insist must remain hot all the time. Put the OS on a non-raid ssd, or at least raid1. The more complicated the raid strategy the slower you're going to be.
Back to top
View user's profile Send private message
keet
Guru
Guru


Joined: 09 Sep 2008
Posts: 527

PostPosted: Sat Dec 30, 2017 12:28 am    Post subject: Reply with quote

Thank you for the recommendation. I was (and still am) having trouble exactly how to accomplish. I want for all the data to be encrypted, as much as possible. I was reading this article for recommendations:

https://wiki.archlinux.org/index.php/Dm-crypt/Encrypting_an_entire_system

I would prefer to have one encrypted LUKS volumes with multiple LVM volumes inside it (inheriting the overall encryption). However, it sounds like that's impossible with multiple disks, unless one can encrypt a volume group, which I didn't see in that article (alternately, I could encrypt the volumes separately, but I'm not sure that I want to do that). However, it does mention LUKS on software RAID, and I assume that I could put LVM inside the LUKS container. I ran this command:

Code:
mdadm --create /dev/md0 --verbose --level=5 --raid-devices=3 /dev/sda3 /dev/sdb3 /dev/sdc3


Once it's done, I'll encrypt /dev/md0, then configure LVM inside that encrypted container. Hopefully this makes sense. I should note that I have three identical (spinning) hard drives.
Back to top
View user's profile Send private message
keet
Guru
Guru


Joined: 09 Sep 2008
Posts: 527

PostPosted: Sat Dec 30, 2017 3:34 am    Post subject: Reply with quote

Since some of your signatures mention Z.F.S... does anyone here know whether 0.7.5. includes native encryption? Also, it's marked as testing/unstable, but I'm guessing thati it 's actually reliable and just that every version is keyworded for Gentoo?
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2302

PostPosted: Sat Dec 30, 2017 6:50 am    Post subject: Reply with quote

You want to encrypt a host filesystem which contains guests or you want to encrypt a volume on a guest?

IMO you need to define what it is you need to have encrypted and only encrypt that. I have never found a need to encrypt a filesystem, only individual files. So I don't have experience for what you're saying.

What I know is that encryption=slower and encrypting a filesystem on the host which will then contain guests will be slowest of all.

IMO it makes no sense to encrypt application binaries unless you need to hide the fact that your computer had any bytes at all written to the disk. The CIA may find that useful, but I can't see how anyone else would.

The way I understand it, encrypting a volume group should be possible.

In my limited understanding of encrypted filesystems, the encryption key should be in the possession of the user, not on the disk itself. It should be inserted when the system boots, then removed once the system is up to prevent theft. I may have that wrong, but it seems to me that if you have the key on the system then it boots automatically anyway, and you'd just as well not encrypt it in the first place.

If you're serious about this encryption thing you may want to consider getting hardware acceleration for encryption. For example, Intel's QuickAssist can be found in a couple different PCI cards and also embedded into some of the atom chips. c2x58 and c3x58 for example.
Back to top
View user's profile Send private message
szatox
Veteran
Veteran


Joined: 27 Aug 2013
Posts: 1648

PostPosted: Sat Dec 30, 2017 12:09 pm    Post subject: Reply with quote

Quote:
If you're serious about this encryption thing you may want to consider getting hardware acceleration for encryption. For example, Intel's QuickAssist can be found in a couple different PCI cards and also embedded into some of the atom chips. c2x58 and c3x58 for example.
Many CPUs those days provide hardware support for AES.

Quote:
it seems to me that if you have the key on the system then it boots automatically anyway, and you'd just as well not encrypt it in the first place.
Yup. Either make it password-protected (Hint: LUKS) or find some other way to provide the key on-demand. E.g. there were some tricks for burning keys into a separate USB flash drive.

Quote:
IMO it makes no sense to encrypt application binaries unless you need to hide the fact that your computer had any bytes at all written to the disk.
Pretty much the same requirements a good backup strategy has: if you have to think about it, you're gonna fail.
Full disk encryption is easy to setup and doesn't bother you when you do your thing.
Back to top
View user's profile Send private message
keet
Guru
Guru


Joined: 09 Sep 2008
Posts: 527

PostPosted: Sat Dec 30, 2017 5:43 pm    Post subject: Reply with quote

1clue, it sounds to me like you are discouraging full-disk encryption. However, it doesn't seem that much more difficult simply to encrypt the whole thing. If I encrypt selectively, it just becomes more complicated. Full-disk encryption (apart from the bootloader) has been easy and reliable in my experience; I've never had any problems with it.

As szatox said, I'm going to keep the decryption key separate (written on a flash drive, or as a password...).

The part that complicates it for me is that I have multiple disks that I want to encrypt as one container. Making them into a RAID array seems to accomplish this as well as giving me some redundancy/protection against disk failure (which is certainly not to say that I'm going to eschew backups; I'll definitely still back it up periodically).

Thus, what I'm doing at the moment is a RAID 5 array (/dev/md0) using all three drives, which I'm encrypting using LUKS (/dev/mapper/encryptedmd0), and then LVM inside that encrypted container. That way (almost) everything is encrypted as one container and I have the benefits of RAID 5 and LVM.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 12356

PostPosted: Sat Dec 30, 2017 6:03 pm    Post subject: Reply with quote

To elaborate on szatox's comment regarding why encryption should be done on a broad basis:
  • Filesystem or block device encryption can be transparent to the user applications, so it works automatically for programs without requiring dedicated support in each application.
  • Some programs have a habit of writing to files other than just the ones the user specifically directs. For example, many editors keep a "recently opened files" list, and some also keep a "recently copied text snippets" list. If you care about encrypting a given file, you likely also care about encrypting any content that comes from that file, so you want the snippets from that file encrypted even when they are not in that file. If all places the editor writes its automatic files are backed by encrypted filesystems/block devices, then the snippets are protected even when separated from their original file.
  • When using file-specific encryption, any temporary decrypted copies required for working with other tools (text editor, browser, word processor, etc.) need to be securely purged when the user is done. This may or may not be easy depending on where the temporaries are stored. Even when it is easy, it's an extra manual step.
This does not even get into the philosophical issues regarding deniability, metadata leaks, etc.

The only substantial upside I see to encrypted files is that it is easier to leave them secured while the system is otherwise operational. For example, even on a system with full-disk encryption, I might store my password database in a separately encrypted file, so that it's not trivially readable to all my programs whenever I'm logged in, but is instead open only when I choose to open it.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum