Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Hardware review: SuperMicro A1SRM-LN7F-2758
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Off the Wall
View previous topic :: View next topic  
Author Message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2433

PostPosted: Thu Apr 05, 2018 7:52 pm    Post subject: Hardware review: SuperMicro A1SRM-LN7F-2758 Reply with quote

I ordered one of these in the last couple days of 2014. I recieved it in January the next year.

https://www.supermicro.com/products/motherboard/Atom/X10/A1SRM-LN7F-2758.cfm . I have 16GB ecc memory in it in 2 sticks. I also have an OCZ Vector 150 and an older spinning rust drive. It's in a SuperMicro 1u rack mount case. All in I'm at like USD $1,200.00.

Things I like about this system:


  1. Up to 64 GB RAM
  2. ECC or non-ECC memory
  3. Low power system, 20w TDP for CPU, <80w typical use for the whole system.
  4. It has an ASPEED IPMI chip with the ability to send a GUI console over the network. This system has NEVER had a monitor, mouse, keyboard or removable drive attached.


What I DON'T like about the system:


  1. The lack of VT-d support. I would very much like to donate a couple NICs to specific VMs here.
  2. The inconsistency of the QuickAssist implementations in IBM hardware.

    Uses:
    This system had a couple different operating systems on it in the first few days, including Ubuntu Server and pfSense, just to see what happened.

    pfSense I never got working. After some forum posts indicated that, while this board is very similar to a pfSense supported system, it's different enough that there was a significant amount of work to make it go. I believe that may be fixed now, but when I wrote it off it had not been resolved.

    Ubuntu Server was just to see how everything came up, and of course it worked fine but I didn't want Ubuntu on this setup so off it came.

    There are several hardware issues with the Intel Atom c2000 chips, none of which are SuperMicro's fault. The ones that come to mind:


    1. https://www.theregister.co.uk/2017/02/06/cisco_intel_decline_to_link_product_warning_to_faulty_chip/
    2. https://www.anandtech.com/show/11110/semi-critical-intel-atom-c2000-flaw-discovered
    3. Due to the way the QuickAssist (QAT) hardware cryptography/compression acceleration works, the performance in real-world VPNs is not quite as good as you'd expect.
    4. QAT hardware is implemented differently in every revision of hardware. Libraries must code for all these differences. The good part here is that IBM supplies the libraries for the most part. The bad news is it's a proprietary blob, if that matters to you.


    These are all problems with the atom c2000 series chips and with QuickAssist, wherever it's implemented.

    Full disclosure: My system has been affected by the flaw described by the anandtech link. I have an RMA in right now.

    So here's my take:

    There are a few applications that make sense for this type of hardware. The ones I'm interested in are:


    1. An Antsle-style microserver for virtualization in small offices or nerd homes.
    2. A firewall/router/IDS/IPS
    3. A NAS or SAN


    Microserver: https://antsle.com/ makes microservers based on C2000 chips. It appears to be some sort of QEMU and/or app server and/or docker.io implementation. Antsle OS is based on Gentoo. If you want a bunch of VMs or services that will be internal-only in a relatively small capacity then this type of thing is very interesting. Most VMs in a non-enterprise setting want a lot of threads but they don't need huge cores.

    Firewall/router/IDS/IPS: The board I have is literally designed to be enterprise networking hardware. It has 7 NICs: 1x system admin port and 3 pairs of NICs which can be used to support failover systems in hardware, or do load balancing type things.

    NAS/SAN: This is a communications processor, and one other place this type of chip is used is in network storage devices. This board has more NICs and fewer SATA slots, but it has BIOS settings for iscsi and it would, IMO, make a great SAN or NAS for a small office.

    My experience:

    My experiments with networking and piping encrypted/compressed data have been fantastic, but I don't really have enough high-speed hardware in my home to fully test it. Also, running a pipe of encrypted data through a network card or just to a disk is not the same as a VPN with real traffic. I have piped /dev/random through an encrypt and compress pipe, through a monitor app, and to /dev/null at about 2.5 gbps. I've squezed 3 processes through 3 different nics to an i7 box, and got about 2 gbps. That said the other box is an old i7 920 with 5x Realtek cards. The same tests for encryption and compression show the i7 to be slower than the c2758, and Realtek generates many more interrupts than the built-in Intel NICs that are on the SuperMicro board.

    VPN:

    I had this system in place as a firewall/router for awhile at my home. I bought the system when my ISP announced that gigabit Internet would soon be available in my area. It's available now, but I don't have it.

    At any rate, I had OpenVPN on this system, and did a VPN tunnel test. Having an established OpenVPN tunnel between my home and office, I saturated the other side (a SOHO router) in short order. Using one of our cloud-based test servers, I connected to the VPN at my home and quickly matched the actual packets sent to my maximum tested throughput via speedtest.net. That was 70 mbps. The SuperMicro board had very low system load throughout. So FWIW my VPN tests are entirely inadequate.

    Recommendation:

    Given the known issues with c2000 processors I would recommend that you go with c3000 processors if you want this type of system. C3000 processors offer:

    1. SATA3 and USB3 instead of rev 2. My c2000 system has 2x sata3 ports, 4x sata2.
    2. VT-d support to donate NICs and other hardware to a VM guest. My biggest gripe with my board is that the c2000 does not support VT-d.
    3. 2-16 cores instead of 2-8 cores.
    4. M.2 PCIE-3 interfaces
    5. 10 GBPS ports, or a combination, or 1 GBPS ports.
    6. Presumably the known hardware flaws of C2000 are fixed.


    In hindsight if I could trade my existing board for something else, I'd pick a C3000 board, sacrifice QuickAssist for a bunch of SATA3 ports and sacrifice some of the 1000-T NICs for some 10 GBPS NICs. Ideally I would have 2x gigabit NICs (1 for dedicated management NIC and the other for a WAN interface) and at least 2 10-gigabit NICs. I'd also like to go to 16 cores.
[/list]
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1606
Location: U.S.A.

PostPosted: Fri Apr 06, 2018 1:41 am    Post subject: Reply with quote

Nice write up.

Get some drives and put FreeNAS on it. That's a suitable role for a machine like that not having a lot of cpu but engineered for reliability.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2433

PostPosted: Fri Apr 06, 2018 3:06 am    Post subject: Reply with quote

Thanks.

A 2-core version of this would be a nice NAS device for 3 or 4 gigabit NICs. It's a kvm box which has a NAS vm.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 17502

PostPosted: Fri Apr 06, 2018 4:49 am    Post subject: Re: Hardware review: SuperMicro A1SRM-LN7F-2758 Reply with quote

1clue wrote:
In hindsight if I could trade my existing board for something else, I'd pick a C3000 board, sacrifice QuickAssist for a bunch of SATA3 ports and sacrifice some of the 1000-T NICs for some 10 GBPS NICs. Ideally I would have 2x gigabit NICs (1 for dedicated management NIC and the other for a WAN interface) and at least 2 10-gigabit NICs. I'd also like to go to 16 cores.
Interesting. Did you buy direct, or through a reseller? I hate having to deal with sales just to see if something is viable. I'm guessing the network devices are all on board? How loud does it get?
_________________
It is what it is out there. So whatever it is, it is.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2433

PostPosted: Fri Apr 06, 2018 4:59 am    Post subject: Reply with quote

I bought through amazon.

7x gigabit nics on-board. Every device on the board has good linux drivers and is a good quality device. The only proprietary blobs are for quickassist. If you give up that by going to a different chip then you can get more sata or network ports.

This chip is 20w tdp. I chose a 1u case but you could very easily make this system fanless.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 17502

PostPosted: Fri Apr 06, 2018 5:18 am    Post subject: Reply with quote

Thanks. I've so far avoided going the server route due to noise. I suspect it probably isn't too loud, given its TDP.

I may have to see what is on Amazon. The price and Intel are my two major issues.
_________________
It is what it is out there. So whatever it is, it is.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2433

PostPosted: Fri Apr 06, 2018 1:36 pm    Post subject: Reply with quote

Even as a 1u the only time I ever hear it is when it's emerging software. The heaviest network tests I gave it, the fans were on their lowest setting. If you use IPMI to pin the fans to medium or high, then you definitely hear it.

The system is sitting on a wire rack in my basement. My desk is right next to the rack. My i7 with extra-huge fans makes more noise than this system.

The Antsle systems advertise that they're silent and fanless, so I know that these systems can be built that way.
Back to top
View user's profile Send private message
SiberianSniper
Guru
Guru


Joined: 06 Apr 2006
Posts: 362
Location: Dayton, OH, USA

PostPosted: Fri Apr 06, 2018 1:37 pm    Post subject: Reply with quote

Never imagined myself writing a review for these computers, but here goes... We have a couple of those at work (not exactly that motherboard, but they're SuperMicro 1U servers with the c2000-series chips). One running Ubuntu, the other running FreeBSD. Here are a few random passing thoughts:

1) As they're production servers, I've been afraid of rebooting them after hearing about the c2000 flaw, especially since they both had over a year of uptime. However, the FreeBSD one needed to be rebooted for the version upgrade, so I pulled the trigger and..... not a problem. It's been rebooted a second time since then without issue.

2) They're plugged into an 8-port KVM. The video works on both, but the keyboard and mouse only work on one. Though they're not identical models, they're not sufficiently different as to why I can figure out the issue. That said, I haven't had to use a physical keyboard on it since it was first installed, so it's more of a curiosity than an annoyance.

3) They're low power, but not particularly quiet. That isn't helped by the fact that I found some extra fans and installed them just because these computers live in a closet.

4) The FreeBSD machine is running as a firewall and has been rock solid. The Linux one is running our web and email servers (small company, only a handful of accounts). In retrospect, I'd probably put the email server on a Xeon/Opteron computer if I had the chance. The web serving is snappy, but the email server's webmail sometimes hangs for ten seconds when pulling up the log in page. I suspect it's a software issue, but can't rule out hardware yet. But, as it's just an annoyance, we don't see a need to upgrade. This pretty much falls in line with 1clue's recommendation.

5) Most of these come with 2 NICs, but some have four in case you need that. It's handy for a firewall (WAN, LAN, DMZ, spare/guest)

6) Would I have one at home? Probably not, as modern day routers are good enough to handle media and NAS, but if I needed anything more, I'd definitely consider it.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2433

PostPosted: Fri Apr 06, 2018 1:56 pm    Post subject: Reply with quote

@SiberianSniper,

Why do you use a KVM? Don't your boxes have an ASPEED video controller? That's a BMC that lets you connect with IPMI through one of your NICs. As my board has 7 NICs I've configured my BIOS to dedicate that NIC to management. It doesn't go through a normal switch, instead straight to an extra NIC on my workstation.

My system has never had a keyboard or monitor or mouse attached. The IPMIView software from SuperMicro gives you gui console access through that IPMI interface.

The 10 second delay for a login page sounds like a software issue. Do you have it doing DNS lookups on all new connections? Just log IP addresses and times, and that saves a significant amount of time, especially if every IP in your office doesn't have a DNS entry.

I've rebooted this system any time I needed to without problem until recently. And when I started to have the problem I was rebooting constantly, trying to get my system to boot without the grub boot loader. Since that bug mentions the boot process and overheating that resistor, I'm thinking that my experiments had something to do with the flaw being triggered on my system. And toward the end there I had the system on the table, cover off because I couldn't get to the BIOS without removing all the hard drives. Then I'd make changes to the bios, reconnect the hard drives and reset. So it was likely not properly cooled either since the fans went straight to open air rather than forcing air through the CPU heat sink.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2433

PostPosted: Fri Apr 06, 2018 3:34 pm    Post subject: Reply with quote

SiberianSniper wrote:

6) Would I have one at home? Probably not, as modern day routers are good enough to handle media and NAS, but if I needed anything more, I'd definitely consider it.


Actually I was trying my hardest not to have one in my home.

Some years before this I went through a process of trying to find an adequate firewall for my purposes. I read a lot of reviews of SOHO routers and some low-end commercial stuff. When my ISP announced gigabit bandwidth coming soon, I added that to the mix. I wanted very good vpn performance, really good VLAN management and a stateful firewall with IDS/IPS. I also wanted a 2-stage setup, with Linux at one point and something like pfSense on the other.

SOHO routers are clearly not there. Even with DD-WRT and the like the hardware just wasn't up to the task. Not only that but with some experimentation with DD-WRT the gui was not consistent with the command line, and while it was OK for things most people do with a SOHO router the features I was interested in were just not done well.

So I started looking at pfSense hardware, and lurking on their forum and eventually asking questions. Looked at routerboard.com, all that. But in both cases you're dealing with a commercial solution that pretty much locks you into whatever they have. Same thing with small business solutions.

I figured this system would do nicely. The only thing that threw a wrench into it was the lack of VT-d, so I couldn't properly donate NICs to a VM. The C3000 series allows that.

At any rate, if you wanted a gigabit firewall/router/vpn/IDS/IPS system with logging, my board would very definitely handle that.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 17502

PostPosted: Fri Apr 06, 2018 4:03 pm    Post subject: Reply with quote

SiberianSniper wrote:
6) Would I have one at home? Probably not, as modern day routers are good enough to handle media and NAS
I just haven't found one I'd want to use for any more than the basic routing and/or switching.


1clue wrote:
The IPMIView software from SuperMicro gives you gui console access through that IPMI interface.
But not only a GUI, a CLI as well? I've never seen an IPMI GUI I wanted to use.
_________________
It is what it is out there. So whatever it is, it is.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2433

PostPosted: Fri Apr 06, 2018 4:36 pm    Post subject: Reply with quote

pjp wrote:
SiberianSniper wrote:
6) Would I have one at home? Probably not, as modern day routers are good enough to handle media and NAS
I just haven't found one I'd want to use for any more than the basic routing and/or switching.


1clue wrote:
The IPMIView software from SuperMicro gives you gui console access through that IPMI interface.
But not only a GUI, a CLI as well? I've never seen an IPMI GUI I wanted to use.


It has a text console too, but I never got it to work. Admittedly I didn't care enough to try much. There may be a telnet/ssh-style terminal interface but last resort you could hook a serial cable to it using a dedicated rj45 serial port on the board.

The GUI client is extremely similar to rdesktop in terms of user experience. You login to the system with IPMIView, then start the gui console (either before or after it's powered on). You can plug in an ISO image into a virtual DVD drive if you want to. Or just power it on. The GUI client shows the BIOS splash and everything else you would see if you had a monitor attached. If the BIOS shows a graphic image rather than just text, that's what you see.

The board has an old-school D-Sub connector for a monitor if you want, but I've never used it and will never use it.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 17502

PostPosted: Fri Apr 06, 2018 5:34 pm    Post subject: Reply with quote

Thanks, I'll keep that in mind.

I've primarily used ILO and ILOM with direct network connections and ssh, although many include some degree of https support (though it is poor IMO). CLI also allows BIOS access, but not the graphics.

My use of IPMI has only been with ILOM and ILO. KVM (kbd/video/mouse) over Ethernet would be nice, but is too expensive.
_________________
It is what it is out there. So whatever it is, it is.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2433

PostPosted: Fri Apr 06, 2018 7:48 pm    Post subject: Reply with quote

With this ASPEED chip there is absolutely no reason to have a kvm switch. This setup is much more convenient, much better in every way. Assuming you have an extra NIC more than your intended application requires, you can make a control-only network which does not route traffic anywhere else, and have a secure set of computers which can access the IPMI interface. I don't need 7x NICs on a home router when I have perfectly good switches, and my workstation has multiple NICs.

At any rate with this IPMI interface you can have however many servers with dedicated IPMI interfaces, and however many clients connected from wherever the administrators sit. I can't imagine a KVM switch like that. And again, it has a plain-old RJ45 TTY port too, in case you want to go that way.
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1606
Location: U.S.A.

PostPosted: Fri Apr 06, 2018 11:49 pm    Post subject: Reply with quote

1clue wrote:
Thanks.

A 2-core version of this would be a nice NAS device for 3 or 4 gigabit NICs. It's a kvm box which has a NAS vm.

It doesn't have enough CPU too be useful as virtualization host.
Back to top
View user's profile Send private message
bunder
Bodhisattva
Bodhisattva


Joined: 10 Apr 2004
Posts: 5743

PostPosted: Sat Apr 07, 2018 9:33 am    Post subject: Reply with quote

question about QAT... don't you need an addon card to make that work?
_________________
overlay | patches
Neddyseagoon wrote:
The problem with leaving is that you can only do it once and it reduces your influence.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2433

PostPosted: Sun Apr 08, 2018 12:27 am    Post subject: Reply with quote

Bones McCracker wrote:
1clue wrote:
Thanks.

A 2-core version of this would be a nice NAS device for 3 or 4 gigabit NICs. It's a kvm box which has a NAS vm.

It doesn't have enough CPU too be useful as virtualization host.


https://antsle.com/ is a gentoo-based product on a c2000 processor which is, specifically, a personal virtualization host. They seem to be selling products.

Based on my experience with the box, the VM has most of the characteristics of the bare metal. If you're doing heavy compiling then it's going to be slow. If you have a bunch of communication-oriented VMs then it does really well.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2433

PostPosted: Sun Apr 08, 2018 12:39 am    Post subject: Reply with quote

bunder wrote:
question about QAT... don't you need an addon card to make that work?


For bigger systems I believe the only real way is to use an add-on card. The Atom is an SoC. The c2*58 models and a few more have QAT built into the SoC. It is a separate processor independent of the CPU, just like if it were an add-on board.

So as a comparison you have the AES instructions in most Intel chips these days. They're built-into-the-cpu the same as a move or compare operation, or multiply or shift. Each core can run these operations, and more cores means more parallelism. These instructions are per-core operations just like anything else, and they work like you might think.

The QuickAssist module, built into the Atom or separate as per the PCI boards you've no doubt read about, are a separate device even though it's built into the physical chip of the processor. You set up a job, you tell the hardware to go and then you either wait or go do something else until the job notifies you that it's done.

In the reality of working with it, you choose to compile QAT into the kernel or not, and you choose to build the encryption libraries with it or not. There are relatively few places in the installed software where you need to make the adjustment. If you chose QAT blobs then it's pretty much all done for you, and you do things using the regular Linux system libraries for encryption and compression.

QAT excels at large blocks of data all being encrypted or decrypted at once, and/or compressed/decompressed at once. The smaller the job, the worse it performs because the overhead of setting up the job is essentially the same no matter what. So a typical VPN might suffer a bit when dealing with an interactive user. That said, it's still faster than an equivalent chip without QAT.

Running Gentoo, my c2758 system compiles roughly half the speed of my i7 920. Encryption benchmarks, however, the c2758 performs slightly better than the i7, both encryption and compression. Which is not bad considering it's i7 vs atom, roughly 600w system power use vs roughly 80w on the atom.
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 5327

PostPosted: Sun Apr 08, 2018 7:19 am    Post subject: Reply with quote

2.5Gbps /dev/random, those are some impressive numbers. I just checked my router (an AMD E350, getting pretty old): it generates 950kB/s with haveged running, 1 byte per second without...
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2433

PostPosted: Sun Apr 08, 2018 6:59 pm    Post subject: Reply with quote

Ant P. wrote:
2.5Gbps /dev/random, those are some impressive numbers. I just checked my router (an AMD E350, getting pretty old): it generates 950kB/s with haveged running, 1 byte per second without...


I've never been an amd guy. I don't know what an e350 is, but your 1 byte per second number is surely due to some sort of misconfiguration or something.

I found this, it doesn't do vpn throughput or any of that but it looks to provide standardized benchmarks.

https://www.servethehome.com/intel-atom-c2758-benchmarks-8-core-rangeley-tested/

As I said the throughput I have was me hacking a test together. Don't recall exactly what the pipe was but it was /dev/random, through an encryption, through a compression and then through a stream bandwidth app, and then to /dev/null. I got the pipe example from a google search, then I experimented with it to get everything involved. My actual network tests are limited by my other hardware.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2433

PostPosted: Sun Apr 08, 2018 9:04 pm    Post subject: Reply with quote

Here's a VPN benchmark for a very similar board with the same processor, a supported setup by pfSense.

https://store.netgate.com/pfSense/C2758.aspx

The numbers there make me pretty sure my personal tests were trash. IDK if I got the units wrong or what, but there's no way I actually got 2.5 gbps with one of those encryption methods. I'm pretty sure I did this with a non-AES cipher because I wanted to check my QAT hardware, but I seriously doubt that would perform that much better, no matter how easy an algorithm I chose.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Off the Wall All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum