DIY virtualized router with real hardware

Message

1clue · Post by **1clue** » Sat Oct 12, 2013 3:09 pm

Renamed from "Need hardware recommendations for DIY router" since that no longer defines the project.

Really, what I need is a router. If somebody knows of off-the-shelf hardware for under $300 then that would be fine too.

This is for my home office, and I need big router features and little router price.

I just bought a Cisco EA6500 (to put behind my other EA6500) but now, before I get it out of the box, I realize that's not going to work, I still don't get the features I want.

Here's what I need:

802.1q (VLANs)
VPN support
Gigabit speeds full duplex
IPV6
firewall that can do rules based on VLANs
A good DHCP+DNS that's better than what normally comes on SOHO equipment.
Low latency

It would be neat to have something like Gentoo on it rather than DD-WRT. I would just stick DD-WRT on what I just bought, but it's not supported.

All the PC-based linux routers I've done before had a high latency for some reason. Of course last time I did it was almost a decade ago.

Any ideas would be appreciated.

frostschutz · Post by **frostschutz** » Sat Oct 12, 2013 3:19 pm

I'd use a cheap TP-Link router with OpenWRT on it, but maybe that's just me...

666threesixes666 · Post by **666threesixes666** » Sat Oct 12, 2013 4:22 pm

i started collecting and documenting technologies revolving around your plan....

802.1q (VLANs)
VPN support
Gigabit speeds full duplex
IPV6
firewall that can do rules based on VLANs
A good DHCP+DNS that's better than what normally comes on SOHO equipment.
Low latency

vlans didnt get 2, vpn only got to hamachi/haguichi... gigabit is hardware problem need 2 gigabit add ins, if motherboard doesnt support it, gigabit switch.... ipv6 supported out of kernel, firewall iptables & "webmin" are good sauce, based on vlans no idea though i imagine so... dnsmasq wiki page handles dhcp + dns, defacto openwrt/ddwrt dhcp dns server... low latency, just build your own box and make your serious linux machine act as router.

so i mean its over 50% of what your post is asking for though not 100% the spirit of the open source community is take what others have done, and build upon it.

Atom2 · Post by **Atom2** » Sat Oct 12, 2013 7:51 pm

I'd see two possible routes to go down - although none involves gentoo:

1.) go with pfsense and buy an ALIX board: The pfsense software is freely available, very comprehensive, easily configurable through a web interface and probably one of the best firewall and router distributions available. There's even a dedicated book available covering all the configuration; the latest version based upon the 2.1 release is about to be released in due course. BTW pfsense is based on BSD.
ALIX is one recommeneded hardware partner. You'll find more info about the ALIX hardware and other hardware options here and if I recall correctly you should be able to find something within your budget.

2.) Use the Asus RT-N66U and reflash its firmware with dd-wrt. The (wireless) router hardware is very good, serves the 5 GHz and 2.4 GHz band at the same time and provides upt to 450 MBit/s wireless speed. All ports are Gigabit LAN and dd-wrt runs very stable. Pricewise the Asus hardware is definitely well within your budget.

1clue · Post by **1clue** » Sat Oct 12, 2013 10:20 pm

My old "inside" router died, and I want to get something better. I tried DD-WRT on that thing, and it was flaky. I'll use DD-WRT again if I have to but so far none of the hardware I'm interested in is listed in their database.

My existing device (which I'm going to keep) has 802.11ac, which is almost a gigabit in wireless, dual band. I have that part covered, I'm not experiencing any sort of latency there. Problem is, this one doesn't have any VLAN support, so I'm kind of in trouble. My old one had VLANs.

I've got a whole lot of home devices, and I have some hardware for my work that I want to isolate from all that. The nature of my work, I need a clean network for my job. I also need some isolation within the work environment for testing purposes.

For this piece, if I get anything with a radio on it I will probably turn the radio off. I want a secure zone behind the normal router, and I want it to actually be somewhat secure, so wires only.

I'm looking at the ALIX stuff now. This is looking really promising. These boards, they say Linux so maybe Gentoo would be an option. I don't know anything about pfSense yet, so I've got some reading to do. Building my own switch out of boards is ideal, I just haven't been able to figure out where to get hardware.

Keep the ideas coming.

Thanks.

666threesixes666 · Post by **666threesixes666** » Sun Oct 13, 2013 2:57 am

i was looking at some interesting articles about vlans and trunking.

http://www.techrepublic.com/article/use ... ltihoming/
http://wiki.openwrt.org/doc/networking/ ... interfaces
http://www.linuxjournal.com/article/7268?page=0,0

gentoo can do ospf bgp & rip via the packages quagga & bird. id really like to get some collaboration on working out corporate grade routing / networking on gentoo. i knew it at some point on corporate grade equipment, but i wanted to migrate away from that way back when.

1clue · Post by **1clue** » Sun Oct 13, 2013 4:44 am

Right now, I'm really curious about pfSense, but I don't think it will work on a SOHO router appliance. I've browsed their forum a bit and don't see much.

I don't like the ALIX boards for this because I want more VLANs than the board has NICs. I have 4 VLANs planned. I also can't really find anything promising as hardware for my price point.

I'm currently fixated on what seems to be my best option: DD-WRT is on the verge of releasing a build for an EA6500. I have 2 of them, one of which is still in the box since I'm still trying to figure out if there's a better solution out there.

I'm hoping that DD-WRT has improved somewhat since I tried it last. Or better yet, I can find a 4+nics (not including uplink) board that takes a real Linux distro or pfSense for under $300.

Atom2 · Post by **Atom2** » Sun Oct 13, 2013 2:07 pm

1clue wrote:Right now, I'm really curious about pfSense, but I don't think it will work on a SOHO router appliance. I've browsed their forum a bit and don't see much.

That's my understanding as well.

1clue wrote:I don't like the ALIX boards for this because I want more VLANs than the board has NICs. I have 4 VLANs planned. I also can't really find anything promising as hardware for my price point.
[...]
Or better yet, I can find a 4+nics (not including uplink) board that takes a real Linux distro or pfSense for under $300.

Have a look at standard PC/server hardware that can be expanded with a dual link (or quad link) Ethernet card. An example might be this. Bearing in mind that this is from ebay.de you should be able to get something similar much cheaper your side of the pond (assuming you are in the US as your budget was quoted in $).

Add a Dual Link Intel PCIe Ethernet card like this one plus a cheaper PCI uplink card (in case you need 4 Gigabit links for VLANs plus 1 additional Ethernet link for the internet uplink) and you should be ready to go.

Another option might be a Quad Link Gigabit Ethernet card, although and unfortunately those are usually in another leauge price-wise so might not be within your budget.

Having said that, I am confident that the former solution (dual link PCIe plus single link PCI) should still work out within your budget.

Also have a read here. There is an idea about sharing the WAN uplink with one of the VLANs through a VLAN capable switch which then goes untagged to the ADSL modem. I haven't tried that myself, but that might work as well. In that post, they also discuss a number of hardware alternatives with more than 2 LAN ports.

I can only vouch for pfSense - it is indeed a great and very stable solution.

1clue · Post by **1clue** » Mon Oct 14, 2013 2:55 am

Yes, I'm in the USA.

The budget is a relatively soft one, but I don't see why I should have to pay much more for what I'm looking at.

The hardware on the EA6500 is adequate to what I need, I just need one that works the way I want it to.

I haven't really found a regular PC box suitable for this sort of thing. All the barebones systems seem to have no more than 2 NICs on them, but I think if I went to mini-itx or something I might find better, there's really no need to be super compact I guess.

For some reason I'm getting irritable about this. I want something like a switch or SOHO router that can run a real distro. Gentoo or maybe a firewall/router-specific distro. I've already spent 2 days on this, which means I should have bought a USD$600 pfSense pre-installed machine to begin with and spent the two days making money instead of unsuccessfully trying not to spend it.

Maybe I'm just tired.

Carnildo · Post by **Carnildo** » Mon Oct 14, 2013 11:03 pm

1clue wrote:I haven't really found a regular PC box suitable for this sort of thing. All the barebones systems seem to have no more than 2 NICs on them, but I think if I went to mini-itx or something I might find better, there's really no need to be super compact I guess.

Is there some reason why you can't take something with a full-sized mainboard, stick a half-dozen single-port Ethernet cards in it, and call it good?

1clue · Post by **1clue** » Tue Oct 15, 2013 2:04 am

Other than the size of it, not really. I don't have a whole lot of room, and I don't want to fill up that space with an AT-sized 6-port switch. I'd rather put in another server or five.

I'm sorry but I can't get the image of a giant antique out of my head. I'd rather push my budget and get something that's better suited to the task.

While I haven't tried pfSense yet, I'm interested in seeing what it does. Maybe I'll put it on a VM? Or I guess I never looked for a demo app.

1clue · Post by **1clue** » Thu Oct 17, 2013 6:17 am

Actually at this point it's looking pretty nice to get a couple of these: http://www.newegg.com/Product/Product.a ... 6833114037
and then find a small board to plug them into.

Just so happens that DD-WRT got a beta of the EA6500 firmware going a couple days ago, so I wasted a couple days trying to make that work. Very buggy and inconsistent, which is what I thought about it before with my wrt610n.

So I got fed up with that, and downloaded a pfSense image. md5 didn't match up, and half the mirrors seem to be dead. So I got fed up with that too.

I think I've blown a week looking for something.

Carnildo is sounding pretty darn smart right now. I KNOW I could do this with Linux on a full board.

Maybe what I should do is buy my new workstation and stuff a bunch of nics in my old i7/12g. Just a little bit overkill, but it's hardware I have, mostly.

AngelKnight · Post by **AngelKnight** » Thu Oct 17, 2013 10:46 am

1clue wrote:Actually at this point it's looking pretty nice to get a couple of these: http://www.newegg.com/Product/Product.a ... 6833114037
and then find a small board to plug them into.

Just so happens that DD-WRT got a beta of the EA6500 firmware going a couple days ago, so I wasted a couple days trying to make that work. Very buggy and inconsistent, which is what I thought about it before with my wrt610n.

So I got fed up with that, and downloaded a pfSense image. md5 didn't match up, and half the mirrors seem to be dead. So I got fed up with that too.

I think I've blown a week looking for something.

Carnildo is sounding pretty darn smart right now. I KNOW I could do this with Linux on a full board.

Maybe what I should do is buy my new workstation and stuff a bunch of nics in my old i7/12g. Just a little bit overkill, but it's hardware I have, mostly.

Did this lately on a home build, using a Soekris Net6501. Having 4 1gbps ports onboard is nice. I think it ticks every box on your list. It ran up a bit higher than $300 since I insisted on having 2x spinning media rather than booting from USB/flash, along with getting 802.11 from the vendor.

Soekris and ALIX boards can run any Linux you want. Even Gentoo or something else source-based if you don't mind the wait.

If you haven't done so already, you might try posting on the Kernel & Hardware forum.

666threesixes666 · Post by **666threesixes666** » Thu Oct 17, 2013 3:21 pm

i agree with the philosophy of reusing old hardware, you could even get a full on work station out of the router its self, not needing to turn on another system to make a quick post to gentoo forums. i disagree with that philosophy as old equipment usually power hogs. old equipment does 1 thing though, gives you experience and time to hunt down micro devices that will fit your specifications. one of my friends on facebook uses a micro pc as a router, ill ask em what kind it is. in the mean time, ill leave you with this....

http://en.wikipedia.org/wiki/Mini-ITX#History

1clue · Post by **1clue** » Thu Oct 17, 2013 4:56 pm

@AngelKnight, this is looking promising.

I'd seen ALIX before from earlier in the thread, but not yet Soekris. I'll go fixate on that for awhile.

Ideally, I would have enough to be my router and nothing else on the system, so I would actually prefer a non-spinning disk. But so far everything I look at is double or triple my budget, or more. Breaking it by a hundred bucks or so isn't going to kill me. I'm beginning to realize that my budget isn't very realistic.

I'm still interested in any small boards anybody knows about with lots of ethernet ports and an ability to run real Linux. Especially handy would be a way to add a 4x ethernet in addition to whatever is on the board.

Thanks.

Post by **NeddySeagoon** » Thu Oct 17, 2013 6:38 pm

1clue,

I'm using a HP Microserver with a 4 port intel NIC. I got my Microserver in the days of £100 cashback in the UK, which made it a very good deal.
It runs gentoo-hardened and host about half a dozen KVMs. I did max out the RAM to do that. One of the KVMs is my media server, so its full of HDDs too.
One KVM is my router.

I don't know what costs are like today in USD

1clue · Post by **1clue** » Thu Oct 17, 2013 6:52 pm

NeddySeagoon wrote:1clue,

I'm using a HP Microserver with a 4 port intel NIC. I got my Microserver in the days of £100 cashback in the UK, which made it a very good deal.
It runs gentoo-hardened and host about half a dozen KVMs. I did max out the RAM to do that. One of the KVMs is my media server, so its full of HDDs too.
One KVM is my router.

I don't know what costs are like today in USD

So, about that:

Don't you consider it a security risk to have a high-function computer as the Internet gateway?
About using a VM as a router: What do you do, give the VM the extra NICs? All the NICs?
Can you/do you use VLANs and trunking?

If that sort of thing works, then I'm all for it. My old i7 is perfect, I'm already using it for a bunch of virtualization and all I'll need to do is add some NICs.

1clue · Post by **1clue** » Thu Oct 17, 2013 6:57 pm

I could get a couple of those four-port nics and that would be the end of my purchases for that.

))

zeronullity · Post by **zeronullity** » Thu Oct 17, 2013 7:53 pm

I've been down this road before myself.. although your performance will increase significantly, so will your power consumption. Even a OLD pentium 4 system with custom compiled kernel/OS will smoke a retail based router any day. 1 to 2ms response. times vs. 0.1 ms or less response times. You might have other options other then DD-WRT.. tomato,OpenWRT, etc. If you can custom compile for your retail router thats the best way to go.. only install packages that you need like with OpenWRT. There is no need to spend a lot of money unless your upgrading to AC wireless. $20 or old scrap pentium 4 configured for power savings.. should do the trick.. just expect to pay in the long run in energy cost.. get a few decent ethernet cards like e1000, and a wireless N/AC card.. and do a minimum low-latency kernel setup. I've yet to find a retail router that out performs my P4 box thats sitting in the garage due to energy cost.. and it's simply loaded with e1000 cards with custom Gentoo/kernel.

FYI - I found that performance wise DD-WRT is extremely buggy especially on routers like Belkin.. compared to other firmware like Tomato by shibby.

Post by **NeddySeagoon** » Thu Oct 17, 2013 8:37 pm

1clue,

The bare metal install is gentoo-hardened with only enough functionality to support KVMs.
Applications are run in KVMs.

I did intend to use PCI pass through to donate the 4 port NIC to the router KVM but there is a hardware bug in its PCIe bridge, which means its not safe to do that. Its fixed in later versions of the card. Instead the interfaces have to be donated to bridges for the router to pick up.

The bare hardware has no internet access at all. The internet is provided by a PPPoE link which has its endpoint in the router VM.
I don't see it as a security risk but its quite a single point of failure risk. My setup may help you get started but be warned that the link is out of date.

There is a mail KVM that runs qmail, http-replicator and a portage mirror, so the bare hardware can't even update unless both the mail server and router KVMs are both running. On boot it complains about not being able to find any timeservers.

The router is shorewall, set up so that everything that is not explicity permitted is denied.

There is an Apache webserver in the mix too but its on a Raspberry Pi running Gentoo rather than a KVM, since Intel shell code won't run on an ARM system.
The Pi is powered from a USB port on the HP Microserver.

I don't use VLANs yet but the support is there if I need to. I've recently got fed up enough with telephone cold callers to get a SIP gateway.
To use that properly I expect I will need to use VLANs for QoS. The idea is to set up POTS as outgoing only.

1clue · Post by **1clue** » Thu Oct 17, 2013 9:05 pm

@zeronullity,

I already have a Linksys EA6500AC, which is wireless AC. I messed with DD-WRT on that and decided it has nothing for me. DD-WRT is not only unstable but inconsistent even with itself. Messing with the VLANs page, I experimented with plugging and unplugging cables that the label for the WAN was correct, but the "status light" under it was way over on the other side. When messing with VLAN settings, the check under the WAN matched to the WAN and the other ports were reversed in order.

I tried for the command line at that point, figured out that there's not enough room in there to do anything, and there's no real organization to what's going on in there anyway.

At least with a mainstream Linux kernel everything makes sense and there are tons of examples.

I started on this quest because my existing VLAN-capable router broke, and my linksys was not up to the task. After having looked at all the open source firmwares out there I'm convinced that that hardware is not adequate. I would like a few more ports anyway.

1clue · Post by **1clue** » Thu Oct 17, 2013 9:17 pm

@NeddySeagoon,

My intent at this point is to put the new box (whatever it is) on the outside, dedicate a port with its own VLAN over to the Linksys, so I can use that, stock, for home use. Everything else will go to my office equipment. That will have no wireless access and will not be accessible from "home" except for maybe a home-only DMZ.

I have another box on the horizon, I'll have 3 virtualization hosts at that point and one of them is a laptop, they'll need to have VLAN support. Maybe I could figure out some sort of failover VM on the other side, if the main box breaks I just move the cable...

I will DEFINITELY give your link a read when I get the chance. I thought about a virtualized router but didn't think it would be practical or really useful. Maybe I need to bone up on my networking and virtualization knowledge. But the idea of it being a VM means I'm not limited in any real way regarding hardware which is great.

I've been staring at raspberry pi for awhile now, haven't had time to mess with it. Aside of its namesake being one of my favorite dessert foods, I think they're really cool gadgets as well. I can't think how many times I've wanted a controller like that.

Things are getting way too complicated right now, I need to slow down and take baby steps. First to read your link and then maybe to set up a hardened Gentoo VM, then maybe add a 4-port NIC to it. Never done hardened anything before, but maybe it's time.

As usual, you've expanded my horizon a lot, changed a bunch of my parameters and I need to re-think this with more information.

Thanks.

zeronullity · Post by **zeronullity** » Thu Oct 17, 2013 9:33 pm

I'd give Tomato by shibby a try first maybe with a thumbdrive for extra space.. not sure if it has all the support
your looking for based on flash size/ram.. etc. As far as I know EA6500 is currently supported. Yeh, hard to beat custom hardware /w custom kernel in both security
& performance, providing you set it up correctly that is. Old scrap system with Asus AC wireless card and Intel gigabit cards should cost less then that retail AC router.
Retail routers are horrible for anything cpu intensive.. i.e. file transfers from usb, etc. can easily halt all performance on the network, what little there is already.

1clue · Post by **1clue** » Thu Oct 17, 2013 11:30 pm

I think I've given up on SOHO hardware for what I'm thinking about. What I have is one of the higher performance appliances out there, and it's great as far as home use but everything I try on it comes up lacking for work. I'm all about leaving that in place to handle the wireless and digital cockroaches.

I don't even want wireless access on the business side of things. Everything has ethernet, and will continue to have it. If you have wireless access to a network, it is not secure. The end. I'm freaking right now because my work equipment is wirelessly accessible.

Frankly I was going down the SOHO firmware route because I didn't want to pay for a real switch/router, and I didn't think a virtualized one would work.

NeddySeagon's idea really sounds perfect for me. I'll try to figure out a failover method to reduce the chance of a disaster, but I can't imagine it getting much better than this.

I'm still interested in small appliance hardware that can handle a full distro. Here are some specs about what I'm interested in:

4g minimum memory
16g flash
2 usb ports
one or more PCIe slots

As another point, I'm also interested in a screaming fast barebones, with an i7 and capable of 32g ram. Needs at least 3 monitor ports, DVI or HDMI or display port. No analog. Sata3 and lots of usb3. This is obviously not the router, I want a sick workstation that might be luggable.

Thanks.

zeronullity · Post by **zeronullity** » Fri Oct 18, 2013 12:39 am

I don't know about luggable but I like my x8dal-i.. although you can get a workstation with even more
then 24 cores now if needed.. it does support up to 96GB of ram. It's all about how much $$$$ you wan
t to dish out. I was only suggesting installing wireless card to replace the EA6500 wireless AC fuction..
however if not needed, don't install, and I agree.. wired is much more secure especially when combined
with encryption. Although no system/network is 100% secure.. It's all about time/methods of cracking it..
which can include physical access. I believe at some time in US history we used electromagnetic surveillance
to spy on Russian telecommunications that way they couldn't say we were taping their physical lines.. so even
phyiscal lines are not secure. =) Any ways you have plenty of x86 based systems to choose from, that should
work for your needs. Just do your research on specs/performance & you should be fine. - GL