| View previous topic :: View next topic |
| Author |
Message |
KShots
Guru
Joined: 09 Oct 2003
Posts: 591
Location: Florida
|
 Posted: Sat Aug 27, 2016 12:37 am Post subject: |
 |
|
Grr... no dice. Still the same error 
Now that it's finished booting, I have the following in my kernel config for IOMMU: | Code: | gorgon ~ # gunzip -c /proc/config.gz | grep -i iommu
# CONFIG_GART_IOMMU is not set
# CONFIG_CALGARY_IOMMU is not set
CONFIG_IOMMU_HELPER=y
CONFIG_IOMMU_API=y
CONFIG_IOMMU_SUPPORT=y
# Generic IOMMU Pagetable Support
CONFIG_IOMMU_IOVA=y
# CONFIG_AMD_IOMMU is not set
CONFIG_INTEL_IOMMU=y
CONFIG_INTEL_IOMMU_SVM=y
CONFIG_INTEL_IOMMU_DEFAULT_ON=y
CONFIG_INTEL_IOMMU_FLOPPY_WA=y
# CONFIG_IOMMU_STRESS is not set |
_________________
Life without passion is death in disguise
Last edited by KShots on Sat Aug 27, 2016 12:39 am; edited 1 time in total |
|
| Back to top |
|
 |
Atom2
Apprentice
Joined: 01 Aug 2011
Posts: 185
|
| Posted: Sat Aug 27, 2016 12:39 am Post subject: |
|
|
| Could you post your /etc/xen/xl.conf file? |
|
| Back to top |
|
|
KShots
Guru
Joined: 09 Oct 2003
Posts: 591
Location: Florida
|
| Posted: Sat Aug 27, 2016 12:41 am Post subject: |
|
|
| Atom2 wrote: | | Could you post your /etc/xen/xl.conf file? |
Sure: | Code: | gorgon ~ # cat /etc/xen/xl.conf
## Global XL config file ##
# Control whether dom0 is ballooned down when xen doesn't have enough
# free memory to create a domain. "auto" means only balloon if dom0
# starts with all the host's memory.
autoballoon="auto"
# full path of the lockfile used by xl during domain creation
lockfile="/var/lock/xl"
# default output format used by "xl list -l"
#output_format="json"
# first block device to be used for temporary VM disk mounts
#blkdev_start="xvda"
# default option to run hotplug scripts from xl
# if disabled the old behaviour will be used, and hotplug scripts will be
# launched by udev.
#run_hotplug_scripts=1
# default backend domain to connect guest vifs to. This can be any
# valid domain identifier.
#vif.default.backend="0"
# default gateway device to use with vif-route hotplug script
#vif.default.gatewaydev="eth0"
# default vif script to use if none is specified in the guest config
vif.default.script="vif-bridge"
# default bridge device to use with vif-bridge hotplug scripts
#vif.default.bridge="xenbr0"
# Reserve a claim of memory when launching a guest. This guarantees immediate
# feedback whether the guest can be launched due to memory exhaustion
# (which can take a long time to find out if launching huge guests).
# see xl.conf(5) for details.
#claim_mode=1 |
_________________
Life without passion is death in disguise |
|
| Back to top |
|
|
KShots
Guru
Joined: 09 Oct 2003
Posts: 591
Location: Florida
|
| Posted: Sat Aug 27, 2016 12:44 am Post subject: |
|
|
From the xen source code: | Code: | + /* We cannot have PoD and PCI device assignment at the same time
+ * for HVM guest. It was reported that VT-d engine cannot
+ * work with PoD enabled because it needs to populated entire page
+ * table for guest. Also a quick grep through AMD IOMMU related
+ * code suggests it has not coped with PoD as well. Just to stay
+ * on the safe side, we disable PCI device assignment with PoD all
+ * together, regardless of the underlying IOMMU in use.
+ */ |
... Which leaves me scratching my head and saying "What's PoD"? A BIOS option?
_________________
Life without passion is death in disguise |
|
| Back to top |
|
|
Atom2
Apprentice
Joined: 01 Aug 2011
Posts: 185
|
| Posted: Sat Aug 27, 2016 12:47 am Post subject: |
|
|
It's a long shot, but your message with PoD might be related to ballooning the dom0. PoD could stand for Paging on Demand.
I suggest you try to disable ballooning by settingin /etc/xen/xl.conf. If you do that, you also should assign a fixed amount of memory to dom0 through the grub (or uefi) command line like so:changing the parameter to what you think is reasonable (obviously depending on your total RAM for the system). It is anyways recommended practice to limit dom0's memory. |
|
| Back to top |
|
|
KShots
Guru
Joined: 09 Oct 2003
Posts: 591
Location: Florida
|
| Posted: Sat Aug 27, 2016 12:53 am Post subject: |
|
|
Got it... it's associated with memory ballooning. By declaring my memory at a static value rather than allowing it to allocate on demand, I got rid of that error. Oddly enough, I had to reduce the memory from 1024 to 256 in order for it to be able to allocate... the machine isn't very busy and has 16GB of RAM, so I'm not sure why it couldn't allocate 1G, even if it has to be contiguous.
At any rate, the VM fired up and it now sees a physical device (em0) to configure. Let's see what happens when I configure it (fingers crossed)
_________________
Life without passion is death in disguise |
|
| Back to top |
|
|
KShots
Guru
Joined: 09 Oct 2003
Posts: 591
Location: Florida
|
| Posted: Sat Aug 27, 2016 1:09 am Post subject: |
|
|
Well... something isn't right. I can reach the internet via the firewall and the new physical device... but the bridged virtual device does not pass any traffic anymore (even ICMP). I can only access the firewall via the serial console, no LAN traffic gets through. On a hunch, I tried resetting to factory default settings and reconfigured, but it still doesn't work. Not sure what's going on here
_________________
Life without passion is death in disguise |
|
| Back to top |
|
|
KShots
Guru
Joined: 09 Oct 2003
Posts: 591
Location: Florida
|
| Posted: Sat Aug 27, 2016 1:12 am Post subject: |
|
|
| Atom2 wrote: | It's a long shot, but your message with PoD might be related to ballooning the dom0. PoD could stand for Paging on Demand.
I suggest you try to disable ballooning by settingin /etc/xen/xl.conf. If you do that, you also should assign a fixed amount of memory to dom0 through the grub (or uefi) command line like so:changing the parameter to what you think is reasonable (obviously depending on your total RAM for the system). It is anyways recommended practice to limit dom0's memory. |
Let me give this a shot... it should give me more breathing room than 256M and will likely be more reliable as well if it requires it to be contiguous memory.
_________________
Life without passion is death in disguise |
|
| Back to top |
|
|
Atom2
Apprentice
Joined: 01 Aug 2011
Posts: 185
|
| Posted: Sat Aug 27, 2016 1:16 am Post subject: |
|
|
| KShots wrote: | | I can reach the internet via the firewall and the new physical device... |
That's an improvment | KShots wrote: | | but the bridged virtual device does not pass any traffic anymore (even ICMP). I can only access the firewall via the serial console, no LAN traffic gets through. On a hunch, I tried resetting to factory default settings and reconfigured, but it still doesn't work. Not sure what's going on here |
That's strange because you haven't changed anything related to the bridge or the vif interface, right?
Is your network configuration right? LAN side and vif to pfSense is one subnet (with the vif only configured within pfSense), the physical device on pfSense is on another subnet (i.e. your external IP address for the routing to happen). Can you ping any other systems on the LAN from dom0? |
|
| Back to top |
|
|
KShots
Guru
Joined: 09 Oct 2003
Posts: 591
Location: Florida
|
| Posted: Sat Aug 27, 2016 1:26 am Post subject: |
|
|
| Atom2 wrote: | | KShots wrote: | | I can reach the internet via the firewall and the new physical device... |
That's an improvment | KShots wrote: | | but the bridged virtual device does not pass any traffic anymore (even ICMP). I can only access the firewall via the serial console, no LAN traffic gets through. On a hunch, I tried resetting to factory default settings and reconfigured, but it still doesn't work. Not sure what's going on here |
That's strange because you haven't changed anything related to the bridge or the vif interface, right?
Is your network configuration right? LAN side and vif to pfSense is one subnet (with the vif only configured within pfSense), the physical device on pfSense is on another subnet (i.e. your external IP address for the routing to happen). Can you ping any other systems on the LAN from dom0? |
Yeah, my physical device that was passed to the domU has a public IP address on it. The virtual bridged device has a class A (10.x.y.z) address, of which both the dom0 and the domU have addresses on the same subnet on this bridge. The dom0 can communicate across its LAN just fine, but it can't communicate with domU.
Only thing I can think of is the changes we made to the script and pfsense.cfg files... but I don't see why those would cause this, it's the only thing I see that's related.
EDIT: Bingo... something's not right about those changes. Reverted back to the original config without checksum offloading fixes and LAN hosts can now ping the domU... but dom0 cannot ping domU yet
On a side note, I limited the host (dom0) memory to 4G and pushed the domU up to 512M so it's at least running at the minimum requirement for pfsense (it was complaining about having less than 512M).
_________________
Life without passion is death in disguise |
|
| Back to top |
|
|
Atom2
Apprentice
Joined: 01 Aug 2011
Posts: 185
|
| Posted: Sat Aug 27, 2016 1:49 am Post subject: |
|
|
That doesn't really make sense to me. All LAN devices go through the physical NIC that makes up the bridge on dom0. So how could the LAN devices be able to ping the pfSense domU when the dom0 (through which all traffic flows by utilizing the bridge) appears not to be able to ping the pfSense domU?
In any case, it's already pretty late in my time zone, I'll need some sleep and maybe this enlightens me. I'll let you alone for now and have another look tomorrow. I think we have nevertheless made some good progress so far.
Atom2 |
|
| Back to top |
|
|
KShots
Guru
Joined: 09 Oct 2003
Posts: 591
Location: Florida
|
| Posted: Sat Aug 27, 2016 2:48 am Post subject: |
|
|
| Atom2 wrote: | That doesn't really make sense to me. All LAN devices go through the physical NIC that makes up the bridge on dom0. So how could the LAN devices be able to ping the pfSense domU when the dom0 (through which all traffic flows by utilizing the bridge) appears not to be able to ping the pfSense domU?
In any case, it's already pretty late in my time zone, I'll need some sleep and maybe this enlightens me. I'll let you alone for now and have another look tomorrow. I think we have nevertheless made some good progress so far.
Atom2 |
Yeah, you've been a lot of help, I'm miles ahead of where I was before. Thanks a million.
On the upside, reverting to the older script (after moving the ethtool command to the bottom) did indeed get things working. The dom0 can communicate to the domU, and it can also use the domU as a gateway and send/receive TCP traffic to the internet at large now. I'd say this is pretty much mission accomplished.
I'd much rather use a script similar to what you're using, though, as what I've done is not going to pick up on any updates/changes as xen updates over time. It's a standalone script that doesn't reference the original. I like the way you have yours set up so when xen updates via portage, you simply call the updated script and then run ethtool. The only thing I can figure is that I'm not quite understanding the xenstored infrastructure, and I may be somehow misusing the vifname parameter in the cfg file. I note that despite setting vifname=pfsense.0, I still have a vif[domIndex].0 interface, so it's definitely not doing what I thought it would do. On the other hand, the vif-bridge script insists that when executed, the vif variable will be defined to point to the appropriate interface... maybe I don't need the xenstore-read command if I can simply call ifname=$vif
EDIT: Success, using $vif rather than detecting via xenstore-read got it working properly. It appears I'm golden at this point. Also, you're right, I do not need to disable checksums on the domU side, only on the dom0 side.
_________________
Life without passion is death in disguise |
|
| Back to top |
|
|
|
Other Things Gentoo
