[workaround] investigating transfer slowdown on PXE boot

s|mon · Last edited by s|mon on Mon May 03, 2021 7:00 am; edited 1 time in total

I've set up a PXE server with the intention to quickly allow booting to a live linux for investigations or memtest etc.
I've used my gentoo home server which already had dnsmasq running with following guides
https://wiki.gentoo.org/wiki/PXE and https://wiki.ubuntu.com/LiveCDNetboot

I got it working so far to offer a 32bit older ubuntu 16.04 (i had a very old laptop for entry tests which has no 64bit cpu) and 64 bit versions 18.4 and 20.4 of the ubunutu livecds and memtest86+.

With memtest and the old 16.04 it was working fine.
Then i tried on my main machine to boot the 20.04 version and it got stuck on loading initrd via tftp. Some more time spent looking at logs without much hints i used a third machine to test.
No change it still seemed to be stuck with loading initrd. Then i observed some interesting hint watching the network throughput on the server being very slow.
I switched to the 18.04 image (half the size) and waited a bit longer. Success!

Next step was to try 20.04 again and go for dinner. When i returned also this had booted and after further tests i found it took about 8min to load the initrd (87MB).
Further investingation showed that the initial transfer of the kernel (12MB) was substantially faster with 17 seconds.
So the transfer got slower over time.

Then i used atftp to test a retreive from gentoo on my main system and it transfered also the initrd in 40sec.
So i assume it has to be from the way the retreive is done from pxe.

i then did tcpdumps of both which gave me the following hints:
from PXE/uefi

APolozov · Posted: Sat Apr 24, 2021 6:42 am Post subject:

It's true - tftp is very slow, if you use syslinux to boot something then you can download all files over http - this is faster (I on my experimental PXE server use nginx with symlink to the directory tftptoot because some things I download two-way(http and tftp) for compatibility and debuging).
_________________
Excuse my bad English, I only study it.

s|mon · Posted: Sat Apr 24, 2021 11:35 am Post subject:

That would be worth a try. Maybe i find some time to try that tomorrow. Although it is for sure not fast - tftp itself would be sufficient for the start it it would not also have this slow down effect when doing the netboot. If i just start a plain tftp transfer from a running system it is constant and would be usable.

s|mon · Posted: Mon May 03, 2021 7:00 am Post subject:

Just to document for anyone who stumbles over it i tried the path to use http instead (lighttpd) and adapted the default config of pxe to use that for kernel and initrd