I've been looking into this because it's fun. You can run gentoo on anything, but some things are easier than others. AWS is one of those things that is less easy. The cool thing is that you can get yourself set up on a t2.micro instance for free. Then when you're ready graduate to a dozens of cores and GB of memory for whatever your compute load is. I find this very cool.
This guy seems to have it dialled:
https://www.artembutusov.com/gentoo-on-aws/
[edit:]or if that's down here it is on wayback:
https://web.archive.org/web/20200807084 ... oo-on-aws/
And once you've done that you can spin your own AMI using his scripts:
https://www.reddit.com/r/Gentoo/comment ... available/
So this is very cool. In principle using one of his AMI's you would have a stage3 up and running in seconds. Except he isn't distributing his AMI's yet, but that's OK. It would be cool if there were an official gentoo AMI. But I understand that would take someone to do it and maintain it, and that isn't me. I'm starting this thread to have a place for discussion of gentoo AWS installation. Maybe there is already a discussion in the smaller circle. I will post some specific issues below as they arise.
Gentoo on AWS EC2
Message
Gentoo on AWS EC2
Last edited by jesnow on Sun Dec 06, 2020 2:14 pm, edited 1 time in total.
One thing that makes AWS less easy is your lack of access to the boot console. Whether you use Grub or LILO, doesn't matter, you have no ability to select a kernel at boot time. If your new kernel panics once you install it you are screwed. You can't just go back to an old kernel, you have to do a whole rigamarole with a rescue instance that *does* boot, for example here:
https://aws.amazon.com/premiumsupport/k ... ble-mount/
That's kind of harsh! Basically this defeats the purpose of *having* a bootloader. Which is to load different kernels at boot time. To achieve that functionality I think you would need to have a mini-distro with its own sshd and set of keys that was capable of chain booting other kernels. This is kind of how pv-grub works on xen, except that's supposed to run on bare metal. So how many recursive nested dolls of kernels do you need to have? Seems kind of wasteful.
My mother solved this problem in 1967 when she was working on VM/CMS on the IBM 360. She literally explained it to me over breakfast when I was in elementary school. The reinvention of the wheel going on here is remarkable.
Jon.
https://aws.amazon.com/premiumsupport/k ... ble-mount/
That's kind of harsh! Basically this defeats the purpose of *having* a bootloader. Which is to load different kernels at boot time. To achieve that functionality I think you would need to have a mini-distro with its own sshd and set of keys that was capable of chain booting other kernels. This is kind of how pv-grub works on xen, except that's supposed to run on bare metal. So how many recursive nested dolls of kernels do you need to have? Seems kind of wasteful.
My mother solved this problem in 1967 when she was working on VM/CMS on the IBM 360. She literally explained it to me over breakfast when I was in elementary school. The reinvention of the wheel going on here is remarkable.
Jon.
- AlexJGreen
- Tux's lil' helper
- Posts: 149
- Joined: Wed Sep 19, 2018 5:37 pm
1) They are very old, and from an untrusted source.coderanger wrote:I can see a bunch of AMIs marked as Gentoo platform in AWS marketplace. The other thing is that building source based distributions there is a waste of money and time
2) My money, my time. It's very cheap in fact compared to buying a machine. Free in fact until you start a big instance.
3) It's an idea to use a binhosted version of gentoo, like cloveros or calculate. But those don't have AMI's either as far as I can tell.
Again anything useful you can contribute beyond "that's dumb thing to do" would be greatly appreciated.
Jon.
I did not decide yet to automate building and publish every week new AMIs (https://github.com/sormy/gentoo-ami-builder). Based on reddit survey, people won't trust these AMIs unless they are from trusted authority, like Gentoo team. I'm not sure if Gentoo maintainers are interested.
Production-level Gentoo on AWS needs not only AMI for AWS but also a set of cloudformation scripts to demonstrate how to build customized images on schedule in pipelines and how to update other services using new AMIs produced in pipelines. With metrics, monitoring, regression testing etc.
By the way, 2020, people mostly run soft in docker containers and lambdas...
On a side note, interacting with boot loader is not highly needed feature. There is a way to see boot log and it is enough mostly. You must NEVER deploy an AMI that has not been tested before to production.
If you are interested you can contribute to gentoo-ami-builder project
Production-level Gentoo on AWS needs not only AMI for AWS but also a set of cloudformation scripts to demonstrate how to build customized images on schedule in pipelines and how to update other services using new AMIs produced in pipelines. With metrics, monitoring, regression testing etc.
By the way, 2020, people mostly run soft in docker containers and lambdas...
On a side note, interacting with boot loader is not highly needed feature. There is a way to see boot log and it is enough mostly. You must NEVER deploy an AMI that has not been tested before to production.
If you are interested you can contribute to gentoo-ami-builder project
I agree that deploying straight to production is a bad idea, but if you cannot interact with the bootloader, how can you ever test a kernel and recover if it goes badly? I see mention above of using a recovery system to modify the disk of the failed system, but this seems like a very heavyweight solution relative to rebooting, directing the bootloader to use an old kernel, and fixing the problem there.sormy wrote:On a side note, interacting with boot loader is not highly needed feature. There is a way to see boot log and it is enough mostly. You must NEVER deploy an AMI that has not been tested before to production.
Greetings to sormy who is indeed the jedi master. I really enjoyed your blog posts on this topic and am also enjoying messing around with virtual machines. My mother programmed virtual machines under VM/CMS on mainframes in the 1960's and 70's. Her professional organization still gives an award in her name. The subsystem she worked on (JES/2) has the same initials I do. I am told some of her code is still running today on big iron all over the world.
Hu: the AWS way is indeed that you can't interact with the boot loader, so there is no way to choose a custom kernel, or pass a kernel command line parameter, or anything interactive like that at boot time. If your kernel won't boot, then your instance is trashed and you usually have to nuke it ("terminate") and start over. That is a pain in the ass. with very small modifications you can start an instance that boots a second volume, then once everything is perfect, copy the contents of the second volume on top of the first, do # shutdown -r now and hope for the best. This two-stage process is how sormy creates gentoo instances. I'm pretty proud that I got this to work.
Edit: You can detach your trashed root volume, then attach it to your free tier t2.micro instance (this is very quick) then chroot into it, this is also fast if you write a script to do the mounting and unmounting. Change what you have to change (like fix your iptables setup), then unmount, detach and reattach as boot volume to your main instance. This works great, and is better than creating a new instance just for chrooting.
Soup to nuts it cost me about $20 in EC2 charges. It's much cheaper if you power down ("stop instance") when you're not using it. that's a hell of a lot cheaper than buying a machine for occasional use.
Hu: the AWS way is indeed that you can't interact with the boot loader, so there is no way to choose a custom kernel, or pass a kernel command line parameter, or anything interactive like that at boot time. If your kernel won't boot, then your instance is trashed and you usually have to nuke it ("terminate") and start over. That is a pain in the ass. with very small modifications you can start an instance that boots a second volume, then once everything is perfect, copy the contents of the second volume on top of the first, do # shutdown -r now and hope for the best. This two-stage process is how sormy creates gentoo instances. I'm pretty proud that I got this to work.
Edit: You can detach your trashed root volume, then attach it to your free tier t2.micro instance (this is very quick) then chroot into it, this is also fast if you write a script to do the mounting and unmounting. Change what you have to change (like fix your iptables setup), then unmount, detach and reattach as boot volume to your main instance. This works great, and is better than creating a new instance just for chrooting.
Soup to nuts it cost me about $20 in EC2 charges. It's much cheaper if you power down ("stop instance") when you're not using it. that's a hell of a lot cheaper than buying a machine for occasional use.
Last edited by jesnow on Sun Dec 13, 2020 6:26 pm, edited 1 time in total.
My question to both of you is how to get my AWS instance (which I like a lot) to securely appear as a regular node in my home network space. This is like the idea of a vpn in reverse. I don't want my machines to use it to see the world, I want it to pretend to be in my local net. When it comes down to it, if I could have my AWS instance just listen on its own local IP on one port I could get done what I want to do right now (which is have it run distcc). If I even knew what to call what I'm trying to do I would be way ahead.
Many thanks in advance.
Jon.
ps: The AWS model is extremely vulnerable to mistakes with iptables. Anything that might put your instance out of reach of ssh should be avoided. However if you make some rookie mistake configuring ssh or iptables and make your instance unreachable, you can stop that instance, detach the root volume, and attach it as a secondary volume to another instance. for example the t2.micro rescue instance you run 24/7 (it's free) anyway. Then you can fix whatever's wrong detach, reattach and boot. MUCH better than starting from scratch.
Many thanks in advance.
Jon.
ps: The AWS model is extremely vulnerable to mistakes with iptables. Anything that might put your instance out of reach of ssh should be avoided. However if you make some rookie mistake configuring ssh or iptables and make your instance unreachable, you can stop that instance, detach the root volume, and attach it as a secondary volume to another instance. for example the t2.micro rescue instance you run 24/7 (it's free) anyway. Then you can fix whatever's wrong detach, reattach and boot. MUCH better than starting from scratch.
This thread appears not to have set the world on fire, and sormy has now taken down his older blog on this topic. I have posted the wayback machine link to it at the top of the thread. I now have a very nice gentoo machine set up on aws, and make the following notes:
1) I have 2 AWS machines. One is a t2.micro (1 vcpu, 1GB) the other is a t2.xlarge (8vcpu, 32GB). Using sormy's method I have the XL (carapaz) running gentoo and the Micro (roglic) just runing Amazon Linux 2. This is free and it's Very Useful.
2) Costs are not bad. I keep Carapaz stopped most of the time except when I want to use it. Costs seem to be about $10/mo just to keep gentoo updated.
3) Getting access to roglic is easy, it runs full time and never changes its ip address, so I can set it up in .ssh/config as an alias and bobs your uncle. Carapaz is harder because it gets a new ip address every time I start it. So I used dyndns and ddclient to set up a permanent hostname in my domain (vesarius.net). This works great, now I can log into carapaz.vesarius.net whenever I start it, and I don't have to worry about what the actual ip is for the vm, or even care. I have to wait 30 extra seconds for the ddclient to announce itself to google domains.
4) Where I'm stuck is actually using it for anything. The idea with AWS instances is you run customer-facing services on them. I'm not at all interested in that. I have an actual steel and hard drives server (merckx.vesarius.net) that does all of that, and my needs are very modest. Instead, I'd like to run distcc (for example) on the EC2 instance. But distcc has horrible security. You basically cant run it the normal way over the internet (horrors! might as well run telnet as root with no password), and the ssh interface is super poorly documented.
5) What I'd really like is some easy way for Carapaz (or for that matter any instance) to simply appear in my local net as a node, just as if it were a box I just bought at best buy. This it turns out is quite a challenge. There are a lot of solutions that start with the word "just" that don't work. As in "just run wireguard". Or "just run the amazon site-to-site vpn tunnel". Or "Just run an ssh tunnel". These things are much easier said than done, *and* you're still left with lots of limitations. What I want is a "bridging" application that does the same thing for the EC2 that a virtualbox host does for its guest containers in bridging mode: making them appear to the local network as if they were local.
Anybody who has actually figured this part out please post.
Cheers,
Jon.
1) I have 2 AWS machines. One is a t2.micro (1 vcpu, 1GB) the other is a t2.xlarge (8vcpu, 32GB). Using sormy's method I have the XL (carapaz) running gentoo and the Micro (roglic) just runing Amazon Linux 2. This is free and it's Very Useful.
2) Costs are not bad. I keep Carapaz stopped most of the time except when I want to use it. Costs seem to be about $10/mo just to keep gentoo updated.
3) Getting access to roglic is easy, it runs full time and never changes its ip address, so I can set it up in .ssh/config as an alias and bobs your uncle. Carapaz is harder because it gets a new ip address every time I start it. So I used dyndns and ddclient to set up a permanent hostname in my domain (vesarius.net). This works great, now I can log into carapaz.vesarius.net whenever I start it, and I don't have to worry about what the actual ip is for the vm, or even care. I have to wait 30 extra seconds for the ddclient to announce itself to google domains.
4) Where I'm stuck is actually using it for anything. The idea with AWS instances is you run customer-facing services on them. I'm not at all interested in that. I have an actual steel and hard drives server (merckx.vesarius.net) that does all of that, and my needs are very modest. Instead, I'd like to run distcc (for example) on the EC2 instance. But distcc has horrible security. You basically cant run it the normal way over the internet (horrors! might as well run telnet as root with no password), and the ssh interface is super poorly documented.
5) What I'd really like is some easy way for Carapaz (or for that matter any instance) to simply appear in my local net as a node, just as if it were a box I just bought at best buy. This it turns out is quite a challenge. There are a lot of solutions that start with the word "just" that don't work. As in "just run wireguard". Or "just run the amazon site-to-site vpn tunnel". Or "Just run an ssh tunnel". These things are much easier said than done, *and* you're still left with lots of limitations. What I want is a "bridging" application that does the same thing for the EC2 that a virtualbox host does for its guest containers in bridging mode: making them appear to the local network as if they were local.
Anybody who has actually figured this part out please post.
Cheers,
Jon.