| View previous topic :: View next topic |
| Author |
Message |
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3140
|
 Posted: Mon Jan 22, 2024 1:12 am Post subject: git gc fails: pack-objects oom killed |
 |
|
Long story short, full clone of portage takes almost 4GB and garbage collecting it requires around 4GB RAM, which is quite inconvenient on a small machine.
Adding SWAP is not an option, gotta make it work with 2GB of RAM or disable gc completely and do it manually on "borrowed" hardware. Quite frankly 1.5G RAM would be even better.
The operation getting interrupted is git pack-objects. I looked for some settings, allegedly putting limits on pack.windowMemory and pack.threads should do the trick, but even absurdly low values didn't help.
Setting packSizeLimit may or may not come in handy; I don't think it does what someone who suggested it thinks it does. I'm testing it right now, but kinda expect it to be more trouble than its worth.
Looking at it I have an impression that git tries to compress all objects in RAM first, and then write a new pack to disk once it's done with crunching numbers instead of just keeping hot data in RAM and dumping already processed data to disk as it becomes available.
Anyone knows any git options I could use to halve the resources required for garbage collection? I don't care if it doubles the time needed, this CPU doesn't usually work very hard. |
|
| Back to top |
|
 |
pingtoo
l33t
Joined: 10 Sep 2021
Posts: 926
Location: Richmond Hill, Canada
|
| Posted: Mon Jan 22, 2024 10:59 pm Post subject: |
|
|
May be pack.packSizeLimit and pack.windowMemory. I have not need to use them so I don't know for sure, but they looks like what you are asking.
I wonder why not able to use swap, but on the other hand have you consider use zswap? |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3140
|
| Posted: Tue Jan 23, 2024 12:11 am Post subject: |
|
|
With limits on pack.packSizeLimit and pack.windowMemory it failed again; looks like a dead end.
I've been thinking about zswap actually, but decided not to. Git packs are already compressed, which means we're in diminishing returns. A quick test with gzip suggests I could save 5% of RAM usage. I need 50% for this exercise to really make a difference.
There is no one reason why I can't use swap. It's a bunch of BS which should either not be the case or should not matter, but combined make a mess I'll probably have to just abandon, eventually. In the meantime I might as well take the opportunity to try ricing git though. |
|
| Back to top |
|
|
pingtoo
l33t
Joined: 10 Sep 2021
Posts: 926
Location: Richmond Hill, Canada
|
| Posted: Tue Jan 23, 2024 4:00 pm Post subject: |
|
|
| szatox wrote: | With limits on pack.packSizeLimit and pack.windowMemory it failed again; looks like a dead end.
I've been thinking about zswap actually, but decided not to. Git packs are already compressed, which means we're in diminishing returns. A quick test with gzip suggests I could save 5% of RAM usage. I need 50% for this exercise to really make a difference.
There is no one reason why I can't use swap. It's a bunch of BS which should either not be the case or should not matter, but combined make a mess I'll probably have to just abandon, eventually. In the meantime I might as well take the opportunity to try ricing git though. |
Since I don't have such situation I don't have much to offer 
But have you consider use the two + zswap?
Also since swap (zswap) is about memory usage, it is not used as store (Could that you are thinking about ZRAM?) so IMHO whether git packs are compressed or not should not matter. |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3140
|
| Posted: Tue Jan 23, 2024 6:33 pm Post subject: |
|
|
| Quote: | | Also since swap (zswap) is about memory usage, it is not used as store (Could that you are thinking about ZRAM?) so IMHO whether git packs are compressed or not should not matter. |
ZRAM and ZSWAP, in context of this topic, they are the same thing.
I mentioned that git packs are compressed, because it seems that I need about as much memory as the compressed pack takes, which in turn suggests this is what git is actually working with during garbage collection. Compressing RAM full of already compressed data won't make a difference.
For comparison, git log -p decompresses current HEAD into a bit over 18GB of diffs.
Anyway... I decided to put those 18GB of diffs through gzip. Not because it's practical, but simply because diffs are quite verbose.It would be funny if the result turned out to be smaller than pack files..
Alright, I'm off to reading the manual. Maybe I'll find some better options... |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3140
|
| Posted: Tue Jan 23, 2024 10:49 pm Post subject: |
|
|
One setting which did actually help with the immediate issue is:
gc.bigPackThreshold
I've set it to 2GB for now, which made git leave the old 3.1GB pack as is and merge all the other stuff into a second, 0.5GB pack. Should be good for a few years to come now. Not *exactly* what I wanted, but I guess I'll call it good enough. |
|
| Back to top |
|
|
|
Other Things Gentoo
