View previous topic :: View next topic |
Author |
Message |
VinzC Watchman
Joined: 17 Apr 2004 Posts: 5098 Location: Dark side of the mood
|
Posted: Sat Nov 04, 2017 11:29 am Post subject: Using /tmp to store GCC temp. files while kernel compiles? |
|
|
Hi all.
I'm configuring a Gentoo machine with a SSD and a mechanical disk, both are SATA drives. I've used the mechanical drive to store /var, portage files and, well, mainly as much as possible of files that [are supposed to] change a lot of times. However as /usr belongs to the SSD, I wonder if I can optimize even further by sending all temporary compile results to somewhere like /tmp or even a tmpfs when I'm compiling the kernel.
For kicks I've tried setting TMPDIR to /tmp in a file under /etc/env.d/ but it doesn't seem to work (I see no temporary file pop in /tmp). In case you wonder, yes I've even rebooted the machine to have a proper starting point and TMPDIR is indeed set to /tmp. Can it be done at all? and how?
Thanks in advance for any hint/suggestion. _________________ Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
Back to top |
|
|
John R. Graham Administrator
Joined: 08 Mar 2005 Posts: 10589 Location: Somewhere over Atlanta, Georgia
|
Posted: Sat Nov 04, 2017 12:47 pm Post subject: |
|
|
How about mounting /usr/src to some partition or directory on your rotating rust? Do you consider .o files to be temporary? If so, I believe it'd take modifications to the source to achieve what you're literally asking about.
- John _________________ I can confirm that I have received between 0 and 499 National Security Letters. |
|
Back to top |
|
|
Naib Watchman
Joined: 21 May 2004 Posts: 6051 Location: Removed by Neddy
|
Posted: Sat Nov 04, 2017 12:50 pm Post subject: |
|
|
bind mount
mount --bind /usr/src /tmp/kernel_buiild _________________
Quote: | Removed by Chiitoo |
|
|
Back to top |
|
|
krinn Watchman
Joined: 02 May 2003 Posts: 7470
|
Posted: Sat Nov 04, 2017 12:55 pm Post subject: |
|
|
yeah, trying to force TMPDIR=/tmp is ... just what /tmp is made for
those are temp files for you, not really temporary files for kernel, that re-use them on next compile if they haven't change.
and it might (or not) use /tmp itself for some files that are really just garbage, but it's something most users won't see because they have cflags set with -pipe which is to avoid that. |
|
Back to top |
|
|
VinzC Watchman
Joined: 17 Apr 2004 Posts: 5098 Location: Dark side of the mood
|
Posted: Sat Nov 04, 2017 1:24 pm Post subject: |
|
|
John R. Graham wrote: | Do you consider .o files to be temporary? |
krinn wrote: | yeah, trying to force TMPDIR=/tmp is ... just what /tmp is made for
those are temp files for you, not really temporary files for kernel, that re-use them on next compile if they haven't change.
and it might (or not) use /tmp itself for some files that are really just garbage, but it's something most users won't see because they have cflags set with -pipe which is to avoid that. |
No, they're not temporary file for *me*, you probably missed my point. Fact is GCC uses temporary files when compiling — I'm not talking about *.o files and the likes, I'm talking about... well, temporary files used by GCC before it spits out the compiled object files and binaries aso. I haven't checked ps when I compile a kernel, which I probably should have; I just wanted to make sure there's not much more I can do otherwise. If I understand -pipe (see make.conf after all, right?) is there to precisely avoid using temporary files and use pipes instead. So if compiling a kernel always uses -pipe then I'm fine .
I don't even know why I asked in the first time. But well, that's me. Thanks for the hint. _________________ Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
Back to top |
|
|
John R. Graham Administrator
Joined: 08 Mar 2005 Posts: 10589 Location: Somewhere over Atlanta, Georgia
|
Posted: Sat Nov 04, 2017 2:07 pm Post subject: |
|
|
FYI, the kernel makefile doesn't honor (nor know about) make.conf, which is purely a Gentooism. It does honor CFLAGS in the environment, though, soworks. And, no, -pipe is not a default option for the kernel makefile.
- John _________________ I can confirm that I have received between 0 and 499 National Security Letters. |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54232 Location: 56N 3W
|
Posted: Sat Nov 04, 2017 2:40 pm Post subject: |
|
|
<nit>
CFLAGS=-pipe is an advisory not an instruction to gcc.
It means to use pipes between compile phases if memory permits.
</nit>
gcc will silently write intermediate files to hdd if there is not enough RAM to honour the -pipe advisory. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
Ant P. Watchman
Joined: 18 Apr 2009 Posts: 6920
|
Posted: Sat Nov 04, 2017 5:12 pm Post subject: |
|
|
I wouldn't bother with all this, just tweak laptop_mode and related sysctls and the kernel will avoid writing to disk on its own. |
|
Back to top |
|
|
VinzC Watchman
Joined: 17 Apr 2004 Posts: 5098 Location: Dark side of the mood
|
Posted: Mon Nov 13, 2017 9:56 am Post subject: |
|
|
Well, thanks for all the hints, guys.
Ant P. wrote: | I wouldn't bother with all this, just tweak laptop_mode and related sysctls and the kernel will avoid writing to disk on its own. |
Sounds interesting. Could you detail a little? _________________ Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
Back to top |
|
|
grumblebear Apprentice
Joined: 26 Feb 2008 Posts: 202
|
Posted: Mon Nov 13, 2017 11:54 am Post subject: |
|
|
What sort of optimizing is that? A SSD is not as fragile as you might think. Even with the portage tree and daily kernel compiles it will normally last for many, many years.
Besides speaking from personal experience, you can also explore the great wisdom Google has to offer on that topic. |
|
Back to top |
|
|
mv Watchman
Joined: 20 Apr 2005 Posts: 6747
|
Posted: Wed Nov 15, 2017 6:21 am Post subject: |
|
|
John R. Graham wrote: | It does honor CFLAGS in the environment, though |
Are you sure about this? Last time I tried, it did not. Upstream is rather picky about modifying compiler flags and still has not accepted patches which allow to add e.g. -march=native. |
|
Back to top |
|
|
mv Watchman
Joined: 20 Apr 2005 Posts: 6747
|
Posted: Wed Nov 15, 2017 6:28 am Post subject: |
|
|
VinzC wrote: | Fact is GCC uses temporary files when compiling |
AFAIK, gcc always places them in the directory where it creates the output files. To my knowledge there is no option to override that behaviour (gcc would need a mktemp-type algorithm/library to create safe filenames otherwise, but apparently it was decided not to include this complication into a compiler). I concluded this information from the documentation of the -save-temps option on the gcc manpage. |
|
Back to top |
|
|
VinzC Watchman
Joined: 17 Apr 2004 Posts: 5098 Location: Dark side of the mood
|
Posted: Wed Nov 15, 2017 9:13 am Post subject: |
|
|
grumblebear wrote: | What sort of optimizing is that? A SSD is not as fragile as you might think. Even with the portage tree and daily kernel compiles it will normally last for many, many years. |
The kind of optimization that brings me even more many years indeed.
For the record, this is only one step as I also plan to use my SSD in read-only mode, just as I've already done with a Raspberry Pi to protect against power outages. It has worked a treat for years even in an industrial environment without damaging the SD card. But that very part is off-topic and has nothing to do with my initial question, I'm just trying to feed your curiosity . _________________ Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
Back to top |
|
|
grumblebear Apprentice
Joined: 26 Feb 2008 Posts: 202
|
Posted: Wed Nov 15, 2017 1:18 pm Post subject: |
|
|
It just sounded to me as if you were concerned about the lifetime of your SSD. So excuse me for being a little off-topic, but the myths about writing to SSDs are going around a little too long for my taste.
I just want to point out to anyone reading this, that it is just not worth too much effort to avoid writing to SSDs. Even in your case, whatever it might be, I highly doubt you will even get a single year additional lifetime. The propability is much higher that some part of the electronics gets damaged or even the SATA standard gets replaced before the flash cells wear out.
PS
To get back on-topic: You can unpack and compile the kernel sources anywhere you like. Just don't forget to make modules_install and put the kernel image where your bootloader expects to find it. And of course you should put /lib/modules on your mechanical drive or compile a monolithic kernel. |
|
Back to top |
|
|
VinzC Watchman
Joined: 17 Apr 2004 Posts: 5098 Location: Dark side of the mood
|
Posted: Wed Dec 20, 2017 12:33 pm Post subject: |
|
|
grumblebear wrote: | It just sounded to me as if you were concerned about the lifetime of your SSD. |
Thank you for your insights but actually no, I'm not concerned about the lifetime of SSD's. At least not more than usual. Instead, in the very case of an SSD, I've been considering optimizing the Hell out of it so a) write to it only when needed and 2) sustain power outages just like the Raspberry Pi's I once configured. I want to most robust SSD I could ever configure. Just for the sake of it and see how far I can go, if you ask .
For the record I did see SSD's fail BTW. And not just once so I don't especially trust them (or, say, their lifetime expectancy) more than others. I may be wrong but, well, that's my experience.
grumblebear wrote: | To get back on-topic: You can unpack and compile the kernel sources anywhere you like. Just don't forget to make modules_install and put the kernel image where your bootloader expects to find it. And of course you should put /lib/modules on your mechanical drive or compile a monolithic kernel. |
Interesting. However from what I've understood from this thread is that if I can get the kernel make process to use -pipe and as long as there's enough memory, there'll be no temp. (intermediate) files, which is all I'm interested in — I can cope with the rest. _________________ Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
Back to top |
|
|
|