Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Using /tmp to store GCC temp. files while kernel compiles?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
VinzC
Advocate
Advocate


Joined: 17 Apr 2004
Posts: 4863
Location: Dark side of the mood

PostPosted: Sat Nov 04, 2017 11:29 am    Post subject: Using /tmp to store GCC temp. files while kernel compiles? Reply with quote

Hi all.

I'm configuring a Gentoo machine with a SSD and a mechanical disk, both are SATA drives. I've used the mechanical drive to store /var, portage files and, well, mainly as much as possible of files that [are supposed to] change a lot of times. However as /usr belongs to the SSD, I wonder if I can optimize even further by sending all temporary compile results to somewhere like /tmp or even a tmpfs when I'm compiling the kernel.

For kicks I've tried setting TMPDIR to /tmp in a file under /etc/env.d/ but it doesn't seem to work (I see no temporary file pop in /tmp). In case you wonder, yes I've even rebooted the machine to have a proper starting point and TMPDIR is indeed set to /tmp. Can it be done at all? and how?

Thanks in advance for any hint/suggestion.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
Back to top
View user's profile Send private message
John R. Graham
Administrator
Administrator


Joined: 08 Mar 2005
Posts: 9577
Location: Somewhere over Atlanta, Georgia

PostPosted: Sat Nov 04, 2017 12:47 pm    Post subject: Reply with quote

How about mounting /usr/src to some partition or directory on your rotating rust? Do you consider .o files to be temporary? If so, I believe it'd take modifications to the source to achieve what you're literally asking about.

- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters.
Back to top
View user's profile Send private message
Naib
Watchman
Watchman


Joined: 21 May 2004
Posts: 5079
Location: Removed by Neddy

PostPosted: Sat Nov 04, 2017 12:50 pm    Post subject: Reply with quote

bind mount
mount --bind /usr/src /tmp/kernel_buiild
_________________
The best argument against democracy is a five-minute conversation with the average voter
Great Britain is a republic, with a hereditary president, while the United States is a monarchy with an elective king
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 5969

PostPosted: Sat Nov 04, 2017 12:55 pm    Post subject: Reply with quote

yeah, trying to force TMPDIR=/tmp is ... just what /tmp is made for :D

those are temp files for you, not really temporary files for kernel, that re-use them on next compile if they haven't change.
and it might (or not) use /tmp itself for some files that are really just garbage, but it's something most users won't see because they have cflags set with -pipe which is to avoid that.
Back to top
View user's profile Send private message
VinzC
Advocate
Advocate


Joined: 17 Apr 2004
Posts: 4863
Location: Dark side of the mood

PostPosted: Sat Nov 04, 2017 1:24 pm    Post subject: Reply with quote

John R. Graham wrote:
Do you consider .o files to be temporary?

krinn wrote:
yeah, trying to force TMPDIR=/tmp is ... just what /tmp is made for :D

those are temp files for you, not really temporary files for kernel, that re-use them on next compile if they haven't change.
and it might (or not) use /tmp itself for some files that are really just garbage, but it's something most users won't see because they have cflags set with -pipe which is to avoid that.

No, they're not temporary file for *me*, you probably missed my point. Fact is GCC uses temporary files when compiling — I'm not talking about *.o files and the likes, I'm talking about... well, temporary files used by GCC before it spits out the compiled object files and binaries aso. I haven't checked ps when I compile a kernel, which I probably should have; I just wanted to make sure there's not much more I can do otherwise. If I understand -pipe (see make.conf after all, right?) is there to precisely avoid using temporary files and use pipes instead. So if compiling a kernel always uses -pipe then I'm fine :) .

I don't even know why I asked in the first time. But well, that's me. Thanks for the hint.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
Back to top
View user's profile Send private message
John R. Graham
Administrator
Administrator


Joined: 08 Mar 2005
Posts: 9577
Location: Somewhere over Atlanta, Georgia

PostPosted: Sat Nov 04, 2017 2:07 pm    Post subject: Reply with quote

FYI, the kernel makefile doesn't honor (nor know about) make.conf, which is purely a Gentooism. It does honor CFLAGS in the environment, though, so
Code:
CFLAGS=-pipe make
works. And, no, -pipe is not a default option for the kernel makefile.

- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 39299
Location: 56N 3W

PostPosted: Sat Nov 04, 2017 2:40 pm    Post subject: Reply with quote

<nit>
CFLAGS=-pipe is an advisory not an instruction to gcc.
It means to use pipes between compile phases if memory permits.
</nit>

gcc will silently write intermediate files to hdd if there is not enough RAM to honour the -pipe advisory.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Ant P.
Advocate
Advocate


Joined: 18 Apr 2009
Posts: 4528

PostPosted: Sat Nov 04, 2017 5:12 pm    Post subject: Reply with quote

I wouldn't bother with all this, just tweak laptop_mode and related sysctls and the kernel will avoid writing to disk on its own.
_________________
*.ebuild // /etc/service/*
Back to top
View user's profile Send private message
VinzC
Advocate
Advocate


Joined: 17 Apr 2004
Posts: 4863
Location: Dark side of the mood

PostPosted: Mon Nov 13, 2017 9:56 am    Post subject: Reply with quote

Well, thanks for all the hints, guys.

Ant P. wrote:
I wouldn't bother with all this, just tweak laptop_mode and related sysctls and the kernel will avoid writing to disk on its own.

Sounds interesting. Could you detail a little?
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
Back to top
View user's profile Send private message
grumblebear
Tux's lil' helper
Tux's lil' helper


Joined: 26 Feb 2008
Posts: 81

PostPosted: Mon Nov 13, 2017 11:54 am    Post subject: Reply with quote

What sort of optimizing is that? A SSD is not as fragile as you might think. Even with the portage tree and daily kernel compiles it will normally last for many, many years.

Besides speaking from personal experience, you can also explore the great wisdom Google has to offer on that topic.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 5754

PostPosted: Wed Nov 15, 2017 6:21 am    Post subject: Reply with quote

John R. Graham wrote:
It does honor CFLAGS in the environment, though

Are you sure about this? Last time I tried, it did not. Upstream is rather picky about modifying compiler flags and still has not accepted patches which allow to add e.g. -march=native.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 5754

PostPosted: Wed Nov 15, 2017 6:28 am    Post subject: Reply with quote

VinzC wrote:
Fact is GCC uses temporary files when compiling

AFAIK, gcc always places them in the directory where it creates the output files. To my knowledge there is no option to override that behaviour (gcc would need a mktemp-type algorithm/library to create safe filenames otherwise, but apparently it was decided not to include this complication into a compiler). I concluded this information from the documentation of the -save-temps option on the gcc manpage.
Back to top
View user's profile Send private message
VinzC
Advocate
Advocate


Joined: 17 Apr 2004
Posts: 4863
Location: Dark side of the mood

PostPosted: Wed Nov 15, 2017 9:13 am    Post subject: Reply with quote

grumblebear wrote:
What sort of optimizing is that? A SSD is not as fragile as you might think. Even with the portage tree and daily kernel compiles it will normally last for many, many years.

The kind of optimization that brings me even more many years indeed.

For the record, this is only one step as I also plan to use my SSD in read-only mode, just as I've already done with a Raspberry Pi to protect against power outages. It has worked a treat for years even in an industrial environment without damaging the SD card. But that very part is off-topic and has nothing to do with my initial question, I'm just trying to feed your curiosity ;-).
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
Back to top
View user's profile Send private message
grumblebear
Tux's lil' helper
Tux's lil' helper


Joined: 26 Feb 2008
Posts: 81

PostPosted: Wed Nov 15, 2017 1:18 pm    Post subject: Reply with quote

It just sounded to me as if you were concerned about the lifetime of your SSD. So excuse me for being a little off-topic, but the myths about writing to SSDs are going around a little too long for my taste.

I just want to point out to anyone reading this, that it is just not worth too much effort to avoid writing to SSDs. Even in your case, whatever it might be, I highly doubt you will even get a single year additional lifetime. The propability is much higher that some part of the electronics gets damaged or even the SATA standard gets replaced before the flash cells wear out.

PS
To get back on-topic: You can unpack and compile the kernel sources anywhere you like. Just don't forget to make modules_install and put the kernel image where your bootloader expects to find it. And of course you should put /lib/modules on your mechanical drive or compile a monolithic kernel.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum