Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Error and missing path after building ccache
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
Leomania
n00b
n00b


Joined: 16 Dec 2017
Posts: 17

PostPosted: Wed Aug 08, 2018 2:35 pm    Post subject: Error and missing path after building ccache Reply with quote

We use ccache for some specific use cases in our environment, so I built it in the most recent Prefix env. Once the emerge was done, there was an error message:

Code:
>>> Installing (1 of 1) dev-util/ccache-3.3.4-r1::gentoo
Updating masquerade for ccache ...
mkdir: cannot create directory '/usr/lib/ccache': Permission denied
!!! Error: (no message)
Call stack:
    * do_update (compiler-shadow.eselect:115)
    * check_do (core.bash:24)
    * do_action (core.bash:105)
    * main (eselect:181)
exiting
 * To use ccache with **non-Portage** C compiling, add
 * /auto/ins-tools/linux_cel69/usr/lib/ccache/bin to the beginning of
 * your path, before /auto/ins-tools/linux_cel69/usr/bin.

I suspect it was trying to do a mkdir on the actual /usr/lib/ccache rather than ${EPREFIX}/usr/lib/ccache.

This was with ccache version 3.3.4-r1.

Thanks,

- Leo
Back to top
View user's profile Send private message
oreo
n00b
n00b


Joined: 14 Aug 2018
Posts: 31

PostPosted: Wed Aug 15, 2018 7:55 am    Post subject: Reply with quote

Sorry to here you are having trouble with portage. Also, sorry for the delay in response. I had noticed this question the other day and did some research but never got around to responding. My bad. :-( Anyways, assuming you haven't entirely given up:

When investigating a portage error in the package building process, it is a good idea to examine the ebuild file. Ebuilds contain the instructions for installation and removal of any given package. They are located under /usr/portage/{package category}/{package name}/{complete package name}.ebuild. In this case, /mnt/gentoo/usr/portage/dev-util/ccache/ccache-3.3.4-r1.ebuild. The file contains a lot of stuff but only the following interests us here:
Code:
pkg_postinst() {
   if [[ ${ROOT} == / ]]; then
      eselect compiler-shadow update ccache
   fi

   # nuke broken symlinks from previous versions that shouldn't exist
   rm -rf "${EROOT}"/usr/lib/ccache.backup || die

   readme.gentoo_print_elog
}


You can see clearly from the call stack that the error originates from the line "eselect compiler-shadow update ccache." The command automatically selects an option for the module compiler-shadow with the option ccache passed along. The update parameter is similar to "set" but instead of requiring manual selection, the selection process is done manually. Note that the error happens in the post-installation phase, meaning we are done with portages temporary directories.

As this seems to be a permission error, please post the output of:
Code:
ls -l /usr/ | grep lib


Now you shouldn't normally be running into a permissions issue as you are required to run emerge as root. This is probably related to how eselect handles permissions.

[/code]This is an unfinished answer as I have not yet looked at /usr/share/eselect/modules/compiler-shadow.eselect. (The file that handles the module compiler-shadow.) I have also not yet tried emerging the package on my own system. I will try booting up the system in a little bit and see if the installation goes okay.

A possible solution however is to manually create the file /usr/lib/ccache as root and then retry emerging it.

Hope this helps!
_________________
"One does not simply, install gentoo."
Back to top
View user's profile Send private message
Leomania
n00b
n00b


Joined: 16 Dec 2017
Posts: 17

PostPosted: Wed Aug 15, 2018 2:09 pm    Post subject: Reply with quote

oreo wrote:
As this seems to be a permission error, please post the output of:
Code:
ls -l /usr/ | grep lib

Now you shouldn't normally be running into a permissions issue as you are required to run emerge as root. This is probably related to how eselect handles permissions.

Hi oreo,

In my case, I shouldn't be running as root because this is a Gentoo Prefix build, and everything is intended to be built under my $EPREFIX which in this case is /auto/ins-tools/linux_cel69. So the permissions issue is expected if it's trying to do the install from the OS root directory /usr/lib rather than /auto/ins-tools/linux_cel69/usr/lib.

That's what I think needs to be changed - nothing should try to install files outside of $EPREFIX. ccache is the first package that I've seen attempt to do so.

Thanks,

- Leo
Back to top
View user's profile Send private message
oreo
n00b
n00b


Joined: 14 Aug 2018
Posts: 31

PostPosted: Wed Aug 15, 2018 5:58 pm    Post subject: Reply with quote

Oh, what is your reason for not compiling into the root dir? Cross compiling? As for eselect, I'm not entirely sure that it respects portages environment variables. It is after all, separate from portage. (Different package.) It is entirely possible that the eselect module has no option for installing in the EPREFIX. (Or the correct parameters are not passed on.) You can see that the eselect command is surrounded by an if statement. (If ${ROOT} == /) In portage the root variable is very similar to the EPREFIX variable but it is used for mainly for cross compiling. (The equivalent of chrooting into a new environment and then emerging whatever package.) As a temporary fix, I would try digging into the module and hard coding the correct paths. I will try and find time to do that myself but I will not make any promises.

If it is a problem with the module, I will contact the maintainer. However, it is most likely a fault with the ebuild not passing on the correct parameters or variables to the eselect statement. If so I will contact the package maintainer. Just let me know if you find an answer before I do.

Hope this helps.
_________________
"One does not simply, install gentoo."
Back to top
View user's profile Send private message
Leomania
n00b
n00b


Joined: 16 Dec 2017
Posts: 17

PostPosted: Wed Aug 15, 2018 6:35 pm    Post subject: Reply with quote

oreo wrote:
Oh, what is your reason for not compiling into the root dir? Cross compiling?

I'm using Prefix to have a place that appears early in my team's PATH where we pick up common versions of programs for our build environment. It's especially useful when a new python/perl module needs to be installed and be available on every system we use. So it can't be in a standard location off of "/".

oreo wrote:
If it is a problem with the module, I will contact the maintainer. However, it is most likely a fault with the ebuild not passing on the correct parameters or variables to the eselect statement. If so I will contact the package maintainer. Just let me know if you find an answer before I do.

Hope this helps.

I'll look into whether I can make the adjustment, but I first need to see how a working package does it. It'll be a few days before I can look into it, though.

Thanks,

- Leo
Back to top
View user's profile Send private message
oreo
n00b
n00b


Joined: 14 Aug 2018
Posts: 31

PostPosted: Wed Aug 15, 2018 9:18 pm    Post subject: Reply with quote

So I booted up gentoo to see if I could replicate the error, so as a regular user, (part of group wheel and portage) I get errors whenever it tries to pull/write to the emerge-fetch logs. What environment trickery did you have to do to emerge as a normal user? Did you adjust the permissions of the necessary folders? Or do you use alternative tmp directories, log directories, etc. ?

In the mean time, I will install the package as root so I can get a hold of the eselect module. (Doesn't appear to be on my system ATM. I'm guessing installing it will download the module.)
_________________
"One does not simply, install gentoo."
Back to top
View user's profile Send private message
Leomania
n00b
n00b


Joined: 16 Dec 2017
Posts: 17

PostPosted: Thu Aug 16, 2018 8:33 pm    Post subject: Reply with quote

oreo wrote:
So I booted up gentoo to see if I could replicate the error, so as a regular user, (part of group wheel and portage) I get errors whenever it tries to pull/write to the emerge-fetch logs. What environment trickery did you have to do to emerge as a normal user? Did you adjust the permissions of the necessary folders? Or do you use alternative tmp directories, log directories, etc. ?

No trickery; just started the bootstrap-prefix process as a normal user with write permissions in the $EPREFIX directory. This is different than a full Gentoo installation; you'd need to have a prefix env set up then try to emerge ccache to reproduce this, unfortunately.

Regards,

- Leo
Back to top
View user's profile Send private message
oreo
n00b
n00b


Joined: 14 Aug 2018
Posts: 31

PostPosted: Fri Aug 17, 2018 6:19 am    Post subject: Reply with quote

Found it. Here is the problem. This is a snippet of the do_update function in the eselect module "compiler-shadow." I have added my own comments to the snippet to make it easier reading.
Code:
for t in "${tools[@]}"; do # tools has been defined earlier in the script as the first parameter passed. In this instance, "ccache." (eselect compiler-shadow update ccache)
                has "${t}" "${MASQ_TOOLS[@]}" || die -q "Unknown tool: ${t}"
                echo "Updating masquerade for ${t} ..."

                local out_dir_var=MASQ_DIR_${t} # this variable is based on files found in /usr/share/shadowman/tools
                local out_dir=${!out_dir_var}
                local tool_exec=$(type -P "${t}")
                [[ ${tool_exec} ]] || die -q "Tool not found in PATH: ${t}"

                mkdir -p "${out_dir}" || die # this is where you run into trouble.
   

In the eselect manual, there is no --prefix argument or anything of the sorts, so I'm questioning whether eselect modules even support the eprefix. Either way, it seems that the this module is hard-coded to look in /usr/share/shadowman regardless of any eprefix.
Code:
MASQ_MODULEDIR=/usr/share/shadowman

There are various references to that directory and it appears to be finding things there which makes me also question where shadowman installed on your system. Could you verify that it is installed under ${EPREFIX}/usr/share/shadowman and not in root? Unless portage substitutes its eprefix in front of every path, it looks like you have several issues here.

As I do not know if eselect supports portage variables and cannot find any documentation, I will start another thread asking that question. It will attract more attention. Just a little bit more info and I can safely start pestering devs. :D

Sorry for the unsure answer. :( I'm dealing with a lot of unknowns.
_________________
"One does not simply, install gentoo."
Back to top
View user's profile Send private message
oreo
n00b
n00b


Joined: 14 Aug 2018
Posts: 31

PostPosted: Fri Aug 17, 2018 8:48 am    Post subject: Reply with quote

Yay! According to https://wiki.gentoo.org/wiki/Project:Eselect/Developer_guide#General_utility_functions
Quote:
All eselect modules are required to support the ROOT variable. For prefix support, variables EPREFIX and EROOT are also defined and have the same meaning as in ebuilds.

This module doesn't even appear to support the root variable. I did a whereis in nano for the root variable and it came up with nothing. As this module has multiple paths in it, it should at least support the root variable.

Welp, at least I know who to contact. I believe that eselect module is part of the shadowman package. I will see about contacting the developer. Consider this case closed.

In the mean time, I will try and make an alternative module that supports eprefixes. Will post when made. Most likely sometime tomorrow. It is very late for me. *yawn* I'm not sure if you can attach files on this forum though...

Hope I have been useful! :D
_________________
"One does not simply, install gentoo."
Back to top
View user's profile Send private message
oreo
n00b
n00b


Joined: 14 Aug 2018
Posts: 31

PostPosted: Fri Aug 17, 2018 7:28 pm    Post subject: Reply with quote

Editing this line should do the trick. Put "{EROOT}" before the path. (Shorthand for ${ROOT%/}${EPREFIX}/)
Code:
# {{{ substitution variables
MASQ_MODULEDIR="${EROOT}"/usr/share/shadowman
# }}}


If there are any problems let me know.
_________________
"One does not simply, install gentoo."
Back to top
View user's profile Send private message
Leomania
n00b
n00b


Joined: 16 Dec 2017
Posts: 17

PostPosted: Wed Aug 22, 2018 5:37 am    Post subject: Reply with quote

oreo wrote:
Editing this line should do the trick. Put "{EROOT}" before the path. (Shorthand for ${ROOT%/}${EPREFIX}/)
Code:
# {{{ substitution variables
MASQ_MODULEDIR="${EROOT}"/usr/share/shadowman
# }}}

If there are any problems let me know.

Hi oreo,

Which file did you find this in?

Thanks,

- Leo
Back to top
View user's profile Send private message
oreo
n00b
n00b


Joined: 14 Aug 2018
Posts: 31

PostPosted: Wed Aug 22, 2018 6:55 am    Post subject: Reply with quote

Eselect modules are stored in /usr/share/eselect/modules/. I am unsure how their location varies with different eprefixes. In this case, the module in discussion is "compiler-shadow". (eselect update compiler-shadow ccache) The full path is /usr/share/eselect/modules/compiler-shadow on my system.

Hope this helps! :D

P. S. If this fixes the issue for you, can you prefix the title with "[solved]" It helps other people not waste their valuable time. :D (Unfortunately not everyone does it.)
_________________
"One does not simply, install gentoo."
Back to top
View user's profile Send private message
Leomania
n00b
n00b


Joined: 16 Dec 2017
Posts: 17

PostPosted: Wed Aug 22, 2018 3:16 pm    Post subject: Reply with quote

oreo wrote:
Eselect modules are stored in /usr/share/eselect/modules/. I am unsure how their location varies with different eprefixes. In this case, the module in discussion is "compiler-shadow". (eselect update compiler-shadow ccache) The full path is /usr/share/eselect/modules/compiler-shadow on my system.

Thanks for that. Looks like the Prefix env modified the MASQ_MODULEDIR during the build/install process; here's what mine looks like:

Code:
# {{{ substitution variables
MASQ_MODULEDIR=/auto/ins-tools/linux_cel69/usr/share/shadowman
# }}}

which is the correct path to use in my situation.

I'd like to run the ccache emerge again but prevent any cleanup from happening after the error so I can look at the scripts. Is there a flag I can supply to emerge to tell it to not do the cleanup?

Thanks,

- Leo
Back to top
View user's profile Send private message
Leomania
n00b
n00b


Joined: 16 Dec 2017
Posts: 17

PostPosted: Fri Aug 24, 2018 3:16 am    Post subject: Reply with quote

I put some echo statements into compiler-shadow.eselect and found that the variable "MASQ_DIR_ccache" was being set to /usr/lib/ccache/bin in the load_tools function. The reason is that the file usr/share/shadowman/tools/ccache contains exactly this path:
Code:
# cat usr/share/shadowman/tools/ccache
/usr/lib/ccache/bin

which is part of dev-util/ccache-3.3.4-r1.

I don't know if this file should be getting modified before or after being read by compiler-shadow.eselect.

I hacked the script to force the correct directory and got ccache installed. I would like to know how to fix it correctly, though.

Regards,

- Leo
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum