Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[Solved] Impossible to build firefox developper edition
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
Fulgurance
Veteran
Veteran


Joined: 15 Feb 2017
Posts: 1198

PostPosted: Sat Nov 17, 2018 12:05 am    Post subject: [Solved] Impossible to build firefox developper edition Reply with quote

Hello, today, before make firefox developper edition ebuild, i try to build this package just from source to learn how this software work.

But when i try to execute ./mach, i have this error. I think mozzila script have problem.

Output:
Code:
fulgurance@msi-gs73vr-6rf ~/Téléchargements/firefox-64.0 $ ./mach
Traceback (most recent call last):
  File "./mach", line 86, in <module>
    main(sys.argv[1:])
  File "./mach", line 78, in main
    mach = get_mach()
  File "./mach", line 50, in get_mach
    config_status_path = os.path.join(dir_path, 'config.status')
  File "/usr/lib64/python2.7/posixpath.py", line 73, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 18: ordinal not in range(128)


mach script:
Code:
#!/bin/sh
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.

# The beginning of this script is both valid shell and valid python,
# such that the script starts with the shell and is reexecuted with
# the right python.
'''which' python2.7 > /dev/null && exec python2.7 "$0" "$@" || exec python "$0" "$@"
'''

from __future__ import print_function, unicode_literals

import os
import sys

def ancestors(path):
    while path:
        yield path
        (path, child) = os.path.split(path)
        if child == "":
            break

def load_mach(dir_path, mach_path):
    import imp
    with open(mach_path, 'r') as fh:
        imp.load_module('mach_bootstrap', fh, mach_path,
                        ('.py', 'r', imp.PY_SOURCE))
    import mach_bootstrap
    return mach_bootstrap.bootstrap(dir_path)


def check_and_get_mach(dir_path):
    bootstrap_paths = (
        'build/mach_bootstrap.py',
        # test package bootstrap
        'tools/mach_bootstrap.py',
    )
    for bootstrap_path in bootstrap_paths:
        mach_path = os.path.join(dir_path, bootstrap_path)
        if os.path.isfile(mach_path):
            return load_mach(dir_path, mach_path)
    return None


def get_mach():
    # Check whether the current directory is within a mach src or obj dir.
    for dir_path in ancestors(os.getcwd()):
        # If we find a "config.status" and "mozinfo.json" file, we are in the objdir.
        config_status_path = os.path.join(dir_path, 'config.status')
        mozinfo_path = os.path.join(dir_path, 'mozinfo.json')
        if os.path.isfile(config_status_path) and os.path.isfile(mozinfo_path):
            import json
            info = json.load(open(mozinfo_path))
            if 'mozconfig' in info and 'MOZCONFIG' not in os.environ:
                # If the MOZCONFIG environment variable is not already set, set it
                # to the value from mozinfo.json.  This will tell the build system
                # to look for a config file at the path in $MOZCONFIG rather than
                # its default locations.
                #
                # Note: subprocess requires native strings in os.environ on Windows
                os.environ[b'MOZCONFIG'] = str(info['mozconfig'])

            if 'topsrcdir' in info:
                # Continue searching for mach_bootstrap in the source directory.
                dir_path = info['topsrcdir']

        mach = check_and_get_mach(dir_path)
        if mach:
            return mach

    # If we didn't find a source path by scanning for a mozinfo.json, check
    # whether the directory containing this script is a source directory. We
    # follow symlinks so mach can be run even if cwd is outside the srcdir.
    return check_and_get_mach(os.path.dirname(os.path.realpath(__file__)))

def main(args):
    mach = get_mach()
    if not mach:
        print('Could not run mach: No mach source directory found.')
        sys.exit(1)
    sys.exit(mach.run(args))


if __name__ == '__main__':
    main(sys.argv[1:])


Last edited by Fulgurance on Tue Nov 20, 2018 4:09 pm; edited 1 time in total
Back to top
View user's profile Send private message
fedeliallalinea
Administrator
Administrator


Joined: 08 Mar 2003
Posts: 30837
Location: here

PostPosted: Sun Nov 18, 2018 9:55 am    Post subject: Reply with quote

Code:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 18: ordinal not in range(128)

Usually this problem is related to locale, what is your locale -a
_________________
Questions are guaranteed in life; Answers aren't.
Back to top
View user's profile Send private message
Fulgurance
Veteran
Veteran


Joined: 15 Feb 2017
Posts: 1198

PostPosted: Sun Nov 18, 2018 4:25 pm    Post subject: Reply with quote

Oh okay.

Code:
msi-gs73vr-6rf% locale -a
C
fr_FR.utf8
POSIX
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Sun Nov 18, 2018 6:18 pm    Post subject: Reply with quote

Interesting if this is the cause for the outstanding distcc bug for Portage...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Fulgurance
Veteran
Veteran


Joined: 15 Feb 2017
Posts: 1198

PostPosted: Sun Nov 18, 2018 11:43 pm    Post subject: Reply with quote

Don't work for me... or i do something wrong?

Code:
FEATURES="-distcc" ./mach
Traceback (most recent call last):
  File "./mach", line 86, in <module>
    main(sys.argv[1:])
  File "./mach", line 78, in main
    mach = get_mach()
  File "./mach", line 50, in get_mach
    config_status_path = os.path.join(dir_path, 'config.status')
  File "/usr/lib64/python2.7/posixpath.py", line 73, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 18: ordinal not in range(128)
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Sun Nov 18, 2018 11:53 pm    Post subject: Reply with quote

Err...sorry about steering you in the wrong direction, but what fedeliallalinea was implying is that the underlying problem is locales and I just extended it, off topic, to distcc where a lot of us have been having trouble... Since you're building outside of portage, and probably was never using distcc to begin with, this is not related to distcc?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Fulgurance
Veteran
Veteran


Joined: 15 Feb 2017
Posts: 1198

PostPosted: Mon Nov 19, 2018 12:06 am    Post subject: Reply with quote

Yes, i don't have distcc, and i compile directly from source without emerge at moment:

Code:
FEATURES="assume-digests binpkg-logs candy config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync multilib-strict news parallel-fetch preserve-libs protect-owned sandbox selinux sesandbox sfperms sign strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"


But bug report you have posted is very similary.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Mon Nov 19, 2018 12:36 am    Post subject: Reply with quote

What happens if you unset all your LANG* variables? Or are they even set to begin with?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Fulgurance
Veteran
Veteran


Joined: 15 Feb 2017
Posts: 1198

PostPosted: Mon Nov 19, 2018 10:41 am    Post subject: Reply with quote

Code:
 fulgurance@msi-gs73vr-6rf   ~/Téléchargements/firefox-64.0  LANG="" ./mach
Traceback (most recent call last):
  File "./mach", line 86, in <module>
    main(sys.argv[1:])
  File "./mach", line 78, in main
    mach = get_mach()
  File "./mach", line 50, in get_mach
    config_status_path = os.path.join(dir_path, 'config.status')
  File "/usr/lib64/python2.7/posixpath.py", line 73, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 18: ordinal not in range(128)
 fulgurance@msi-gs73vr-6rf   ~/Téléchargements/firefox-64.0  echo $LANG
fr_FR.utf8


Don't see strange block output, it's because i'm testing new package for zsh.

I have the same problem...
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Mon Nov 19, 2018 5:55 pm    Post subject: Reply with quote

I assume there's some other stuff that's done before running 'mach' (which I have no idea what it does other than it does it...)
Will it work if you did a fresh build setup with LANG unset?

I'm crossing my fingers that I have distcc fully working once more with another firefox build, which is in progress right now...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
fedeliallalinea
Administrator
Administrator


Joined: 08 Mar 2003
Posts: 30837
Location: here

PostPosted: Mon Nov 19, 2018 6:01 pm    Post subject: Reply with quote

In moz_pkgsetup function of mozcoreconf eclass set C as LANG
mozcoreconf-v6.eclass:
moz_pkgsetup() {
        # Ensure we use C locale when building
        export LANG="C"
        export LC_ALL="C"
        export LC_MESSAGES="C"
        export LC_CTYPE="C"
...

_________________
Questions are guaranteed in life; Answers aren't.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Mon Nov 19, 2018 6:12 pm    Post subject: Reply with quote

Well, nuts, no idea what's going on now for Fulgurance, but at least for the distcc users now have a hackish solution... Question is whether LANG needs to be propagated to the distcc helpers.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9507
Location: beyond the rim

PostPosted: Tue Nov 20, 2018 11:55 am    Post subject: Reply with quote

The problem is likely the pathname where you're trying to build:

"~/Téléchargements/firefox-64.0"

those french special chars aren't part of ASCII and therefore cause problems. 0xc3 is used by unicode as prefix for a group of those characters.
Back to top
View user's profile Send private message
Fulgurance
Veteran
Veteran


Joined: 15 Feb 2017
Posts: 1198

PostPosted: Tue Nov 20, 2018 3:44 pm    Post subject: Reply with quote

Yes ! It's that ! Thanks !
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Tue Nov 20, 2018 4:59 pm    Post subject: Reply with quote

Thanks, can't believe pathnames are parsed...oh nevermind, python is like that after all :p

Reminds me of a problem I had recently of a python program that I wrote to scrape a website, was working just fine until one day, someone put in unicode and caused my program to crash when it prints out into the logfile.

Misery.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21489

PostPosted: Wed Nov 21, 2018 2:50 am    Post subject: Reply with quote

Python 3 is generally a bit better in this regard. Python 2 sometimes let you get away with unsafe code until you tried to use troublesome characters. Python 3 is more likely to punish you the moment you try to write Unicode unsafe code, regardless of whether you actually have any unsafe characters in the string when you test the program. This makes porting to Python 3 very painful, and makes writing in Python 3 a bit more painful. On the other hand, it means that you're much less likely to consider a script done, then later have it blow up when somebody feeds it non-English text.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Sat Nov 24, 2018 12:10 am    Post subject: Reply with quote

Unfortunately it was python3.6 that I thought my script was working cleanly, then someone put some unicode in and it exploded.

Granted it was an imported library that did not handle the unicode properly. I ended up working around the library problem by stripping the unicode... Not the best way of handling it, but one has to do what one has to do...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21489

PostPosted: Sat Nov 24, 2018 5:40 pm    Post subject: Reply with quote

Interesting. I wrote "more likely" because I wasn't sure whether Python 3 would catch all such cases. Your experience confirms my concern: even with the stricter handling in Python 3, Python code can still be unsafe and run correctly until someone feeds it bad data, rather than always reacting badly to patterns that will mishandle bad data, without regard to whether bad data is present this time.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum