View previous topic :: View next topic |
Author |
Message |
Fulgurance Veteran
Joined: 15 Feb 2017 Posts: 1198
|
Posted: Sat Nov 17, 2018 12:05 am Post subject: [Solved] Impossible to build firefox developper edition |
|
|
Hello, today, before make firefox developper edition ebuild, i try to build this package just from source to learn how this software work.
But when i try to execute ./mach, i have this error. I think mozzila script have problem.
Output:
Code: | fulgurance@msi-gs73vr-6rf ~/Téléchargements/firefox-64.0 $ ./mach
Traceback (most recent call last):
File "./mach", line 86, in <module>
main(sys.argv[1:])
File "./mach", line 78, in main
mach = get_mach()
File "./mach", line 50, in get_mach
config_status_path = os.path.join(dir_path, 'config.status')
File "/usr/lib64/python2.7/posixpath.py", line 73, in join
path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 18: ordinal not in range(128)
|
mach script:
Code: | #!/bin/sh
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
# The beginning of this script is both valid shell and valid python,
# such that the script starts with the shell and is reexecuted with
# the right python.
'''which' python2.7 > /dev/null && exec python2.7 "$0" "$@" || exec python "$0" "$@"
'''
from __future__ import print_function, unicode_literals
import os
import sys
def ancestors(path):
while path:
yield path
(path, child) = os.path.split(path)
if child == "":
break
def load_mach(dir_path, mach_path):
import imp
with open(mach_path, 'r') as fh:
imp.load_module('mach_bootstrap', fh, mach_path,
('.py', 'r', imp.PY_SOURCE))
import mach_bootstrap
return mach_bootstrap.bootstrap(dir_path)
def check_and_get_mach(dir_path):
bootstrap_paths = (
'build/mach_bootstrap.py',
# test package bootstrap
'tools/mach_bootstrap.py',
)
for bootstrap_path in bootstrap_paths:
mach_path = os.path.join(dir_path, bootstrap_path)
if os.path.isfile(mach_path):
return load_mach(dir_path, mach_path)
return None
def get_mach():
# Check whether the current directory is within a mach src or obj dir.
for dir_path in ancestors(os.getcwd()):
# If we find a "config.status" and "mozinfo.json" file, we are in the objdir.
config_status_path = os.path.join(dir_path, 'config.status')
mozinfo_path = os.path.join(dir_path, 'mozinfo.json')
if os.path.isfile(config_status_path) and os.path.isfile(mozinfo_path):
import json
info = json.load(open(mozinfo_path))
if 'mozconfig' in info and 'MOZCONFIG' not in os.environ:
# If the MOZCONFIG environment variable is not already set, set it
# to the value from mozinfo.json. This will tell the build system
# to look for a config file at the path in $MOZCONFIG rather than
# its default locations.
#
# Note: subprocess requires native strings in os.environ on Windows
os.environ[b'MOZCONFIG'] = str(info['mozconfig'])
if 'topsrcdir' in info:
# Continue searching for mach_bootstrap in the source directory.
dir_path = info['topsrcdir']
mach = check_and_get_mach(dir_path)
if mach:
return mach
# If we didn't find a source path by scanning for a mozinfo.json, check
# whether the directory containing this script is a source directory. We
# follow symlinks so mach can be run even if cwd is outside the srcdir.
return check_and_get_mach(os.path.dirname(os.path.realpath(__file__)))
def main(args):
mach = get_mach()
if not mach:
print('Could not run mach: No mach source directory found.')
sys.exit(1)
sys.exit(mach.run(args))
if __name__ == '__main__':
main(sys.argv[1:])
|
Last edited by Fulgurance on Tue Nov 20, 2018 4:09 pm; edited 1 time in total |
|
Back to top |
|
|
fedeliallalinea Administrator
Joined: 08 Mar 2003 Posts: 30837 Location: here
|
Posted: Sun Nov 18, 2018 9:55 am Post subject: |
|
|
Code: | UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 18: ordinal not in range(128) |
Usually this problem is related to locale, what is your locale -a _________________ Questions are guaranteed in life; Answers aren't. |
|
Back to top |
|
|
Fulgurance Veteran
Joined: 15 Feb 2017 Posts: 1198
|
Posted: Sun Nov 18, 2018 4:25 pm Post subject: |
|
|
Oh okay.
Code: | msi-gs73vr-6rf% locale -a
C
fr_FR.utf8
POSIX
|
|
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9645 Location: almost Mile High in the USA
|
Posted: Sun Nov 18, 2018 6:18 pm Post subject: |
|
|
Interesting if this is the cause for the outstanding distcc bug for Portage... _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
Fulgurance Veteran
Joined: 15 Feb 2017 Posts: 1198
|
Posted: Sun Nov 18, 2018 11:43 pm Post subject: |
|
|
Don't work for me... or i do something wrong?
Code: | FEATURES="-distcc" ./mach
Traceback (most recent call last):
File "./mach", line 86, in <module>
main(sys.argv[1:])
File "./mach", line 78, in main
mach = get_mach()
File "./mach", line 50, in get_mach
config_status_path = os.path.join(dir_path, 'config.status')
File "/usr/lib64/python2.7/posixpath.py", line 73, in join
path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 18: ordinal not in range(128) |
|
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9645 Location: almost Mile High in the USA
|
Posted: Sun Nov 18, 2018 11:53 pm Post subject: |
|
|
Err...sorry about steering you in the wrong direction, but what fedeliallalinea was implying is that the underlying problem is locales and I just extended it, off topic, to distcc where a lot of us have been having trouble... Since you're building outside of portage, and probably was never using distcc to begin with, this is not related to distcc? _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
Fulgurance Veteran
Joined: 15 Feb 2017 Posts: 1198
|
Posted: Mon Nov 19, 2018 12:06 am Post subject: |
|
|
Yes, i don't have distcc, and i compile directly from source without emerge at moment:
Code: | FEATURES="assume-digests binpkg-logs candy config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync multilib-strict news parallel-fetch preserve-libs protect-owned sandbox selinux sesandbox sfperms sign strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
|
But bug report you have posted is very similary. |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9645 Location: almost Mile High in the USA
|
Posted: Mon Nov 19, 2018 12:36 am Post subject: |
|
|
What happens if you unset all your LANG* variables? Or are they even set to begin with? _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
Fulgurance Veteran
Joined: 15 Feb 2017 Posts: 1198
|
Posted: Mon Nov 19, 2018 10:41 am Post subject: |
|
|
Code: | fulgurance@msi-gs73vr-6rf ~/Téléchargements/firefox-64.0 LANG="" ./mach
Traceback (most recent call last):
File "./mach", line 86, in <module>
main(sys.argv[1:])
File "./mach", line 78, in main
mach = get_mach()
File "./mach", line 50, in get_mach
config_status_path = os.path.join(dir_path, 'config.status')
File "/usr/lib64/python2.7/posixpath.py", line 73, in join
path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 18: ordinal not in range(128)
fulgurance@msi-gs73vr-6rf ~/Téléchargements/firefox-64.0 echo $LANG
fr_FR.utf8
|
Don't see strange block output, it's because i'm testing new package for zsh.
I have the same problem... |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9645 Location: almost Mile High in the USA
|
Posted: Mon Nov 19, 2018 5:55 pm Post subject: |
|
|
I assume there's some other stuff that's done before running 'mach' (which I have no idea what it does other than it does it...)
Will it work if you did a fresh build setup with LANG unset?
I'm crossing my fingers that I have distcc fully working once more with another firefox build, which is in progress right now... _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
fedeliallalinea Administrator
Joined: 08 Mar 2003 Posts: 30837 Location: here
|
Posted: Mon Nov 19, 2018 6:01 pm Post subject: |
|
|
In moz_pkgsetup function of mozcoreconf eclass set C as LANG
mozcoreconf-v6.eclass: | moz_pkgsetup() {
# Ensure we use C locale when building
export LANG="C"
export LC_ALL="C"
export LC_MESSAGES="C"
export LC_CTYPE="C"
... |
_________________ Questions are guaranteed in life; Answers aren't. |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9645 Location: almost Mile High in the USA
|
Posted: Mon Nov 19, 2018 6:12 pm Post subject: |
|
|
Well, nuts, no idea what's going on now for Fulgurance, but at least for the distcc users now have a hackish solution... Question is whether LANG needs to be propagated to the distcc helpers. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
Genone Retired Dev
Joined: 14 Mar 2003 Posts: 9507 Location: beyond the rim
|
Posted: Tue Nov 20, 2018 11:55 am Post subject: |
|
|
The problem is likely the pathname where you're trying to build:
"~/Téléchargements/firefox-64.0"
those french special chars aren't part of ASCII and therefore cause problems. 0xc3 is used by unicode as prefix for a group of those characters. |
|
Back to top |
|
|
Fulgurance Veteran
Joined: 15 Feb 2017 Posts: 1198
|
Posted: Tue Nov 20, 2018 3:44 pm Post subject: |
|
|
Yes ! It's that ! Thanks ! |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9645 Location: almost Mile High in the USA
|
Posted: Tue Nov 20, 2018 4:59 pm Post subject: |
|
|
Thanks, can't believe pathnames are parsed...oh nevermind, python is like that after all :p
Reminds me of a problem I had recently of a python program that I wrote to scrape a website, was working just fine until one day, someone put in unicode and caused my program to crash when it prints out into the logfile.
Misery. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
Hu Moderator
Joined: 06 Mar 2007 Posts: 21489
|
Posted: Wed Nov 21, 2018 2:50 am Post subject: |
|
|
Python 3 is generally a bit better in this regard. Python 2 sometimes let you get away with unsafe code until you tried to use troublesome characters. Python 3 is more likely to punish you the moment you try to write Unicode unsafe code, regardless of whether you actually have any unsafe characters in the string when you test the program. This makes porting to Python 3 very painful, and makes writing in Python 3 a bit more painful. On the other hand, it means that you're much less likely to consider a script done, then later have it blow up when somebody feeds it non-English text. |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9645 Location: almost Mile High in the USA
|
Posted: Sat Nov 24, 2018 12:10 am Post subject: |
|
|
Unfortunately it was python3.6 that I thought my script was working cleanly, then someone put some unicode in and it exploded.
Granted it was an imported library that did not handle the unicode properly. I ended up working around the library problem by stripping the unicode... Not the best way of handling it, but one has to do what one has to do... _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
Hu Moderator
Joined: 06 Mar 2007 Posts: 21489
|
Posted: Sat Nov 24, 2018 5:40 pm Post subject: |
|
|
Interesting. I wrote "more likely" because I wasn't sure whether Python 3 would catch all such cases. Your experience confirms my concern: even with the stricter handling in Python 3, Python code can still be unsafe and run correctly until someone feeds it bad data, rather than always reacting badly to patterns that will mishandle bad data, without regard to whether bad data is present this time. |
|
Back to top |
|
|
|