| View previous topic :: View next topic |
| Author |
Message |
haarp
Guru
Joined: 31 Oct 2007
Posts: 535
|
 Posted: Fri May 11, 2018 10:49 pm Post subject: emerge is non-deterministic? |
 |
|
I'm experiencing something of a curiosity with portage-2.3.24-r1.
As there's still no recent stable gentoo-sources, I still have 4.12.12 installed. emerge suggests to also install 4.9.95, which is stable. But here's the kicker - not always!
I can run
| Code: | | emerge -pvDNu world |
multiple times, and there's roughly a 50:50 chance that gentoo-sources will be in the list of merge candidates.
Huh?? |
|
| Back to top |
|
 |
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Sat May 12, 2018 12:36 am Post subject: |
|
|
Can we see the output of | Code: | | grep -Fn sys-kernel /var/lib/portage/world |
and the usual emerge --info please. |
|
| Back to top |
|
|
haarp
Guru
Joined: 31 Oct 2007
Posts: 535
|
| Posted: Sat May 12, 2018 12:52 am Post subject: |
|
|
sure thing
| Code: | # grep -Fn sys-kernel /var/lib/portage/world
195:sys-kernel/gentoo-sources
196:sys-kernel/gentoo-sources:4.12.12
197:sys-kernel/linux-firmware |
I remember adding 4.12.12 a long time ago to prevent it from getting unmerged by depclean when I also had 4.14 installed (I don't have it anymore)
| Code: | # emerge --info
Portage 2.3.24 (python 3.5.5-final-0, default/linux/amd64/17.0/desktop, gcc-6.4.0, glibc-2.25-r11, 4.12.12-gentoo-H x86_64)
=================================================================
System uname: Linux-4.12.12-gentoo-H-x86_64-Intel-R-_Core-TM-_i7-3920XM_CPU_@_2.90GHz-with-gentoo-2.4.1
KiB Mem: 24302688 total, 3594640 free
KiB Swap: 33554428 total, 32954860 free
Head commit of repository gentoo: 06fbdeb1f6f84e8a375a850426038ff227a9b280
sh bash 4.4_p12
ld GNU ld (Gentoo 2.29.1 p3) 2.29.1
app-shells/bash: 4.4_p12::gentoo
dev-java/java-config: 2.2.0-r4::gentoo
dev-lang/perl: 5.24.3-r1::gentoo
dev-lang/python: 2.7.14-r1::gentoo, 3.5.5::gentoo, 3.6.5::gentoo
dev-util/cmake: 3.9.6::gentoo
dev-util/pkgconfig: 0.29.2::gentoo
sys-apps/baselayout: 2.4.1-r2::gentoo
sys-apps/openrc: 0.34.11::gentoo
sys-apps/sandbox: 2.13::gentoo
sys-devel/autoconf: 2.13::gentoo, 2.69-r4::gentoo
sys-devel/automake: 1.11.6-r3::gentoo, 1.15.1-r2::gentoo
sys-devel/binutils: 2.29.1-r1::gentoo
sys-devel/gcc: 6.4.0-r1::gentoo
sys-devel/gcc-config: 1.8-r1::gentoo
sys-devel/libtool: 2.4.6-r3::gentoo
sys-devel/make: 4.2.1::gentoo
sys-kernel/linux-headers: 4.16-r1::gentoo (virtual/os-headers)
sys-libs/glibc: 2.25-r11::gentoo
Repositories:
gentoo
location: /var/db/repos/gentoo
sync-type: git
sync-uri: https://anongit.gentoo.org/git/repo/gentoo.git
priority: -1000
bumblebee
location: /var/db/repos/bumblebee
sync-type: laymansync
sync-uri: git://github.com/Bumblebee-Project/bumblebee-gentoo
masters: gentoo
priority: 50
jorgicio
location: /var/db/repos/jorgicio
sync-type: laymansync
sync-uri: https://github.com/jorgicio/jorgicio-gentoo.git
masters: gentoo
priority: 50
rust
location: /var/db/repos/rust
sync-type: laymansync
sync-uri: https://github.com/gentoo/gentoo-rust.git
masters: gentoo
priority: 50
spring
location: /var/db/repos/spring
sync-type: laymansync
sync-uri: git://github.com/springlobby/overlay.git
masters: gentoo
priority: 50
steam-overlay
location: /var/db/repos/steam-overlay
sync-type: laymansync
sync-uri: https://github.com/anyc/steam-overlay.git
masters: gentoo
priority: 50
vmware
location: /var/db/repos/vmware
sync-type: laymansync
sync-uri: git://anongit.gentoo.org/proj/vmware.git
masters: gentoo
priority: 50
vortex
location: /var/db/repos/vortex
sync-type: laymansync
sync-uri: git://github.com/nE0sIghT/vortex-overlay.git
masters: gentoo
priority: 50
x11
location: /var/db/repos/x11
sync-type: laymansync
sync-uri: git://anongit.gentoo.org/proj/x11
masters: gentoo
priority: 50
haarp
location: /var/db/repos/haarp
sync-type: laymansync
sync-uri: git://anongit.gentoo.org/user/haarp.git
masters: gentoo
priority: 100
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe -fno-stack-protector"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -O2 -pipe -fno-stack-protector"
DISTDIR="/var/cache/distfiles"
EMERGE_DEFAULT_OPTS="--ask-enter-invalid --alert --read-news --keep-going --quiet-unmerge-warn"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs cgroup config-protect-if-modified distlocks ebuild-locks fail-clean fixlafiles ipc-sandbox merge-sync multilib-strict news noinfo parallel-fetch parallel-install preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync webrsync-gpg xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
INSTALL_MASK="/etc/systemd /lib/systemd /lib64/systemd /usr/lib/systemd /usr/lib64/systemd"
LANG="en_DK.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j8"
PKGDIR="/var/cache/packages"
PORTAGE_COMPRESS="xz"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X a52 aac acl acpi aio alsa amd64 aot archive berkdb bluetooth bluray branding bzip2 cairo caps cdda cdr cgroups cli consolekit crypt cryptsetup cups custom-cflags custom-optimization cxx dbus dri dts dvd dvdr emboss encode exif fam fat ffmpeg flac fuse gdbm geoip gif git glamor gphoto2 gtk gtkstyle hidpi http2 iconv ipv6 jpeg keybinder lcms libass libcanberra libinput libnotify lm_sensors lz4 lzo mad mng modemmanager modules mp3 mp4 mpeg mtp multilib ncurses networkmanager nls nptl nsplugin ntfs ntfsdecrypt offensive ogg opengl openmp pam pango pcre pdf pgo pm-utils png policykit ppds pulseaudio qrcode qt3support rar readline run-as-root s3tc samba sdl seccomp sftp smp ssh ssl staging startup-notification suid svg symlink syslog tcmalloc tcpd threads thunar tiff truetype udev udisks unicode upower usb v4l vaapi vdpau vorbis wifi wxwidgets x264 x265 xattr xcb xml xscreensaver xv xvid zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon plan sheets stage words" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx mmx mmxext sse sse2 sse3 ssse3 sse4 sse4_1 sse4_2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="pc efi-64" INPUT_DEVICES="libinput wacom" KERNEL="linux" L10N="en de" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-0" POSTGRES_TARGETS="postgres9_5" PYTHON_SINGLE_TARGET="python3_5" PYTHON_TARGETS="python2_7 python3_5" RUBY_TARGETS="ruby23" USERLAND="GNU" VIDEO_CARDS="vesa intel nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset: CC, CPPFLAGS, CTARGET, CXX, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS |
|
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Sat May 12, 2018 3:57 pm Post subject: |
|
|
My gut reaction is: with all those web overlays taking precedence, it's no wonder it's "undeterministic".
When it's changing behaviour in front of you, with no sync or other changes in between, can you get the output of | Code: | | emerge -pv sys-kernel/gentoo-sources |
Can you mask out sys-kernel/* from other overlays? Might need to mask it across all, and then unmask ::gentoo.
See what changes you get, also on an emerge -pv @system (then @world), keeping an eye out for overlay changes. (::fubar -> ::gentoo) |
|
| Back to top |
|
|
haarp
Guru
Joined: 31 Oct 2007
Posts: 535
|
| Posted: Sat May 12, 2018 5:12 pm Post subject: |
|
|
Yes, it's changing behavior during consecutive runs, without any changes to configs, syncing or anything else.
| Code: | $ emerge -pv sys-kernel/gentoo-sources
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild NS ] sys-kernel/gentoo-sources-4.9.95:4.9.95::gentoo [4.12.12:4.12.12::gentoo] USE="experimental symlink -build" 92,990 KiB
Total: 1 package (1 in new slot), Size of downloads: 92,990 KiB |
No surprises here.
Let's mask sys-kernel/* and unmask sys-kernel/*::gentoo. This also unmasks sys-kernel/hardened-sources-4.8.17-r2::gentoo. Now emerge -pvDNu world randomly switches between wanting to merge gentoo-sources and hardened-sources. 
Since hardened-sources are not part of my worldfile, it's presence indicates that this might be the work of virtual/linux-sources-3::gentoo, which I have installed.
To exclude the possibility of overlays being responsible, I've disabled all of them, expect for ::gentoo itself. It still oscillates. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Sun May 13, 2018 1:07 pm Post subject: |
|
|
| haarp wrote: | | To exclude the possibility of overlays being responsible, I've disabled all of them, expect for ::gentoo itself. It still oscillates. |
OK, at this point (if it were my machine) I would /join #gentoo-portage on IRC: chat.freenode.org or .net and wait for zmedico to be around.
Weird bugs like this come up all the time, and while it might just be a config snafu, it could also be a resolver bug.
They tend not to show up, unless under specific circumstance, and it's much easier if the coder can ask questions and get files in realtime (for them; not for the reporter) at the time it's happening.
It's also the quickest way to find the config snafu.
You could file a bug, if you wanted, but it will take an awful lot longer (I'd estimate a couple of months, vs a couple of hours.)
Another approach is to file the bug, then try IRC after you get a response, and it looks like it's going to need more and more info, or there doesn't seem to be much response.
Other than that, hopefully genone will see this topic.
Sorry I cannot be of more help; I don't code on portage, and I've never seen it oscillate between package choice like that.
It smells like a bug to me, fwtw; even if it's just under certain circumstance, it should complain instead, imo.
With hardened-sources, either mask it again specifically after the unmask for ::gentoo, or unmask sys-kernel/gentoo-sources and linux-firmware (not sys-kernel/*)
edit: Hmm, how come you're getting a lower slot for gentoo-sources? |
|
| Back to top |
|
|
saboya
Guru
Joined: 28 Nov 2006
Posts: 552
Location: Brazil
|
| Posted: Sun May 13, 2018 3:22 pm Post subject: |
|
|
| Apparently --pretend is non-deterministic. Yours is not the first post I've seen about this. |
|
| Back to top |
|
|
haarp
Guru
Joined: 31 Oct 2007
Posts: 535
|
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Mon May 14, 2018 2:32 pm Post subject: |
|
|
Non-deterministic wrt the final order it spits out (which isn't precise under --jobs anyway.)
Missing updates are another matter.
Oscillation smells like a bug to me, likely triggered under specific config, with a specific combination of installed packages, etc. |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 9526
Location: beyond the rim
|
| Posted: Wed May 16, 2018 1:51 pm Post subject: |
|
|
| steveL wrote: | | Oscillation smells like a bug to me, likely triggered under specific config, with a specific combination of installed packages, etc. |
If --pretend results change without any (significant) change to system state it's definitely a bug of some kind, as it completely breaks the definition of --pretend. Could simply be a codepath iterating over an unordered data structure, and depending on the actual order packages are included/excluded in the final result. |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Wed May 16, 2018 2:32 pm Post subject: |
|
|
| Genone wrote: | | Could simply be a codepath iterating over an unordered data structure |
I would be surprised if that (i.e. preallocated size or choice of hash function) would be non-deterministic in python if the order of insertion of the data is deterministic. The latter seems likely unless multiple threads are used. And once the metadata is calculated (which presumably spans multiple threads but happens only at the first (few) call(s)): Why should portage use multiple threads before the end of dependency resolution in the presence of GIL? (Which BTW is IMHO a major reason why python was not a fortunate choice as a language for a PM). |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Wed May 16, 2018 4:22 pm Post subject: |
|
|
| Genone wrote: | | If --pretend results change without any (significant) change to system state it's definitely a bug of some kind, as it completely breaks the definition of --pretend. |
Thanks for confirming.
@mv: I used to think that, but then I tried pkgcore. |
|
| Back to top |
|
|
Dr.Willy
Guru
Joined: 15 Jul 2007
Posts: 547
Location: NRW, Germany
|
| Posted: Wed May 16, 2018 5:19 pm Post subject: |
|
|
| steveL wrote: | | @mv: I used to think that, but then I tried pkgcore. |
Well yes, python is great if you replace enough of it with c. |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Wed May 16, 2018 9:25 pm Post subject: |
|
|
| Genone wrote: | | If --pretend results change without any (significant) change to system state it's definitely a bug of some kind, as it completely breaks the definition of --pretend. |
Yes, but it wouldn't be really a surprise as this scenario is causing trouble as user has define this rule:
#1 i want gentoo-sources stable and latest
#2 i want gentoo-sources-4.12.12 which is not stable
If you take try to solve the problem, looking at need #2 is easy, he want 4.12.12 just give him 4.12.12
But if you look at #1, technically user have 0 stable gentoo-sources install (4.12.12 is not stable, and 4.9.95 is a good candidate then), but at the same time, if you just look at "any gentoo sources install?", then 4.12.12 met this.
To me the right answer is what portage is offering "sometimes", having 4.12.12 as per user wish, but also having 4.9.95 (that's the answer to met the "gentoo-sources" put in his world).
The usual, mixing stable and unstable is never good, also for portage it seem. |
|
| Back to top |
|
|
haarp
Guru
Joined: 31 Oct 2007
Posts: 535
|
| Posted: Wed May 16, 2018 9:30 pm Post subject: |
|
|
Yes, my mix of stable/unstable is less than ideal. I accept that some breakage may occur with weird combinations. But I expect breakage to be consistent at least  This bug is not critical, but quite peculiar.
I hope I'll get around to filing a bug report in the coming week.
And thanks for your responses! |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Thu May 17, 2018 1:31 am Post subject: |
|
|
| haarp wrote: | This bug is not critical, but quite peculiar.
I hope I'll get around to filing a bug report in the coming week. |
Agree, this should be fix, mixing stable/unstable make it a bit a pain for user, but portage shouldn't had any handling that. |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21624
|
| Posted: Thu May 17, 2018 1:40 am Post subject: |
|
|
Non-deterministic dictionary key ordering is standard in recent versions of Python. There was a big scare a few years ago about web applications initializing dictionaries with untrusted user input as keys, because a malicious user could submit a set of keys that would cause collisions and force extremely bad lookup performance. The workaround from the language authors was to bias the key->bucket transformation with a random value not knowable to the attacker (generally picked once at program initialization time). For a simple demonstration, run the below program multiple times and observe the output. On patched versions of Python, you get a different ordering of words each time. | Code: | d = {}
with open('/usr/share/dict/words', 'rt') as fp:
for i, line in enumerate(fp):
d[line] = i
print(''.join(d.keys())) |
|
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Thu May 17, 2018 1:03 pm Post subject: |
|
|
| Hu wrote: | | Non-deterministic dictionary key ordering is standard in recent versions of Python. |
Thank you very much for this interesting information. Also the reason for this makes sense. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Thu May 17, 2018 3:20 pm Post subject: |
|
|
| steveL wrote: | | @mv: I used to think that, but then I tried pkgcore. |
| Dr.Willy wrote: | | Well yes, python is great if you replace enough of it with c. |
Heh, that's one way of putting it.
But python works well as a high-level language, interfacing to C for speed where needed; and ferringb has done an excellent job with pkgcore.
So I don't think we can say "python is a bad choice", given the stonking proof of the opposite: the best package manager that ever came out of all the years of Gentoo.
Obviously, that's just my opinion, but it is based on scripting against all of them.
Here's another: ferringb is the most talented programmer ever to accept a Gentoo badge.
Gentoo should have been proud of nurturing his talent, instead of hiding it under a tonne of McCreesh shite. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Thu May 17, 2018 3:22 pm Post subject: |
|
|
| Hu wrote: | | Non-deterministic dictionary key ordering is standard in recent versions of Python. |
Good info, but I really don't see what that has to do with missing a package.
Even if it is part of the symptom, it's still a bug, and I don't think ordering will be relevant once it's fixed (by definition.) |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Thu May 17, 2018 3:41 pm Post subject: |
|
|
| steveL wrote: | | But python works well as a high-level language, interfacing to C for speed where needed |
If all of the dependency resolver is written in C (or C++) then one can argue whether C (or C++) is actually the main implementation language of the pm.
If any part is not written in C/C++ then this part cannot be parallelised.
The problem is that one "step" in dependency resolving is NP complete, and no matter how you do it, this "step" is quite large and (no matter how well the implementation is written) can get roughly 8 times the speed if 8 cores are used...
As someone here and on some ml suggested, this step can be a SAT solver written in a different language (maybe pkgcore goes this route?) but I would then tend to argue that the language in which this solver is implemented is the "main" language of the pm. The other languages (python and shell) or then only "kit" holding the thing together. |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21624
|
| Posted: Fri May 18, 2018 2:11 am Post subject: |
|
|
| steveL wrote: | | Hu wrote: | | Non-deterministic dictionary key ordering is standard in recent versions of Python. |
Good info, but I really don't see what that has to do with missing a package. |
It may not. I was responding to the discussion about why the ordering changes across runs: | Genone wrote: | | steveL wrote: | | Oscillation smells like a bug to me |
Could simply be a codepath iterating over an unordered data structure, and depending on the actual order packages are included/excluded in the final result. |
| mv wrote: | | I would be surprised if that (i.e. preallocated size or choice of hash function) would be non-deterministic in python if the order of insertion of the data is deterministic. |
I cannot comment on why non-deterministic ordering would influence the decision to include/exclude packages, although I like Genone's theory in that regard. For example, if the solver iterates over an unordered dictionary and builds a secondary data structure from visited elements, and the presence of package P1 in that secondary structure eliminates the need for package P2 when it pulls P3 from the dictionary, then the per-run ordering of the dictionary would control whether P2 is used based on whether P1 is found before P3. I am not saying the package manager should work this way, only that it's very plausible that the current implementation does. |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Fri May 18, 2018 4:21 am Post subject: |
|
|
| Hu wrote: | | I cannot comment on why non-deterministic ordering would influence the decision to include/exclude packages |
Here is a concrete example: Assume that there are packages A B in world with dependencies || ( C D ) and || ( D C )), respectively (neither C nor D being installed nor having further dependencies for simplicity). Then whether C or D is preferred depends on whether A or B is processed first.
However, in the case of hardened-sources it really seems to be a bug: eix --deps hardened-sources shows that only virtual/gentoo-sources might pull it in, and gentoo-sources has no || dependencies (and all of the dependencies of gentoo-sources should already be satisfied, anyway). So portage should never have a reason to pull in hardened-sources. (Assuming the information is right that this happens without overlays.) |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 9526
Location: beyond the rim
|
| Posted: Fri May 18, 2018 2:13 pm Post subject: |
|
|
| steveL wrote: | | Hu wrote: | | Non-deterministic dictionary key ordering is standard in recent versions of Python. |
Good info, but I really don't see what that has to do with missing a package.
Even if it is part of the symptom, it's still a bug, and I don't think ordering will be relevant once it's fixed (by definition.) |
Well, I was just thinking loud. If the input (=system state) of a function (=emerge call) doesn't change, what else could cause it to return different results? Some kind of (implicit) RNG application. And I had recently stumbled across a comment about the internal python hash function to be salted per process leading to different orders within unordered data structures. So that seemed like a realistic cause which could be easily overlooked if it only triggered in rare situations.
Constant input should lead to constant results for --pretend, no matter if the result is correct. Otherwise --pretend is broken by definition as you cannot rely on portage to actually show the action that would be performed. |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Fri May 18, 2018 4:05 pm Post subject: |
|
|
| Hu wrote: | | For a simple demonstration, run the below program multiple times and observe the output. On patched versions of Python, you get a different ordering of words each time. |
it randomize with python3 but not when i use python2, so that bug could be in portage for years (but not seen as long as user use python2)
that's not a good point for python, i could understand they need to fix this, but changing a function like that is not good ; why didn't they create a new randomize function doing the same but with salt? Users in need for it have to use the new one and everyone is happy. |
|
| Back to top |
|
|
|
Portage & Programming
