pye - "pick your emerge" script

[Monkeywrench]

awesome script dr_shadow! I'm currently kind of learning python (my first language, actually) but I'd have no idea how to put this kind of thing together... really cool

The only things I would like to see would be colored output, specifically having the USE flags and package size in different colors, and having the options that the user invokes for pye extended to each emerge. What I mean is, for example, if I run "pye -upvD world" and emerge a few packages, one of the packages might say
" * Processed 347 info files: 1 errors; run with emerge --verbose to view errors."
even if I did pass "-v" to pye. But these are both small issues.

Thanks again for the script

[Pink]

Great script, discovered it today and for what I've used it for so far it's been great.

Big thanks from me

[dr_strange]

Thanks for the thanks:-)

[zsoltika]

Some improvemnet suggestion:

• - pye should have a "--version" param which displays pye's version.
  - If there are nothing to update like below it should exit except asking me to choose from nothing.

  Code: Select all

  #pye -uUDv system                                                  
  One moment...
  These are the packages that I would merge, in order:
  Calculating system dependencies  ...done!
  Total size of downloads: 0 kB
  Enter the numbers of packages you want to emerge or unmerge (in case of prune),comma-separated (you can also specify ranges, like 1,4-8,12 etc.). Alternatively, you can specify packages to be excluded from the list; for this, start your first item with an ! (e.g. !2,4,8). For all packages write 'a'. If you want to quit, press q:

Cheers from Budapest

[STEDevil]

hi,
imho, if you call just "pye", the script calls "emerge -Up world", and when you agree it does "emerge -U world", right?

If that´s right, it´s maybe the wrong way, ´cause -U can really break some things. (for me, it did once - so i looked for this option in your script before running it )
For me, i changed it to -uD, so i get the up- AND downgrades AND the dependencies.

No, pye anything just runs emerge -p anything; you should add any other flags yourself.

If that is the case, then perhaps the very first sentence in the very first post in this thread should be updated?

The following script basically gives you a numbered list of emerge -pU world output, and you can choose what to emerge by typing in the numbers simply...

Also, it would probably also be a good idea to update the entry for pye in this thread http://forums.gentoo.org/viewtopic.php?t=67849
(but hopefully the ebuild for this little script will be added soon anyway =).

BTW, you haven't been thinking of adding a functionallity like ehush ( http://forums.gentoo.org/viewtopic.php?t=76161 ) to your script? Could be added as an option when proceding with the actual emerges, eg "sh" or "s hush". That would be really greate.

Visszlát

[dr_strange]

Some improvemnet suggestion:

• - pye should have a "--version" param which displays pye's version.
  - If there are nothing to update like below it should exit except asking me to choose from nothing.

  Code: Select all

  #pye -uUDv system                                                  
  One moment...
  These are the packages that I would merge, in order:
  Calculating system dependencies  ...done!
  Total size of downloads: 0 kB
  Enter the numbers of packages you want to emerge or unmerge (in case of prune),comma-separated (you can also specify ranges, like 1,4-8,12 etc.). Alternatively, you can specify packages to be excluded from the list; for this, start your first item with an ! (e.g. !2,4,8). For all packages write 'a'. If you want to quit, press q:

Cheers from Budapest

Good ideas, thanks. I am back from vacation now, and will start thinking about implementing them and all the other suggestions in this thread.

Re-cheers from Budapest, see you on #gentoo-hu on freenode!

[dr_strange]

Okay, I've decided to unleash version 0.8.4 onto the unsuspecting masses... There are quite a few changes:

- the script now uses the getopt.getopt() function for option parsing
- you have the following options now:
-v or --version: prints version
-h or --help
-s or --sync: makes emerge rsync before anything else
-e or --emergeopts: here you list the opts you would give to emerge: for instance pye -e uDv world would be equivalent with calling emerge -puDv world
-i or --info <package> will print all info about <package> that can be found in /var/db/pkg
-c or --category: will list all packages available in a portage category; e.g. pye -c games-fps will give you the list of all FPS games in portage, to choose from
-p or --prune: lists all installed packages, and the user can choose which nes to unmerge

When the packages are listed, you will have the following possibilities:

Code: Select all

             * you can simply enter the numbers comma-separated, e.g. 1,3,5,6,8
             * you can also specify ranges: 2,4-7,9,11-16
             * "a" or "all" merges all packages in the list
             * "!" excludes the specified packages (and merges the rest): !4,6-8,13
             * "x" excludes _and_ masks them in /etc/portage/package.mask or ~/.pye/pye.mask
             * "q" quits pye

- Pye now boasts of a config file! It would reside in ~/.pye/pye.conf (you should manually create the dir and file for the time being), and can contain the following variables:

Code: Select all

EMERGEOPTS = uDv
SILENT = no
MASKFILE_ETC_PORTAGE_PACKAGEMASK = no

With EMERGEOPTS you can specify your standard options for emerge (pye -e overrides this).
If SILENT is yes, pye will not print the list of choices as shown above.
The maskfile is a new feature. When you specify the packages to be excluded with an x (e.g. x3,6-9,23), pye will enter these packages either into /etc/portage/package.mask or ~/.pye/pye.mask, depending on the variable MASKFILE_ETC_PORTAGE_PACKAGEMASK (if yes, the former, if no, the latter). Those packages will not show up when you dp pye world the next time.
At the end, pye now gives emerge times for the individual packages.

The script is available from here. Please test extensively and suggest improvements.

TODO:

- --purge option to totally unmerge a package
- --sqlite backend
- HUSH variable to oppress emerge output

Enjoy,

dr_strange

[dr_strange]

further TODOs:

TODO:

- --purge option to totally unmerge a package
- --sqlite backend
- HUSH variable to oppress emerge output
- beautify output of --info
- provide some alternative output for --info if package is not installed
- colorize package listing if --v in emergeopts

[nouse66]

i just grabbed your newest version and it's not running for me:

Code: Select all

aaron/ # pye world

Traceback (most recent call last):
  File "/usr/sbin/pye", line 179, in ?
    conf, defaultopts, silent, maskfile = parse_pyeconf(confpath)
  File "/usr/sbin/pye", line 48, in parse_pyeconf
    return confvar, optsvar, silentvar, maskfilevar
UnboundLocalError: local variable 'confvar' referenced before assignment

that's all i get, any idea what's wrong?

thanks,
aaron

[iverson0881]

Code: Select all

bash-2.05b# ./pye
Traceback (most recent call last):
  File "./pye", line 179, in ?
    conf, defaultopts, silent, maskfile = parse_pyeconf(confpath)
  File "./pye", line 48, in parse_pyeconf
    return confvar, optsvar, silentvar, maskfilevar
UnboundLocalError: local variable 'confvar' referenced before assignment

Yup having problems here too

NEVERMIND it works perfectly just make sure you create your ~/.pye/pye.conf as told

[dr_strange]

OK, another TODO: make an ebuild that installs pye + creates the conf file and empty mask file for the user

thx guys

[Chickpea]

I just working with this today and I get a couple of errors. If I use pye alone i get this error message:

Code: Select all

These are the packages that I would merge, in order:

Calculating dependencies   
emerge: there are no masked or unmasked ebuilds to satisfy "w".

!!! Error calculating dependencies. Please correct.
No packages to emerge.

if I try to use this option:

Code: Select all

  -p or --prune will list all installed packages, from which you can pick the ones
                        to unmerge (proceed carefully!)

Code: Select all

emergeopts: 
Traceback (most recent call last):
  File "/usr/bin/pye", line 242, in ?
    print "emergeopts: ", green(emergeopts)
NameError: name 'emergeopts' is not defined

What is going on?

[asph]

Code: Select all

bash-2.05b# ./pye
Traceback (most recent call last):
  File "./pye", line 179, in ?
    conf, defaultopts, silent, maskfile = parse_pyeconf(confpath)
  File "./pye", line 48, in parse_pyeconf
    return confvar, optsvar, silentvar, maskfilevar
UnboundLocalError: local variable 'confvar' referenced before assignment

Yup having problems here too

NEVERMIND it works perfectly just make sure you create your ~/.pye/pye.conf as told

same error here.. do we have to create the config file?
a suggestion may be that pye creates the config file with default settings if it's not here, or ask you the values (showing defaults) the first time you run it

[dr_strange]

yes, please, for the time being do create the conf file, I will amend the script and post a new version soon

[asph]

i created the config file, now i get another error:

emerge: there are no ebuilds to satisfy "w".

[Keffin]

i created the config file, now i get another error:

emerge: there are no ebuilds to satisfy "w".

I got that when I just ran "pye". I ran "pye world" and got past that, then got a ~/.pye/pye.mask doesn't exist error. I created that then it worked like a charm.

[stahlsau]

hi,
just wanna say: nice script, dr_strange, thank you!
I´m using this for a long time embedded into my own update-script, works very well. Recently, i had to do a "emerge -e world", which in former times kept failing at the moment i went to bed. With "pye -e world" and "step-emerge" there are no problems, and i see the other day which packages failed overnight. Very nice!
(and please excuse my language, it´s early in the morning )

[asph]

works fine now, steps i followed:

Code: Select all

# cd
# wget http://magenta.linuxforum.hu/downloads/pye
# chmod +x pye
# mkdir ~/.pye
# echo "EMERGEOPTS = uDv" > ~/.pye/pye.conf
# echo "SILENT = no" >> ~/.pye/pye.conf
# echo "MASKFILE_ETC_PORTAGE_PACKAGEMASK = no" >> ~/.pye/pye.conf
# touch ~/.pye/pye.mask
# ./pye world

thanks for pye, works fine

[dr_strange]

hi,
just wanna say: nice script, dr_strange, thank you!
I´m using this for a long time embedded into my own update-script, works very well. Recently, i had to do a "emerge -e world", which in former times kept failing at the moment i went to bed. With "pye -e world" and "step-emerge" there are no problems, and i see the other day which packages failed overnight. Very nice!
(and please excuse my language, it´s early in the morning )

glad you like it; please also test the new version, I'd like to catch as many bugs as possible

[Chickpea]

The only thing that I noticed and dont like with the current version (0.8.4) is that pye -p doesn't work and if you try pye prune, it lists the dups (much like it works now)

According to the help pye outputs pye -p or pye --prune should list all of the packages installed on your system giving you an opportunity to remove packages in the same numbering fashion.

It has worked like that in the past but the current script doesn't do that.

[dr_strange]

Version 0.8.4.1 is up

Two bugfixes:

- pye --prune now works (I hope )
- the script now creates a default ~/.pye directory and pye.conf and pye.mask files if they do not exist

[Chickpea]

Version 0.8.4.1 is up

Two bugfixes:

- pye --prune now works (I hope )
- the script now creates a default ~/.pye directory and pye.conf and pye.mask files if they do not exist

Just wanted to say that the bug fixes worked. I dont get the previous errors I received when running the script...especially the --prune option.

Great work!

[hepta_sean]

Version 0.8.4.1 is up

I already posted some problems a few months ago. They are still there with this version:

1.) You only record category and package, but not the version in the merge list. Therefore, you can only emerge the latest (unmasked) version with pye, not any previous version (this also happens in world updates, when an old slot is to be updated):

Code: Select all

sean@lois ~/bin $ pye-0.8.4.1 =gtk+-1.2.10-r11
One moment...

These are the packages that I would merge, in order:

Calculating dependencies  ...done!
0): [ebuild   R   ] x11-libs/gtk+-1.2.10-r11

You can now enter the numbers of packages you want to emerge/unmerge/exclude.
...
Enter your choices now: a

I'm about to emerge the following packages: x11-libs/gtk+
Is this correct? Choose an option: e(merge)/n(o)/q(uit): q

sean@lois ~/bin $ emerge -p x11-libs/gtk+

These are the packages that I would merge, in order:

Calculating dependencies ...done!
[ebuild   R   ] x11-libs/gtk+-2.4.13

2.) mergethis() gives false positives on remerges, because it only checks, if the latest version is installed:

Code: Select all

sean@lois ~/bin $ pye-0.8.4.1 gtk+
One moment...

These are the packages that I would merge, in order:

Calculating dependencies  ...done!
0): [ebuild   R   ] x11-libs/gtk+-2.4.13

You can now enter the numbers of packages you want to emerge/unmerge/exclude.
...
Enter your choices now: a

I'm about to emerge the following packages: x11-libs/gtk+
Is this correct? Choose an option: e(merge)/n(o)/q(uit): e

emerge: root access required.
This is digest of pye on Wed, 10 Nov 2004 00:05:47:

x11-libs/gtk+: Merged in 0.02 minutes
Check /var/log/emerge for details. Exiting.

3.) pye does not work with binary packages:

Code: Select all

sean@lois ~/bin $ pye-0.8.4.1 -e k gtk+
One moment...

These are the packages that I would merge, in order:

Calculating dependencies  ...done!
[binary   R   ] x11-libs/gtk+-2.4.13

No packages to emerge.

4.) Since you call emerge directly with each package, pye enters every package into the world file. Over time the world file will contain every single package of the system. Please use "--oneshot".

5.) With the changed way of giving options to emerge, I cannot give long options anymore (some options like "--oneshot" or "--newuse" do not have short variants, so this would be necessary).

Would be nice, if you could consider these issues.

[dr_strange]

Thank you all for the input and suggestions. My Gentoo PC is currently suffering from a serious malaise, but as soon as it gets repaired I will return to my script.

[theDreamer]

pye is a very nice tool - good work.

though, i have one problem with it:

the usage of the tool is manually emerging parts of the world/system list (mostly).
When emerging using -D you fill the world file with system sub packages you dont want manually add but that should be dependent on other packages.

for example:
the output of pye world results in both evolution and evolution-data-server.
when using pye without explicitly saying not emerging evolution-data-server (which is a sub-package of evolution) it will actually add it to the world file.
this is just a small example for expanding the world file to hold unnecessary packages.

This is standing against the logic of using the system dependencies and having minimal world file.
there's another nice tool that talks about what i've tried to explain:
http://forums.gentoo.org/viewtopic.php?t=142475

enough talking, i think that a small fix for this is required and also simple:
when emerging the packages you should add --oneshot.

what do you think?