View previous topic :: View next topic |
Author |
Message |
_______0 Guru
Joined: 15 Oct 2012 Posts: 521
|
Posted: Sun Jan 06, 2013 8:59 pm Post subject: how to transform a column list into usable command line? |
|
|
hi,
I want to generate a list of of packages (or any files for that matter) with eix. Currently is possible but if I want to use that list in command line is not possible because it's a column and command line can't process columns.
What I am trying to do is with eix to list packages from a certain category that are NOT installed, but shockingly has options for everything except for 'NOT installed' :/
This list is needed because some packages I want to discard but most not, so eix alone can't do it. I was passing eix to emerge like this emerge `eix fumanchu`.
Anyone know how to transform a list like this:
to this??
My current crudish solution is to do eix -I -C >foo then eix -C > bar and diff foo bar > final, and at last manually tweak final and pass it to emerge with `cat final`. I know, is not the smartest way to achieve this.
thnks |
|
Back to top |
|
|
kimmie Guru
Joined: 08 Sep 2004 Posts: 531 Location: Australia
|
Posted: Sun Jan 06, 2013 11:01 pm Post subject: |
|
|
When you use bash command substitution $(..) (or even the old shell backticks `..`), all whitespace is coalesced to single spaces. Newlines are treated as whitespace, so they disappear.
So:
Code: | $ equery -q l -f x11-drivers
x11-drivers/nvidia-drivers-310.19
x11-drivers/xf86-input-evdev-2.7.3
x11-drivers/xf86-input-synaptics-1.6.2-r1
$ echo $(equery -q l -f x11-drivers)
x11-drivers/nvidia-drivers-310.19 x11-drivers/xf86-input-evdev-2.7.3 x11-drivers/xf86-input-synaptics-1.6.2-r1
|
If your output contains non-newline whitespace, you'll have to be careful about quoting. |
|
Back to top |
|
|
_______0 Guru
Joined: 15 Oct 2012 Posts: 521
|
Posted: Mon Jan 07, 2013 8:39 am Post subject: |
|
|
amazing it works!!
thanks a lot.
just one more thing. Why can't command line process columns? Is more natural than serialized items. For example I do ls on a directory with ogg files, then I highlight the list and it becomes a column:
Code: |
1.mp3
2.mp3
3.mp3
... |
there's no way to paste columns to a command on the terminal. If not current shell design is fundamentally flawed. Serialized items, visually, are a pain to parse (humanly), and wastes a lot of time. |
|
Back to top |
|
|
kimmie Guru
Joined: 08 Sep 2004 Posts: 531 Location: Australia
|
Posted: Mon Jan 07, 2013 11:23 am Post subject: |
|
|
I see the argument pasting issue as the terminal, the shell and the user all having different views of a stream of text, more than as a flaw in the shell. The terminal is 2D, the shell is 1D, and only you can say whether the end of your selection should terminate the command when pasted....
I think some xterm variants may translate newlines into spaces (I'm not sure which ones, and it probably depends on a using ctrl or alt-paste). Or, app-misc/screen will virtualise your terminal session, and has a vi-like copy mode where you can scroll and search terminal output and also copy and paste. It will do that sort of translation. screen is really handy if you do lots of terminal work, especially if your session is on a remote system. Bit of a learning curve though.
From the shell end, I find myself using cat or xargs fairly often. xargs takes arguments from stdin, and builds a command, so:
Code: | $ mkdir test
$ cd test
$ touch a b abc "def ghi"
$ ls -1
a
abc
b
def ghi
$ cat | xargs -d '\n' du -sh
a <--- cut and paste above output
abc
b
def ghi
^D <--- hit ^D to terminate input to cat
0 a
0 abc
0 b
0 def ghi
|
The -d option tells xargs that each line is an argument, so that will handle the file "def ghi" correctly.
You can use a shell alias to save typing:
Code: | $ alias pastie="cat | xargs -d '\n'"
$ pastie ls -lh
a <--- copy and paste
abc
b
def ghi
^D <--- hit ^D
-rw-r--r-- 1 gid gid 0 Jan 7 21:47 a
-rw-r--r-- 1 gid gid 0 Jan 7 21:47 abc
-rw-r--r-- 1 gid gid 0 Jan 7 21:47 b
-rw-r--r-- 1 gid gid 0 Jan 7 21:47 def ghi
|
Sometimes, just using $(cat) is handier:
Code: | $ for x in $(cat) ; do echo "Processing $x ..." ; done
a <--- paste
abc
b
def ghi
^D <-- hit ^D
Processing a ...
Processing abc ...
Processing b ...
Processing def ...
Processing ghi ... |
Since that won't handle "def ghi" correctly, sometimes you need:
Code: | $ while read x ; do echo "Processing $x ..." ; done
a
abc
b
def ghi
^D
Processing a ...
Processing abc ...
Processing b ...
Processing def ghi ...
|
|
|
Back to top |
|
|
mv Watchman
Joined: 20 Apr 2005 Posts: 6780
|
Posted: Mon Jan 07, 2013 2:01 pm Post subject: |
|
|
You should really get used to read manuals. Of course, eix is able to print uninstalled packages (e.g. ... --and --not --installed), and it can produce it in almost any format you want; it could even print you uninstalled slots if you dive into details of the documentation, although this would be a bit more tricky and require to build your custom FORMAT string. |
|
Back to top |
|
|
_______0 Guru
Joined: 15 Oct 2012 Posts: 521
|
Posted: Mon Jan 07, 2013 4:22 pm Post subject: |
|
|
it was tricky though, looks like the option order matters here, by crazy recombination of option I discovered something that appears to yield the proper results:
Code: | eix --only-names -C fonts --not -I |
by the way, I dunno wtf this means:
Code: | EXPRESSION ::= [ --not | -! ] BRACE_OR_TEST |
EXPRESSION [ --and| -a ] EXPRESSION |
EXPRESSION [ --or | -o ] EXPRESSION
BRACE_OR_TEST ::= --open|-( EXPRESSION --close|-) |
TEST_WITH_OPTIONS
TEST_WITH_OPTIONS ::= [TEST_OPTIONS] [PATTERN]
|
why the hell are brackets, parethesis and pipes in there?? are they usable or what? dis crazy!! |
|
Back to top |
|
|
mv Watchman
Joined: 20 Apr 2005 Posts: 6780
|
Posted: Mon Jan 07, 2013 4:46 pm Post subject: |
|
|
_______0 wrote: | it was tricky though, looks like the option order matters here |
Of course. It is a list of expressions. In a shell the list of expressions also differs from
Quote: | By the way, I dunno wtf this means: |
It is a grammar written in (a somewhat sloppy variant of) Backus-Naur Form (BNR) which is a wide-spread form (and essentiall the only simple form) to describe the syntax of complex expressions, see e.g. the Wiki entry on BNR. You will meet this in many manpages, e.g. in |
|
Back to top |
|
|
py-ro Veteran
Joined: 24 Sep 2002 Posts: 1734 Location: Velbert
|
Posted: Mon Jan 07, 2013 8:49 pm Post subject: |
|
|
If you use bash, try CTRL-X CTRL-E, your favorite editor will open and you can paste your arguments easily.
[EDIT]Swapped keys[/EDIT] |
|
Back to top |
|
|
_______0 Guru
Joined: 15 Oct 2012 Posts: 521
|
Posted: Tue Jan 08, 2013 12:42 pm Post subject: |
|
|
py-ro wrote: | If you use bash, try CTRL-X CTRL-E, your favorite editor will open and you can paste your arguments easily.
[EDIT]Swapped keys[/EDIT] |
uh???
k, found out a simpler way to transform columnated list to serial:
Code: | echo $(echo '<paste huge column paste>a
cmdsubst>b
cmdsubst>c
cmdsubst>d
cmdsubst>e
cmdsubst>f
cmdsubst>g
cmdsubst>h') |
Compared with cat | xargs ... is simpler since you can save two extra commands, | and xargs with complicated escaped option. But nice to know different ways of achieving this.
kimmie wrote: | The terminal is 2D, the shell is 1D |
^^^ that made me lol!! poor shell
What about the reverse? how to transform a serial list into a column?? IS IT TOO MUCH ASKING THIS NOW??? |
|
Back to top |
|
|
John R. Graham Administrator
Joined: 08 Mar 2005 Posts: 10654 Location: Somewhere over Atlanta, Georgia
|
Posted: Tue Jan 08, 2013 3:07 pm Post subject: |
|
|
TIMTOWTDI. Here're just a few:
From the command line: Code: | ~ $ awk 'BEGIN { for (i=1; i <= ARGC; i++) print ARGV[i] }' The quick brown fox jumped over the lazy dog.
The
quick
brown
fox
jumped
over
the
lazy
dog. | Or, from a pipe: Code: | ~ $ echo "The quick brown fox jumped over the lazy dog." | awk '{ for (i=1; i <= NF; i++) print $i }'
The
quick
brown
fox
jumped
over
the
lazy
dog. | Or, more simply (but more abstrusely), with sed: Code: | ~ $ echo "The quick brown fox jumped over the lazy dog." | sed 's/\s/\n/g'
The
quick
brown
fox
jumped
over
the
lazy
dog. | There are more but I'm going to quit now.
- John _________________ I can confirm that I have received between 0 and 499 National Security Letters. |
|
Back to top |
|
|
papu l33t
Joined: 25 Jan 2008 Posts: 729 Location: Sota algun pi o alzina...
|
Posted: Mon Jan 14, 2013 1:29 pm Post subject: |
|
|
hi, i like pass to file a list of filter atoms bases/atoms versions , with all dependencies of a previous just installed package.
something like this:
Code: | enric@Ordinador ~ $ sudo emerge -pv kscd
These are the packages that would be merged, in order:
Calculating dependencies ... done!
[ebuild N ] media-libs/libdiscid-0.3.0 USE="-static-libs" 0 kB
[ebuild N ] media-libs/musicbrainz-3.0.3:3 USE="{-test}" 0 kB
[ebuild N ] kde-base/libkcompactdisc-4.9.5:4 USE="alsa (-aqua) -debug" 0 kB
[ebuild N ] kde-base/kscd-4.9.5:4 USE="(-aqua) -debug" 0 kB
Total: 4 packages (4 new), Size of downloads: 0 kB |
Code: |
enric@Ordinador ~ $ sudo emerge -pv kscd|cut -d']' -f2 |cut -d' ' -f2 > uninstall_kscd
enric@Ordinador ~ $ cat uninstall_kscd
are
dependencies
media-libs/libdiscid-0.3.0
media-libs/musicbrainz-3.0.3:3
kde-base/libkcompactdisc-4.9.5:4
kde-base/kscd-4.9.5:4
4 |
but in an optimized and clean manner, i don't know how to use text filters like sed o awk to do this.
and then uninstall this packages with this:
Code: | enric@Ordinador ~ $ sudo emerge -Cp `< uninstall_kscd`
This action can remove important packages! In order to be safer, use
* `emerge -pv --depclean <atom>` to check for reverse dependencies before
* removing packages.
>>> These are the packages that would be unmerged:
media-libs/libdiscid
selected: 0.3.0
protected: none
omitted: none
media-libs/musicbrainz
selected: 3.0.3
protected: none
omitted: none
kde-base/libkcompactdisc
selected: 4.9.5
protected: none
omitted: none
kde-base/kscd
selected: 4.9.5
protected: none
omitted: none
All selected packages: kde-base/kscd-4.9.5 kde-base/libkcompactdisc-4.9.5 media-libs/libdiscid-0.3.0 media-libs/musicbrainz-3.0.3
>>> 'Selected' packages are slated for removal.
>>> 'Protected' and 'omitted' packages will not be removed. |
thanks...a lot! ad1 _________________ --so ~amd64 & openrc --cpu 7700 non-x --ram 2x16GB --gpu RX 470 |
|
Back to top |
|
|
mv Watchman
Joined: 20 Apr 2005 Posts: 6780
|
Posted: Mon Jan 14, 2013 2:07 pm Post subject: |
|
|
The eix --pipe parsing algorithm is quite smart, so you can simply use eix:
Code: | sudo emerge -pv kscd | eix '-|*' --format '<markedversions:EQNAMEVERSION>' |
|
|
Back to top |
|
|
papu l33t
Joined: 25 Jan 2008 Posts: 729 Location: Sota algun pi o alzina...
|
Posted: Mon Jan 14, 2013 2:48 pm Post subject: |
|
|
mv wrote: | The eix --pipe parsing algorithm is quite smart, so you can simply use eix:
Code: | sudo emerge -pv kscd | eix '-|*' --format '<markedversions:EQNAMEVERSION>' |
|
thanks, but this only read first package of the list:
Code: |
enric@Ordinador ~ $ sudo emerge -pv kscd | eix '-|*' --format '<markedversions:EQNAMEVERSION>'
=media-libs/libdiscid-0.3.0 |
Code: | enric@Ordinador ~ $ sudo emerge -pv kscd
These are the packages that would be merged, in order:
Calculating dependencies ... done!
[ebuild N ] media-libs/libdiscid-0.3.0 USE="-static-libs" 0 kB
[ebuild N ] media-libs/musicbrainz-3.0.3:3 USE="{-test}" 0 kB
[ebuild N ] kde-base/libkcompactdisc-4.9.5:4 USE="alsa (-aqua) -debug" 0 kB
[ebuild N ] kde-base/kscd-4.9.5:4 USE="(-aqua) -debug" 0 kB
Total: 4 packages (4 new), Size of downloads: 0 kB |
_________________ --so ~amd64 & openrc --cpu 7700 non-x --ram 2x16GB --gpu RX 470 |
|
Back to top |
|
|
mv Watchman
Joined: 20 Apr 2005 Posts: 6780
|
Posted: Mon Jan 14, 2013 3:54 pm Post subject: |
|
|
papu wrote: | thanks, but this only read first package of the list: |
You are right, this is due to changed output of portage: The appended slot in the versionnumber is confusing eix. I guess eix needs to be fixed. |
|
Back to top |
|
|
papu l33t
Joined: 25 Jan 2008 Posts: 729 Location: Sota algun pi o alzina...
|
Posted: Mon Jan 14, 2013 5:36 pm Post subject: |
|
|
mv wrote: | papu wrote: | thanks, but this only read first package of the list: |
You are right, this is due to changed output of portage: The appended slot in the versionnumber is confusing eix. I guess eix needs to be fixed. |
,
i ask you, ¿what do you recomended to me to start learning filter text, sed or awk or... ? i don't need advanced think, only to take some control of this stuff on command line
thanks, a lot... _________________ --so ~amd64 & openrc --cpu 7700 non-x --ram 2x16GB --gpu RX 470 |
|
Back to top |
|
|
mv Watchman
Joined: 20 Apr 2005 Posts: 6780
|
Posted: Mon Jan 14, 2013 6:18 pm Post subject: |
|
|
In general, you can use any of sed, awk, perl (or even python or ruby) for simple modifications of such a type. However, dedicated tools like eix are preferrable if they are suited to your task, because you can expect that they will be adopted to changes of portage. For instance, although it is unfortunate, that just now it turned out that eix has problems with the new portage output, this will be fixed in the next eix release (>=0.28.1 which will be soon in the tree - the fix is already in git): For your own scripts you cannot expect that they will fix themselves automatically when portage changes its output format in a new version...
Edit: Until the new version is available to you, you can use something like
Code: | sudo emerge -pv kscd | sed -n -e 's!^.* \([^ ]*/[^ ]*\).*$!=\1!p' |
|
|
Back to top |
|
|
papu l33t
Joined: 25 Jan 2008 Posts: 729 Location: Sota algun pi o alzina...
|
Posted: Tue Jan 15, 2013 12:17 am Post subject: |
|
|
mv wrote: | In general, you can use any of sed, awk, perl (or even python or ruby) for simple modifications of such a type. However, dedicated tools like eix are preferrable if they are suited to your task, because you can expect that they will be adopted to changes of portage. For instance, although it is unfortunate, that just now it turned out that eix has problems with the new portage output, this will be fixed in the next eix release (>=0.28.1 which will be soon in the tree - the fix is already in git): For your own scripts you cannot expect that they will fix themselves automatically when portage changes its output format in a new version...
Edit: Until the new version is available to you, you can use something like
Code: | sudo emerge -pv kscd | sed -n -e 's!^.* \([^ ]*/[^ ]*\).*$!=\1!p' |
|
ok, thank you very much my friend,it is i was looking for , i am going to learn about sed.
with the today update, the eix way works properly too.
_________________ --so ~amd64 & openrc --cpu 7700 non-x --ram 2x16GB --gpu RX 470 |
|
Back to top |
|
|
paulj Guru
Joined: 30 Sep 2004 Posts: 525 Location: Wales, UK
|
Posted: Fri Jan 18, 2013 6:16 am Post subject: |
|
|
_______0 wrote: |
by the way, I dunno wtf this means:
Code: | EXPRESSION ::= [ --not | -! ] BRACE_OR_TEST |
EXPRESSION [ --and| -a ] EXPRESSION |
EXPRESSION [ --or | -o ] EXPRESSION
BRACE_OR_TEST ::= --open|-( EXPRESSION --close|-) |
TEST_WITH_OPTIONS
TEST_WITH_OPTIONS ::= [TEST_OPTIONS] [PATTERN]
|
why the hell are brackets, parethesis and pipes in there?? are they usable or what? dis crazy!! |
It's a grammar definition. The brackets, parenthesis and pipes are read as follows: the brackets include an either/or part, so means you can use --not or -!. The pipe symbol is the OR symbol in this case. I would read the EXPRESSION part as follows: Where EXPRESSION can be used, I can also use either --not or -! followed by a BRACE_OR_TEST OR EXPRESSION followed by --and or -a followed by an EXPRESSION, OR an EXPRESSION followed by --or or -o followed by EXPRESSION. It is a of course recursive in nature, so you can nest the terms. For example: produces a long list of 143 matches. I want to find packages containing gnome and extras: Code: | eix gnome --and extras | This lists three packages. If I didn't want packages containing python: Code: | eix gnome --and extras --and --not python | As expected based on the previous result, this lists two packages.
I can recommend the Dragon Compiler Book if you want to learn more about grammar and syntax definition! |
|
Back to top |
|
|
|