Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Help: bash, sed, awk, anything for beginners
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Off the Wall
View previous topic :: View next topic  
Author Message
fluxbox
n00b
n00b


Joined: 09 Jun 2018
Posts: 0

PostPosted: Sat Jun 09, 2018 10:56 pm    Post subject: Help: bash, sed, awk, anything for beginners Reply with quote

Code:
alpha01first.foo  bravo02second.foo  echo05fifth.foo


The above should look like the below:

Code:
first01last.foo first02last.foo first05last.foo


01, 02, and 05 are suppose to be constants.

Therefore renaming in sequence will not work.

I want to know how to change just the first and last variables.

Prefer the answer be in bash, sed, awk, or anything for beginners.
Back to top
View user's profile Send private message
Dr.Willy
Guru
Guru


Joined: 15 Jul 2007
Posts: 457
Location: NRW, Germany

PostPosted: Sat Jun 09, 2018 11:18 pm    Post subject: Reply with quote

What?
Back to top
View user's profile Send private message
Naib
Watchman
Watchman


Joined: 21 May 2004
Posts: 5433
Location: Removed by Neddy

PostPosted: Sun Jun 10, 2018 12:20 am    Post subject: Reply with quote

can the extension be assumed to be constant? if so...

Code:
for f in *.foo; do mv "$f" "first${f//[^0-9]/}last.foo"; done

_________________
The best argument against democracy is a five-minute conversation with the average voter
Great Britain is a republic, with a hereditary president, while the United States is a monarchy with an elective king
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1605
Location: U.S.A.

PostPosted: Tue Jun 12, 2018 1:41 am    Post subject: Reply with quote

Ask Naib; he's good at this kind of thing, if you're okay with a quick 80% solution. You might have to clarify what you're trying to do, because your sample didn't provide adequate input and output and instead used pseudo-variables without any form of convention to actually denote them, which is why neither of these guys understood what you are trying to do.
_________________
patrix_neo wrote:
The human thought: I cannot win.
The ratbrain in me : I can only go forward and that's it.
Back to top
View user's profile Send private message
desultory
Administrator
Administrator


Joined: 04 Nov 2005
Posts: 9018

PostPosted: Tue Jun 12, 2018 3:47 am    Post subject: Reply with quote

Have you considered mmv (app-misc/mmv)?
Back to top
View user's profile Send private message
fluxbox
n00b
n00b


Joined: 09 Jun 2018
Posts: 0

PostPosted: Wed Jun 13, 2018 10:22 am    Post subject: Reply with quote

Bones McCracker wrote:
Ask Naib; he's good at this kind of thing, if you're okay with a quick 80% solution. You might have to clarify what you're trying to do, because your sample didn't provide adequate input and output and instead used pseudo-variables without any form of convention to actually denote them, which is why neither of these guys understood what you are trying to do.


I apologize for not being clear. I was trying to make it short.

I have some files that look like this:

artbook-page-07-scanned-by.jpg
artbook-page-19-scanned-by.jpg
artbook-page-23-scanned-by.jpg

Or this:

magazine page 07 scanned by.jpg
magazine page 19 scanned by.jpg
magazine page 23 scanned by.jpg

Or this:

vol.2.pg.07.scanned.by.jpg
vol.2.pg.19.scanned.by.jpg
vol.2.pg.23.scanned.by.jpg

I want them to look like this:

page_07_useful.jpg
page_19_useful.jpg
page_23_useful.jpg

I want to keep the original page number for reference.
I hope this is more clear.
Back to top
View user's profile Send private message
Naib
Watchman
Watchman


Joined: 21 May 2004
Posts: 5433
Location: Removed by Neddy

PostPosted: Wed Jun 13, 2018 11:43 am    Post subject: Reply with quote

uuur, that 2nd set of numbers breaks my bash 1liners :) mmv should be able to do this otherwise awk BUT even then some rules/limitations on the number of number sets would be required
_________________
The best argument against democracy is a five-minute conversation with the average voter
Great Britain is a republic, with a hereditary president, while the United States is a monarchy with an elective king
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1605
Location: U.S.A.

PostPosted: Thu Jun 14, 2018 2:23 am    Post subject: Reply with quote

fluxbox wrote:
Bones McCracker wrote:
Ask Naib; he's good at this kind of thing, if you're okay with a quick 80% solution. You might have to clarify what you're trying to do, because your sample didn't provide adequate input and output and instead used pseudo-variables without any form of convention to actually denote them, which is why neither of these guys understood what you are trying to do.


I apologize for not being clear. I was trying to make it short.

I have some files that look like this:

artbook-page-07-scanned-by.jpg
artbook-page-19-scanned-by.jpg
artbook-page-23-scanned-by.jpg

Or this:

magazine page 07 scanned by.jpg
magazine page 19 scanned by.jpg
magazine page 23 scanned by.jpg

Or this:

vol.2.pg.07.scanned.by.jpg
vol.2.pg.19.scanned.by.jpg
vol.2.pg.23.scanned.by.jpg

I want them to look like this:

page_07_useful.jpg
page_19_useful.jpg
page_23_useful.jpg

I want to keep the original page number for reference.
I hope this is more clear.

Two questions for further clarification:

Is the string "page" a literal (same always, for all), or is varable (parsed dynamically from the filenames)?

Is the string "useful" a literal (same always, for all); if not, where does it come from?
_________________
patrix_neo wrote:
The human thought: I cannot win.
The ratbrain in me : I can only go forward and that's it.
Back to top
View user's profile Send private message
Dr.Willy
Guru
Guru


Joined: 15 Jul 2007
Posts: 457
Location: NRW, Germany

PostPosted: Thu Jun 14, 2018 9:21 am    Post subject: Reply with quote

Bones McCracker wrote:
Two questions for further clarification:
Is the string "page" a literal (same always, for all), or is varable (parsed dynamically from the filenames)?
Is the string "useful" a literal (same always, for all); if not, where does it come from?

And here we have an example of a man whose job it is to explain to other people what they actually want.
I should take notes, but I find that most of the time I just don't have the patience.
Back to top
View user's profile Send private message
fluxbox
n00b
n00b


Joined: 09 Jun 2018
Posts: 0

PostPosted: Thu Jun 14, 2018 10:31 am    Post subject: Reply with quote

Bones McCracker wrote:

Two questions for further clarification:

Is the string "page" a literal (same always, for all), or is varable (parsed dynamically from the filenames)?

Is the string "useful" a literal (same always, for all); if not, where does it come from?


The "page_" is a literal. The "_useful" is a literal.
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1605
Location: U.S.A.

PostPosted: Thu Jun 14, 2018 11:05 pm    Post subject: Reply with quote

Then this is trivial. You can probably even do it with bash built-in string parsing features. Somebody explain it to him.

One more useful bit of info: are the files in one directory or scattered? Are there files in the same place(s) that don't match the pattern and which should be ignored?
_________________
patrix_neo wrote:
The human thought: I cannot win.
The ratbrain in me : I can only go forward and that's it.
Back to top
View user's profile Send private message
desultory
Administrator
Administrator


Joined: 04 Nov 2005
Posts: 9018

PostPosted: Fri Jun 15, 2018 2:49 am    Post subject: Reply with quote

Bones McCracker wrote:
Then this is trivial. You can probably even do it with bash built-in string parsing features.
Agreed, it would indeed be doable, though it would also be more work than necessary. I stick by my prior suggestion as the easiest way that satisfies the requirements mentioned so far, after all you can run it from within bash.
Bones McCracker wrote:
Somebody explain it to him.
Thanks, I needed a laugh and that delivered.
Bones McCracker wrote:
Are there files in the same place(s) that don't match the pattern and which should be ignored?
That could be something that would make mmv a decidedly less easy option, typically still usable, but not nearly as easily.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 17344

PostPosted: Fri Jun 15, 2018 3:22 am    Post subject: Reply with quote

I'm still not exactly clear on what the start and finish is. With the following files, it appears that there would be collisions in renaming them to page_*_useful.jpg:
Code:
$ cat list_of_files
artbook-page-07-scanned-by.jpg
artbook-page-19-scanned-by.jpg
artbook-page-23-scanned-by.jpg

magazine page 07 scanned by.jpg
magazine page 19 scanned by.jpg
magazine page 23 scanned by.jpg

vol.2.pg.07.scanned.by.jpg
vol.2.pg.19.scanned.by.jpg
vol.2.pg.23.scanned.by.jpg
That said, sometimes I like the "get it done" solution (I've never used mmv and don't have it installed, so...):
Code:
$ sed -n 's/\(.*\)\([0-9][0-9]\)\(.scanned.by.jpg\)/mv \1\2\3 page_\2_useful.jpg/p' list_of_files
mv artbook-page-07-scanned-by.jpg page_07_useful.jpg
mv artbook-page-19-scanned-by.jpg page_19_useful.jpg
mv artbook-page-23-scanned-by.jpg page_23_useful.jpg
mv magazine page 07 scanned by.jpg page_07_useful.jpg
mv magazine page 19 scanned by.jpg page_19_useful.jpg
mv magazine page 23 scanned by.jpg page_23_useful.jpg
mv vol.2.pg.07.scanned.by.jpg page_07_useful.jpg
mv vol.2.pg.19.scanned.by.jpg page_19_useful.jpg
mv vol.2.pg.23.scanned.by.jpg page_23_useful.jpg
The files with spaces would need to be handled separately, but that's a starting point.
_________________
The whole system has to go. The modern criminal justice system is incompatible with Neuroscience. --Sapolsky
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6062
Location: Room 101

PostPosted: Wed Jun 20, 2018 6:19 pm    Post subject: Reply with quote

fluxbox ...

given the various examples, I think you're asking a "teach a man to fish" question, and not simply looking to solve this one particular problem. I do this kind of thing all the time (interactively), and it's worth a little effort to get familiar with what's involved ... and the various tools you might choose to get familiar with. I've never used app-misc/mmv but at a quick look it seems like an attempt to emulate zsh's "zmv" (given bash doesn't do this sort of thing natively), and as we've seen from Naib and pjp's examples, shell or sed (or awk) could be used for this, but these generally require a good grasp of syntax, quoting rules, etc, to get started. So, while any one of those may be worth the effort, I'm going to make the case that it's the shell you're using that is the problem, and suggest you take a look at app-shells/zsh ... if only because it makes doing this sort of thing (and many others) easy (both to learn, and to apply).

'zmv' is made available in the shell by 'autoload -Uz zmv', normally you'd set this in ~/.zshrc.

fluxbox wrote:
artbook-page-07-scanned-by.jpg
artbook-page-19-scanned-by.jpg
artbook-page-23-scanned-by.jpg

Code:
% cd $(mktemp -d)
% touch artbook-page-{07,19,23}-scanned-by.jpg
% zmv -nv '(artbook-page-)([0-9][0-9])(-scanned-by).jpg' 'page_$2_useful.jpg'
mv -- artbook-page-07-scanned-by.jpg page_07_useful.jpg
mv -- artbook-page-19-scanned-by.jpg page_19_useful.jpg
mv -- artbook-page-23-scanned-by.jpg page_23_useful.jpg

To explain what we're doing here: the '-n' switch is "no execution: print what would happen, but don't do it", and the '-v' is "verbose". Then we're separating the input into sections, '($1)($2)($3)', any part of these can be reused later, but we only want '$2' (the page number). We could have done the following, and got the same results:

Code:
% zmv -nv '(artbook-)(page)-([0-9][0-9])(-scanned-by).jpg' '$2_$3_useful.jpg'
mv -- artbook-page-07-scanned-by.jpg page_07_useful.jpg
mv -- artbook-page-19-scanned-by.jpg page_19_useful.jpg
mv -- artbook-page-23-scanned-by.jpg page_23_useful.jpg

.... so, how we deal with problem depends on how best to slice & dice the input. Obviously, if you want the 'mv' to be applied you remove the '-n' ("no execution") switch.

fluxbox wrote:
magazine page 07 scanned by.jpg
magazine page 19 scanned by.jpg
magazine page 23 scanned by.jpg

This is a similar problem, and pretty much the same syntax:

Code:
% touch magazine\ page\ {07,19,23}\ scanned\ by.jpg
% zmv -nv '(magazine) (page) ([0-9][0-9]) (scanned by).jpg' '$2_$3_useful.jpg'
mv -- 'magazine page 07 scanned by.jpg' page_07_useful.jpg
mv -- 'magazine page 19 scanned by.jpg' page_19_useful.jpg
mv -- 'magazine page 23 scanned by.jpg' page_23_useful.jpg

Bonus titbit, you can get rid of those pesky spaces (recursively) with the following:

Code:
% zmv -Q '(**/)* *(.D)' '$f:gs/ /_'

... that syntax is a little more complex as we're using the "glob qualifiers" '.D', '$f' (which is the name of each file stored in a variable), and substitution ... but from what you've so far been exposed to you should be able to get the general idea of what's happening: the file pattern ''(**/)* *', any "file" with a space in its name in this directory, or in any directory within this directory, is being mv'ed to that same filename only with space replaced with underscore.

fluxbox wrote:
vol.2.pg.07.scanned.by.jpg
vol.2.pg.19.scanned.by.jpg
vol.2.pg.23.scanned.by.jpg

A similar problem to the others, only sliced & diced in a slightly different way:

Code:
% touch vol.2.pg.{07,19,23}.scanned.by.jpg
% zmv -nv '(vol.2.pg.)([0-9][0-9])(.scanned.by).jpg 'page_$2_useful.jpg'
mv -- vol.2.pg.07.scanned.by.jpg page_07_useful.jpg
mv -- vol.2.pg.19.scanned.by.jpg page_19_useful.jpg
mv -- vol.2.pg.23.scanned.by.jpg page_23_useful.jpg

The one problem with pjp's example is that he ends up with unique files with the same name, and so if the files are in the same directory they would clobber those created by the prior match (ooops!). In such cases you can insert a unique, or random, identifier:

Code:
% zmv -nv '*([0-9][0-9])*.jpg' 'page_$1_useful-$RANDOM.jpg'
mv -- artbook-page-07-scanned-by.jpg page_07_useful-27427.jpg
mv -- artbook-page-19-scanned-by.jpg page_19_useful-3724.jpg
mv -- artbook-page-23-scanned-by.jpg page_23_useful-19546.jpg
mv -- 'magazine page 07 scanned by.jpg' page_07_useful-17859.jpg
mv -- 'magazine page 19 scanned by.jpg' page_19_useful-8579.jpg
mv -- 'magazine page 23 scanned by.jpg' page_23_useful-1459.jpg
mv -- vol.2.pg.07.scanned.by.jpg page_07_useful-28014.jpg
mv -- vol.2.pg.19.scanned.by.jpg page_19_useful-3460.jpg
mv -- vol.2.pg.23.scanned.by.jpg page_23_useful-26201.jpg

.... or you could make a numerical increment with padding to three numerical spaces (assuming there are less than 999 files):

Code:
% i=0 ; zmv -nv '*([0-9][0-9])*.jpg' 'page_$1_useful-${(l:3::0:)$((++i))}.jpg'
mv -- artbook-page-07-scanned-by.jpg page_07_useful-001.jpg
mv -- artbook-page-19-scanned-by.jpg page_19_useful-002.jpg
mv -- artbook-page-23-scanned-by.jpg page_23_useful-003.jpg
mv -- 'magazine page 07 scanned by.jpg' page_07_useful-004.jpg
mv -- 'magazine page 19 scanned by.jpg' page_19_useful-005.jpg
mv -- 'magazine page 23 scanned by.jpg' page_23_useful-006.jpg
mv -- vol.2.pg.07.scanned.by.jpg page_07_useful-007.jpg
mv -- vol.2.pg.19.scanned.by.jpg page_19_useful-008.jpg
mv -- vol.2.pg.23.scanned.by.jpg page_23_useful-009.jpg

Notice that in those two examples we've changed the file match to cope with all the examples you'd provided, we're using globs rather than the filenames.

Homework: from all of the above you should be to work out how to do such a rename matching all of your examples but recursively ... ask if you're not sure.

Hopefully that gives you some idea of how to get started, or at least something to consider along with the other offerings ... zsh is great for interactive use, and you wouldn't notice the difference if switching (most syntax is the same as with bash), however it makes doing simple things like file renaming a doddle.

Happy fishing ... khay
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1605
Location: U.S.A.

PostPosted: Thu Jun 21, 2018 2:11 am    Post subject: Reply with quote

Who is "we"? Like the Royal "We"?
_________________
patrix_neo wrote:
The human thought: I cannot win.
The ratbrain in me : I can only go forward and that's it.
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 5163

PostPosted: Thu Jun 21, 2018 2:28 am    Post subject: Reply with quote

Trying to golf this down with coreutils is fun and all but sometimes it's easier to just hit it with a hammer.

Install dev-perl/rename, then use that to do 's/.+?(\d{2}).+?\.jpg/first_$1_last.jpg/' or whatever
Back to top
View user's profile Send private message
desultory
Administrator
Administrator


Joined: 04 Nov 2005
Posts: 9018

PostPosted: Thu Jun 21, 2018 3:58 am    Post subject: Reply with quote

Bones McCracker wrote:
Who is "we"? Like the Royal "We"?
The querent and the interlocutor, or if you prefer to avoid useful terms deemed obsolescent the engaged interlocutors. It is a common usage.
Back to top
View user's profile Send private message
John-Boy
Guru
Guru


Joined: 23 Jun 2004
Posts: 439
Location: Desperately seeking moksha in all the wrong places

PostPosted: Thu Jun 21, 2018 5:01 am    Post subject: Reply with quote

vbscript will sort that out.
_________________
Like the Roman, I seem to see "the River Tiber foaming with much blood"
Back to top
View user's profile Send private message
Naib
Watchman
Watchman


Joined: 21 May 2004
Posts: 5433
Location: Removed by Neddy

PostPosted: Thu Jun 21, 2018 8:37 am    Post subject: Reply with quote

John-Boy wrote:
vbscript will sort that out.
you shut your mouth!
_________________
The best argument against democracy is a five-minute conversation with the average voter
Great Britain is a republic, with a hereditary president, while the United States is a monarchy with an elective king
Back to top
View user's profile Send private message
patrix_neo
Guru
Guru


Joined: 08 Jan 2004
Posts: 445
Location: The Maldives

PostPosted: Thu Jun 21, 2018 2:32 pm    Post subject: Reply with quote

Why not just rename all files having 00-99?
Problem one : Separators for filenames, like: '-', '.', '<space>', etc.
Split the filename not having numbers 00 - 99 or A-Z,a-z as a separator and then clog it to one string.
WARNING When '.' is a separator, make it not being the last dot in the string of the filename.

Problem two: Finding out which file belongs to what album:
Then compare the filenames before hitting 00 - 99 and shuffle respective string-domain to one album.
Even better, remove the 00 - 99 from the string of the filename since that one can occur otherwise before or after the 'real' number.

Presto aligretto, done!
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6062
Location: Room 101

PostPosted: Thu Jun 21, 2018 4:22 pm    Post subject: Reply with quote

Ant P. wrote:
Trying to golf this down with coreutils is fun and all but sometimes it's easier to just hit it with a hammer. Install dev-perl/rename, then use that to do 's/.+?(\d{2}).+?\.jpg/first_$1_last.jpg/' or whatever

Ant P. ... when all you have is a hammer, everything looks like a nail :) ... rename.pl is fairly good for simple oldname,newname renaming but you loose being able to use the power of shell, and as far as I know doesn't have the ability to test what would be done prior to making those changes.

So, say you want to rename based on the directory they are located in:

Code:
% cd $(mktemp -d)
% mkdir foo bar baz
% touch foo/artbook-page-{07,19,23}-scanned-by.jpg bar/magazine\ page\ {07,19,23}\ scanned\ by.jpg baz/vol.2.pg.{07,19,23}.scanned.by.jpg
% zmv -nv '(**/)*([0-9][0-9])*.jpg' '$1$1:t_$2.jpg'
mv -- 'bar/magazine page 07 scanned by.jpg' bar/bar_07.jpg
mv -- 'bar/magazine page 19 scanned by.jpg' bar/bar_19.jpg
mv -- 'bar/magazine page 23 scanned by.jpg' bar/bar_23.jpg
mv -- baz/vol.2.pg.07.scanned.by.jpg baz/baz_07.jpg
mv -- baz/vol.2.pg.19.scanned.by.jpg baz/baz_19.jpg
mv -- baz/vol.2.pg.23.scanned.by.jpg baz/baz_23.jpg
mv -- foo/artbook-page-07-scanned-by.jpg foo/foo_07.jpg
mv -- foo/artbook-page-19-scanned-by.jpg foo/foo_19.jpg
mv -- foo/artbook-page-23-scanned-by.jpg foo/foo_23.jpg

Not only can we get information from the environment, we can use the shell to match certain criteria, ie, only zero length files (the critieria doesn't matter, there are many we might match, it just suites this examples because all our files are zero length):

Code:
% zmv -Qnv '(**/)*([0-9][0-9])*.jpg*(.L0)' '$1$1:t_$2.jpg'
mv -- 'bar/magazine page 07 scanned by.jpg' bar/bar_07.jpg
mv -- 'bar/magazine page 19 scanned by.jpg' bar/bar_19.jpg
mv -- 'bar/magazine page 23 scanned by.jpg' bar/bar_23.jpg
mv -- baz/vol.2.pg.07.scanned.by.jpg baz/baz_07.jpg
mv -- baz/vol.2.pg.19.scanned.by.jpg baz/baz_19.jpg
mv -- baz/vol.2.pg.23.scanned.by.jpg baz/baz_23.jpg
mv -- foo/artbook-page-07-scanned-by.jpg foo/foo_07.jpg
mv -- foo/artbook-page-19-scanned-by.jpg foo/foo_19.jpg
mv -- foo/artbook-page-23-scanned-by.jpg foo/foo_23.jpg

I could go on, but it should be clear that a lot more is possible than with rename.pl

best ... khay
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Off the Wall All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum