| View previous topic :: View next topic |
| Author |
Message |
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 4849
Location: Somewhere over Atlanta, Georgia
|
 Posted: Wed Jan 25, 2012 4:10 pm Post subject: |
 |
|
How many Gentoo scripters does it take to screw in a light bulb? 
I love puzzles like this. I was about to reply before RazielFMX beat me to it that pure bash didn't have the pattern matching capability to do the whole thing, lacking rich regular expression support. It's difficult in sed and AWK as well but easy in Perl. The complexity comes in because of preserving the single '.' that separates the name from the extension but converting all others to '_'. (Dark Foo, do I have that correct?)
RazielFMX's solution is close but doesn't handle all the special cases (e.g., file name "a.b.c.d").
This is scripting, so my tendency is to use the best tool for each part. For collecting and operating on a list of files, it's hard to beat bash's mechanisms and Perl is unbeatable for pattern matching, so, with the heavy lifting done by a little Perl: | test.pl: | chomp;
s/\s/_/g;
if (($Name, $Ext) = m/^(.*)\.(.*)/) {
$Name =~ s/\./_/g;
print "$Name.$Ext\n";
} else {
print "$_\n";
}
|
then borrowing from Naib's original one liner get's you this | Code: | | for f in *; do mv "$f" $(echo $f | perl -n test.pl); done |
There are endless arguments on efficiency and style, but on the argument of capability, I couldn't come up with a pure bash implementation that would do it at all. Have I missed something?
- John
_________________
Yoda: "Intentionally left blank, this space is."
Last edited by John R. Graham on Wed Jan 25, 2012 6:19 pm; edited 1 time in total |
|
| Back to top |
|
 |
RazielFMX
l33t
Joined: 23 Apr 2005
Posts: 672
Location: NY, USA
|
| Posted: Wed Jan 25, 2012 6:12 pm Post subject: |
|
|
Ah. I missed the '.' in the original request and thought it was a random space before an ill placed end of sentence. My bad. Revised, pure Perl implementation below, with special consideration for hidden files. Also note this code will not work right if you have mixed directories and files in the target. If you need that, I can modify to be directory friendly.
What you have:
| Code: |
$ ls -1a
.
..
a b
a_b_c_d
a.b.c.d
a b. c d.txt
.foo
.foo bar
.foo bar.txt
.foo_foo.txt
.foo.foo.txt
.foo.txt
test.txt
|
When you run:
| Code: |
$ ~/tmp/stripSpaces.pl
[WARN] .foo_foo.txt already exists; skipping rename of '.foo.foo.txt' to '.foo_foo.txt'
[INFO] Renaming 'a b. c d.txt' to 'a_b__c_d.txt'
[INFO] Renaming '.foo bar.txt' to '.foo_bar.txt'
[INFO] Renaming 'a b' to 'a_b'
[INFO] Renaming 'a.b.c.d' to 'a_b_c.d'
[INFO] Renaming '.foo bar' to '.foo_bar'
|
New contents:
| Code: |
$ ls -1a
.
..
a_b
a_b_c_d
a_b_c.d
a_b__c_d.txt
.foo
.foo_bar
.foo_bar.txt
.foo_foo.txt
.foo.foo.txt
.foo.txt
test.txt
|
My source code:
| Code: | #!/usr/bin/perl -w
use strict;
use Cwd qw(getcwd);
use File::Copy qw(move);
my $dir = getcwd();
opendir(DIR, $dir) or die $!;
my $exitCode = 0;
while (my $file = readdir(DIR)) {
next if ($file =~ /^\.+$/o); #Skip special dir refs
my $new;
if ($file =~ /^\.(.+)$/o) {
my $tmp = $1;
$new = '.' . &processFilename($tmp);
} else {
$new = &processFilename($file);
}
next if ($new eq $file); #No changes were made
if (-e $new) {
print STDERR "[WARN] $new already exists; skipping rename of '$file' to '$new'\n";
next;
}
print STDERR "[INFO] Renaming '$file' to '$new'\n";
my $check = move($file, $new);
if (! $check) {
print STDERR "[WARN] Unable to rename '$file' to '$new': $!\n";
# File::Copy::Move can leave artifacts on the FS
if (-e $new) {
unlink $new;
}
$exitCode = -1 if ($exitCode == 0);
}
}
closedir(DIR);
exit($exitCode);
sub processFilename {
my $file = shift;
my $new;
if ($file =~ /^(.+)\.(\S+)$/o) {
my $base = $1;
my $ext = $2;
$base =~ s/(?:\s|\.)/_/go;
$new = join('.', $base, $ext);
} else {
($new = $file) =~ s/(?:\s|\.)/_/go;
}
return $new;
} |
|
|
| Back to top |
|
|
Dark Foo
l33t
Joined: 21 Nov 2008
Posts: 921
Location: 127.0.0.1
|
| Posted: Wed Jan 25, 2012 6:23 pm Post subject: |
|
|
| John R. Graham wrote: | The complexity comes in because of preserving the single '.' that separates the name from the extension but converting all others to '_'. (Dark Foo, do I have that correct?)
|
What have I started
Yes thats correct John, Guess my request was not as simple as i first thought  Still all a learning experience, if i wanted easy I could have stayed with Windows
Thanks for the script RazielFMX, much appreciated 
If i read it right, and I'm probably not, it looks at though it will do files only atm, or will it do folders too?
I could do files then folders but both at same time would be faster for me
_________________
I know 43 ways to kill with a SKITTLE, so taste my rainbow bitch. |
|
| Back to top |
|
|
RazielFMX
l33t
Joined: 23 Apr 2005
Posts: 672
Location: NY, USA
|
| Posted: Wed Jan 25, 2012 9:13 pm Post subject: |
|
|
Do you want it to tree walk, or simply executing only in the current directory is ok?
And you are right, it will not work on directories in the event of a failure, however, it WILL work on directories if everything goes right.
Let me know what your spec is and I will modify it accordingly. |
|
| Back to top |
|
|
Naib
Advocate
Joined: 21 May 2004
Posts: 3891
Location: UK - Birmingham
|
| Posted: Wed Jan 25, 2012 9:20 pm Post subject: |
|
|
To walk a tree with bash(4)
| Code: |
shopt -s globstar; for f in **/*; do t="${f// /_}"; mv "$f" "${t//./_}.${f##*.}"; done |
should do it
_________________
| Quote: | | Voting holds no real power, he who counts the votes has the true power. |
Weaver Projects
whats the difference between 9/11 and a cow?
u stop milking a cow after 10 years |
|
| Back to top |
|
|
Dark Foo
l33t
Joined: 21 Nov 2008
Posts: 921
Location: 127.0.0.1
|
| Posted: Wed Jan 25, 2012 9:26 pm Post subject: |
|
|
well seeing as i am lazy i would like to start from my raid5 and have it look at everything below
/storage
/pictures
/music
/3D_APPS
so on and so forth
I will have to take a look in the morning - thanks to everyone who has posted suggestions, i really appreciate your help
_________________
I know 43 ways to kill with a SKITTLE, so taste my rainbow bitch. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 4849
Location: Somewhere over Atlanta, Georgia
|
| Posted: Wed Jan 25, 2012 9:27 pm Post subject: |
|
|
Naib, with your latest script, test file name "a b c.d e f" is renamed to "a_b_c_d_e_f.d e f", not exactly what the OP is requesting.
- John
_________________
Yoda: "Intentionally left blank, this space is." |
|
| Back to top |
|
|
Naib
Advocate
Joined: 21 May 2004
Posts: 3891
Location: UK - Birmingham
|
| Posted: Wed Jan 25, 2012 11:09 pm Post subject: |
|
|
well some assumptions will have to be made. its from windows so the last decimal is going to be use for extension .###
_________________
| Quote: | | Voting holds no real power, he who counts the votes has the true power. |
Weaver Projects
whats the difference between 9/11 and a cow?
u stop milking a cow after 10 years |
|
| Back to top |
|
|
BoneKracker
Veteran
Joined: 14 Mar 2006
Posts: 1271
Location: U.S.A.
|
| Posted: Wed Jan 25, 2012 11:18 pm Post subject: |
|
|
Ever seen the episode of Star Trek with the "Neural Neutralizer"? Perl is the "Neural Neutralizer" of scripting languages. It can do anything, but it will fry your brain in the process. 
_________________
Obama killed bin Laden like Nixon was the first man on the Moon. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 4849
Location: Somewhere over Atlanta, Georgia
|
| Posted: Thu Jan 26, 2012 1:06 am Post subject: |
|
|
It's kind of like Cyrillic: until you know the symbols, it just looks like gibberish.
Well, actually, it still looks like gibberish. But, it is powerful. Little while ago, I became "upstream" on a medium sized Perl project (app-backup/flexbackup) so I've been practicing.
- John
_________________
Yoda: "Intentionally left blank, this space is." |
|
| Back to top |
|
|
BoneKracker
Veteran
Joined: 14 Mar 2006
Posts: 1271
Location: U.S.A.
|
| Posted: Thu Jan 26, 2012 2:58 am Post subject: |
|
|
I thought there was a 'rename' in perl, specifically for file renaming, perhaps a module.
_________________
Obama killed bin Laden like Nixon was the first man on the Moon. |
|
| Back to top |
|
|
RazielFMX
l33t
Joined: 23 Apr 2005
Posts: 672
Location: NY, USA
|
| Posted: Thu Jan 26, 2012 3:36 am Post subject: |
|
|
| BoneKracker wrote: | | I thought there was a 'rename' in perl, specifically for file renaming, perhaps a module. |
There is, but it is recommend to use the File::Copy::move instead as rename behavior varies from system to system. |
|
| Back to top |
|
|
RazielFMX
l33t
Joined: 23 Apr 2005
Posts: 672
Location: NY, USA
|
| Posted: Thu Jan 26, 2012 4:28 pm Post subject: |
|
|
One fully featured Perl script, as promised.
| Code: |
#!/usr/bin/perl -w
# Author: RazielFMZ 26-Jan-2012
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
#
use strict;
use Getopt::Long qw(:config bundling);
use File::Copy qw(move);
use File::Find qw(find);
use File::Basename qw(basename);
#
# Usage
#
my $opts = {};
&GetOptions(
$opts,
"help|h",
"dir=s@",
"recurse|r"
) or die &getUsage();
if (defined $opts->{'help'} && $opts->{'help'}) {
print STDERR &getUsage();
exit(1);
}
if (! defined $opts->{'dir'}) {
die "--dir is required";
} else {
#Validate
foreach my $dir (@{$opts->{'dir'}}) {
if (! -d $dir) {
die "$dir is not a valid directory";
}
}
}
#
# Main
#
my $exitCode = 0;
if ($opts->{'recurse'}) {
print STDERR "[INFO] Operating in recursive mode, delegating to find\n";
find({
wanted => sub {
my $base = basename($File::Find::name);
##Skip magic dirs
if ($base !~ /^\.+$/o) {
my $check = &processEntry($File::Find::name);
if (! $check) {
$exitCode = -1 if ($exitCode == 0);
}
}
},
follow => 0,
bydepth => 1,
no_chdir => 1,
}, @{$opts->{'dir'}});
} else {
print STDERR "[INFO] Recursive mode disabled\n";
foreach my $dir (@{$opts->{'dir'}}) {
my $dc = opendir(DIR, $dir);
if (! $dc) {
print STDERR "[ERROR] Failure to open $dir: $!\n";
$exitCode = -1 if ($exitCode == 0);
next;
}
while (my $entry = readdir(DIR)) {
next if ($entry =~ /^\.+$/o);
my $tmp = join('/', $dir, $entry);
my $check = &processEntry($tmp);
if (! $check) {
$exitCode = -1 if ($exitCode == 0);
}
}
closedir(DIR);
}
}
exit($exitCode);
#
# Methods
#
sub processEntry {
my $file = shift;
my $new;
if ($file =~ /^\.(.+)$/o) {
my $tmp = $1;
$new = '.' . &processFilename($tmp);
} else {
$new = &processFilename($file);
}
return 1 if ($new eq $file);
if (-e $new) {
print STDERR "[WARN] $new already exists; skipping rename of '$file' to '$new'\n";
return 1;
}
print STDERR "[INFO] Renaming '$file' to '$new'\n";
my $check = move($file, $new);
if (! $check) {
print STDERR "[WARN] Unable to rename '$file' to '$new': $!\n";
return undef;
}
return 1;
}
sub processFilename {
my $file = shift;
my $new;
if ($file =~ /^(.+)\.(\S+)$/o) {
my $base = $1;
my $ext = $2;
$base =~ s/(?:\s|\.)/_/go;
$new = join('.', $base, $ext);
} else {
($new = $file) =~ s/(?:\s|\.)/_/go;
}
return $new;
}
sub getUsage {
my $scriptname = basename($0);
my $usage = <<_USAGE_END_;
Usage: $scriptname
h|help : Print this screen
dir <dir> : Directory to execute under, this can be specified multiple
times for more than one directory.
r|recurse : Recursively walk the directory tree. Symlinks are ignored.
_USAGE_END_
return $usage;
}
|
You can turn on and off the recursive mode, and specify multiple directories by specifying multiple --dir arguments.
| Code: |
$ ./stripSpaces.pl --help
Usage: stripSpaces.pl
h|help : Print this screen
dir <dir> : Directory to execute under, this can be specified multiple
times for more than one directory.
r|recurse : Recursively walk the directory tree. Symlinks are ignored.
|
Last edited by RazielFMX on Thu Jan 26, 2012 4:35 pm; edited 1 time in total |
|
| Back to top |
|
|
Dark Foo
l33t
Joined: 21 Nov 2008
Posts: 921
Location: 127.0.0.1
|
| Posted: Thu Jan 26, 2012 4:34 pm Post subject: |
|
|
sweet - will try it out shortly and report back
_________________
I know 43 ways to kill with a SKITTLE, so taste my rainbow bitch. |
|
| Back to top |
|
|
RazielFMX
l33t
Joined: 23 Apr 2005
Posts: 672
Location: NY, USA
|
| Posted: Thu Jan 26, 2012 4:35 pm Post subject: |
|
|
| There was a small bug I just patched and did an in-post edit. Please use the newest version |
|
| Back to top |
|
|
Dark Foo
l33t
Joined: 21 Nov 2008
Posts: 921
Location: 127.0.0.1
|
| Posted: Fri Jan 27, 2012 5:46 pm Post subject: |
|
|
works quite well, it has left a few folders with a dot in its name, what would i have to add for it to change - to _ ?
_________________
I know 43 ways to kill with a SKITTLE, so taste my rainbow bitch. |
|
| Back to top |
|
|
RazielFMX
l33t
Joined: 23 Apr 2005
Posts: 672
Location: NY, USA
|
| Posted: Fri Jan 27, 2012 9:16 pm Post subject: |
|
|
it leaves the . on folders due to it thinking it is an extension. I have patched it (with support for symlinks as well) and added the '-' character. I switched the regex to be globally stored so you or I can tweak it easily and not have look into the methods.
| Code: |
#!/usr/bin/perl -w
# Author: RazielFMZ 27-Jan-2012, v1.1
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
#
use strict;
use Getopt::Long qw(:config bundling);
use File::Copy qw(move);
use File::Find qw(find);
use File::Basename qw(basename);
#Global Vars
our $RENAME_REGEX = qr/(?:\s|[.-])/;
#
# Usage
#
my $opts = {};
&GetOptions(
$opts,
"help|h",
"dir=s@",
"recurse|r"
) or die &getUsage();
if (defined $opts->{'help'} && $opts->{'help'}) {
print STDERR &getUsage();
exit(1);
}
if (! defined $opts->{'dir'}) {
die "--dir is required";
} else {
#Validate
foreach my $dir (@{$opts->{'dir'}}) {
if (! -d $dir) {
die "$dir is not a valid directory";
}
}
}
#
# Main
#
my $exitCode = 0;
if ($opts->{'recurse'}) {
print STDERR "[INFO] Operating in recursive mode, delegating to find\n";
find({
wanted => sub {
my $base = basename($File::Find::name);
##Skip magic dirs
if ($base !~ /^\.+$/o) {
my $check = &processEntry($File::Find::name);
if (! $check) {
$exitCode = -1 if ($exitCode == 0);
}
}
},
follow => 0,
bydepth => 1,
no_chdir => 1,
}, @{$opts->{'dir'}});
} else {
print STDERR "[INFO] Recursive mode disabled\n";
foreach my $dir (@{$opts->{'dir'}}) {
my $dc = opendir(DIR, $dir);
if (! $dc) {
print STDERR "[ERROR] Failure to open $dir: $!\n";
$exitCode = -1 if ($exitCode == 0);
next;
}
while (my $entry = readdir(DIR)) {
next if ($entry =~ /^\.+$/o);
my $tmp = join('/', $dir, $entry);
my $check = &processEntry($tmp);
if (! $check) {
$exitCode = -1 if ($exitCode == 0);
}
}
closedir(DIR);
}
}
exit($exitCode);
#
# Methods
#
sub processEntry {
my $file = shift;
my $treatAsDir;
if (-l $file) {
my $target = readlink($file);
#If readlink returns undef and we know we have a symlink,
#we have a link with no target, just treat is like a file
$treatAsDir = (defined $target && -d $target) ? 1 : 0;
} elsif (-d $file) {
$treatAsDir = 1;
} else {
$treatAsDir = 0;
}
my $new;
if ($file =~ /^\.(.+)$/o) {
my $tmp = $1;
$new = '.' . &processFilename($tmp, $treatAsDir);
} else {
$new = &processFilename($file, $treatAsDir);
}
return 1 if ($new eq $file);
if (-e $new) {
print STDERR "[WARN] $new already exists; skipping rename of '$file' to '$new'\n";
return 1;
}
print STDERR "[INFO] Renaming '$file' to '$new'\n";
my $check = move($file, $new);
if (! $check) {
print STDERR "[WARN] Unable to rename '$file' to '$new': $!\n";
return undef;
}
return 1;
}
sub processFilename {
my $file = shift;
my $treatAsDir = shift;
my $new;
if ((! $treatAsDir) && $file =~ /^(.+)\.(\S+)$/o) {
my $base = $1;
my $ext = $2;
$base =~ s/$RENAME_REGEX/_/go;
$new = join('.', $base, $ext);
} else {
($new = $file) =~ s/$RENAME_REGEX/_/go;
}
return $new;
}
sub getUsage {
my $scriptname = basename($0);
my $usage = <<_USAGE_END_;
Usage: $scriptname
h|help : Print this screen
dir <dir> : Directory to execute under, this can be specified multiple
times for more than one directory.
r|recurse : Recursively walk the directory tree. Symlinks are ignored.
_USAGE_END_
return $usage;
}
|
|
|
| Back to top |
|
|
Dark Foo
l33t
Joined: 21 Nov 2008
Posts: 921
Location: 127.0.0.1
|
| Posted: Fri Jan 27, 2012 9:17 pm Post subject: |
|
|
Thanks, will test it out in the morning, rather tired atm, but not as tired as if I had to rename all the files, your script it a god send
_________________
I know 43 ways to kill with a SKITTLE, so taste my rainbow bitch. |
|
| Back to top |
|
|
RazielFMX
l33t
Joined: 23 Apr 2005
Posts: 672
Location: NY, USA
|
| Posted: Fri Jan 27, 2012 9:24 pm Post subject: |
|
|
| Dark Foo wrote: | | Thanks, will test it out in the morning, rather tired atm, but not as tired as if I had to rename all the files, your script it a god send |
Anytime! |
|
| Back to top |
|
|
Dark Foo
l33t
Joined: 21 Nov 2008
Posts: 921
Location: 127.0.0.1
|
| Posted: Sat Jan 28, 2012 1:35 pm Post subject: |
|
|
Looking good so far, have only tested it on one dir, just in case but looks to be performing its job perfectly
_________________
I know 43 ways to kill with a SKITTLE, so taste my rainbow bitch. |
|
| Back to top |
|
|
RazielFMX
l33t
Joined: 23 Apr 2005
Posts: 672
Location: NY, USA
|
| Posted: Mon Jan 30, 2012 7:25 pm Post subject: |
|
|
| Glad to hear it. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Desktop Environments
