| View previous topic :: View next topic |
| Author |
Message |
Cr0t
l33t
Joined: 27 Apr 2002
Posts: 944
Location: USA
|
 Posted: Wed Apr 30, 2014 2:16 pm Post subject: box.com sync script |
 |
|
I have a 50GB box.com account, which I use for my picture/video backups. The problem with box.com is, that you can't have files, which are bigger than 250MB. I wrote a script, which splits the file and pushes it to box.com.
A couple of test files: | Code: | 10:09:51^cr0t@ninapie:~/test > l *
4328061 -rw-r--r-- 1 cr0t t0 954M Apr 30 10:07 foo.iso
4327059 -rw-r--r-- 1 cr0t t0 977K Apr 30 10:08 pic1.png
4327058 -rw-r--r-- 1 cr0t t0 52K Apr 30 10:08 pic2.png
d1:
total 21M
4333795 drwxr-xr-x 2 cr0t t0 4.0K Apr 30 10:09 ./
4327056 drwxr-xr-x 3 cr0t t0 4.0K Apr 30 10:08 ../
4327688 -rw-r--r-- 1 cr0t t0 2.0M Apr 30 10:09 pic1.png
4327304 -rw-r--r-- 1 cr0t t0 20M Apr 30 10:09 pic33.png
10:09:53^cr0t@ninapie:~/test > |
Running the script: | Code: | 10:09:53^cr0t@ninapie:~/test > ../boxSync.sh
We have 5 files to sync to box.com.
#5: rsyncing file ./d1/pic33.png... transfered 20004982 Bytes... DONE
#4: rsyncing file ./d1/pic1.png... transfered 2000588 Bytes... DONE
#3: rsyncing file ./foo.iso... splitting file... transfered 1000244418 Bytes... DONE
#2: rsyncing file ./pic2.png... transfered 52844 Bytes... DONE
#1: rsyncing file ./pic1.png... transfered 1000340 Bytes... DONE
We sync'ed 1023303172 Bytes to box.com
10:12:07^cr0t@ninapie:~/test > |
the actual box.com folder | Code: | 10:12:38^cr0t@ninapie:~/test > l ../box.com/*
4064301 -rw-r--r-- 1 cr0t t0 250M Apr 30 10:11 ../box.com/foo.iso__split__aa
4064311 -rw-r--r-- 1 cr0t t0 250M Apr 30 10:12 ../box.com/foo.iso__split__ab
4064460 -rw-r--r-- 1 cr0t t0 250M Apr 30 10:12 ../box.com/foo.iso__split__ac
4064489 -rw-r--r-- 1 cr0t t0 204M Apr 30 10:12 ../box.com/foo.iso__split__ad
4064496 -rw-r--r-- 1 cr0t t0 977K Apr 30 10:08 ../box.com/pic1.png
4064495 -rw-r--r-- 1 cr0t t0 52K Apr 30 10:08 ../box.com/pic2.png
../box.com/d1:
total 21M
4071767 drwxr-xr-x 2 cr0t t0 4.0K Apr 30 10:11 ./
4071766 drwxr-xr-x 3 cr0t t0 4.0K Apr 30 10:12 ../
4064276 -rw-r--r-- 1 cr0t t0 2.0M Apr 30 10:09 pic1.png
4063927 -rw-r--r-- 1 cr0t t0 20M Apr 30 10:09 pic33.png
10:12:42^cr0t@ninapie:~/test > |
If you want to put foo.iso together, you will just have to cat it back together | Code: | 10:13:56^cr0t@ninapie:~/box.com > cat foo.iso__split__a* > foo.iso
10:14:04^cr0t@ninapie:~/box.com > md5sum foo.iso
e37115d4da0e187130ab645dee4f14ed foo.iso
10:14:11^cr0t@ninapie:~/box.com > md5sum ../test/foo.iso
e37115d4da0e187130ab645dee4f14ed ../test/foo.iso
10:14:18^cr0t@ninapie:~/box.com > |
Now the script | Code: | 10:14:38^cr0t@ninapie:~ > cat boxSync.sh
#!/bin/sh
MAXFILESIZE=262144000
DIRECTORY=/home/cr0t/box.com
SAVEIFS=$IFS
IFS=$(echo -en "\n\b")
list=`find -type f`
count=`echo $list | wc -w`
total=0
echo "We have $count files to sync to box.com."
for line in $list;
do
echo -n \#$count\: rsyncing file "$line"...
target=`dirname "$line"`
mkdir -p "$DIRECTORY"/$target
size=`stat -c %s $line`
if [ "$size" -ge "$MAXFILESIZE" ];
then
mkdir -p "$DIRECTORY"/`dirname "$line"`
split -b $MAXFILESIZE "$line" "$line"__split__
echo -n " splitting file..."
size=`rsync -av --size-only "$line"\_\_split\_\_* "$DIRECTORY"/$target | grep sent | head -1 | awk '{print $2}' | sed s/\,//g`
echo -n " transfered $size Bytes... "
rm -rf "$line"\_\_split\_\_*
else
size=`rsync -av --size-only "$line" "$DIRECTORY"/$target | grep sent | head -1 | awk '{print $2}' | sed s/\,//g`
echo -n " transfered $size Bytes... "
fi
echo " DONE"
let "total+=$size"
let "count-=1"
done
echo "We sync'ed $total Bytes to box.com"
IFS=$SAVEIFS
10:14:40^cr0t@ninapie:~ > |
The script compares via the file size
_________________
cya |
|
| Back to top |
|
 |
windex
n00b
Joined: 09 Dec 2012
Posts: 70
|
| Posted: Sat May 10, 2014 11:58 pm Post subject: |
|
|
| Thanks! |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
