Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
box.com sync script
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
Cr0t
l33t
l33t


Joined: 27 Apr 2002
Posts: 881
Location: USA

PostPosted: Wed Apr 30, 2014 2:16 pm    Post subject: box.com sync script Reply with quote

I have a 50GB box.com account, which I use for my picture/video backups. The problem with box.com is, that you can't have files, which are bigger than 250MB. I wrote a script, which splits the file and pushes it to box.com.

A couple of test files:
Code:
10:09:51^cr0t@ninapie:~/test > l *
4328061 -rw-r--r-- 1 cr0t t0 954M Apr 30 10:07 foo.iso
4327059 -rw-r--r-- 1 cr0t t0 977K Apr 30 10:08 pic1.png
4327058 -rw-r--r-- 1 cr0t t0  52K Apr 30 10:08 pic2.png

d1:
total 21M
4333795 drwxr-xr-x 2 cr0t t0 4.0K Apr 30 10:09 ./
4327056 drwxr-xr-x 3 cr0t t0 4.0K Apr 30 10:08 ../
4327688 -rw-r--r-- 1 cr0t t0 2.0M Apr 30 10:09 pic1.png
4327304 -rw-r--r-- 1 cr0t t0  20M Apr 30 10:09 pic33.png
10:09:53^cr0t@ninapie:~/test >
Running the script:
Code:
10:09:53^cr0t@ninapie:~/test > ../boxSync.sh
We have 5 files to sync to box.com.
#5: rsyncing file ./d1/pic33.png... transfered 20004982 Bytes...  DONE
#4: rsyncing file ./d1/pic1.png... transfered 2000588 Bytes...  DONE
#3: rsyncing file ./foo.iso... splitting file... transfered 1000244418 Bytes...  DONE
#2: rsyncing file ./pic2.png... transfered 52844 Bytes...  DONE
#1: rsyncing file ./pic1.png... transfered 1000340 Bytes...  DONE
We sync'ed 1023303172 Bytes to box.com
10:12:07^cr0t@ninapie:~/test >
the actual box.com folder
Code:
10:12:38^cr0t@ninapie:~/test > l ../box.com/*
4064301 -rw-r--r-- 1 cr0t t0 250M Apr 30 10:11 ../box.com/foo.iso__split__aa
4064311 -rw-r--r-- 1 cr0t t0 250M Apr 30 10:12 ../box.com/foo.iso__split__ab
4064460 -rw-r--r-- 1 cr0t t0 250M Apr 30 10:12 ../box.com/foo.iso__split__ac
4064489 -rw-r--r-- 1 cr0t t0 204M Apr 30 10:12 ../box.com/foo.iso__split__ad
4064496 -rw-r--r-- 1 cr0t t0 977K Apr 30 10:08 ../box.com/pic1.png
4064495 -rw-r--r-- 1 cr0t t0  52K Apr 30 10:08 ../box.com/pic2.png

../box.com/d1:
total 21M
4071767 drwxr-xr-x 2 cr0t t0 4.0K Apr 30 10:11 ./
4071766 drwxr-xr-x 3 cr0t t0 4.0K Apr 30 10:12 ../
4064276 -rw-r--r-- 1 cr0t t0 2.0M Apr 30 10:09 pic1.png
4063927 -rw-r--r-- 1 cr0t t0  20M Apr 30 10:09 pic33.png
10:12:42^cr0t@ninapie:~/test >
If you want to put foo.iso together, you will just have to cat it back together
Code:
10:13:56^cr0t@ninapie:~/box.com > cat foo.iso__split__a* > foo.iso
10:14:04^cr0t@ninapie:~/box.com > md5sum foo.iso
e37115d4da0e187130ab645dee4f14ed  foo.iso
10:14:11^cr0t@ninapie:~/box.com > md5sum ../test/foo.iso
e37115d4da0e187130ab645dee4f14ed  ../test/foo.iso
10:14:18^cr0t@ninapie:~/box.com >
Now the script
Code:
10:14:38^cr0t@ninapie:~ > cat boxSync.sh
#!/bin/sh

MAXFILESIZE=262144000
DIRECTORY=/home/cr0t/box.com

SAVEIFS=$IFS
IFS=$(echo -en "\n\b")

list=`find -type f`
count=`echo $list | wc -w`
total=0

echo "We have $count files to sync to box.com."

for line in $list;
do
        echo -n \#$count\: rsyncing file "$line"...
        target=`dirname "$line"`
        mkdir -p "$DIRECTORY"/$target
        size=`stat -c %s $line`
        if [ "$size" -ge "$MAXFILESIZE" ];
        then
                mkdir -p "$DIRECTORY"/`dirname "$line"`
                split -b $MAXFILESIZE "$line" "$line"__split__
                echo -n " splitting file..."
                size=`rsync -av --size-only "$line"\_\_split\_\_* "$DIRECTORY"/$target | grep sent | head -1 | awk '{print $2}' | sed s/\,//g`
                echo -n " transfered $size Bytes... "
                rm -rf "$line"\_\_split\_\_*
        else
                size=`rsync -av --size-only "$line" "$DIRECTORY"/$target | grep sent | head -1 | awk '{print $2}' | sed s/\,//g`
                echo -n " transfered $size Bytes... "
        fi
        echo " DONE"
        let "total+=$size"
        let "count-=1"
done
echo "We sync'ed $total Bytes to box.com"

IFS=$SAVEIFS
10:14:40^cr0t@ninapie:~ >
The script compares via the file size
_________________
cya
    ©®0t
Back to top
View user's profile Send private message
windex
n00b
n00b


Joined: 09 Dec 2012
Posts: 70

PostPosted: Sat May 10, 2014 11:58 pm    Post subject: Reply with quote

Thanks!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum