Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Is there any way to set my own dropbox-like server? [SOLVED]
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Off the Wall
View previous topic :: View next topic  
Author Message
Fran
Guru
Guru


Joined: 29 Feb 2004
Posts: 524
Location: Coruña (Spain)

PostPosted: Fri Oct 29, 2010 8:59 am    Post subject: Is there any way to set my own dropbox-like server? [SOLVED] Reply with quote

I've been testing dropbox and it's great (I regularly use 3 computers). But I don't trust them much. Also, I have a public, fixed IP with TBs of free space and huge bandwith (my desktop computer at work :)), so I feel stupid paying them to get more than 2GB.

I've read about combining git/rsync and incron, but no actual examples/howtos. There are also a couple of mono programs that seem to be what I'm looking for (novell ifolder, sparkleshare)... but mono? Ugh. Unless someone tells me that they work great I'd rather search for alternatives.

So, is there any client AND server open source program that does the same as dropbox and... well, "just works", like dropbox?


Last edited by Fran on Fri Feb 01, 2013 9:13 am; edited 1 time in total
Back to top
View user's profile Send private message
dreadlorde
Apprentice
Apprentice


Joined: 16 Dec 2008
Posts: 243
Location: /adm/timezone/US_Michigan

PostPosted: Fri Oct 29, 2010 12:30 pm    Post subject: Reply with quote

vsftp?
_________________
Ludwig von Mises Institute
Quote:
I am not to be a shepherd, I am not to be a grave-digger. No longer will I speak to the people; for the last time I have spoken to the dead.
Back to top
View user's profile Send private message
sts
Tux's lil' helper
Tux's lil' helper


Joined: 02 Jul 2007
Posts: 97

PostPosted: Fri Oct 29, 2010 1:20 pm    Post subject: Reply with quote

There's Unison which is really old but probably still works and I'm pretty sure there was an open source dropbox clone that you could run on your own servers but now I can't find it... so google around for that.
Back to top
View user's profile Send private message
Fran
Guru
Guru


Joined: 29 Feb 2004
Posts: 524
Location: Coruña (Spain)

PostPosted: Fri Oct 29, 2010 1:22 pm    Post subject: Reply with quote

dreadlorde wrote:
vsftp?

Erm... what does vsftp have to do with dropbox? Does it upload a file automatically when I change it locally, so I don't need to remember to do it? Does it transfer only the part of the file that changed, so it doesn't hog my limited upload? Does it delete a file in one client when I delete it in the other, so everything is kept in sync? If not, then it doesn't "do the same as dropbox", which is what I'm looking for.
Back to top
View user's profile Send private message
Fran
Guru
Guru


Joined: 29 Feb 2004
Posts: 524
Location: Coruña (Spain)

PostPosted: Fri Oct 29, 2010 1:39 pm    Post subject: Reply with quote

sts wrote:
There's Unison which is really old but probably still works and I'm pretty sure there was an open source dropbox clone that you could run on your own servers but now I can't find it... so google around for that.

If the open source alternative is lsyncd, I've tried it and it sucks as a dropbox alternative. It's only one way (i.e., if I create a file in client A it's uploaded to the sever, but client B doesn't download it. In fact if I restart lsyncd in B, the file is deleted again in the server).

Unison doesn't watch directories (so you're stuck with cron) and doesn't keep old versions of modified/deleted files, IIRC.
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1569
Location: U.S.A.

PostPosted: Fri Oct 29, 2010 2:15 pm    Post subject: Reply with quote

Fran wrote:
dreadlorde wrote:
vsftp?

Erm... what does vsftp have to do with dropbox? Does it upload a file automatically when I change it locally, so I don't need to remember to do it? Does it transfer only the part of the file that changed, so it doesn't hog my limited upload? Does it delete a file in one client when I delete it in the other, so everything is kept in sync? If not, then it doesn't "do the same as dropbox", which is what I'm looking for.

Well, the concept of a "drop-box" predates that product by like a decade, and ftp has been the traditional way to do such things.

From what you are describing, some rsync-based product may be answer. Another approach would be a distributed filesystem.
_________________
"The accumulation of all power, legislative, executive, and judiciary in the same hands...may justly be pronounced the very definition of tyranny."
--James Madison, Federalist 46
Back to top
View user's profile Send private message
zixnub
n00b
n00b


Joined: 27 Dec 2007
Posts: 66
Location: Brasschaat, Belgium

PostPosted: Fri Oct 29, 2010 2:21 pm    Post subject: Re: Is there any way to set my own dropbox-like server? Reply with quote

Fran wrote:
I've been testing dropbox and it's great (I regularly use 3 computers). But I don't trust them much. Also, I have a public, fixed IP with TBs of free space and huge bandwith (my desktop computer at work :)), so I feel stupid paying them to get more than 2GB.

I've read about combining git/rsync and incron, but no actual examples/howtos. There are also a couple of mono programs that seem to be what I'm looking for (novell ifolder, sparkleshare)... but mono? Ugh. Unless someone tells me that they work great I'd rather search for alternatives.

So, is there any client AND server open source program that does the same as dropbox and... well, "just works", like dropbox?


Short answer: no

Long answer: A senior colleague of mine is looking for people to start an open source dropbox clone. He's got the entire architecture in his head, but since he's burried in work, it will most likely never get of the ground.
_________________
https://github.com/Zubnix/trinityshell
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16117
Location: Colorado

PostPosted: Fri Oct 29, 2010 4:08 pm    Post subject: Re: Is there any way to set my own dropbox-like server? Reply with quote

zixnub wrote:
He's got the entire architecture in his head, but since he's burried in work, it will most likely never get of the ground.
He could always put it in a wiki. Documentation is often lacking when it comes to open source. Pony up.
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
poly_poly-man
Advocate
Advocate


Joined: 06 Dec 2006
Posts: 2477
Location: RIT, NY, US

PostPosted: Fri Oct 29, 2010 5:18 pm    Post subject: Reply with quote

nfs?
_________________
iVBORw0KGgoAAAANSUhEUgAAA

avatar: new version of logo - see topic 838248. Potentially still a WiP.
Back to top
View user's profile Send private message
Fran
Guru
Guru


Joined: 29 Feb 2004
Posts: 524
Location: Coruña (Spain)

PostPosted: Fri Oct 29, 2010 5:41 pm    Post subject: Reply with quote

poly_poly-man wrote:
nfs?

Nope. I use nfs at work, where I have a 1gbps local network between my desktop and server. At home with my 6Mbps/600kbps cable it'd be a bit... cumbersome. I need the files stored in my hard disks. In all of them.

Quote:
Short answer: no

:(
Back to top
View user's profile Send private message
shickapooka800
Guru
Guru


Joined: 05 Dec 2004
Posts: 304
Location: no

PostPosted: Fri Oct 29, 2010 6:09 pm    Post subject: Reply with quote

how about a clever OpenAFS setup? it is a network filesystem, but the client has a large cache (oversimplified a lot, but whatever). you could make the cache that you keep with you the same size as the shared volume. so it would act basically as a synced backup whenever you connected.

I kinda do a similar thing with windows shares. at work, "My Documents" on my machine is mapped to a network drive with offline capability. if I lose connectivity I can still open and edit all files that the OS cached, and I can create new ones. When I regain connectivity, changed files and new files are synced in both ways (new file on server but not on client becomes available but only in meta data, it won't actually sync until I open the file or access it in some way).

note that with the "both ways" setup, some intelligence needs to be there to merge things. I don't know how intelligent windows is at this, and I don't want to know. So I just tread carefully if I am on multiple machines and editing files on the same share. I don't want to know what happens if I edit a file offline, in two places then have both of them sync.....

edit: I'm pretty sure there actually isn't any intelligence and it is in fact a "first come first serve" architecture.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16117
Location: Colorado

PostPosted: Fri Oct 29, 2010 8:57 pm    Post subject: Re: Is there any way to set my own dropbox-like server? Reply with quote

Fran wrote:
There are also a couple of mono programs that seem to be what I'm looking for (novell ifolder, sparkleshare)... but mono? Ugh. Unless someone tells me that they work great I'd rather search for alternatives.
Mono is a language. Unless there is a "real" reason to avoid them, why not try them out? Especially given that the opportunities seem limited (or nil)?

That said, I'm looking to see what I can find regarding the git option you mentioned.

I thought there was an open source alternative too, but I can't think of it (or what I was thinking of).
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16117
Location: Colorado

PostPosted: Fri Oct 29, 2010 11:33 pm    Post subject: Reply with quote

Duplicity might be what I was thinking of as an alternative to dropbox.
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
Fran
Guru
Guru


Joined: 29 Feb 2004
Posts: 524
Location: Coruña (Spain)

PostPosted: Tue Jan 22, 2013 2:44 pm    Post subject: Reply with quote

Necromancy FTW. I've just found a dropbox replacement that works quite well: http://www.mayrhofer.eu.org/dvcs-autosync

And it not only "just works", but is also extremely simple :) (<1000 python SLOC). It uses git (duh) and jabber (genius!) to propagate local changes. The idea is simple: all computers log in a jabber server using the same account. When a change occurs (inotify) the local computer pushes the changes to the git server and sends a jabber message to itself, which is received by the rest (since all use the same account), which pull the changes.

I've been testing it for a day; it seems robust (since it's based on git it's almost impossible to lose any data) and fast. I've been removing, editing, moving, etc. files, and everything stays synced. The only thing that doesn't seem to work well is removing directories (maybe adding a -r to the git rm command would make it work).
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1569
Location: U.S.A.

PostPosted: Tue Jan 22, 2013 9:50 pm    Post subject: Reply with quote

Fran wrote:
It uses git (duh) and jabber (genius!) to propagate local changes. The idea is simple: all computers log in a jabber server using the same account. When a change occurs (inotify) the local computer pushes the changes to the git server and sends a jabber message to itself, which is received by the rest (since all use the same account), which pull the changes.

Wow. All it's missing is The Clapper.
_________________
"The accumulation of all power, legislative, executive, and judiciary in the same hands...may justly be pronounced the very definition of tyranny."
--James Madison, Federalist 46
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16117
Location: Colorado

PostPosted: Thu Jan 24, 2013 12:10 am    Post subject: Reply with quote

Fran wrote:
Necromancy FTW. I've just found a dropbox replacement that works quite well: http://www.mayrhofer.eu.org/dvcs-autosync
Thanks for the update... this looks pretty cool. And it appears they have a Windows version. Have you tried it too?
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
Fran
Guru
Guru


Joined: 29 Feb 2004
Posts: 524
Location: Coruña (Spain)

PostPosted: Sun Jan 27, 2013 10:21 am    Post subject: Reply with quote

pjp wrote:
Fran wrote:
Necromancy FTW. I've just found a dropbox replacement that works quite well: http://www.mayrhofer.eu.org/dvcs-autosync
Thanks for the update... this looks pretty cool. And it appears they have a Windows version. Have you tried it too?

Nope, I just boot to windows for steam :) (I have nothing else installed).

After a few days of use I have uninstalled dropbox. DVCS-autosync is not as set-and-forget as dropbox, since you may find the occasional conflict that must be resolved by hand (dropbox creates those "conflited copy" files) but it's not a big problem if you know git, and those conflicts are very rare. And there are nice advantages over dropbox aside from the obvious "unlimited" free storage and guaranteed privacy, like using .gitignore to avoid syncing certain files (.o, .pyc, *~, *.swp, etc).

BTW: removing directories works, but you must let git do it (i.e., you simply remove all files in the directory).

And for people trying it out, there is a bug in the current git version. Patch:
Code:
--- dvcs-autosync.orig/dvcs-autosync   2013-01-22 18:24:15.687684141 +0100
+++ dvcs-autosync/dvcs-autosync   2013-01-22 18:24:28.934003345 +0100
@@ -536,7 +536,7 @@
             try:
                 if event.src_pathname:
                     logging.debug('Detected moved file from %s to %s', event.src_pathname, event.pathname)
-                    self._handler.queue_action(event, cmd_move, [event.src_pathname, event.pathname], act_on_dirs=True)
+                    self.handler._queue_action(event, cmd_move, [event.src_pathname, event.pathname], act_on_dirs=True)
                 else:
                     logging.debug('Moved file to %s, but unknown source, will simply add new file', event.pathname)
                     self.handler._queue_action(event, cmd_add, [event.pathname], act_on_dirs=True)

I've already notified upstream.
Back to top
View user's profile Send private message
Dr.Willy
Guru
Guru


Joined: 15 Jul 2007
Posts: 353
Location: NRW, Germany

PostPosted: Sun Jan 27, 2013 11:09 am    Post subject: Reply with quote

Fran wrote:
And for people trying it out, there is a bug in the current git version. Patch:
Code:
--- dvcs-autosync.orig/dvcs-autosync   2013-01-22 18:24:15.687684141 +0100
+++ dvcs-autosync/dvcs-autosync   2013-01-22 18:24:28.934003345 +0100
@@ -536,7 +536,7 @@
             try:
                 if event.src_pathname:
                     logging.debug('Detected moved file from %s to %s', event.src_pathname, event.pathname)
-                    self._handler.queue_action(event, cmd_move, [event.src_pathname, event.pathname], act_on_dirs=True)
+                    self.handler._queue_action(event, cmd_move, [event.src_pathname, event.pathname], act_on_dirs=True)
                 else:
                     logging.debug('Moved file to %s, but unknown source, will simply add new file', event.pathname)
                     self.handler._queue_action(event, cmd_add, [event.pathname], act_on_dirs=True)

I've already notified upstream.

<3 dynamic typing
Back to top
View user's profile Send private message
Fran
Guru
Guru


Joined: 29 Feb 2004
Posts: 524
Location: Coruña (Spain)

PostPosted: Sun Jan 27, 2013 12:29 pm    Post subject: Reply with quote

Dr.Willy wrote:
<3 dynamic typing

Yep, that was my first thought when I found the bug.
Back to top
View user's profile Send private message
Dr.Willy
Guru
Guru


Joined: 15 Jul 2007
Posts: 353
Location: NRW, Germany

PostPosted: Sun Jan 27, 2013 1:02 pm    Post subject: Reply with quote

I have never understood the purpose of dynamic typing.
I mean, I really like python, but the dynamic typing and (partly as a consequence) the way object orientation is done make python really awkward to use sometimes.
At the end of the day the computer has to make something meaningful of your
Code:
var x = 'c' + 5
statement. If it can't, what's the point of telling you later rather than sooner?
Back to top
View user's profile Send private message
Prenj
n00b
n00b


Joined: 20 Nov 2011
Posts: 13

PostPosted: Sun Jan 27, 2013 1:09 pm    Post subject: Reply with quote

Dr.Willy wrote:
I have never understood the purpose of dynamic typing.
I mean, I really like python, but the dynamic typing and (partly as a consequence) the way object orientation is done make python really awkward to use sometimes.
At the end of the day the computer has to make something meaningful of your
Code:
var x = 'c' + 5
statement. If it can't, what's the point of telling you later rather than sooner?


It's sloppy programming. Just because dynamic typing exists, it doesn't mean it's there to compensate for people not thinking clearly.
Back to top
View user's profile Send private message
Dr.Willy
Guru
Guru


Joined: 15 Jul 2007
Posts: 353
Location: NRW, Germany

PostPosted: Sun Jan 27, 2013 1:54 pm    Post subject: Reply with quote

Prenj wrote:
Just because dynamic typing exists, it doesn't mean it's there to compensate for people not thinking clearly.

But that seems to be it's foremost application.
Back to top
View user's profile Send private message
Prenj
n00b
n00b


Joined: 20 Nov 2011
Posts: 13

PostPosted: Sun Jan 27, 2013 1:55 pm    Post subject: Reply with quote

Dr.Willy wrote:
Prenj wrote:
Just because dynamic typing exists, it doesn't mean it's there to compensate for people not thinking clearly.

But that seems to be it's foremost application.


Well yeah, hence the results. :lol:
Back to top
View user's profile Send private message
Dr.Willy
Guru
Guru


Joined: 15 Jul 2007
Posts: 353
Location: NRW, Germany

PostPosted: Tue Jan 29, 2013 7:39 pm    Post subject: Reply with quote

Hey Fran, here's something I stumbled upon: http://blog.bittorrent.com/2013/01/24/test-bittorrent-sync-pre-alpha/
Might be interesting once this goes stable.
Back to top
View user's profile Send private message
Fran
Guru
Guru


Joined: 29 Feb 2004
Posts: 524
Location: Coruña (Spain)

PostPosted: Tue Jan 29, 2013 9:50 pm    Post subject: Reply with quote

Dr.Willy wrote:
Hey Fran, here's something I stumbled upon: http://blog.bittorrent.com/2013/01/24/test-bittorrent-sync-pre-alpha/
Might be interesting once this goes stable.

Yeah, saw it yesterday. Looks interesting. I hope it implements some kind of versioning and not just a glorified rsync.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Off the Wall All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum