| View previous topic :: View next topic |
| Author |
Message |
Fran
Guru
Joined: 29 Feb 2004
Posts: 530
Location: Coruña (Spain)
|
 Posted: Fri Oct 29, 2010 8:59 am Post subject: Is there any way to set my own dropbox-like server? [SOLVED] |
 |
|
I've been testing dropbox and it's great (I regularly use 3 computers). But I don't trust them much. Also, I have a public, fixed IP with TBs of free space and huge bandwith (my desktop computer at work  ), so I feel stupid paying them to get more than 2GB.
I've read about combining git/rsync and incron, but no actual examples/howtos. There are also a couple of mono programs that seem to be what I'm looking for (novell ifolder, sparkleshare)... but mono? Ugh. Unless someone tells me that they work great I'd rather search for alternatives.
So, is there any client AND server open source program that does the same as dropbox and... well, "just works", like dropbox?
Last edited by Fran on Fri Feb 01, 2013 9:13 am; edited 1 time in total |
|
| Back to top |
|
 |
dreadlorde
Apprentice
Joined: 16 Dec 2008
Posts: 243
Location: /adm/timezone/US_Michigan
|
| Posted: Fri Oct 29, 2010 12:30 pm Post subject: |
|
|
vsftp?
_________________
Ludwig von Mises Institute | Quote: | | I am not to be a shepherd, I am not to be a grave-digger. No longer will I speak to the people; for the last time I have spoken to the dead. |
|
|
| Back to top |
|
|
sts
Tux's lil' helper
Joined: 02 Jul 2007
Posts: 97
|
| Posted: Fri Oct 29, 2010 1:20 pm Post subject: |
|
|
| There's Unison which is really old but probably still works and I'm pretty sure there was an open source dropbox clone that you could run on your own servers but now I can't find it... so google around for that. |
|
| Back to top |
|
|
Fran
Guru
Joined: 29 Feb 2004
Posts: 530
Location: Coruña (Spain)
|
| Posted: Fri Oct 29, 2010 1:22 pm Post subject: |
|
|
Erm... what does vsftp have to do with dropbox? Does it upload a file automatically when I change it locally, so I don't need to remember to do it? Does it transfer only the part of the file that changed, so it doesn't hog my limited upload? Does it delete a file in one client when I delete it in the other, so everything is kept in sync? If not, then it doesn't "do the same as dropbox", which is what I'm looking for. |
|
| Back to top |
|
|
Fran
Guru
Joined: 29 Feb 2004
Posts: 530
Location: Coruña (Spain)
|
| Posted: Fri Oct 29, 2010 1:39 pm Post subject: |
|
|
| sts wrote: | | There's Unison which is really old but probably still works and I'm pretty sure there was an open source dropbox clone that you could run on your own servers but now I can't find it... so google around for that. |
If the open source alternative is lsyncd, I've tried it and it sucks as a dropbox alternative. It's only one way (i.e., if I create a file in client A it's uploaded to the sever, but client B doesn't download it. In fact if I restart lsyncd in B, the file is deleted again in the server).
Unison doesn't watch directories (so you're stuck with cron) and doesn't keep old versions of modified/deleted files, IIRC. |
|
| Back to top |
|
|
Bones McCracker
Veteran
Joined: 14 Mar 2006
Posts: 1605
Location: U.S.A.
|
| Posted: Fri Oct 29, 2010 2:15 pm Post subject: |
|
|
| Fran wrote: |
Erm... what does vsftp have to do with dropbox? Does it upload a file automatically when I change it locally, so I don't need to remember to do it? Does it transfer only the part of the file that changed, so it doesn't hog my limited upload? Does it delete a file in one client when I delete it in the other, so everything is kept in sync? If not, then it doesn't "do the same as dropbox", which is what I'm looking for. |
Well, the concept of a "drop-box" predates that product by like a decade, and ftp has been the traditional way to do such things.
From what you are describing, some rsync-based product may be answer. Another approach would be a distributed filesystem.
_________________
| patrix_neo wrote: | The human thought: I cannot win.
The ratbrain in me : I can only go forward and that's it. |
|
|
| Back to top |
|
|
zixnub
n00b
Joined: 27 Dec 2007
Posts: 67
Location: Brasschaat, Belgium
|
| Posted: Fri Oct 29, 2010 2:21 pm Post subject: Re: Is there any way to set my own dropbox-like server? |
|
|
| Fran wrote: | I've been testing dropbox and it's great (I regularly use 3 computers). But I don't trust them much. Also, I have a public, fixed IP with TBs of free space and huge bandwith (my desktop computer at work ), so I feel stupid paying them to get more than 2GB.
I've read about combining git/rsync and incron, but no actual examples/howtos. There are also a couple of mono programs that seem to be what I'm looking for (novell ifolder, sparkleshare)... but mono? Ugh. Unless someone tells me that they work great I'd rather search for alternatives.
So, is there any client AND server open source program that does the same as dropbox and... well, "just works", like dropbox? |
Short answer: no
Long answer: A senior colleague of mine is looking for people to start an open source dropbox clone. He's got the entire architecture in his head, but since he's burried in work, it will most likely never get of the ground.
_________________
https://github.com/udevbe/greenfield
https://github.com/udevbe/westfield |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 17260
|
| Posted: Fri Oct 29, 2010 4:08 pm Post subject: Re: Is there any way to set my own dropbox-like server? |
|
|
| zixnub wrote: | | He's got the entire architecture in his head, but since he's burried in work, it will most likely never get of the ground. |
He could always put it in a wiki. Documentation is often lacking when it comes to open source. Pony up.
_________________
The whole system has to go. The modern criminal justice system is incompatible with Neuroscience. --Sapolsky |
|
| Back to top |
|
|
poly_poly-man
Advocate
Joined: 06 Dec 2006
Posts: 2477
Location: RIT, NY, US
|
| Posted: Fri Oct 29, 2010 5:18 pm Post subject: |
|
|
nfs?
_________________
iVBORw0KGgoAAAANSUhEUgAAA
avatar: new version of logo - see topic 838248. Potentially still a WiP. |
|
| Back to top |
|
|
Fran
Guru
Joined: 29 Feb 2004
Posts: 530
Location: Coruña (Spain)
|
| Posted: Fri Oct 29, 2010 5:41 pm Post subject: |
|
|
| poly_poly-man wrote: | | nfs? |
Nope. I use nfs at work, where I have a 1gbps local network between my desktop and server. At home with my 6Mbps/600kbps cable it'd be a bit... cumbersome. I need the files stored in my hard disks. In all of them.
 |
|
| Back to top |
|
|
shickapooka800
Guru
Joined: 05 Dec 2004
Posts: 304
Location: no
|
| Posted: Fri Oct 29, 2010 6:09 pm Post subject: |
|
|
how about a clever OpenAFS setup? it is a network filesystem, but the client has a large cache (oversimplified a lot, but whatever). you could make the cache that you keep with you the same size as the shared volume. so it would act basically as a synced backup whenever you connected.
I kinda do a similar thing with windows shares. at work, "My Documents" on my machine is mapped to a network drive with offline capability. if I lose connectivity I can still open and edit all files that the OS cached, and I can create new ones. When I regain connectivity, changed files and new files are synced in both ways (new file on server but not on client becomes available but only in meta data, it won't actually sync until I open the file or access it in some way).
note that with the "both ways" setup, some intelligence needs to be there to merge things. I don't know how intelligent windows is at this, and I don't want to know. So I just tread carefully if I am on multiple machines and editing files on the same share. I don't want to know what happens if I edit a file offline, in two places then have both of them sync.....
edit: I'm pretty sure there actually isn't any intelligence and it is in fact a "first come first serve" architecture. |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 17260
|
| Posted: Fri Oct 29, 2010 8:57 pm Post subject: Re: Is there any way to set my own dropbox-like server? |
|
|
| Fran wrote: | | There are also a couple of mono programs that seem to be what I'm looking for (novell ifolder, sparkleshare)... but mono? Ugh. Unless someone tells me that they work great I'd rather search for alternatives. |
Mono is a language. Unless there is a "real" reason to avoid them, why not try them out? Especially given that the opportunities seem limited (or nil)?
That said, I'm looking to see what I can find regarding the git option you mentioned.
I thought there was an open source alternative too, but I can't think of it (or what I was thinking of).
_________________
The whole system has to go. The modern criminal justice system is incompatible with Neuroscience. --Sapolsky |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 17260
|
| Posted: Fri Oct 29, 2010 11:33 pm Post subject: |
|
|
Duplicity might be what I was thinking of as an alternative to dropbox.
_________________
The whole system has to go. The modern criminal justice system is incompatible with Neuroscience. --Sapolsky |
|
| Back to top |
|
|
Fran
Guru
Joined: 29 Feb 2004
Posts: 530
Location: Coruña (Spain)
|
| Posted: Tue Jan 22, 2013 2:44 pm Post subject: |
|
|
Necromancy FTW. I've just found a dropbox replacement that works quite well: http://www.mayrhofer.eu.org/dvcs-autosync
And it not only "just works", but is also extremely simple (<1000 python SLOC). It uses git (duh) and jabber (genius!) to propagate local changes. The idea is simple: all computers log in a jabber server using the same account. When a change occurs (inotify) the local computer pushes the changes to the git server and sends a jabber message to itself, which is received by the rest (since all use the same account), which pull the changes.
I've been testing it for a day; it seems robust (since it's based on git it's almost impossible to lose any data) and fast. I've been removing, editing, moving, etc. files, and everything stays synced. The only thing that doesn't seem to work well is removing directories (maybe adding a -r to the git rm command would make it work). |
|
| Back to top |
|
|
Bones McCracker
Veteran
Joined: 14 Mar 2006
Posts: 1605
Location: U.S.A.
|
| Posted: Tue Jan 22, 2013 9:50 pm Post subject: |
|
|
| Fran wrote: | | It uses git (duh) and jabber (genius!) to propagate local changes. The idea is simple: all computers log in a jabber server using the same account. When a change occurs (inotify) the local computer pushes the changes to the git server and sends a jabber message to itself, which is received by the rest (since all use the same account), which pull the changes. |
Wow. All it's missing is The Clapper.
_________________
| patrix_neo wrote: | The human thought: I cannot win.
The ratbrain in me : I can only go forward and that's it. |
|
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 17260
|
| Posted: Thu Jan 24, 2013 12:10 am Post subject: |
|
|
Thanks for the update... this looks pretty cool. And it appears they have a Windows version. Have you tried it too?
_________________
The whole system has to go. The modern criminal justice system is incompatible with Neuroscience. --Sapolsky |
|
| Back to top |
|
|
Fran
Guru
Joined: 29 Feb 2004
Posts: 530
Location: Coruña (Spain)
|
| Posted: Sun Jan 27, 2013 10:21 am Post subject: |
|
|
| pjp wrote: | | Thanks for the update... this looks pretty cool. And it appears they have a Windows version. Have you tried it too? |
Nope, I just boot to windows for steam (I have nothing else installed).
After a few days of use I have uninstalled dropbox. DVCS-autosync is not as set-and-forget as dropbox, since you may find the occasional conflict that must be resolved by hand (dropbox creates those "conflited copy" files) but it's not a big problem if you know git, and those conflicts are very rare. And there are nice advantages over dropbox aside from the obvious "unlimited" free storage and guaranteed privacy, like using .gitignore to avoid syncing certain files (.o, .pyc, *~, *.swp, etc).
BTW: removing directories works, but you must let git do it (i.e., you simply remove all files in the directory).
And for people trying it out, there is a bug in the current git version. Patch:
| Code: | --- dvcs-autosync.orig/dvcs-autosync 2013-01-22 18:24:15.687684141 +0100
+++ dvcs-autosync/dvcs-autosync 2013-01-22 18:24:28.934003345 +0100
@@ -536,7 +536,7 @@
try:
if event.src_pathname:
logging.debug('Detected moved file from %s to %s', event.src_pathname, event.pathname)
- self._handler.queue_action(event, cmd_move, [event.src_pathname, event.pathname], act_on_dirs=True)
+ self.handler._queue_action(event, cmd_move, [event.src_pathname, event.pathname], act_on_dirs=True)
else:
logging.debug('Moved file to %s, but unknown source, will simply add new file', event.pathname)
self.handler._queue_action(event, cmd_add, [event.pathname], act_on_dirs=True)
|
I've already notified upstream. |
|
| Back to top |
|
|
Dr.Willy
Guru
Joined: 15 Jul 2007
Posts: 450
Location: NRW, Germany
|
| Posted: Sun Jan 27, 2013 11:09 am Post subject: |
|
|
| Fran wrote: | And for people trying it out, there is a bug in the current git version. Patch:
| Code: | --- dvcs-autosync.orig/dvcs-autosync 2013-01-22 18:24:15.687684141 +0100
+++ dvcs-autosync/dvcs-autosync 2013-01-22 18:24:28.934003345 +0100
@@ -536,7 +536,7 @@
try:
if event.src_pathname:
logging.debug('Detected moved file from %s to %s', event.src_pathname, event.pathname)
- self._handler.queue_action(event, cmd_move, [event.src_pathname, event.pathname], act_on_dirs=True)
+ self.handler._queue_action(event, cmd_move, [event.src_pathname, event.pathname], act_on_dirs=True)
else:
logging.debug('Moved file to %s, but unknown source, will simply add new file', event.pathname)
self.handler._queue_action(event, cmd_add, [event.pathname], act_on_dirs=True)
|
I've already notified upstream. |
<3 dynamic typing |
|
| Back to top |
|
|
Fran
Guru
Joined: 29 Feb 2004
Posts: 530
Location: Coruña (Spain)
|
| Posted: Sun Jan 27, 2013 12:29 pm Post subject: |
|
|
| Dr.Willy wrote: | | <3 dynamic typing |
Yep, that was my first thought when I found the bug. |
|
| Back to top |
|
|
Dr.Willy
Guru
Joined: 15 Jul 2007
Posts: 450
Location: NRW, Germany
|
| Posted: Sun Jan 27, 2013 1:02 pm Post subject: |
|
|
I have never understood the purpose of dynamic typing.
I mean, I really like python, but the dynamic typing and (partly as a consequence) the way object orientation is done make python really awkward to use sometimes.
At the end of the day the computer has to make something meaningful of your statement. If it can't, what's the point of telling you later rather than sooner? |
|
| Back to top |
|
|
Prenj
n00b
Joined: 20 Nov 2011
Posts: 16
|
| Posted: Sun Jan 27, 2013 1:09 pm Post subject: |
|
|
| Dr.Willy wrote: | I have never understood the purpose of dynamic typing.
I mean, I really like python, but the dynamic typing and (partly as a consequence) the way object orientation is done make python really awkward to use sometimes.
At the end of the day the computer has to make something meaningful of your statement. If it can't, what's the point of telling you later rather than sooner? |
It's sloppy programming. Just because dynamic typing exists, it doesn't mean it's there to compensate for people not thinking clearly. |
|
| Back to top |
|
|
Dr.Willy
Guru
Joined: 15 Jul 2007
Posts: 450
Location: NRW, Germany
|
| Posted: Sun Jan 27, 2013 1:54 pm Post subject: |
|
|
| Prenj wrote: | | Just because dynamic typing exists, it doesn't mean it's there to compensate for people not thinking clearly. |
But that seems to be it's foremost application. |
|
| Back to top |
|
|
Prenj
n00b
Joined: 20 Nov 2011
Posts: 16
|
| Posted: Sun Jan 27, 2013 1:55 pm Post subject: |
|
|
| Dr.Willy wrote: | | Prenj wrote: | | Just because dynamic typing exists, it doesn't mean it's there to compensate for people not thinking clearly. |
But that seems to be it's foremost application. |
Well yeah, hence the results.  |
|
| Back to top |
|
|
Dr.Willy
Guru
Joined: 15 Jul 2007
Posts: 450
Location: NRW, Germany
|
|
| Back to top |
|
|
Fran
Guru
Joined: 29 Feb 2004
Posts: 530
Location: Coruña (Spain)
|
| Posted: Tue Jan 29, 2013 9:50 pm Post subject: |
|
|
Yeah, saw it yesterday. Looks interesting. I hope it implements some kind of versioning and not just a glorified rsync. |
|
| Back to top |
|
|
|
Off the Wall
