snarfed.org

draw group stream of consciousness

backup subversion fsfs with rsync

Wed, 01 Jan 2003 [comments (5)] [history] [rdf] [raw]

/space/subversion.png

I use Subversion for source control, and it's great. I recently migrated my repository to use the new FSFS backing store, which stores the repository directly on the filesystem, instead of in Berkeley DB.

One of the many advantages of FSFS is that it's very easy to backup. The repository files can be copied directly. The only tricks are to copy the db/current file first, and not to copy the transactions/ directory. I backup my repository with rsync, like so:

rsync ... /path/to/repo/db/current backup:~/repo/db
rsync ... --exclude transactions/ --exclude db/current \
          /path/to/repo backup:~/


See also:

comment bubble OpenID mike, Thu 08 Feb 2007

You really dont want to do this. You can corrupt your svn database if rsync only gets part of a tranction. A better way is to script backing up the svn repository manually

comment bubble OpenID joe, Wed 28 Feb 2007

Agreed about the corruption.  I would instead recommend first making a hotcopy of the working repository and then doing an rsync on that.

comment bubble OpenID AlberT, Fri 27 Apr 2007

The svn "Note: Backup" section seems to agree with this article ...

the repository will not be corrupted as you don't copty transictions and copy current before the rest ...

"So, if you are using standard backup tools to make backups of a FSFS repository, configure the software to copy the "current" file before the numbered revision files, if possible, and configure it not to copy the "transactions" directory.  If you can't do those things, use "svnadmin hotcopy", or be prepared to cope with the very occasional need for repair of the repository upon restoring it from backup."

comment bubble OpenID jonvel, Wed 28 May 2008

I'd like to say that I've been using a very similar strategy for backing up my FSFS Svn Repository (going on about 2 GB total data, 4800+ revisions).  I've restored this from backup, and it works fine.  No corruption issues that anyone is complaining about, even with a "live" system that I've TRIED to corrupt - ie making a large (~1500 files) checkin while the backup was in process.  Yes, when I restored the backup (to a third server), the latest update (the ~1.5k files) wasn't there, but I expected that to happen.  re-synching after the update, however, did give me the files I was expecting, and the update I expected.

Since my SVN repo is running on a windows machine, I can't use rsync.  But I CAN use winscp's scripting capability.  The commands are essentially the same:

=== begin winscp scriptfile ===
option batch on
option confirm off
option transfer binary
option include current
synchronize remote c:\mytemprepo /backup/location
option include clear
option exclude transactions
option exclude current
synchronize remote c:\mytemprepo /backup/location
exit
=== end winscp script ===
called with:
winscp.com <session_id> /script=winscpscript > logfile

Now a backup takes a whopping < 5 minutes to run (well, the first one was very long).

comment bubble OpenID jonvel, Wed 28 May 2008

I'd like to say that I've been using a very similar strategy for backing up my FSFS Svn Repository (going on about 2 GB total data, 4800+ revisions).  I've restored this from backup, and it works fine.  No corruption issues that anyone is complaining about, even with a "live" system that I've TRIED to corrupt - ie making a large (~1500 files) checkin while the backup was in process.  Yes, when I restored the backup (to a third server), the latest update (the ~1.5k files) wasn't there, but I expected that to happen.  re-synching after the update, however, did give me the files I was expecting, and the update I expected.

Since my SVN repo is running on a windows machine, I can't use rsync.  But I CAN use winscp's scripting capability.  The commands are essentially the same:

=== begin winscp scriptfile ===
option batch on
option confirm off
option transfer binary
option include current
synchronize remote c:\mytemprepo /backup/location
option include clear
option exclude transactions
option exclude current
synchronize remote c:\mytemprepo /backup/location
exit
=== end winscp script ===
called with:
winscp.com <session_id> /script=winscpscript > logfile

Now a backup takes a whopping < 5 minutes to run (well, the first one was very long).

Post a comment...



Simple HTML and wiki markup are allowed.