Moving

It's funny sometimes. Even in the 21st century, we don't really have a reliable way of moving a lot of bits around quickly.

I have been trying to push some data between two boxes for about two or three days now, and due to the standard Murphy's Law events of spurious "Access denied" errors, timeouts, and other TCP/IP effluvia, I haven't really found a robust way to get my data from Point A to Point B.

First I tried Rich Copy, which was touted with the slogan "Trust me when I tell you, this is the answer to all your file copying needs." Indeed, Rich Copy is probably great for moving files around. I've had some minor success with it in the past. But I quickly discovered that Rich Copy was failing to give me a rich experience in the area of restartability.

Restartability is incredibly important for data transfer. See sentence #1. I am completely certain that at some point, any transfer of more than a handful of files can and will be interrupted. What then? A smart application picks up where it leaves off. For whatever reason, Rich Copy was starting from scratch every time, which probably is because of an oversight on my part somewhere in the unintuitive configuration settings.

No matter, because there's always Robocopy. Robocopy has proven itself pretty smart with restartability...most of the time. In this case, it found itself alternating between calling the destination file older and newer than the source file, despite the fact they were identical files on two NTFS partitions. This suggests that maybe the root cause is something as simple as one of the clocks on one of the servers was off. See sentence #1.

So Robocopy would also restart the transfer from scratch. No es bueno.

Finally, I tried my old standby, rsync. I was really hoping that I could leverage a utility that had some sane looping concepts built into it; something that could spot that there was a problem, work around it, and keep chugging along. Alas, rsync is good for many things, but synchronizing isn't one of them: the word "retry" appears nowhere in the rsync(1) man page.

No need to fear, I kludged something together with spit and bailing wire. The oh-so-amazing twenty-first technology that will reliably move files from A to B? A batch script.

@ECHO ON
:DOIT
rsync -avP --modify-window=5 /src/dir /dst/
IF 0 == %ERRORLEVEL% GOTO EXIT

:ERROR
GOTO DOIT

:EXIT
EXIT /B

It really makes me wonder sometimes. Is this the best we can do for easy, reliable file copy operations? It's ugly and it's slow. On the other hand, the big sophisticated applications that are designed to work around this aren't going to be any faster if they can't figure out how to avoid copying what they've already copied before.

Also considered, but dismissed due to its immaturity: SharpBITS.NET. And nothing that involves typing "Import-Module BITSTransfer" is ready for hundreds of files yet, nor have I been convinced that it persists after a reboot.