[CLUE-Tech] how to handle open files - upload server

[Greg Knaddison]

Quoting Nate Duehr <nate at natetech.com>:

> 
> The case I'm wondering about: How to detect in that regular cron job if 
> the file is open (and probably growing) right at the time because it's 
> being uploaded right then... so the cron doesn't attempt to move it to 
> the other machine in the middle of an upload.
> 

I'm not 100% clear on the example, but I have two possible ideas on how to solve
it.  

If the files are being uploaded by reliable users (or a script) then the files
can be uploaded with a temporary file name.  Once they are finished, the
person/script renames them to a name that your cron job will look for.  For
example, I upload my file1.sxi.tmp file.  The cron script looks for *.sxi files
to move and finds none so it does nothing.  Then the file finishes uploading and
I rename it to file1.sxi  and the cron job kicks up and finds the file and
handles it.

The more code intensive solution is to create a hash table (two column text
file) of file name and file size.  Have the cron job run every hour and compare
the current name/size to the  name/size in the hash table.  If they are the
same, then assume the file is "stable" and can be moved.  If they are different
then do nothing with the file.

You'd also have to deal with cleaning up your hash table somehow, but that seems
simple enough.

Good luck,
Greg