[CLUE-Tech] Re: [CO-SAGE] how to handle open files - upload server

[Alan Peery]

Nate Duehr wrote:

>The case I'm wondering about: How to detect in that regular cron job if 
>the file is open (and probably growing) right at the time because it's 
>being uploaded right then... so the cron doesn't attempt to move it to 
>the other machine in the middle of an upload.
>  
>
Check the modification time of the file.  > 5 minutes ago, no writer is 
still around and it is safe to move. 

It is much easier if you have control of the clients uploading.  If you 
do (you own their source code), then have them upload 
<random_file_name>.tmp files.  When finished, a rename from *.tmp to 
<intended_unique_name>.fld (for example) is nearly an atomic operation.  
You can then have a cron job delete any *.tmp files older than 1 day and 
have a very sensible operation.  This worked properly for me with 20+ 
clients delivering gigabytes of data hourly over over months.

Of course, there is the minor issue that FTP sends it passwords in the 
clear.  (The solution is left as an exercise to the reader. ;-) )

Alan