Rsync files younger or older than

Today I needed to copy all files from a certain folder, but younger than 1 month  to a remote machine.

I knew how to use find of course… and rsync is pretty easy too… Combining these 2 needs some explanation:

Finding files younger than 31 days is performed like this:

Rsyncing all files from a folder is pretty easy too:

Now, if you would like to rsync only the filers younger thn 31 days from your source folder, you can combine rsync and find in this way:

However… this is not 100 working as intended… This will skip filename with spaces, as the space are not escaped during the rsync…

The solution for this is to split the one liner above into 2 seperate commands again:

1) perform the find, and write the output to a tempfile:

2) perform the rsync with the –from-files parameter:

(do not forget the . in the command to declare a ‘source dir’)

Remark: When using relative paths, you could/should change the /path/to/folder to path/to/folder and put a . instead of a / in your rsync command
E.g. like this:

12 thoughts on “Rsync files younger or older than”

  1. Doesn’t work for me.

    Tries to find the destination path with the first two directories doubled up:

    /Volumes/Data/Volumes/Data/Transfer/Work

    should be looking at just:

    /Volumes/Data/Transfer/Work

     

    So failed with no such file or directory

    I checked the output file and this lists all the files i want to rsync correctly, i.e. –

    /Volumes/Data/Transfer/Work/suchandsuchfile.txt

  2. Good idea! Just a remark when writing youngest filenames to a file and rsync those: Though rsync had the -a option (that implies -t) it did not preserve the timestamps of the destination files.

  3. Withdraw my last comment, it DOES preserve timestamps I just happened to choose files of the last year and did not carefully look at the year 🙁

  4. This has one major flaw.  The -d option (delete files in destination that don’t exist in source) becomes unusable because you are rsyncing files one by one instead of rsyncing folders.  I’d like to see a way to say, rsync this folder (and subfolders) but ignore files and folders older than XX days.

     

    1. Hey Jazzy,

      Yes, this -d option is of course something that lacks in the script itself… Haven’t found a way to do this, but never needed it anyway 🙂

      About the Ignore files, you might be able to use something like: –exclude-from=/path/to/exclusion-file, created with a find command on local or remote server?

  5. Even that doesnt work for me properly. First command workis as expected.

    # find /var/testing/* -type d -mtime 1 >/var/output.txt

    # cat /var/output.txt
    /var/testing/lmn
    /var/testing/lmn/a
    /var/testing/lmn/a/b
    /var/testing/lmn/a/b/c
    /var/testing/lmn/a/b/c/d
    /var/testing/lmn/a/b/c/d/e
    /var/testing/lmn/a/b/c/d/e/f
    /var/testing/lmn/new
    /var/testing/lmn/new1

    However, the 2nd command even though calls the file /var/outut.txt, you can see /var completely gets copied to the estination. I am trying to copy only the data under /var/testing/*.

    # rsync -Ravh –files-from=/var/output.txt / /tmp/xyz
    building file list … done
    var/
    var/testing/
    var/testing/lmn/
    var/testing/lmn/a/
    var/testing/lmn/a/b/
    var/testing/lmn/a/b/c/
    var/testing/lmn/a/b/c/d/
    var/testing/lmn/a/b/c/d/e/
    var/testing/lmn/a/b/c/d/e/f/
    var/testing/lmn/new/
    var/testing/lmn/new1/

    sent 286 bytes received 49 bytes 670.00 bytes/sec
    total size is 0 speedup is 0.00
    uatnim[/var/testing] #

     

    1. it is kind of normal that all subfolders are copied and created too.
      Add in a
      cd /var/testing
      and change your find command to

      find . -type d -mtime 1 >/var/output.txt

  6. I use rsync .. find .. and the –delete option doesn’t work. To prevent the folder from growing I need a working –delete. How to get the –delete working?

  7. It’s me, guest, again. I solved is by additionally deleting the files in a second command by using find again:

    Here’s this second command:

    num_of_the_n_newest_files_to_keep=10

    delete_every_file_except_the_newest_n_files=$((find "/home/.../Backup" -iname "NAME_*.bak" |wc -l – $num_of_the_n_newest_files_to_keep))
    if [ “$delete_every_file_except_the_newest_n_files” -gt “0” ]; then # if are there any files to delete
    find “/home/…/…” -type f -not -path “*/folder_to_ignore/*” -printf “%p\n” | sort -n | head -n $delete_every_file_except_the_newest_n_files | xargs rm
    fi

    PS: Ignore the “folder_to_ignore”, it’s an internal thing 🙂

    1. Hi you again, guest! 🙂

      You might try:
      ls -1tr | head -n -10 | xargs -d '\n' rm -f --

      Which is easier, less code and should also be faster.

      – where the ls output’s in 1 column, and sorted by time in reverse order.
      – we remove the 10 newest files with the head pipe
      – we send all output to xargs that rm’s these files, where the -d add a new line option and handles files with spaces in their names.

Leave a Reply

Your email address will not be published. Required fields are marked *