archivetar wraps mpiFileUtils. Thus we are able to use many of the options in dfind to filter the initial list of files to only archive a subset of data.
NOTE If not using Globus to upload using the --size option you will not have an simple way without manually using dcp with the over.cache created by archivetar. So it is not recommended unless using Globus to upload the data to another location.
Currently archivetar understands the following filters:
--atime --mtime --ctime --user --group
Multiple filters use logical and, eg --atime +180 --user brockp will archive only files accessed more than 180days ago AND owned by user brockp.
Filters are only applied in the initial scan. They are ignored if used with the --list option.
It is possible to use filters with archivepurge archive all files from a specific user and delete. Use --save-list rather than --save-purge-list because the first has ALL files to be archived, not just those in tars.
# find and archive all files owned by user `brockp` in given group space.
# scan once and get meta-data but do not archive
archivetar --prefix brockp-archive --user brockp --dryrun --save-list
# actually archive using list created above update timestamp
archivetar --prefix brockp-archive --list brockp-archive-<timestamp>.cache
--source <UUID> --destination <UUID> --destination-path
/path/on/dest/brockp-archive/ --size 1G
# once all transfer above finish delete file in the initial list
archivepurge --purge-list brockp-archive-<timestamp>.cache