Comparing Files Against Local Store When Downloading From Artifactory

Let’s say you want to download a large amount of files from Artifactory to your local storage. However, you don’t want to download files you already have, or that have been modified more recently locally.

Artifactory, or its tools, don’t have any built-in functionality that will compare your local files’ timestamp to what’s available on your server. However, we can get creative and achieve Timestampped CLI Downlaods

As a workaround we will be using File Specs on download with the JFrog CLI. A combination of using file specs along with Unix/Linux commands can achieve the desired result.

To recap, the requirements are the following:

• Download the files according to a specific set of requirements
• Download the file if it doesn’t already exist
• If the file already exists, only download it if it’s newer than the locally available one

What we are able to do is create a script. For name let’s name it timestampped_download.sh. It contains the following one-line script:

If you are not familiar with the CLI, we are essentially telling it to download files according to a Spec File (linked earlier) which will define the artifacts we want and the criteria to meet. The spec-vars flag allows us to pass variables as key-value pairs that can then be replaced inside the spec at run time.

These spec files can work as templates for different situations that are needed, and in general, even outside of this use case, they can be pretty useful to have around.

We can now run the script with the following arguments, referenced in the code as $1, $2, and $3

$ test.sh <repo name> <artifact name> <template>

Where template is a spec file, for example in this case we used the following file which we named template.spec:

Being able to choose which template you want to use allows for an incredible amount of flexibility in customization when downloading large numbers of artifacts. If you want to stick to one template you could also hard code the repo/name/whatever else into the script or use AQL instead of the spec.

Get creative and get downloading!