First, we will generate a list of items. This example will just be generating a root-level list of items. If needed you can head deeper into the folder structure.
This is using the item properties rest API and using the jq utility to get a list of items, url encode them, and output them into a text file.
Next, we will run the copy call on each item generated from the list.
The following is a simple bash loop that reads each line from the text file and performs the copy rest call. I had previously exported the TOKEN variable before running the loop with my user’s token. This example is copying from one repository to another. Go ahead and replace the <REPO_NAME*> sections according to your needs. You may also adjust the sleep parameter as needed.
This will print out each copy command and the results. If there are any issues you can spot the particular command/item and further troubleshoot.
One thing to note is that Artifactory copies/moves items according to Linux convention which can be slightly unintuitive. The copy command will create a folder within a folder for a folder already existing as seen in the following example (note how ‘folder2’ when copied over is moved within the existing ‘folder2’).
Example of copying a folder in Linux:
Original Structure:
Running a copy command will result in a ‘folder2’ being created inside the destination folder not overwriting the existing ‘folder2’ in the ‘newlocation’ location.
So if you are copying to a specific path you should take care to not include the particular folder name at the end of the path of the destination unless you want to copy the directory structure into that particular location.
An example of the potentially unintended nested folder can be seen in action if you add the “$line” variable to the destination section of the curl and run the loop twice. The second loop will attempt to copy the items into the second repository which will already have the folders copied over from the first loop so they will be placed into the folder instead of overwriting the folder
I.e. Adjusted script which can lead to the unintended nested folder structure
Loop 1:
Loop 2:
This is using the item properties rest API and using the jq utility to get a list of items, url encode them, and output them into a text file.
curl -uadmin "<ART_URL>/artifactory/api/storage/<REPO_NAME>/" | jq -r '.children[].uri' | jq -Rr @uri > listOfItems.txt
Next, we will run the copy call on each item generated from the list.
The following is a simple bash loop that reads each line from the text file and performs the copy rest call. I had previously exported the TOKEN variable before running the loop with my user’s token. This example is copying from one repository to another. Go ahead and replace the <REPO_NAME*> sections according to your needs. You may also adjust the sleep parameter as needed.
while read -r line; do echo -e "\n\n `date` - Copying $line\n\n"; curl -XPOST -u<USER>:$TOKEN "<ART_URL>/artifactory/api/copy/<REPO_NAME_1>$line?to=<REPO_NAME_2>"; echo -e "\n\n`date` - Sleeping\n\n"; sleep 3; done < listOfItems.txt
This will print out each copy command and the results. If there are any issues you can spot the particular command/item and further troubleshoot.
One thing to note is that Artifactory copies/moves items according to Linux convention which can be slightly unintuitive. The copy command will create a folder within a folder for a folder already existing as seen in the following example (note how ‘folder2’ when copied over is moved within the existing ‘folder2’).
Example of copying a folder in Linux:
Original Structure:
$ tree . ├── folder1 │ └── folder2 │ └── test.txt └── newlocation └── folder2 4 directories, 1 file
Running a copy command will result in a ‘folder2’ being created inside the destination folder not overwriting the existing ‘folder2’ in the ‘newlocation’ location.
$ cp -r folder1/folder2/ newlocation/folder2/ $ tree . ├── folder1 │ └── folder2 │ └── test.txt └── newlocation └── folder2 └── folder2 └── test.txt 5 directories, 2 files
So if you are copying to a specific path you should take care to not include the particular folder name at the end of the path of the destination unless you want to copy the directory structure into that particular location.
An example of the potentially unintended nested folder can be seen in action if you add the “$line” variable to the destination section of the curl and run the loop twice. The second loop will attempt to copy the items into the second repository which will already have the folders copied over from the first loop so they will be placed into the folder instead of overwriting the folder
I.e. Adjusted script which can lead to the unintended nested folder structure
while read -r line; do echo -e "\n\n `date` - Copying $line\n\n"; curl -XPOST -u<USER>:$TOKEN "<ART_URL>/artifactory/api/copy/<REPO_NAME_1>$line?to=<REPO_NAME_2>$line"; echo -e "\n\n`date` - Sleeping\n\n"; sleep 3; done < listOfItems.txt
Loop 1:
Loop 2: