Demo for xnat downloader

The xnat_downloader tool allows us to use code to download MRI data from the storage space that MRRF sends our data to (called XNAT) and store it on our computers in both it's "raw" and "reconstructed" forms. Access the source data for xnat_downloader here.

The "raw" form for our MR image data are referred to as DICOM images. DICOM stands for Digital Imaging and Communication in Medicine, and provides includes metadata about the image and the raw components of the image data. DICOM data have a file extension of .dcm and all the DICOM files that make up an image are stored in a folder looking something like this:

We don't typically look at the raw form of the header data. Rather it's common to use a tool that takes the DICOM files as input and produces output that includes (a) meta-data about the image, extracted from the DICOM header data, (b) a reconstructed image in the form of a NIFTI file. The tool most commonly used for this is dcm2niix, developed by Dr. Chris Rorden.

Fortunately, the xnat_downloader tool both downloads data from our xnat storage space AND reconstructs it with dcm2niix. Here are steps for using xnat_downloader:

1. Obtain access to an executable source of xnat_downloader.

• You can download this container to your server using a bash terminal using this command: wget -O xnat_downloader-v0.2.8.sif https://osf.io/47vs5/download
• For example, in our lab we use a singularity container, which is like a mini virtual machine stored on our server. We store the container in a folder that contains all the singularity containers our lab uses, for example the path is /Volumes/vosslabhpc/UniversalSoftware/SingularityContainers/xnat_downloader-v0.2.8.sif.

2. Create a config file that will be your input to xnat_downloader and store this as a .json file. Here are example contents of a config file:

	"server": "https://rpacs.iibi.uiowa.edu/xnat",
	"destination": "/out",
	"project": "PSYCH_4025",
	"subjects": ["20231017_1","DEMO"],
	"sub_dict": {
							"20231017_1":"001",
							"DEMO":"002" 
						},				
	"scan_dict": {
        "func-bold_task-REST": "func-bold_task-rest",
        "anat-T1w": "anat-T1w",
        "func-bold_task-taskswitch":"func-bold_task-flanker" 
	              }
}

Usage notes:

• The project field can be found in the xnat gui at the MRRF website
• the subjects field is a list of subjects as they are named in your subject list on xnat
• the sub_dict field is a key-value pair where you can link the original name with a new name you want for the image when stored, with format of old:new
• the scan_dict field is a key-value pair for scan names as stored on xnat and how you want to store the file

3. Create a shell script that you can use to run xnat_downloader from a place that you have access to the code. Here is an example template for usage with a singularity container:

#!/usr/bin/env bash
#$ -pe smp 8
#$ -q all.q
#$ -m bea
#$ -M [email protected]
#$ -e /Shared/logDirectory
OMP_NUM_THREADS=10
singularity run --cleanenv -B /Shared/DirectoryRoot/:/out \
        /Shared/SingularityContainerLocation/xnat_downloader-v0.2.8.sif \
        -i /Shared/ConfigFileLocations/Project_download_config.json \
        -c /Users/HAWKID/xnat_pass_config.json


# DirectoryRoot should be a folder that exists

Here is the template filled out for downloading participants listed in the config file above:

#!/usr/bin/env bash
#$ -pe smp 8
#$ -q all.q
#$ -m bea
#$ -M [email protected]
#$ -e /Shared//vosslabhpc/Projects/CourseData/xnat_downloader_demo

OMP_NUM_THREADS=10
singularity run --cleanenv -B /Shared/vosslabhpc/Projects/CourseData/xnat_downloader_demo/bids:/out \
        /Shared/vosslabhpc/UniversalSoftware/SingularityContainers/xnat_downloader-v0.2.8.sif \
        -i /Shared/vosslabhpc/Projects/CourseData/xnat_downloader_demo/download_config.json

Usage notes:

• when running this script (stored as a .sh shell script) you will be prompted for your xnat user name and password. You will only be prompted once for one call of the config file. However, if you want this script to be able to run automatically without user interaction, you use an additional -c flag for a user/password configuration file. This would require storing your username and password so you should think about feasibility of this information to stay safe. Example format for the user config file:

{"password": "PASSWORD", "user": "HAWKID", "server": "https://rpacs.iibi.uiowa.edu/xnat"}