This page will provide you with information on how to use a Command line script function to upload and download data from the UNSW Data Archive. The Command line script is for users with a technical background and with large volumes of data. The Command line script allows automation, easier repeatability, and control.

This page contains the following elements:

 

Preparing to use the Data Archive

Before you start uploading your research data to the Data Archive, make sure it is organised and in alignment with UNSW recommendations and, that you know the level of access you have to the Collections inside the Data Archive. Read the Preparing data for upload and the RDMP and the Data Archive pages for more information.

Note: a word of caution; it is recommended that you retain a local copy of the research data you have uploaded to the Data Archive for two weeks, before local deletion, to ensure that all three copies of your data have been created and one copy has been taken off-site. See the When should I delete / remove the local copy of my data? FAQ for more information.

 

Should I use the Command line script?

The UNSW Data Archive Web application provides access to an interface enabling you to upload, download, locate, and review your research data. The Data Archive Web application is suitable for researchers who wish to search the Data Archive and to either upload or download less than 50 gigabytes (GB) of data and/or less then 1000 files (during a single session).

Researchers who frequently upload or download data, and/or generate datasets, may wish to schedule a regular upload or download of data using the Command Line Script or use SFTP rather than manually uploading and downloading data via the Web application.

Note: The script can be used for automation of large, manual, uploads of data to the store.

  • Researchers who upload or download more than 50 GB of data and/or more than 1000 files, at a time, please see the SFTP Client guide.
  • Researchers who wish to schedule a regular upload or download of data, please follow the steps set out below.

 

System/setup requirements

To upload and download research data using the Command line script you will need:

  • To be using one of the operating systems below:
    • Window 7 Home Premium and Enterprise Editions
    • Mac OS X 10.6.8
    • Centos 5.9
  • The latest version of Java.
  • An active UNSW zID and zPass. To verify your zID/zPass visit the UNSW Identity management page.
  • The source and destination paths of the data you wish to upload and/or download.

Contact your local IT support or the UNSW IT Service Centre (9385 1333) to have Java installed, to obtain the script, and to discuss other operating system compatibility.

 

Help and further information: