-
Notifications
You must be signed in to change notification settings - Fork 1
Contributing Datasets to the NGDAC
The following steps must be completed to submit glider data to the Integrated Ocean Observing System (IOOS) National Glider Data Assembly Center (NGDAC):
- Create trajectory feature type NetCDF files that that conform the IOOS-specification. (Jump)
- Register with the NGDAC as a data provider (Jump)
- Contact the DAC administrators to acquire a WMO id for the deployment and then register the deployment. (Jump)
- Push the NetCDF files to the NGDAC. (Jump)
The full registration process should take approximately 24 hours to complete, so it's highly recommended to begin the process as early as possible prior to the glider deployment.
Once uploaded, deployment datasets may be accessed via THREDDS at tds.gliders.ioos.us. Current services include:
- OPeNDAP
- HTTP download
- NcML
- Attributes Convention Dataset Discovery Service (ACDD)
- ncISO
- NetCDF Sensor Observation Service (ncSOS)
The NGDAC accepts files that conform to a community developed trajectory feature type NetCDF file specification. This specification was designed to be consistent with the NODC, Australian IMOS and the EGO-GROOM NetCDF descriptions. The format achieves the following:
- Preserves raw data sampling frequencies in the original data files
- Meets CF metadata conventions
- Provides required deployment metadata, in the form of global and variable attributes, that allows for harvesting of datasets by digital libraries
- Allows aggregation of individual files into deployment datasets using the Thematic Realtime Environmental Data Distribution Service, TDS.
The current file description (CDL, NcML and NetCDF) can be found here.
An established data provider is someone who has contacted the DAC and has requested an account. All data provider registration requests should be directed to:
John Kerfoot
Institute of Marine & Coastal Sciences
Rutgers University
(848) 932-3344
or
David Foster
Applied Science Associates
(401) 789-6224
The following information must be provided for each request:
- Contact name
- Contact organization
- Email address
- Telephone number
A user account will be created to allow the authorized data provider to push data to the NGDAC via sftp or ftp.
Registering a new deployment is a 2-step process. Gliders must be registered and assigned a WMO id, in order for the datasets to be broadcast on the Global Telecommunication System. This registration process will ultimately be automated via a web form. Currently, however, please direct all glider registration requests to the contacts listed above.
The following information must be provided for each request:
- Data provider (from Step #1)
- Glider name
- Approximate deployment date
- Approximate deployment location (GPS coordinates)
Once the request is received, we will forward the request to the National Data Buoy Center for a new WMO id. Once the WMO id has been assigned, the deployment must be registered prior to uploading NetCDF files.
We've created a web page to simplify this task. Once you've logged in using your data provider credentials (top right of the page), click the Your Missions link. This will take you to the deployment registration form. You must specify a unique deployment name and the WMO id assigned to this glider for the new deployment. Then click Register Deployment to create the deployment NetCDF directory on the ftp server. This directory is the directory that the NetCDF files should be uploaded to.
After the deployment has been registered, you can click on the deployment name to take you to the deployment metadata page. Information on the deployment (i.e.: operator name and deployment start date and location) should be filled in. Once the deployment has been completed (glider recovered), click the Completed check box to denote that the data is ready for archiving by NODC.
The data provider user account provides sftp/ftp push access to the user's home directory:
$USERNAME/upload
where $USERNAME is the user name for the account.
A new directory should be created, by the user, for each glider deployment. The following convention is strongly recommended:
$USERNAME/upload/$GLIDER-YYYYMMDDThhmm
where $GLIDER is the name of the registered glider and YYYYMMDDThhmm is the timestamp corresponding to the start of the deployment.
Please remember to use the proper file naming convention and to include the wmoid.txt file containing the assigned WMO id for that deployment.
The resulting deployment directory structure will look something like this:
USERNAME/upload/$GLIDER-YYYYMMDDThhmm
segment1.nc
segment2.nc
segment3.nc
...
wmoid.txt
Currently, files may be overwritten, but not deleted from the server.
The github site contains a generic ftp script, written in Perl, that can be used to upload the files to the NGDAC. The script requires the following non-core modules:
You must specify your credentials in the $USER and $PASS variables contained in the script.
Please email John Kerfoot (IMCS/Rutgers University) with any questions regarding the script.
A registered data provider may point a browser to ftp.gliders.ioos.us, enter their credentials, and view the status (number of files uploaded and last update time) of each of the deployment datasets.