Documentation for DEX products
This guide is for our current DEX Upload API users who were onboarded to the API before 10/1/2024.
The CDC Data Exchange (DEX) Upload API is an open-source service created to support public health Data Senders in their effort to share critical public health data with internal CDC Programs. The open-source model allows users to tailor the tool to fit specific data needs.
The DEX Upload API service is a highly scalable, reliable means of transiting files of nearly any type and size from public health partners to the CDC, even when sent over unreliable network connections. As a current user, your organization will continue to benefit from the DEX Upload API’s robust list of features.
Resumable Uploads: Implementation of the Tus service as a library in a Golang application, facilitating high-volume file uploads over HTTP.
Authentication: OAuth provider approval enforcement to ensure security.
Metadata Verification: Enforcement of upload submission metadata standards.
File Delivery: Routing of uploaded files to configured target destinations.
File Observability: Upload lifecycle event tracking and endpoints for health check, information, and application version.
Retry Delivery: Tool that delivers files to target destinations that uploaded successfully but were unsuccessful in delivery.
User Interface: Facilitates uploads and observability within a user interface.
DEX Upload API is built on Tus, an open-source protocol that allows data submitters to resume data transfers after a communication failure between the submitter and CDC. This feature is essential for data submitters with large amounts of data to share. The DEX Upload API also:
To learn more about the product and view code examples, you can visit the following DEX Upload API resources:
DEX Upload API GitHub repository
DEX Upload API Configuration README
DEX OpenAPI (Swagger) specification
DEX Upload API offers different environments that authenticated users will use during different parts of their onboarding process:
Users start in the DEX Upload API STG Environment. Here, the DEX Upload API Team will work with you to conduct end-to-end testing with production-like volume in real-time. We’ll work out any remaining defects before moving to the final environment.
All deployment activities have been sequenced, reviewed, and confirmed by all DEX team members. As a result, teams in the PRD Environment are approved to begin data transfers to the CDC. The DEX Upload API Team will work with you to determine when you’ll make your transfers.
DEX Upload API offers different endpoints that authenticated users will need during different parts of the file upload process (note: to successfully reach each endpoint below, your url should include upload/ before your chosen endpoint.
You will use this endpoint to upload multiple files over HTTP. Using this endpoint exposes a server that implements the Tus Resumable Upload protocol. You must implement a Tus client to connect in order to communicate in a way that Tus understands. The endpoint uses a combination of POST, PATCH, and OPTION requests to upload files. This endpoint authenticates users via bearer tokens in the Authorization header.
You will use this endpoint to authenticate with the DEX Upload API. You must have a SAMS system account in order to use this endpoint, as it uses SAMS as the identity provider.
To get a token, enter your username and password in an HTTP POST request. This request routes to the Upload API Management layer, which contains the client credentials needed for the Password Grant flow. With the combination of your user and client credentials, SAMS issues a short-lived authentication token, as well as a refresh token. You can use this token in future requests to the Upload API by providing it as a Bearer token in the Authorization header of your HTTP requests.
This endpoint allows you to check the status of the Upload API system, and its dependent systems. You can send an HTTP GET request to this endpoint and receive a 200 response with JSON in the body containing the overall status of the system, as well as statuses of dependent systems. This endpoint authenticates users via bearer tokens in the Authorization header.
| Production | api.cdc.gov/upload/info/{upload-id} |
| Staging | apistg.cdc.gov/upload/info/{upload-id} |
| Test | apitst.cdc.gov/upload/info/{upload-id} |
| {upload-id} | Universally Unique Identifier (UUID) of the upload assigned by the upload server |
This endpoint serves as a mechanism for you to verify that your upload has completed successfully and that they are available in the underlying storage. You can send an HTTP GET request to this endpoint providing your unique ID of your upload as a path parameter.
This endpoint authenticates users via bearer tokens in the Authorization header.
Users can access the /upload/info/{uploadId} endpoint to have objects returned from the API that can provide information about a wide range of actions and issues.
The Manifest Object provides metadata about the uploaded file. Metadata returned includes the following:
All metadata fields submitted, including –
The File_Info Object returns the size of the uploaded file in bytes at the time of the GET request. Metadata returned includes the following:
Size_Bytes: value returned will vary based on upload progress
Updated_At: value returned will vary based on upload progress
The Upload_Status Object provides the current status of the upload at the time of the GET request. Metadata returned includes the following:
Status: value returned will vary based on upload progress
Chunk_Received_At: returns the most recent successful upload chunk processed date/time; if the upload is complete, this value is the date/time when the last chunk completed.
The Deliveries Object provides information about the target file delivery for the upload. Metadata returned includes the following:
Status: value returned will vary based on target delivery success
Name: returns the DEX assigned name of the program delivery target location
Location: returns the target location to which the file was delivered
Delivered_At: returns the datetime of file delivery to the target location
Issues: value returned will include target delivery failure messaging
Current users are already sending data with the DEX Upload API. As a result, the following information will help users access the API.
All current users will use the Secure Access Management System (SAMS) Access to authenticate into the DEX Upload API environments. The DEX Upload API uses a password grant flow, where users exchange their SAMS credentials for an authentication token and a refresh token. The authentication token is valid for one hour. After expiring, the user will need to use the refresh token to get a new auth token. Authentication is managed through the auth API located at the /oauth endpoint.
To login, provide your SAMS username and password to request an authorization token. This can be done by sending a POST request to the /oauth API endpoint.
Your request must have the following header:
Content-Type: application/x-www-form-urlencoded
Provide your SAMS credentials in the following query parameters:
An example request URL would look like this:
https://<hostname>/oauth?username=<your-username>\&password=<your-password>
You can authenticate into the DEX Upload API Staging Environment (STG) for testing or the DEX Upload API Production Environment (PRD) to begin real-time, real-world transfer of your public health data to the CDC.
You can also read more about the DEX Upload API’s OAuth requirements in our GitHub repository.
As an existing DEX Upload API user, you should have your storage locations configured. In the event that you need to add or confirm locations, please consult the DEX Upload API guidance on configuring upload routing and delivery targets.
You can read more about our metadata requirements in the DEX Upload API Metadata Requirements section below.
Users will upload files to the DEX Upload API using their chosen Tus resumable upload client. The DEX Upload API accepts uploads using Java, JavaScript, Python, and directly in the user’s browser. Users can view example scripts for each option that can help them build their upload correctly:
Use the /info endpoint to confirm the status of your file transfer. You’ll see one of two responses:
Metadata plays an important role in how the DEX Upload API organizes, manages, and routes public health data. As an existing user, the DEX Upload API Team has already worked with you to ensure that your program can meet the requirements for submitting your metadata to the CDC. Your Sender Manifest is the approved metadata for your organization. The Sender Manifest accompanies file uploads initiated by senders.
Values in the Sender Manifest are validated against expected values provided by the program and by senders during onboarding. Please review the Sender Manifest Schema Definition Json as well as the following fields that are required for all submissions:
| Required Metadata | |||
|---|---|---|---|
| Metadata field name | Definition and purpose | Type | How to use this field |
| data_stream_id (example: celr) | Identifies the CDC program the file should be contained in | String | This value was determined during onboarding |
| data_stream_route (example: hl7v2) | Defines the path that data takes through DEX to arrive at the CDC – this can be the name of the internal folder destination for the data file | String | This value was determined during onboarding |
| sender_id (example: APHL) | Identifies the sender, machine, or intermediary that’s sending data | Array | This value was determined during onboarding |
| data_producer_id (example: FL-SPHL) | Identifies the public health authority that supplies the data | Array | This value was determined during onboarding (Some data will be sent directly by senders who will not have a data_producer_id – if this is you, enter “null”) |
| Jurisdiction (example: FL) | Defines the city, county, state, tribe, or territory where the data originates | Array | This value is decided by you and the DEX Upload API Team during onboarding. (If you don’t track this information, the value is “null”) |
| received_filename (example: filename.hl7) | Identifies the name of the file being sent | String | The DEX Team worked with you to create unique markers for your file name during your onboarding process to help organize your data submissions. |
| Version (example: 2.0) | Defines the version of the metadata being sent | String | This value was determined during onboarding |
| supporting_metadata | Metadata specific to your program that you track | String or Array | If you track information for your organization outside the fields above, you can include it under this field |
There are characters that should never be used when naming your metadata. Using these characters could result in upload failures. Those characters are:
| Character | Character Description |
|---|---|
| / | forward slash |
The DEX Upload API Team can work with you to ensure that your metadata meets our requirements, but you should also review your naming conventions to make sure that you are not using these special characters – especially in your supporting metadata.
An issue with metadata may result in an error response such as the one shown below:
Error: tus: unexpected response while creating upload
"upload_id":"ba2ba966-508c-46e8-b3a9-aa272c7c4162", "validation_errors":
["validation failure","sender_id had disallowed value RI-YYY",
"validation failure","data_producer_id had disallowed value RI-ZZZ",
"validation failure","field jurisdiction was missing"]
Provided metadata:
{ "version": "2.0", "data_stream_id": "ehdi", "data_stream_route": "csv", "received_filename": "example.csv", "data_producer_id": "RI-ZZZ", "sender_id": "RI-YYY", "meta_ext_objectkey": "test-obj-key", "meta_ext_file_timestamp": "test-timestamp", "meta_ext_uploadid": "test-upload-id", "meta_ext_source": "test-src", "meta_ext_filestatus": "test-file-status" }
In this example, the client has failed to provide some metadata that is required for jurisdiction, and in addition, they provided metadata values that were not allowed for sender_id and data_producer_id. Responses will vary depending on the Tus client that you decide to use. When your upload is successful, Tus clients that provide status callbacks will indicate the last upload attempt was successful. Others may finish silently, so be sure you choose the client that meets your needs.
You can decide which Tus client you want to use to send your data. When an error occurs, all Tus clients provide feedback. Typically, this feedback arrives as exceptions or error handler callbacks, like a 404 error. The response will indicate the reason for the error.
DEX uses Akamai’s Let’s Encrypt integration to obtain, deploy, and manage SSL/TLS certificates through the Let’s Encrypt Certificate Authority. All Upload API users must add this certificate to their certificate store and ensure that their trust store is set to trust “Let’s Encrypt” root certificates.
If you have a new addition to your team, you’ll need to get your new team member set up with a SAMS account to be able to transfer and view data. To register a new user for SAMS, complete the following steps:
An existing user from your organization should email the DEX Upload API Team at with the name and email address of your new user.
The DEX Upload API Team will create a custom invitation for your new user. The invitation will arrive in the email you provided to the DEX Team from sams-no-reply@cdc.gov. The email subject will be “U.S. Centers for Disease Control: SAMS Partner Portal – Invitation to Register.”
Can’t find the invitation?
If your new user doesn’t see their invitation in their inbox or their invitation expired, contact the DEX Team to request that they resend it. If the invitation is not working, reach out to the SAMS Helpdesk at 877-681-2901 or samshelp@cdc.gov.
The invitation includes a username and temporary password. They’ll need to have these ready when they register for SAMS. They’ll also need:
Click the registration link in the invitation. Enter the username and temporary password; the user will be prompted to change their password. Next, they’ll need to validate their legal name, organization, and contact information. To complete the registration, click the “submit” button. \
Once the steps above have been completed, the DEX Upload API Team will be notified to approve the access request. This may take a few minutes or a few days, given how busy the team is. Your new users will receive an email from sams-no-reply@cdc.gov with the subject “U.S. Centers for Disease Control: SAMS Partner Portal – SAMS Activity Authorization” to confirm their approval.
If you need additional information or have questions, please contact the DEX Upload API Team.
| Report an incident: | dexuploadapi@cdc.gov |
| Contact the team: | dexuploadapi@cdc.gov |
Users can see real-time product updates in the public DEX Upload API GitHub Repository. Specific information about product releases and new features can be found in the DEX Upload Release Notes. To see a history of product changes, visit the DEX Upload API Change Log.