DIBBs Ingestion Service (3.0.0)

Download OpenAPI specification:Download

Getting Started with the DIBBs Ingestion Service

Introduction

The DIBBs Ingestion service offers a REST API with endpoints for standardization and harmonization of FHIR messages. It offers name standardization, date of birth (DoB) standardization, phone number standardization, geocoding, and several utilities for working with FHIR servers.

Running the Ingestion Service

You can run the Ingestion service using Docker, another OCI container runtime (e.g., Podman), or directly from the Python source code.

To run the Ingestion service with Docker follow these steps.

  1. Confirm that you have Docker installed by running docker -v. If you don’t see a response similar to what’s shown below, follow these instructions to install Docker.
❯ docker -v
Docker version 20.10.21, build baeda1f
  1. Download a copy of the Docker image from the PHDI repository by running docker pull ghcr.io/cdcgov/dibbs-ecr-viewer/ingestion:latest.
  2. Run the service with docker run -p 8080:8080 ghcr.io/cdcgov/dibbs-ecr-viewer/ingestion:latest.

Congratulations, the ingestion service should now be running on localhost:8080!

Running from Python Source Code

We recommend running the ingestion service from a container, but if that isn't feasible for a given use case, it may also be run directly from Python using the steps below.

  1. Ensure that both Git and Python 3.13 or higher are installed.
  2. Clone the PHDI repository with git clone https://github.com/CDCgov/dibbs-ecr-viewer.
  3. Navigate to /dibbs-ecr-viewer/containers/ingestion/.
  4. Make a fresh virtual environment with python -m venv .venv.
  5. Activate the virtual environment with source .venv/bin/activate (MacOS and Linux), venv\Scripts\activate (Windows Command Prompt), or .venv\Scripts\Activate.ps1 (Windows PowerShell).
  6. Install all of the Python dependencies for the ingestion service with pip install -r requirements.txt into your virtual environment.
  7. Run the FHIR Converter on localhost:8080 with python -m uvicorn app.main:app --host 0.0.0.0 --port 8080.

Building the Docker Image

To build the Docker image for the Ingestion service from source code instead of downloading it from the PHDI repository, follow these steps.

  1. Ensure that both Git and Docker are installed.
  2. Clone the PHDI repository with git clone https://github.com/CDCgov/dibbs-ecr-viewer.
  3. Navigate to /dibbs-ecr-viewer/containers/ingestion/.
  4. Run docker build -t ingestion ..

The API

When viewing these docs from the /redoc endpoint on a running instance of the ingestion service or the PHDI website, detailed documentation on the API will be available below.

Diagrams

flowchart LR

  subgraph requests["Requests"]
    direction TB
    subgraph GET["fas:fa-download <code>GET</code>"]
      hc["<code>/</code>\n(health check)"]

    end
    subgraph POST-standardization["fas:fa-upload <code>POST Standardization</code>"]
      ecr["<code>/fhir/harmonization/<br>standardization/standardize_names<//code>"]
      phones["<code>/fhir/harmonization/<br>standardization/standardize_phones<//code>"]
      dob["<code>/fhir/harmonization/<br>standardization/standardize_dob<//code>"]
    end
    subgraph POST-geospatial["fas:fa-upload <code>POST Geospatial</code>"]
      geo["<code>/fhir/geospatial/geocode/geocode_bundle<//code>"]
    end
    subgraph POST-linkage["fas:fa-upload <code>POST Linkage</code>"]
      link["<code>/fhir/linkage/link/add_patient_identifier_in_bundle<//code>"]
    end
  end


  subgraph service[REST API Service]
    direction TB
    subgraph mr["fab:fa-docker container"]
      viewer["fab:fa-python <code>ingestion<br>HTTP:3000/</code>"]
    end
    subgraph smarty["fab:fa-docker smarty"]
      smarty-api["fab:fa-python <code>Smarty API</code>"]
    end
    mr <--> |<br><code>GET /geocodeAddress</code>| smarty

  end

  subgraph response["Responses"]
    subgraph JSON["fa:fa-file-alt <code>JSON</code>"]
      rsp-hc["fa:fa-file-code <code>OK</code> fa:fa-thumbs-up"]
      fhirdata["fa:fa-file-code FHIR Data"]
      post-ecr["200"]
    end
  end

hc -.-> mr -.-> rsp-hc
POST-standardization ===> mr ===> post-ecr
POST-linkage ==> mr ===> post-ecr
POST-geospatial --> mr ---> post-ecr

Health Check

This endpoint checks service status. If an HTTP 200 status code is returned along with '{"status": "OK"}' then the service is available and running properly.

Responses

Response samples

Content type
application/json
{
  • "status": "OK"
}

fhir/harmonization

Standardize Names Endpoint

This endpoint standardizes the names in the provided FHIR bundle or resource.

Inputs and Outputs

  • :param input: A dictionary with the schema specified by the StandardizeNamesInput model.
  • :return: A FHIR bundle or resource with standardized names.
Request Body schema: application/json
data
required
object (Data)

A FHIR resource or bundle in JSON format.

Trim (boolean) or Trim (null) (Trim)
Default: true

When true, leading and trailing spaces are removed.

Overwrite (boolean) or Overwrite (null) (Overwrite)
Default: true

If true, data is modified in-place; if false, a copy of data is modified and returned.

Case (string) or Case (null) (Case)
Default: "upper"

The type of casing that should be used.

Remove Numbers (boolean) or Remove Numbers (null) (Remove Numbers)
Default: true

If true, delete numeric characters; if false leave numbers in place.

Responses

Request samples

Content type
application/json
{
  • "data": {
    },
  • "trim": true,
  • "overwrite": true,
  • "case": "upper",
  • "remove_numbers": true
}

Response samples

Content type
application/json
{
  • "status_code": 200,
  • "bundle": {
    }
}

Standardize Phones Endpoint

This endpoint standardizes the phone numbers in the provided FHIR bundle or resource.

The endpoint requires an address so that country code can be generated.

Inputs and Outputs

  • :param input: A dictionary with the schema specified by the StandardizePhonesInput model.
  • :return: A FHIR bundle with standardized phone numbers.
Request Body schema: application/json
data
required
object (Data)

A FHIR resource or bundle in JSON format.

Overwrite (boolean) or Overwrite (null) (Overwrite)
Default: true

If true, data is modified in-place; if false, a copy of data is modified and returned.

Responses

Request samples

Content type
application/json
{
  • "data": {
    },
  • "overwrite": true
}

Response samples

Content type
application/json
{
  • "status_code": 200,
  • "bundle": {
    }
}

Standardize Dob Endpoint

This endpoint standardizes the patient date of birth in the provided FHIR bundle or resourceDates are changed to the FHIR standard of YYYY-MM-DD.

The endpoint returns a FHIR bundle with standardized birth dates.

Request Body schema: application/json
data
required
object (Data)

A FHIR resource or bundle in JSON format.

Overwrite (boolean) or Overwrite (null) (Overwrite)
Default: true

If true, data is modified in-place; if false, a copy of data is modified and returned.

Format (string) or Format (null) (Format)
Default: "%Y-%m-%d"

The date format that the input DOB is supplied in.

Responses

Request samples

Content type
application/json
{
  • "data": {
    },
  • "overwrite": true,
  • "format": "%m/%d/%Y"
}

Response samples

Content type
application/json
{
  • "status_code": 200,
  • "bundle": {
    }
}

fhir/geospatial

Geocode Bundle Endpoint

Given a FHIR bundle and a specified geocoding method, this endpoint geocodes all patient addresses across all patient resources in the bundle.

Two geocoding methods are currently supported — Smarty and the U.S. Census.

If using the Smarty provider, the request parameter: smarty_auth_id, smarty_auth_token and license_type must be provided. If they aren't provided as request parameters, then the service will attempt to obtain them through environment variables. If they can’t be found in either the request parameters or environment variables, an HTTP 400 status will be returned.

Request Body schema: application/json
bundle
required
object (Bundle)

A FHIR resource or bundle in JSON format.

geocode_method
required
string (Geocode Method)
Enum: "smarty" "census"

The geocoding service to be used.

Smarty Auth Id (string) or Smarty Auth Id (null) (Smarty Auth Id)

Authentication ID for the geocoding service. Must be provided in the request body or set as an environment variable of the service if geocode_method is smarty.

Smarty Auth Token (string) or Smarty Auth Token (null) (Smarty Auth Token)

Authentication Token for the geocoding service. Must be provided in the request body or set as an environment variable of the service if geocode_method is smarty.

License Type (string) or License Type (null) (License Type)
Default: "us-rooftop-geocoding-enterprise-cloud"

License type for the geocoding service. Must be provided in the request body or set as an environment variable of the service if geocode_method is smarty.

Overwrite (boolean) or Overwrite (null) (Overwrite)
Default: true

If true, data is modified in-place; if false, a copy of data modified and returned.

Responses

Request samples

Content type
application/json
{
  • "bundle": { },
  • "geocode_method": "smarty",
  • "smarty_auth_id": "string",
  • "smarty_auth_token": "string",
  • "license_type": "us-rooftop-geocoding-enterprise-cloud",
  • "overwrite": true
}

Response samples

Content type
application/json
Example
{
  • "status_code": 200,
  • "bundle": {
    }
}

fhir/linkage

Add Patient Identifier In Bundle Endpoint

This endpoint adds a salted hash identifier to every patient resource in a FHIR bundle . If a salt isn't provided in the request, the value of the 'SALT_STR' environment variable will be used. In the case where a salt isn't provided and 'SALT_STR' isn't defined, an HTTP 500 status code will be returned.

Inputs and Outputs

  • :param input: A JSON formatted request body with schema specified by the AddPatientIdentifierInBundleInput model.
  • :return: A FHIR bundle where every patient resource contains a hashed identifier.
Request Body schema: application/json
bundle
required
object (Bundle)

A FHIR bundle

Salt Str (string) or Salt Str (null) (Salt Str)
Default: ""

The salt to use with the hash. This is intended to prevent reverse engineering of the PII used to create the hash.

Overwrite (boolean) or Overwrite (null) (Overwrite)
Default: true

If true, data is modified in-place; if false, a copy of data modified and returned.

Responses

Request samples

Content type
application/json
{
  • "bundle": { },
  • "salt_str": "",
  • "overwrite": true
}

Response samples

Content type
application/json
{
  • "status_code": 0,
  • "message": "string",
  • "bundle": { }
}

fhir/transport

Upload Bundle To Fhir Server Endpoint

This endpoint uploads all of the resources in a FHIR bundle to a FHIR server.

Inputs and Outputs

  • :param input: A JSON formated request body with schema specified by the UploadBundleToFhirServerInput model.
  • :return: A dictionary containing the status code and body of the response received from the FHIR server.
Request Body schema: application/json
bundle
required
object (Bundle)

A FHIR bundle (type 'batch' or 'transaction') to post. Each entry in the bundle must contain a request element in addition to a resource. The FHIR API provides additional details on creating FHIR-conformant batch/transaction bundles.

Cred Manager (string) or Cred Manager (null) (Cred Manager)

The credential manager used to authenticate to the FHIR server.

Fhir Url (string) or Fhir Url (null) (Fhir Url)

The url of the FHIR server to upload to.

Responses

Request samples

Content type
application/json
{
  • "bundle": { },
  • "cred_manager": "azure",
  • "fhir_url": "string"
}

Response samples

Content type
application/json
{
  • "status_code": 0,
  • "message": "string",
  • "bundle": { }
}

cloud/storage

Write Blob To Cloud Storage Endpoint

This endpoint uploads the information from a blob into a specified cloud provider's storage organizing it by a bucket name as well as a file name.

Inputs and Outputs

  • :param input: A JSON formatted request body (blob) with schema specified by the WriteBlobToStorageInput model.
  • :return: A dictionary containing the status code and body of the response received from the cloud provider.
Request Body schema: application/json
blob
required
object (Blob)

Contents of a blob to be written to cloud storage.

Cloud Provider (string) or Cloud Provider (null) (Cloud Provider)

The cloud provider hosting the storage resource that the blob will be uploaded to. Must be provided in the request body or set as an environment variable of the service.

Bucket Name (string) or Bucket Name (null) (Bucket Name)

Name of the cloud storage bucket that the blob should be uploaded to. Must be provided in the request body or set as an environment variable of the service.

file_name
required
string (File Name)

Name of the blob

Storage Account Url (string) or Storage Account Url (null) (Storage Account Url)
Default: ""

The URL of an Azure storage account. Must be provided in the request body or set as an environment variable of the service is 'cloud_provider' is 'azure'.

Responses

Request samples

Content type
application/json
{
  • "blob": { },
  • "cloud_provider": "azure",
  • "bucket_name": "string",
  • "file_name": "string",
  • "storage_account_url": ""
}

Response samples

Content type
application/json
{
  • "status_code": 0,
  • "message": "string",
  • "bundle": { }
}