blob

Functions for interacting with Azure Blob Storage.

Functions

create_storage_container_if_not_exists(→ None)	Create an Azure blob storage container if it does not already exist.
upload_to_storage_container(→ None)	Upload a file or list of files to an Azure blob storage container.
download_from_storage_container(→ None)	Download a list of files from an Azure blob storage container.
get_node_mount_config(...)	Get configuration for mounting Azure Blob Storage containers to Azure Batch nodes via blobfuse.

Module Contents

create_storage_container_if_not_exists(blob_storage_container_name: str, blob_service_client: azure.storage.blob.BlobServiceClient) → None

Create an Azure blob storage container if it does not already exist.

Parameters

blob_storage_container_name

Name of the storage container.

blob_service_client

The blob service client to use when looking for and potentially creating the storage container.

Returns

None

None on success.

upload_to_storage_container(file_paths: str | list[str], blob_storage_container_name: str, blob_service_client: azure.storage.blob.BlobServiceClient, local_root_dir: str = '.', remote_root_dir: str = '.') → None

Upload a file or list of files to an Azure blob storage container. This function preserves relative directory structure among the uploaded files within the storage container.

Parameters

file_paths

File or list of files to upload, as string paths relative to local_upload_dir. A single string will be coerced to a length-one list.

blob_storage_container_name

Name of the blob storage container to which to upload the files. Must already exist.

blob_service_client

BlobServiceClient to use when uploading.

local_root_dir

Root directory for the relative file paths in local storage. Default "." (use the local working directory).

remote_root_dir

Root directory for the relative file paths within the blob storage container. Default "." (start at the blob storage container root).

Returns

None

None on success.

Raises

Error if the blob storage container does not exist.

download_from_storage_container(file_paths: list[str], blob_storage_container_name: str, blob_service_client: azure.storage.blob.BlobServiceClient = None, local_root_dir: str = '.', remote_root_dir: str = '.', **kwargs) → None

Download a list of files from an Azure blob storage container. Preserves relative directory structure.

Parameters

file_paths

File or list of files to upload, as string paths relative to local_upload_dir. A single string will be coerced to a length-one list.

blob_storage_container_name

Name of the blob storage container from which to download the files. Must already exist.

blob_service_client

BlobServiceClient to use when downloading. If None, attempt to create one via client.get_blob_service_client() using provided **kwargs, if any. Default None.

local_root_dir

Root directory for the relative file paths in local storage. Default "." (use the local working directory).

remote_root_dir

Root directory for the relative file paths within the blob storage container. Default "." (start at the blob storage container root).

**kwargs

Keyword arguments passed to get_blob_service_client().

Returns

None

None on success.

Raises

Error if the blob storage container does not exist.

get_node_mount_config(storage_containers: str | list[str], account_names: str | list[str], identity_references: azure.batch.models.ComputeNodeIdentityReference | list[azure.batch.models.ComputeNodeIdentityReference], shared_relative_mount_path: str = '', mount_names: list[str] = None, blobfuse_options: str | list[str] = None, **kwargs) → azure.batch.models.MountConfiguration

Get configuration for mounting Azure Blob Storage containers to Azure Batch nodes via blobfuse.

Parameters

storage_containers

Name(s) of the Azure Blob storage container(s) to mount.

account_names

Name(s) of the Azure Blob storage account(s) in which to look for the storage container(s). If a single value, look for all storage containers within the same storage account. Otherwise, look for each container within the corresponding account. The function will raise an error if there is more than one account_names value but a different number of storage_containers, as then the mapping is ambiguous.

identity_references

Valid models.ComputeNodeIdentityReference objects for the node to use when connecting to the storage_containers, or an iterable of such objects with one object for each of the storage_containers.

shared_relative_mount_path

Path relative to the fsmounts directory within the running node at which to mount the storage containers. Default “”, i.e. mount within fsmounts itself.

mount_names

Iterable of names (or paths) for the individual mounted storage containers relative to the shared_relative_mount_path. If None, use the storage container names given in

storage_containers as the mount_names.

blobfuse_options

Additional options passed to blobfuse. Default None.

**kwargs

Additional keyword arguments passed to the models.AzureBlobFileSystemConfiguration constructor.

Returns

list[models.MountConfiguration]

A list of instantiated models.MountConfiguration objects describing the desired storage container mounts.