Azure Blob Removal and Backup/Restore with PowerShell

Azure Blob Storage is a great service for storing unstructured object data in the cloud. In the development phase of  an application utilizing Azure Blob Storage, it can be necessary to wipe out an Azure test account as they get cluttered with test data. In a production application we need a way to backup valuable blob data and restore as needed. In this blog post, I will show you how to do each of these actions using Azure PowerShell.

Removal

This script will remove all containers from an Azure instance. This is useful for clearing out Azure development accounts.

First we need to connect to our Azure storage account and create a storage context:

$AzureAccountName = "TestAzureAccount"
$AzureKey = "azurekey"
$Ctx = New-AzureStorageContext $AzureAccountName -StorageAccountKey $AzureKey

Next we loop through all Azure containers in the account and delete them along with their contents. The -Force parameter is added so we are not prompted when deleting containers:

Get-AzureStorageContainer -Context $Ctx |  % {
    "Deleting container " + $_.Name
    Remove-AzureStorageContainer -Name $_.Name -Context $Ctx -Force
}

This will indiscriminately remove all containers from a storage account. Think twice before running this in production!

Backup

Azure blob storage does not have a built in service to backup blobs. On a production account we need a way to backup blobs in case of data corruption or other disaster scenarios. This script will download all blobs from a given Azure storage account to the local file system.

First we choose a local folder to download our blobs to and create a storage context:

$LocalFolder = "C:/AzureBackup/"
$AzureAccountName = "TestAzureAccount"
$AzureKey = "azurekey"
$Ctx = New-AzureStorageContext $AzureAccountName -StorageAccountKey $AzureKey

Next we create a folder for each container:

Get-AzureStorageContainer -Context $Ctx | % {
    $containerName = $_.Name
    $destination = $LocalFolder + $containerName
    "Creating folder for container " + $containerName
    New-Item -Path $destination -ItemType Directory -Force > $null

Finally we download each blob from the container into the container's folder. Forward slashes in blob names will translate to folders in the container directory.

    Get-AzureStorageBlob -Container $containerName -Context $Ctx | % {
        Get-AzureStorageBlobContent -Destination $destination -Context $Ctx -Blob $_.Name `
            -Container $containerName -Force > $null
        "Created blob " + $_.Name
    } 
}

Restore

In a disaster scenario we may need a way to restore the blob backup to an Azure storage account. This script will copy all of the files from our local backup to the server.

First we configure our local folder which has our blob backup and create a storage context:

$LocalFolder = "C:/AzureBackup/"
$AzureAccountName = "TestAzureAccount"
$AzureKey = "azurekey"
$Ctx = New-AzureStorageContext $AzureAccountName -StorageAccountKey $AzureKey

Next we create a container for each top-level folder in our backup directory if it does not already exist:

ls $LocalFolder | % {
    try
    {
        New-AzureStorageContainer -Name $_.Name -Context $Ctx -ErrorAction Stop
        "Creating container " + $_.Name
    }
    catch [Microsoft.WindowsAzure.Commands.Storage.Common.ResourceAlreadyExistException]
    {
        "Container " + $_.Name + " already exists"
    }

We then set the powershell directory to the container directory so that we can get the blob's path relative to the container in the next step:

    $Container = $_.Name
    cd $_.FullName

Finally, we upload each blob in the container's directory to the server container with its full relative path as the blob name:

    ls -File -Recurse | % {
        "Uploading blob " +  $_.FullName
        Set-AzureStorageBlobContent -File $_.FullName -Context $Ctx -Container $Container `
            -Blob (Resolve-Path -Relative $_.FullName) -Force > $null
    } 
}

Notes

This is a simplified script that is meant to be used as a basis for a backup and restore process for Azure blob storage. You will likely need to filter containers and blobs to download and restore only what you need. On a production server, the backup and restore could be very heavy. Consider an incremental backup and restore approach using Azure upload and modified dates to check for updates.


[original publish date: 10/03/16]