[Windows Azure] How to use the Windows Azure Blob Storage Service in .NET

How to use the Windows Azure Blob Storage Service in .NET

version 1.7 version 2.0

This guide will demonstrate how to perform common scenarios using the Windows Azure Blob storage service. The samples are written in C# and use the Windows Azure Storage Client Library for .NET. The scenarios covered include uploading, listing,downloading, and deleting blobs. For more information on blobs, see the Next steps section.

Table of contents

• What is Blob Storage
• Concepts
• Create a Windows Azure Storage account
• Setup a storage connection string
• How to: Programmatically access blob storage
• How to: Create a container
• How to: Upload a blob into a container
• How to: List the blobs in a container
• How to: Download blobs
• How to: Delete blobs
• Next steps

What is Blob Storage

Windows Azure Blob storage is a service for storing large amounts of unstructured data that can be accessed from anywhere in the world via HTTP or HTTPS. A single blob can be hundreds of gigabytes in size, and a single storage account can contain up to 100TB of blobs. Common uses of Blob storage include:

• Serving images or documents directly to a browser
• Storing files for distributed access
• Streaming video and audio
• Performing secure backup and disaster recovery
• Storing data for analysis by an on-premises or Windows Azure-hosted service

You can use Blob storage to expose data publicly to the world or privately for internal application storage.

Concepts

The Blob service contains the following components:

• Storage Account: All access to Windows Azure Storage is done through a storage account. This is the highest level of the namespace for accessing blobs. An account can contain an unlimited number of containers, as long as their total size is under 100TB.

• Container: A container provides a grouping of a set of blobs. All blobs must be in a container. An account can contain an unlimited number of containers. A container can store an unlimited number of blobs.

• Blob: A file of any type and size. There are two types of blobs that can be stored in Windows Azure Storage: block and page blobs. Most files are block blobs. A single block blob can be up to 200GB in size. This tutorial uses block blobs. Page blobs, another blob type, can be up to 1TB in size, and are more efficient when ranges of bytes in a file are modified frequently. For more information about blobs, see Understanding Block Blobs and Page Blobs.

• URL format: Blobs are addressable using the following URL format:
  http://<storage account>.blob.core.windows.net/<container>/<blob>

  The following URL could be used to address one of the blobs in the diagram above:
  http://sally.blob.core.windows.net/movies/MOV1.AVI

Create a Windows Azure Storage account

To use storage operations, you need a Windows Azure storage account. You can create a storage account by following these steps. (You can also create a storage account using the REST API.)

1. Log into the Windows Azure Management Portal.

2. At the bottom of the navigation pane, click NEW.

   

3. Click DATA SERVICES, then STORAGE, and then click QUICK CREATE.

   

4. In URL, type a subdomain name to use in the URI for the storage account. The entry can contain from 3-24 lowercase letters and numbers. This value becomes the host name within the URI that is used to address Blob, Queue, or Table resources for the subscription.

5. Choose a Region/Affinity Group in which to locate the storage. If you will be using storage from your Windows Azure application, select the same region where you will deploy your application.

6. Optionally, you can enable geo-replication.

7. Click CREATE STORAGE ACCOUNT.

Setup a storage connection string

The Windows Azure Storage Client Library for .NET supports using a storage connection string to configure endpoints and credentials for accessing storage services. You can put your storage connection string in a configuration file, rather than hard-coding it in code:

• When using Windows Azure Cloud Services, it is recommended you store your connection string using the Windows Azure service configuration system (*.csdef and *.cscfg files).
• When using Windows Azure Web Sites, Windows Azure Virtual Machines, or building .NET applications that are intended to run outside of Windows Azure, it is recommended you store your connection string using the .NET configuration system (e.g. web.config or app.config file).

Retrieval of your connection string is shown later in this guide.

Configuring your connection string when using Cloud Services

The service configuration mechanism is unique to Windows Azure Cloud Services projects and enables you to dynamically change configuration settings from the Windows Azure Management Portal without redeploying your application.

To configure your connection string in the Windows Azure service configuration:

1. Within the Solution Explorer of Visual Studio, in the Roles folder of your Windows Azure Deployment Project, right-click your web role or worker role and click Properties.
   

2. Click the Settings tab and press the Add Setting button.
   

   A new Setting1 entry will then show up in the settings grid.

3. In the Type drop-down of the new Setting1 entry, choose Connection String.
   

4. Click the ... button at the right end of the Setting1 entry. The Storage Account Connection String dialog will open.

5. Choose whether you want to target the storage emulator (Windows Azure storage simulated on your local machine) or a storage account in the cloud. The code in this guide works with either option. Enter the Primary Access Key value copied from the earlier step in this tutorial if you wish to store blob data in the storage account we created earlier on Windows Azure.
   

6. Change the entry Name from Setting1 to a friendlier name like StorageConnectionString. You will reference this connection string later in the code in this guide.
   

Configuring your connection string using .NET configuration

If you are writing an application that is not a Windows Azure cloud service, (see previous section), it is recommended you use the .NET configuration system (e.g. web.config or app.config). This includes Windows Azure Web Sites or Windows Azure Virtual Machines, as well as applications designed to run outside of Windows Azure. You store the connection string using the<appSettings> element as follows:

<configuration><appSettings><addkey="StorageConnectionString"value="DefaultEndpointsProtocol=https;AccountName=[AccountName];AccountKey=[AccountKey"/></appSettings></configuration>

Read Configuring Connection Strings for more information on storage connection strings.

You are now ready to perform the how-to tasks in this guide.

How to: Programmatically access blob storage

Obtaining the assembly

You can use NuGet to obtain the Microsoft.WindowsAzure.Storage.dll assembly. Right-click your project in Solution Explorer and choose Manage NuGet Packages. Search online for "WindowsAzure.Storage" and click Install to install the Windows Azure Storage package and dependencies.

Namespace declarations

Add the following namespace declarations to the top of any C# file in which you wish to programmatically access Windows Azure Storage:

usingMicrosoft.WindowsAzure.Storage;usingMicrosoft.WindowsAzure.Storage.Auth;usingMicrosoft.WindowsAzure.Storage.Blob;

Make sure you reference the Microsoft.WindowsAzure.Storage.dll assembly.

Retrieving your connection string

You can use the CloudStorageAccount type to represent your Storage Account information. If you are using a Windows Azure project template and/or have a reference to Microsoft.WindowsAzure.CloudConfigurationManager, you can you use theCloudConfigurationManager type to retrieve your storage connection string and storage account information from the Windows Azure service configuration:

CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));

If you are creating an application with no reference to Microsoft.WindowsAzure.CloudConfigurationManager, and your connection string is located in the web.config or app.config as show above, then you can use ConfigurationManager to retrieve the connection string. You will need to add a reference to System.Configuration.dll to your project, and add another namespace declaration for it:

usingSystem.Configuration;...CloudStorageAccount storageAccount =CloudStorageAccount.Parse(ConfigurationManager.ConnectionStrings["StorageConnectionString"].ConnectionString);

A CloudBlobClient type allows you to retrieve objects that represent containers and blobs stored within the Blob Storage Service. The following code creates a CloudBlobClient object using the storage account object we retrieved above:

CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

ODataLib dependencies

ODataLib dependencies in the Storage Client Library for .NET are resolved through the ODataLib (version 5.0.2) packages available through NuGet and not WCF Data Services. The ODataLib libraries can be downloaded directly or referenced by your code project through NuGet. The specific ODataLib packages are OData, Edm, and Spatial.

How to: Create a container

All storage blobs reside in a container. You can use a CloudBlobClient object to get a reference to the container you want to use. You can create the container if it doesn't exist:

// Retrieve storage account from connection string.CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the blob client.CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();// Retrieve a reference to a container. CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");// Create the container if it doesn't already exist.
container.CreateIfNotExists();

By default, the new container is private and you must specify your storage access key to download blobs from this container. If you want to make the files within the container available to everyone, you can set the container to be public using the following code:

container.SetPermissions(newBlobContainerPermissions{PublicAccess=BlobContainerPublicAccessType.Blob});

Anyone on the Internet can see blobs in a public container, but you can modify or delete them only if you have the appropriate access key.

How to: Upload a blob into a container

Windows Azure Blob Storage supports block blobs and page blobs. In the majority of cases, block blob is the recommended type to use.

To upload a file to a block blob, get a container reference and use it to get a block blob reference. Once you have a blob reference, you can upload any stream of data to it by calling the UploadFromStream method. This operation will create the blob if it didn't previously exist, or overwrite it if it does exist. The following example shows how to upload a blob into a container and assumes that the container was already created.

// Retrieve storage account from connection string.CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the blob client.CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();// Retrieve reference to a previously created container.CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");// Retrieve reference to a blob named "myblob".CloudBlockBlob blockBlob = container.GetBlockBlobReference("myblob");// Create or overwrite the "myblob" blob with contents from a local file.using(var fileStream =System.IO.File.OpenRead(@"path\myfile")){
    blockBlob.UploadFromStream(fileStream);}

How to: List the blobs in a container

To list the blobs in a container, first get a container reference. You can then use the container's ListBlobs method to retrieve the blobs and/or directories within it. To access the rich set of properties and methods for a returned IListBlobItem, you must cast it to a CloudBlockBlob, CloudPageBlob, or CloudBlobDirectory object. If the type is unknown, you can use a type check to determine which to cast it to. The following code demonstrates how to retrieve and output the URI of each item in thephotos container:

// Retrieve storage account from connection string.CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the blob client. CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();// Retrieve reference to a previously created container.CloudBlobContainer container = blobClient.GetContainerReference("photos");// Loop over items within the container and output the length and URI.foreach(IListBlobItem item in container.ListBlobs(null,false)){if(item.GetType()==typeof(CloudBlockBlob)){CloudBlockBlob blob =(CloudBlockBlob)item;Console.WriteLine("Block blob of length {0}: {1}", blob.Properties.Length, blob.Uri);}elseif(item.GetType()==typeof(CloudPageBlob)){CloudPageBlob pageBlob =(CloudPageBlob)item;Console.WriteLine("Page blob of length {0}: {1}", pageBlob.Properties.Length, pageBlob.Uri);}elseif(item.GetType()==typeof(CloudBlobDirectory)){CloudBlobDirectory directory =(CloudBlobDirectory)item;Console.WriteLine("Directory: {0}", directory.Uri);}}

As shown above, the blob service has the concept of directories within containers, as well. This is so that you can organize your blobs in a more folder-like structure. For example, consider the following set of block blobs in a container named photos:

photo1.jpg
2010/architecture/description.txt
2010/architecture/photo3.jpg
2010/architecture/photo4.jpg
2011/architecture/photo5.jpg
2011/architecture/photo6.jpg
2011/architecture/description.txt
2011/photo7.jpg

When you call ListBlobs on the 'photos' container (as in the above sample), the collection returned will containCloudBlobDirectory and CloudBlockBlob objects representing the directories and blobs contained at the top level. Here would be the resulting output:

Directory: https://<accountname>.blob.core.windows.net/photos/2010/Directory: https://<accountname>.blob.core.windows.net/photos/2011/Block blob of length 505623: https://<accountname>.blob.core.windows.net/photos/photo1.jpg

Optionally, you can set the UseFlatBlobListing parameter of of the ListBlobs method to true. This would result in every blob being returned as a CloudBlockBlob , regardless of directory. Here would be the call to ListBlobs:

// Loop over items within the container and output the length and URI.foreach(IListBlobItem item in container.ListBlobs(null,true)){...}

and here would be the results:

Block blob of length 4: https://<accountname>.blob.core.windows.net/photos/2010/architecture/description.txtBlock blob of length 314618: https://<accountname>.blob.core.windows.net/photos/2010/architecture/photo3.jpgBlock blob of length 522713: https://<accountname>.blob.core.windows.net/photos/2010/architecture/photo4.jpgBlock blob of length 4: https://<accountname>.blob.core.windows.net/photos/2011/architecture/description.txtBlock blob of length 419048: https://<accountname>.blob.core.windows.net/photos/2011/architecture/photo5.jpgBlock blob of length 506388: https://<accountname>.blob.core.windows.net/photos/2011/architecture/photo6.jpgBlock blob of length 399751: https://<accountname>.blob.core.windows.net/photos/2011/photo7.jpgBlock blob of length 505623: https://<accountname>.blob.core.windows.net/photos/photo1.jpg

For more information, see CloudBlobContainer.ListBlobs.

How to: Download blobs

To download blobs, first retrieve a blob reference and then call the DownloadToStream method. The following example uses the DownloadToStream method to transfer the blob contents to a stream object that you can then persist to a local file.

// Retrieve storage account from connection string.CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the blob client.CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();// Retrieve reference to a previously created container.CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");// Retrieve reference to a blob named "photo1.jpg".CloudBlockBlob blockBlob = container.GetBlockBlobReference("photo1.jpg");// Save blob contents to a file.using(var fileStream =System.IO.File.OpenWrite(@"path\myfile")){
    blockBlob.DownloadToStream(fileStream);}

You can also use the DownloadToStream method to download the contents of a blob as a text string.

// Retrieve storage account from connection string.CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the blob client.CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();// Retrieve reference to a previously created container.CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");// Retrieve reference to a blob named "myblob.txt"CloudBlockBlob blockBlob2 = container.GetBlockBlobReference("myblob.txt");string text;using(var memoryStream =newMemoryStream()){
    blockBlob2.DownloadToStream(memoryStream);
    text =System.Text.Encoding.UTF8.GetString(memoryStream.ToArray());}

How to: Delete blobs

To delete a blob, first get a blob reference and then call the Delete method on it.

// Retrieve storage account from connection string.CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the blob client.CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();// Retrieve reference to a previously created container.CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");// Retrieve reference to a blob named "myblob.txt".CloudBlockBlob blockBlob = container.GetBlockBlobReference("myblob.txt");// Delete the blob.
blockBlob.Delete();

Next steps

Now that you've learned the basics of blob storage, follow these links to learn how to do more complex storage tasks.

• View the blob service reference documentation for complete details about available APIs:
  • Storage Client Library for .NET reference
  • REST API reference
• Learn about more advanced tasks you can perform with Windows Azure Storage at Storing and Accessing Data in Windows Azure.
• View more feature guides to learn about additional options for storing data in Windows Azure.
  • Use Table Storage to store structured data.
  • Use SQL Database to store relational data.