[Windows Azure] How to use the Table Storage Service

How to use the Table Storage Service

version 1.7 version 2.0

This guide will show you how to perform common scenarios using the Windows Azure Table Storage Service. The samples are written in C# code and use the Windows Azure Storage Client Library for .NET. The scenarios covered include creating and deleting a table, as well as working with table entities. For more information on tables, see the Next steps section.

Table of Contents

• What is the Table Service
• Concepts
• Create a Windows Azure Storage account
• Setup a storage connection string
• How to: Programmatically access table storage
• How to: Create a table
• How to: Add an entity to a table
• How to: Insert a batch of entities
• How to: Retrieve all entities in a partition
• How to: Retrieve a range of entities in a partition
• How to: Retrieve a single entity
• How to: Replace an entity
• How to: Insert-or-replace an entity
• How to: Query a subset of entity properties
• How to: Delete an entity
• How to: Delete a table
• Next steps

What is the Table Service

The Windows Azure Table storage service stores large amounts of structured data. The service is a NoSQL datastore which accepts authenticated calls from inside and outside the Windows Azure cloud. Windows Azure tables are ideal for storing structured, non-relational data. Common uses of the Table service include:

• Storing TBs of structured data capable of serving web scale applications
• Storing datasets that don't require complex joins, foreign keys, or stored procedures and can be denormalized for fast access
• Quickly querying data using a clustered index
• Accessing data using the OData protocol and LINQ queries with WCF Data Service .NET Libraries

You can use the Table service to store and query huge sets of structured, non-relational data, and your tables will scale as demand increases.

Concepts

The Table service contains the following components:

• URL format: Code addresses tables in an account using this address format:
  http://<storage account>.table.core.windows.net/<table>

  You can address Azure tables directly using this address with the OData protocol. For more information, see OData.org

• Storage Account: All access to Windows Azure Storage is done through a storage account. The total size of blob, table, and queue contents in a storage account cannot exceed 100TB.

• Table: A table is a collection of entities. Tables don't enforce a schema on entities, which means a single table can contain entities that have different sets of properties. An account can contain many tables, the size of which is only limited by the 100TB storage account limit.

• Entity: An entity is a set of properties, similar to a database row. An entity can be up to 1MB in size.

• Properties: A property is a name-value pair. Each entity can include up to 252 properties to store data. Each entity also has 3 system properties that specify a partition key, a row key, and a timestamp. Entities with the same partition key can be queried more quickly, and inserted/updated in atomic operations. An entity's row key is its unique identifier within a partition.

Create a Windows Azure Storage account

To use storage operations, you need a Windows Azure storage account. You can create a storage account by following these steps. (You can also create a storage account using the REST API.)

1. Log into the Windows Azure Management Portal.

2. At the bottom of the navigation pane, click NEW.

   

3. Click DATA SERVICES, then STORAGE, and then click QUICK CREATE.

   

4. In URL, type a subdomain name to use in the URI for the storage account. The entry can contain from 3-24 lowercase letters and numbers. This value becomes the host name within the URI that is used to address Blob, Queue, or Table resources for the subscription.

5. Choose a Region/Affinity Group in which to locate the storage. If you will be using storage from your Windows Azure application, select the same region where you will deploy your application.

6. Optionally, you can enable geo-replication.

7. Click CREATE STORAGE ACCOUNT.

Setup a storage connection string

The Windows Azure Storage Client Library for .NET supports using a storage connection string to configure endpoints and credentials for accessing storage services. You can put your storage connection string in a configuration file, rather than hard-coding it in code:

• When using Windows Azure Cloud Services, it is recommended you store your connection string using the Windows Azure service configuration system (*.csdef and *.cscfg files).
• When using Windows Azure Web Sites, Windows Azure Virtual Machines, or building .NET applications that are intended to run outside of Windows Azure, it is recommended you store your connection string using the .NET configuration system (e.g. web.config or app.config file).

In both cases, you can retrieve your connection string using the CloudConfigurationManager.GetSetting method, as shown later in this guide.

Configuring your connection string when using Cloud Services

The service configuration mechanism is unique to Windows Azure Cloud Services projects and enables you to dynamically change configuration settings from the Windows Azure Management Portal without redeploying your application.

To configure your connection string in the Windows Azure service configuration:

1. Within the Solution Explorer of Visual Studio, in the Roles folder of your Windows Azure Deployment Project, right-click your web role or worker role and click Properties.
   

2. Click the Settings tab and press the Add Setting button.
   

   A new Setting1 entry will then show up in the settings grid.

3. In the Type drop-down of the new Setting1 entry, choose Connection String.
   

4. Click the ... button at the right end of the Setting1 entry. The Storage Account Connection String dialog will open.

5. Choose whether you want to target the storage emulator (the Windows Azure storage simulated on your local machine) or an actual storage account in the cloud. The code in this guide works with either option. Enter the Primary Access Key value copied from the earlier step in this tutorial if you wish to store blob data in the storage account we created earlier on Windows Azure.
   

6. Change the entry Name from Setting1 to a "friendlier" name like StorageConnectionString. You will reference this connection string later in the code in this guide.
   

Configuring your connection string using .NET configuration

If you are writing an application that is not a Windows Azure cloud service, (see previous section), it is recommended you use the .NET configuration system (e.g. web.config or app.config). This includes Windows Azure Web Sites or Windows Azure Virtual Machines, as well as applications designed to run outside of Windows Azure. You store the connection string using the<appSettings> element as follows:

<configuration><appSettings><addkey="StorageConnectionString"value="DefaultEndpointsProtocol=https;AccountName=[AccountName];AccountKey=[AccountKey]"/></appSettings></configuration>

Read Configuring Connection Strings for more information on storage connection strings.

You are now ready to perform the how-to tasks in this guide.

How to: Programmatically access table storage

Obtaining the assembly

You can use NuGet to obtain the Microsoft.WindowsAzure.Storage.dll assembly. Right-click your project in Solution Explorer and choose Manage NuGet Packages. Search online for "WindowsAzure.Storage" and click Install to install the Windows Azure Storage package and dependencies.

Namespace declarations

Add the following code namespace declarations to the top of any C# file in which you wish to programmatically access Windows Azure Storage:

usingMicrosoft.WindowsAzure.Storage;usingMicrosoft.WindowsAzure.Storage.Auth;usingMicrosoft.WindowsAzure.Storage.Table;

Make sure you reference the Microsoft.WindowsAzure.Storage.dll assembly.

Retrieving your connection string

You can use the CloudStorageAccount type to represent your Storage Account information. If you are using a Windows Azure project template and/or have a reference to the Microsoft.WindowsAzure.CloudConfigurationManager namespace, you can you use the CloudConfigurationManager type to retrieve your storage connection string and storage account information from the Windows Azure service configuration:

CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));

If you are creating an application with no reference to Microsoft.WindowsAzure.CloudConfigurationManager, and your connection string is located in the web.config or app.config as show above, then you can use ConfigurationManager to retrieve the connection string. You will need to add a reference to System.Configuration.dll to your project and add another namespace declaration for it:

usingSystem.Configuration;...CloudStorageAccount storageAccount =CloudStorageAccount.Parse(ConfigurationManager.ConnectionStrings["StorageConnectionString"].ConnectionString);

ODataLib dependencies

ODataLib dependencies in the Storage Client Library for .NET are resolved through the ODataLib (version 5.0.2) packages available through NuGet and not WCF Data Services. The ODataLib libraries can be downloaded directly or referenced by your code project through NuGet. The specific ODataLib packages are OData, Edm, and Spatial.

How to: Create a table

A CloudTableClient object lets you get reference objects for tables and entities. The following code creates aCloudTableClient object and uses it to create a new table. All code in this guide assumes that the application being built is a Windows Azure Cloud Service project and uses a storage connection string stored in the Windows Azure application's service configuration.

// Retrieve the storage account from the connection string.CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the table client.CloudTableClient tableClient = storageAccount.CreateCloudTableClient();// Create the table if it doesn't exist.CloudTable table = tableClient.GetTableReference("people");
table.CreateIfNotExists();

How to: Add an entity to a table

Entities map to C# objects using a custom class derived from TableEntity. To add an entity to a table, create a class that defines the properties of your entity. The following code defines an entity class that uses the customer's first name as the row key and last name as the partition key. Together, an entity's partition and row key uniquely identify the entity in the table. Entities with the same partition key can be queried faster than those with different partition keys, but using diverse partition keys allows for greater parallel operation scalability. For any property that should be stored in the table service, the property must be a public property of a supported type that exposes both get and set. Also, your entity type must expose a parameter-less constructor.

publicclassCustomerEntity:TableEntity{publicCustomerEntity(string lastName,string firstName){this.PartitionKey= lastName;this.RowKey= firstName;}publicCustomerEntity(){}publicstringEmail{get;set;}publicstringPhoneNumber{get;set;}}

Table operations involving entities are performed using the CloudTable object you created in "How to: Create a Table." The operation to be performed is represented by a TableOperation object. The following code example shows the creation of theCloudTable object and then a CustomerEntity object. To prepare the operation, a TableOperation is created to insert the customer entity into the table. Finally, the operation is executed by calling CloudTable.Execute.

// Retrieve the storage account from the connection string.CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the table client.CloudTableClient tableClient = storageAccount.CreateCloudTableClient();// Create the CloudTable object that represents the "people" table.CloudTable table = tableClient.GetTableReference("people");// Create a new customer entity.CustomerEntity customer1 =newCustomerEntity("Harp","Walter");
customer1.Email="Walter@contoso.com";
customer1.PhoneNumber="425-555-0101";// Create the TableOperation that inserts the customer entity.TableOperation insertOperation =TableOperation.Insert(customer1);// Execute the insert operation.
table.Execute(insertOperation);

How to: Insert a batch of entities

You can insert a batch of entities into a table in one write operation. Some other notes on batch operations:

1. You can perform updates, deletes, and inserts in the same single batch operation.
2. A single batch operation can include up to 100 entities.
3. All entities in a single batch operation must have the same partition key.
4. While it is possible to perform a query as a batch operation, it must be the only operation in the batch.

The following code example creates two entity objects and adds each to a TableBatchOperation using the Insert method. Then CloudTable.Execute is called to execute the operation.

// Retrieve the storage account from the connection string.CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the table client.CloudTableClient tableClient = storageAccount.CreateCloudTableClient();// Create the CloudTable object that represents the "people" table.CloudTable table = tableClient.GetTableReference("people");// Create the batch operation.TableBatchOperation batchOperation =newTableBatchOperation();// Create a customer entity and add it to the table.CustomerEntity customer1 =newCustomerEntity("Smith","Jeff");
customer1.Email="Jeff@contoso.com";
customer1.PhoneNumber="425-555-0104";// Create another customer entity and add it to the table.CustomerEntity customer2 =newCustomerEntity("Smith","Ben");
customer2.Email="Ben@contoso.com";
customer2.PhoneNumber="425-555-0102";// Add both customer entities to the batch insert operation.
batchOperation.Insert(customer1);
batchOperation.Insert(customer2);// Execute the batch operation.
table.ExecuteBatch(batchOperation);

How to: Retrieve all entities in a partition

To query a table for all entities in a partition, use a TableQuery object. The following code example specifies a filter for entities where 'Smith' is the partition key. This example prints the fields of each entity in the query results to the console.

// Retrieve the storage account from the connection string.CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the table client.CloudTableClient tableClient = storageAccount.CreateCloudTableClient();// Create the CloudTable object that represents the "people" table.CloudTable table = tableClient.GetTableReference("people");// Construct the query operation for all customer entities where PartitionKey="Smith".TableQuery<CustomerEntity> query =newTableQuery<CustomerEntity>().Where(TableQuery.GenerateFilterCondition("PartitionKey",QueryComparisons.Equal,"Smith"));// Print the fields for each customer.foreach(CustomerEntity entity in table.ExecuteQuery(query)){Console.WriteLine("{0}, {1}\t{2}\t{3}", entity.PartitionKey, entity.RowKey,
        entity.Email, entity.PhoneNumber);}

How to: Retrieve a range of entities in a partition

If you don't want to query all the entities in a partition, you can specify a range by combining the partition key filter with a row key filter. The following code example uses two filters to get all entities in partition 'Smith' where the row key (first name) starts with a letter earlier than 'E' in the alphabet and then prints the query results.

// Retrieve the storage account from the connection string.CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the table client.CloudTableClient tableClient = storageAccount.CreateCloudTableClient();//Create the CloudTable object that represents the "people" table.CloudTable table = tableClient.GetTableReference("people");// Create the table query.TableQuery<CustomerEntity> rangeQuery =newTableQuery<CustomerEntity>().Where(TableQuery.CombineFilters(TableQuery.GenerateFilterCondition("PartitionKey",QueryComparisons.Equal,"Smith"),TableOperators.And,TableQuery.GenerateFilterCondition("RowKey",QueryComparisons.LessThan,"E")));// Loop through the results, displaying information about the entity.foreach(CustomerEntity entity in table.ExecuteQuery(rangeQuery)){Console.WriteLine("{0}, {1}\t{2}\t{3}", entity.PartitionKey, entity.RowKey,
        entity.Email, entity.PhoneNumber);}

How to: Retrieve a single entity

You can write a query to retrieve a single, specific entity. The following code uses a TableOperation to specify the customer 'Ben Smith'. This method returns just one entity, rather than a collection, and the returned value in TableResult.Result is aCustomerEntity. Specifying both partition and row keys in a query is the fastest way to retrieve a single entity from the Table service.

// Retrieve the storage account from the connection string.CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the table client.CloudTableClient tableClient = storageAccount.CreateCloudTableClient();// Create the CloudTable object that represents the "people" table.CloudTable table = tableClient.GetTableReference("people");// Create a retrieve operation that takes a customer entity.TableOperation retrieveOperation =TableOperation.Retrieve<CustomerEntity>("Smith","Ben");// Execute the retrieve operation.TableResult retrievedResult = table.Execute(retrieveOperation);// Print the phone number of the result.if(retrievedResult.Result!=null)Console.WriteLine(((CustomerEntity)retrievedResult.Result).PhoneNumber);elseConsole.WriteLine("The phone number could not be retrieved.");

How to: Replace an entity

To update an entity, retrieve it from the table service, modify the entity object, and then save the changes back to the table service. The following code changes an existing customer's phone number. Instead of calling Insert, this code uses Replace. This causes the entity to be fully replaced on the server, unless the entity on the server has changed since it was retrieved, in which case the operation will fail. This failure is to prevent your application from inadvertently overwriting a change made between the retrieval and update by another component of your application. The proper handling of this failure is to retrieve the entity again, make your changes (if still valid), and then perform another Replace operation. The next section will show you how to override this behavior.

// Retrieve the storage account from the connection string.CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the table clientCloudTableClient tableClient = storageAccount.CreateCloudTableClient();// Create the CloudTable object that represents the "people" table.CloudTable table = tableClient.GetTableReference("people");// Create a retrieve operation that takes a customer entity.TableOperation retrieveOperation =TableOperation.Retrieve<CustomerEntity>("Smith","Ben");// Execute the operation.TableResult retrievedResult = table.Execute(retrieveOperation);// Assign the result to a CustomerEntity object.CustomerEntity updateEntity =(CustomerEntity)retrievedResult.Result;if(updateEntity !=null){// Change the phone number.
   updateEntity.PhoneNumber="425-555-0105";// Create the InsertOrReplace TableOperationTableOperation updateOperation =TableOperation.Replace(updateEntity);// Execute the operation.
   table.Execute(updateOperation);Console.WriteLine("Entity updated.");}elseConsole.WriteLine("Entity could not be retrieved.");

How to: Insert-or-replace an entity

Replace operations will fail if the entity has been changed since it was retrieved from the server. Furthermore, you must retrieve the entity from the server first in order for the Replace to be successful. Sometimes, however, you don't know if the entity exists on the server and the current values stored in it are irrelevant - your update should overwrite them all. To accomplish this, you would use an InsertOrReplace operation. This operation inserts the entity if it doesn't exist, or replaces it if it does, regardless of when the last update was made. In the following code example, the customer entity for Ben Smith is still retrieved, but it is then saved back to the server using InsertOrReplace. Any updates made to the entity between the retrieval and update operation will be overwritten.

// Retrieve the storage account from the connection string.CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the table client.CloudTableClient tableClient = storageAccount.CreateCloudTableClient();// Create the CloudTable object that represents the "people" table.CloudTable table = tableClient.GetTableReference("people");// Create a retrieve operation that takes a customer entity.TableOperation retrieveOperation =TableOperation.Retrieve<CustomerEntity>("Smith","Ben");// Execute the operation.TableResult retrievedResult = table.Execute(retrieveOperation);// Assign the result to a CustomerEntity object.CustomerEntity updateEntity =(CustomerEntity)retrievedResult.Result;if(updateEntity !=null){// Change the phone number.
   updateEntity.PhoneNumber="425-555-1234";// Create the InsertOrReplace TableOperationTableOperation insertOrReplaceOperation =TableOperation.InsertOrReplace(updateEntity);// Execute the operation.
   table.Execute(insertOrReplaceOperation);Console.WriteLine("Entity was updated.");}elseConsole.WriteLine("Entity could not be retrieved.");

How to: Query a subset of entity properties

A table query can retrieve just a few properties from an entity instead of all the entity properties. This technique, called projection, reduces bandwidth and can improve query performance, especially for large entities. The query in the following code returns only the email addresses of entities in the table. This is done by using a query of DynamicTableEntity and also anEntityResolver. You can learn more about projection in this blog post. Note that projection is not supported on the local storage emulator, so this code runs only when using an account on the table service.

// Retrieve storage account from connection stringCloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the table clientCloudTableClient tableClient = storageAccount.CreateCloudTableClient();//Create the CloudTable that represents the "people" table.CloudTable table = tableClient.GetTableReference("people");// Define the query, and only select the Email propertyTableQuery<DynamicTableEntity> projectionQuery =newTableQuery<DynamicTableEntity>().Select(newstring[]{"Email"});// Define an entity resolver to work with the entity after retrieval.EntityResolver<string> resolver =(pk, rk, ts, props, etag)=> props.ContainsKey("Email")? props["Email"].StringValue:null;foreach(string projectedEmail in table.ExecuteQuery(projectionQuery, resolver,null,null)){Console.WriteLine(projectedEmail);}

How to: Delete an entity

You can easily delete an entity after you have retrieved it, using the same pattern shown for updating an entity. The following code retrieves and deletes a customer entity.

// Retrieve storage account from connection stringCloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the table clientCloudTableClient tableClient = storageAccount.CreateCloudTableClient();//Create the CloudTable that represents the "people" table.CloudTable table = tableClient.GetTableReference("people");// Create a retrieve operation that expects a customer entity.TableOperation retrieveOperation =TableOperation.Retrieve<CustomerEntity>("Smith","Ben");// Execute the operation.TableResult retrievedResult = table.Execute(retrieveOperation);// Assign the result to a CustomerEntity.CustomerEntity deleteEntity =(CustomerEntity)retrievedResult.Result;// Create the Delete TableOperation.if(deleteEntity !=null){TableOperation deleteOperation =TableOperation.Delete(deleteEntity);// Execute the operation.
   table.Execute(deleteOperation);Console.WriteLine("Entity deleted.");}elseConsole.WriteLine("Could not retrieve the entity.");

How to: Delete a table

Finally, the following code example deletes a table from a storage account. A table which has been deleted will be unavailable to be recreated for a period of time following the deletion.

// Retrieve the storage account from the connection string.CloudStorageAccount storageAccount =CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));// Create the table client.CloudTableClient tableClient = storageAccount.CreateCloudTableClient();//Create the CloudTable that represents the "people" table.CloudTable table = tableClient.GetTableReference("people");// Delete the table it if exists.
table.DeleteIfExists();

Next steps

Now that you've learned the basics of table storage, follow these links to learn how to do more complex storage tasks.

• View the blob service reference documentation for complete details about available APIs:
  • Storage Client Library for .NET reference
  • REST API reference
• Learn about more advanced tasks you can perform with Windows Azure Storage at Storing and Accessing Data in Windows Azure.
• View more feature guides to learn about additional options for storing data in Windows Azure.
  • Use Blob Storage to store unstructured data.
  • Use SQL Database to store relational data.