Inside TSQL Querying - Chapter 2. Physical Query Processing

Summary Description

The SQL language is spoken by most database experts, and all relational database products include some dialect of the SQL standard. Nevertheless, each product has its own particular query-processing mechanism. Understanding the way a database engine processes queries helps software architects, designers, and programmers make good choices when designing database schemas and writing queries.

When a query reaches the database engine, the SQL Server performs two major steps to produce the desired query result. The first step is query compilation, which generates a query plan, and the second step is the execution of the query plan.

Query compilation in SQL Server 2005 consists of three steps: parsing, algebrization, and query optimization. After those steps are completed, the compiler stores the optimized query plan in the procedure cache. There, the execution engine copies the plan into its executable form and subsequently executes the steps in the query plan to produce the query result. If the same query or stored procedure is executed again and the plan is located in the procedure cache, the compilation step is skipped and the query or stored procedure proceeds directly to execution reusing the stored plan.

Commands Generating Various Formats of a Showplan

Content	Text	XML	Graphical
Operators	SET SHOWPLAN_TEXT ON	N/A	N/A
Operators and estimated costs	SET SHOWPLAN_ALL ON	SET SHOWPLAN_XML ON	Display Estimated Execution Plan in Management Studio
Run-time info	SET STATISTICS PROFILE ON	SET STATISTICS XML ON	Include Actual Execution Plan in Management Studio

Showplan-Related Trace Event Classes

Trace Event Class	Compile or Run	Includes run-time info	Includes XML showplan	Generates trace against SQL Server 2000
Showplan All	Run	No	No	Yes
Showplan All for Query Compile	Compile	No	No	No
Showplan Statistics Profile	Run	Yes[1]	No	Yes
Showplan Text	Run	No	No	Yes
Showplan Text (Unencoded)	Run	No	No	Yes[2]
Showplan XML	Run	No	Yes	No
Showplan XML for Query Compile	Compile	No	Yes	No
Showplan XML Statistics Profile	Run	Yes[3]	Yes	No
Performance Statistics	Compile and Run[4]	Yes[5]	Yes	No

Update Plans

The query optimizer must take care of several specific issues when optimizing INSERT, UPDATE, and DELETEor, in other words, data modifyingstatements. Here I will describe the techniques employed by the SQL Server to process these statements.

The IUD (shorthand I will use for "INSERT, UPDATE, and DELETE") plans have two stages. The first stage is read only, and it determines which rows need to be inserted/updated/deleted by generating a data stream describing the changes to be made. For INSERTs, the data stream contains column values; for DELETEs, it has the table keys; and for UPDATEs, it has both the table keys and the values of changed columns. The second stage applies changes in the data stream to the table; additionally, it takes actions necessary to preserve data integrity by performing constraint validation, it maintains nonclustered indexes and indexed views, and it fires triggers if they exist. Usually the UPDATE and DELETE query plans contain two references to the target table: the first reference is used to identify the affected rows and the second to perform the change. The INSERT plans contain only one reference to the target table unless the same target table also participates in generating the inserted rows.

In some simple cases, SQL Server merges the read and write stages of the IUD plans together. This is the case, for example, when inserting values directly into a table (a process known as a scalar insert) or updating/deleting rows identified by a value of a primary key on the target table.

The Assert operator is automatically included in the query plans in the second phase if SQL Server needs to perform constraint validation. SQL Server validates the CHECK constraints for INSERTs and UPDATEs by evaluating a usually inexpensive scalar expression on each affected row and column. Foreign key constraints are enforced on INSERTs and UPDATEs to the table containing the foreign key constraint, and they're enforced on UPDATEs and DELETEs to the table containing the referenced key. The related table that is not the target of the IUD operation is scanned to verify the constraint; therefore, data access is involved. Declaring a primary key automatically creates a unique index on the key columns, but this is not the case for a foreign key. UPDATEs and DELETEs of referenced keys must access the foreign key table for each updated or deleted primary key value, either to validate nonexistence of the removed key or to propagate the change if it is a cascading referential integrity constraint. Therefore, you should ensure there is an index on the foreign key if you plan to perform UPDATEs affecting the key values or DELETEs from the primary table.

In addition to performing the IUD operation on the clustered index or heap, processing of the INSERT and DELETE queries also maintains all nonclustered indexes, and the UPDATE queries maintain indexes containing the modified columns. Because nonclustered indexes include the clustered index and partitioning keys to allow efficient access to the table row, updating columns that participate in the clustered index or in the partitioning key is expensive because it requires maintenance of all indexes. Updating the partitioning key might also cause rows to move between partitions. Therefore, when you have a choice, choose clustering and partitioning keys that you don't plan to update.

Note

SQL Server 2005 restricts the partitioning keys to a single column; therefore, "partitioning key" and "partitioning column" are synonyms.

In general, the performance of IUD statements is closely tied to the number of maintained indexes that include the target columns, because those must all be modified. Performing single-row INSERT and DELETE operations to an index requires a single index-tree traversal. SQL Server implements update to an index or partitioning key as a DELETE followed by an INSERT therefore, it is roughly twice as expensive as a nonkey UPDATE.

Others

You can use DBCC FREEPROCCACHE to clear up the procedure cache to ensure the compilation will take place afterwards. However, you should be careful using the DBCC FREEPROCCACHE on production servers because it will delete the contents of the procedure cache, and all statements and stored procedures will have to be compiled anew

The DOP is 0 (which means serial plan), which shows the actual degree of parallelism (or DOP, which is the number of concurrent threads working on the single query) of the execution

Observe that in the preceding example I used SET SHOWPLAN_TEXT OFF following the query. This is because SET SHOWPLAN_TEXT ON is not only causing the query plan to show up, it is also turning off query execution for the connection. Query execution will stay turned off until SQL Server executes SET SHOWPLAN_TEXT OFF on the same connection

When examining the plan for a particular query, a good way to find potential problems is to find the biggest discrepancies between the optimizer's estimates and the real number of executes and returned rows. Here we must be careful because the optimizer's estimate in the column EstimateRows is per estimated execution, while the Rows in the showplan output mentioned previously is the cumulative number of rows returned by the operator from all its executions. Therefore, to assess the optimizer's discrepancy we must multiply the EstimateRows by EstimateExecutions and compare the result with the actual number of all rows returned in the Rows column of the SET STATISTICS PROFILE output.

After the query optimizer produces the plan for the batch or stored procedure, the plan is placed in the procedure cache. You can examine the procedure cache using several dynamic management views (DMV) and functions (DMF), DBCC PROCCACHE, and the deprecated catalog view sys.syscacheobjects. I will show how you can access a showplan in XML format for the queries currently in the procedure cache.