SQL Server性能计数器收集汇总方案（Reporting Service）

通过收集计数器信息，并将计数器信息汇总为不同粒度存储，以Reporting Service报表服务器显示。以下是计数器收集汇总的基本架构。

笔者需要收集的SQL Server计数器包括：SQL Server计数器属性详解

一、SQL Server计数器基本架构图

1、生产服务器通过ODBC开发数据库互联ODBC配置访问监控监控服务器的SQLPerfData，将数据写入该数据库。详情请见计数器部署 SQL Server性能计数器部署（批量）

2、通过作业调用存储过程spb_Perf_CounterDataCollect将数据从SQLPerfData写入SQLPerfDataStat数据库。

 /****************************** 功能描述：<性能计数器15秒统计>

 *   创建者：<HuangCH〉

 *   创建日期：<2014-09-22>

 *   备注说明：<根据具体业务而定，每分钟一次>

 ##########

 Change Log

 ##########

 Date                 Changer             Description

 --------------------------------------------------

 <2014-09-22>        <HuangCH>            <新建>

 --------------------------------------------------

 ***************************/

 ALTER PROC [dbo].[spb_Perf_CounterDataCollect]

 as

 --定义当前时间

 SET NOCOUNT ON

 DECLARE @FifteenSec DATETIME

         ,@OneMinute DATETIME

         ,@FiveMinute DATETIME

         ,@HalfHour DATETIME

         ,@OneHour DATETIME

         ,@SixHour DATETIME

         ,@OldFifteenSec DATETIME

 DECLARE @MaxRecordIndex INT

 DECLARE @MinRecordIndex INT

 SET @FifteenSec=CONVERT(VARCHAR(23),GETDATE(),120)

 SET @OneMinute=CONVERT(VARCHAR(16),@FifteenSec,120)

 SET @FiveMinute=Dateadd(mi,Datepart(mi,@OneMinute) / 5 * 5 - Datepart(mi,@OneMinute),@OneMinute)

 SET @HalfHour=Dateadd(mi,Datepart(mi,@OneMinute) / 30 * 30 - Datepart(mi,@OneMinute),@OneMinute)

 SET @OneHour=CONVERT(VARCHAR(14),@FifteenSec,120)+'00:00'

 SET @SixHour=Dateadd(HH,Datepart(mi,@OneHour) / 30 * 30 - Datepart(mi,@OneHour),@OneHour)

 ------进入循环收集处理-----------------------------------------------

 DECLARE @MachineName SYSNAME

 DECLARE CUR_COUNTERDATA CURSOR FOR

 SELECT DISTINCT  MachineName

 FROM [dbo].[CounterDetails_Collect] WITH(NOLOCK)

 OPEN CUR_COUNTERDATA

 FETCH NEXT FROM CUR_COUNTERDATA INTO @MachineName

 WHILE @@FETCH_STATUS=0

 BEGIN

     IF NOT EXISTS (SELECT TOP 1 1 FROM [dbo].[CounterDetails_Dts] WHERE [MachineName]=@MachineName)

     BEGIN

         --添加未插入的数据

         INSERT INTO [dbo].[CounterDetails_Dts]

         SELECT @MachineName,0,@FifteenSec,@OneMinute,@FiveMinute,@HalfHour,@OneHour,@SixHour,@OneMinute,@OneMinute,@FifteenSec 

         FETCH NEXT FROM CUR_COUNTERDATA INTO @MachineName

         CONTINUE

     END 

 SELECT @MinRecordIndex=[LastRecordIndex],@OldFifteenSec=LastFifteenSec  FROM [dbo].[CounterDetails_Dts] WITH(NOLOCK)

 WHERE [MachineName]=@MachineName

 IF DATEDIFF(MI,@OldFifteenSec,@FifteenSec)>25

 BEGIN

   SET  @MinRecordIndex=0

 END

 SELECT @MaxRecordIndex=MAX([RecordIndex]),@FifteenSec=MAX(CONVERT(DATETIME,LEFT(A.CounterDateTime,23)))

 FROM [SQLPerfData].[dbo].[CounterData]  A WITH(NOLOCK)

 WHERE CounterID=(SELECT TOP 1 CounterID FROM [dbo].[CounterDetails_Collect] B WITH(NOLOCK) WHERE B.MachineName=@MachineName)

 AND [RecordIndex]>@MinRecordIndex

 --若由于重启导致index恢复，则要重新计算

 IF @MaxRecordIndex IS NULL

 BEGIN

 SET @MaxRecordIndex=@MinRecordIndex--上次最大值

 SELECT @MinRecordIndex=NumberOfRecords FROM --当前最大值

 (

 SELECT ROW_NUMBER() OVER(ORDER BY LogStartTime DESC) ROW_ID,NumberOfRecords

 FROM [SQLPerfData].[dbo].[DisplayToID]

 WHERE GUID IN(

 SELECT DISTINCT A.GUID FROM [SQLPerfData].[dbo].[CounterData]  A WITH(NOLOCK)

 JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK)  ON A.CounterID=B.CounterID

 WHERE   B.MachineName=@MachineName)

 ) AA

 WHERE ROW_ID=1

 --删除已经收集的

 DELETE A FROM [SQLPerfData].[dbo].[CounterData]  A WITH(NOLOCK)

 JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK)  ON A.CounterID=B.CounterID

 WHERE A.[RecordIndex]>@MinRecordIndex AND A.[RecordIndex]<=@MaxRecordIndex AND  B.MachineName=@MachineName

 --恢复当前有效值

 SET @MaxRecordIndex=@MinRecordIndex

 SET @MinRecordIndex=0

 END

 IF @MaxRecordIndex IS NOT NULL

 BEGIN

 BEGIN TRY

     --BEGIN TRAN

         ----收集

         INSERT INTO [dbo].[CounterData_FifteenSeconds](CounterID,CounterDateTime,CounterValue)

         SELECT A.CounterID,CONVERT(DATETIME,LEFT(A.CounterDateTime,23)),A.CounterValue

         FROM [SQLPerfData].[dbo].[CounterData]  A WITH(NOLOCK)

         JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK)  ON A.CounterID=B.CounterID

         WHERE A.[RecordIndex]>@MinRecordIndex AND A.[RecordIndex]<=@MaxRecordIndex AND  B.MachineName=@MachineName

         UPDATE  [dbo].[CounterDetails_Dts]

         SET LastFifteenSec=@FifteenSec,[LastRecordIndex]=@MaxRecordIndex

         WHERE MachineName=@MachineName

         --COMMIT

 END TRY

 BEGIN CATCH

     IF @@TRANCOUNT>0

        COMMIT;

     --THROW

 END CATCH

 --更新

 /*

 UPDATE  [dbo].[CounterDetails_Dts]

 SET LastFifteenSec=@FifteenSec,[LastRecordIndex]=@MaxRecordIndex

 WHERE [CounterID]=@CounterID */

 --删除30分钟内的数据

 END

 DELETE A

 FROM [CounterData_FifteenSeconds]   A

 JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK)  ON A.CounterID=B.CounterID

 WHERE B.MachineName=@MachineName AND CounterDateTime<DATEADD(WEEK,-1,@FifteenSec)

 FETCH NEXT FROM CUR_COUNTERDATA INTO @MachineName

 END

 CLOSE CUR_COUNTERDATA

 DEALLOCATE CUR_COUNTERDATA

 --监控

spb_Perf_CounterDataCollect

3、汇总取均值，将15秒钟数据汇总成1分钟，5分钟，半小时，一小时，六小时数据。各个粒度的数据用于不同时段的报表显示。例如1分钟数据汇总可用于显示一天内的报表，一个月的报表则需要一小时的数据才能正常显示。通过作业调用spb_Perf_CounterDataDts汇总。

 /****************************** 功能描述：<性能计数器15秒统计>

 *   创建者：<HuangCH〉

 *   创建日期：<2014-09-22>

 *   备注说明：<根据具体业务而定，每分钟一次>

 ##########

 Change Log

 ##########

 Date                 Changer             Description

 --------------------------------------------------

 <2014-09-22>        <HuangCH>            <新建>

 --------------------------------------------------

 ***************************/

 ALTER PROC [dbo].[spb_Perf_CounterDataDts]

 as

 --定义当前时间

 SET NOCOUNT ON

 DECLARE @OneMinute DATETIME

         ,@FiveMinute DATETIME

         ,@HalfHour DATETIME

         ,@OneHour DATETIME

         ,@SixHour DATETIME

 DECLARE @OldOneMinute  datetime

         ,@OldFiveMinute datetime

         ,@OldHalfHour datetime

         ,@OldOneHour datetime

         ,@OldSixHour datetime

         ,@OldFifteenSec DATETIME

 --MachineName

 DECLARE @MachineName SYSNAME

 DECLARE CUR_CounterDataDts CURSOR FOR

 SELECT DISTINCT  MachineName

 FROM [dbo].[CounterDetails_Dts] WITH(NOLOCK)

 OPEN CUR_CounterDataDts

 FETCH NEXT FROM CUR_CounterDataDts INTO @MachineName

 WHILE @@FETCH_STATUS=0

     BEGIN

         SELECT

         @OldFifteenSec= LastFifteenSec,

         @OldOneMinute= LastOneMinute,

         @OldFiveMinute= LastFiveMinute,

         @OldHalfHour=LastHalfHour,

         @OldOneHour=LastOneHour,

         @OldSixHour=LastSixHour

         FROM [dbo].[CounterDetails_Dts] WITH(NOLOCK)

         WHERE MachineName=@MachineName

         SET @OneMinute=CONVERT(VARCHAR(16),@OldFifteenSec,120)

         SET @FiveMinute=Dateadd(mi,Datepart(mi,@OneMinute) / 5 * 5 - Datepart(mi,@OneMinute),@OneMinute)

         SET @HalfHour=Dateadd(mi,Datepart(mi,@OneMinute) / 30 * 30 - Datepart(mi,@OneMinute),@OneMinute)

         SET @OneHour=CONVERT(VARCHAR(14),@OneMinute,120)+'00:00'

         SET @SixHour=Dateadd(HH,Datepart(HH,@OneHour) / 6 * 6 -Datepart(hh,@OneHour),@OneHour)

         --超过60分钟未收集，则只收集60分钟以内的数据

         IF DATEDIFF(MI,@OldOneMinute,@OneMinute)>60

         BEGIN

           SET @OldOneMinute=DATEADD(MI,-60,@OneMinute)

         END

         UPDATE [dbo].[CounterDetails_Dts]

         SET LastOneMinute=@OneMinute,LastFiveMinute=@FiveMinute,LastHalfHour=@HalfHour,LastOneHour=@OneHour,LastSixHour=@SixHour

         WHERE MachineName=@MachineName

         BEGIN TRY

             --一分钟

             INSERT INTO [dbo].[CounterData_OneMinute]

             SELECT A.CounterID,CONVERT(VARCHAR(16),A.CounterDateTime,120),AVG(A.CounterValue)

             FROM [dbo].[CounterData_FifteenSeconds]  A WITH(NOLOCK)

             JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK) ON A.CounterID=B.CounterID

             WHERE B.MachineName=@MachineName AND A.CounterDateTime <= @OneMinute And A.CounterDateTime > @OldOneMinute

             GROUP BY A.CounterID,CONVERT(VARCHAR(16),A.CounterDateTime,120)

         END TRY

         BEGIN CATCH

             IF @@ROWCOUNT>0

               COMMIT

         END CATCH

         BEGIN TRY

             --五分钟

             INSERT INTO [dbo].[CounterData_FiveMinute]

             SELECT A.CounterID,Dateadd(mi,Datepart(mi,A.CounterDateTime) / 5 * 5 - Datepart(mi,A.CounterDateTime),CONVERT(DATETIME,CONVERT(CHAR(16),A.CounterDateTime,120))),AVG(A.CounterValue)

             FROM [dbo].[CounterData_OneMinute]  A WITH(NOLOCK)

             JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK) ON A.CounterID=B.CounterID

             WHERE B.MachineName=@MachineName AND A.CounterDateTime <= @FiveMinute And A.CounterDateTime > @OldFiveMinute

             GROUP BY A.CounterID,Dateadd(mi,Datepart(mi,A.CounterDateTime) / 5 * 5 - Datepart(mi,A.CounterDateTime),CONVERT(DATETIME,CONVERT(CHAR(16),A.CounterDateTime,120)))

         END TRY

         BEGIN CATCH

             IF @@ROWCOUNT>0

               COMMIT

         END CATCH

         BEGIN TRY

             --半个小时

             INSERT INTO [dbo].[CounterData_HalfHour]

             SELECT A.CounterID,Dateadd(mi,Datepart(mi,A.CounterDateTime) / 30 * 30 - Datepart(mi,A.CounterDateTime),CONVERT(DATETIME,CONVERT(CHAR(16),A.CounterDateTime,120))),AVG(A.CounterValue)

             FROM [dbo].[CounterData_OneMinute]  A WITH(NOLOCK)

             JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK) ON A.CounterID=B.CounterID

             WHERE B.MachineName=@MachineName AND A.CounterDateTime <= @HalfHour And A.CounterDateTime > @OldHalfHour

             GROUP BY A.CounterID,Dateadd(mi,Datepart(mi,A.CounterDateTime) / 30 * 30 - Datepart(mi,A.CounterDateTime),CONVERT(DATETIME,CONVERT(CHAR(16),A.CounterDateTime,120)))

         END TRY

         BEGIN CATCH

             IF @@ROWCOUNT>0

               COMMIT

         END CATCH

         BEGIN TRY

             --一小时

             INSERT INTO [dbo].[CounterData_OneHour]

             SELECT A.CounterID,CONVERT(VARCHAR(14),A.CounterDateTime,120)+'00:00',AVG(A.CounterValue)

             FROM [dbo].[CounterData_OneMinute]  A WITH(NOLOCK)

             JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK) ON A.CounterID=B.CounterID

             WHERE B.MachineName=@MachineName AND A.CounterDateTime <= @OneHour And A.CounterDateTime > @OldOneHour

             GROUP BY A.CounterID,CONVERT(VARCHAR(14),A.CounterDateTime,120)

         END TRY

         BEGIN CATCH

             IF @@ROWCOUNT>0

               COMMIT

         END CATCH            

         BEGIN TRY

             --六小时

             INSERT INTO [dbo].[CounterData_SixHour]

             SELECT A.CounterID,Dateadd(HH,Datepart(HH,A.CounterDateTime) / 6 * 6 -Datepart(hh,A.CounterDateTime),A.CounterDateTime),AVG(A.CounterValue)

             FROM [dbo].[CounterData_OneMinute]  A WITH(NOLOCK)

             JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK) ON A.CounterID=B.CounterID

             WHERE B.MachineName=@MachineName AND A.CounterDateTime <= @SixHour And A.CounterDateTime > @OldSixHour

             GROUP BY A.CounterID,Dateadd(HH,Datepart(HH,A.CounterDateTime) / 6 * 6 -Datepart(hh,A.CounterDateTime),A.CounterDateTime)

         END TRY

         BEGIN CATCH

             IF @@ROWCOUNT>0

               COMMIT

         END CATCH

     FETCH NEXT FROM CUR_CounterDataDts INTO @MachineName

     END

 CLOSE CUR_CounterDataDts

 DEALLOCATE CUR_CounterDataDts

spb_Perf_CounterDataDts

4、计算历史数据为基线数据，与当前数据对比。SQL Server基线算法（同比和环比）

二、SQL Server汇总统计架构

如上图所示，计数器架构分别存储三个数据库SQLPerfData、SQLPerfDataStat、SQLPerfDataStat_Histroy；这三个计数器的功能具体如下：

1、 SQLPerfData接收计数器信息

DisplayToID：计数器自动生成，记录计数器启动信息

CounterDetails：计数器自动生成，记录计数器的基本信息

CounterData：记录计数器传入的数据

2、 SQLPerfDataStat统计计数器信息

基本信息模块：

CounterDetails_Collect：用于同步 CounterDetails表数据，用于轮训统计

CounterDetails_Collect_bak：无用计数器备份

CounterDetails_Dts：统计传输控制表，以服务器为单位，记录服务器统计传输时间。

CounterDetails_Report：同步CounterDetails_Collect表信息，通过优化，用于报表显示作用。

数据模块：

CounterData_FifteenSeconds：15秒数据收集，用于同步CounterData表数据。

CounterData_OneMinute：一分钟数据收集，15秒数据均值计算而成；适合查看当天数据。保留7天数据。

CounterData_FiveMinute：五分钟数据收集，一分钟数据均值计算而成；适合查看4天内数据。保留7天数据。

CounterData_HalfHour：30分钟数据收集，一分钟数据均值计算而成；适合查看一周内数据。保留7天数据。

CounterData_OneHour：一小时数据收集，一分钟数据均值计算而成；适合查看一个月内数据。保留永久

CounterData_SixHour：六小时数据收集，一分钟数据均值计算而成；适合查看一年内数据。保留永久

基线模块：

CBaseCounterData_OneMinute：同比基线，可以计算未来一周的数据走势。保留7天数据。

RBaseCounterData_OneMinute：环比基线，可以计算未来一天的数据走势。保留7天数据。

手动维护模块：

CounterTypeDetails：计数器类型和描述信息记录表。

MonitorContorl：手动维护，计数器报警监控控制表。

3、SQLPerfDataStat_Histroy归档计数器信息

CounterData_OneMinute_bak：一分钟数据备份，保留永久。

CounterData_FiveMinute_bak：五分钟数据备份，保留永久。

CounterData_HalfHour_bak：30分钟数据备份，保留永久。

基线模块：

CBaseCounterData_OneMinute_bak：同比基线备份，保留7天。

RBaseCounterData_OneMinute_bak：环比基线备份，保留7天。