等待事件：enq: HW - contention和enq: TM - contention

　　今天生成了生产库前几日的AWR报告，发现等待事件中出现了一个陌生的event--enq: HW - contention，google一下是ASSM(Auto Segment Space Management)的表空间上扩展时引起的等待。通常是有LOB大对象的表的并发插入、更新引起的LOB Segment High Water Mark并发争用；了解了之后开始解决问题。

　　原因：为防止多个进程同时修改HWM而提供的锁称为HW锁。想要移动HWM的进程必须获得HW锁。若在获取HW锁过程中发生争用，则等待enq: HW - contention事件。HW锁争用大部分是大量执行insert所引发的。

1.从语句跟踪
1）首先查出是哪个语句引起的争用

select SQL_ID,EVENT,to_char(sample_time,'yyyy/mm/dd hh24:mi:ss') as dt from v$active_session_history
where event like 'enq: HW%'
order by sample_time desc;

SQL_ID EVENT DT
------------- ---------------------------------------------------------------- ---------------------------------------------------------------------------
ctad8v29f07wm enq: HW - contention 2013/10/18 16:33:00
ctad8v29f07wm enq: HW - contention 2013/10/18 16:33:00
ctad8v29f07wm enq: HW - contention 2013/10/18 16:33:00

.......

2）根据sql_id确定是哪个表的什么字段争用引起等待
SELECT * FROM V$SQLTEXT WHERE SQL_ID='ctad8v29f07wm';

ADDRESS HASH_VALUE SQL_ID COMMAND_TYPE PIECE SQL_TEXT
---------------- ---------- ------------- ------------ ---------- ----------------------------------------------------------------
000000045F9FE0D0 2464161683 ctad8v29f07wm 6 1 :p2))
000000045F9FE0D0 2464161683 ctad8v29f07wm 6 0 UPDATE "SYS_ENTRYDIC" SET "APPSETS" = :p1 WHERE (("BHAPPITEM" =

--至此明白是SYS_ENTRYDIC上的APPSETS字段批量更新或插入引起的等待，结合应用业务也确实如此，此字段更新，插入频率较高。

2.直接定位文件号和块号，得出表名字段名
1）查出哪个文件哪个块
select event,p1,p2,p3 from v$session_wait where event like '%contention%';

EVENT P1 P2 P3
---------------------------------------------------------------- ---------- ---------- ----------
enq: HW - contention 1213661190 6 20974058
enq: HW - contention 1213661190 6 20974058
enq: HW - contention 1213661190 6 20974058
enq: HW - contention 1213661190 6 20974058
enq: HW - contention 1213661190 6 20974058
enq: HW - contention 1213661190 6 20974058
enq: HW - contention 1213661190 6 20974058

select dbms_utility.data_block_address_block(20974058) as blk#,dbms_utility.data_block_address_file(20974058) as file# from dual;

BLK# FILE#
---------- ----------
2538 5

2）得到段名
select owner,segment_name,segment_type, segment_name,block_id,blocks
from dba_extents
where file_id = 5
and 2538 between block_id and block_id+blocks-1;

OWNER SEGMENT_NAME SEGMENT_TYPE SEGMENT_NAME BLOCK_ID BLOCKS
------------------------------ -------------------------------------------------- ------------------ -------------------------------------------------- ---------- ----------
BOLAN SYS_LOB0000075975C00004$$ LOBSEGMENT SYS_LOB0000075975C00004$$ 2536 8

3）确认表名字段名
select table_name,column_name,segment_name,index_name from dba_lobs
where segment_name='SYS_LOB0000075975C00004$$';

TABLE_NAME COLUMN_NAME SEGMENT_NAME INDEX_NAME
------------------------------ -------------------------------------------------- -------------------------------------------------- ------------------------------
SYS_ENTRYDIC APPSETS SYS_LOB0000075975C00004$$ SYS_IL0000075975C00004$$

1和2中相互验证了引起争用的段，解决方式分两种，在ASSM表空间之内的：

a) As temporary workaround, manually add extra space to the LOB segment

ALTER TABLE <lob_table> MODIFY LOB (<column_name>) (allocate extent (size <extent size>));
或者
b) It may related Bug 6376915.
Refer to Note 6376915.8 “Bug 6376915 HW enqueue contention for ASSM LOB segments”
In 10.2.0.4 or above, this fix has been included, and can be enabled by setting event 44951 to a value
between 1 and 1024. A higher value would be more beneficial in reducing contention.
EVENT=”44951 TRACE NAME CONTEXT FOREVER, LEVEL < 1 – 1024 >”
语句如下：
alter system set events '44951 trace name context forever, level 1024';
或者
c) Consider partitioning the LOB in a manner that will evenly distribute concurrent DML across multiple partitions。（分区减少段扩展的等待）

使用MSSM的：

a) As temporary workaround, manually add extra space to the LOB segment

ALTER TABLE <lob_table> MODIFY LOB (<column_name>) (allocate extent (size <extent size>));
或者
b) Consider partitioning the LOB in a manner that will evenly distribute concurrent DML across multiple partitions

我是用 alter system set events '44951 trace name context forever, level 1024'; 的方式解决的

解决前后对比：

未调整前：
Foreground Wait Events

Event                                      Waits　　%Time -outs　　Total Wait Time (s)　　Avg wait (ms)　　Waits /txn　　% DB time
enq: HW - contention 　　　　   243,239 　　     0 　　　　　　 267,844 　　　　　　　1101 　　　　　　0.08 　　　　 30.38
log file sync 　　　　　　　　　   2,879,162         0 　　　　　　 88,962 　　　　　　　　31 　　 　　　　  1.00 　　　　 10.09
SQL*Net message from dblink   107,467,710 　  0 　　　　　　59,295                        1 　　　　　　　  37.41 　　　　6.73

调整后Foreground Wait Events中未发现enq: HW - contention等待事件。

enq: TM - contention

三种可能引起改等待时间的情况：

1.  执行DML 期间，为防止对与DML 相关的数据行进行修改，执行DML 的进程必须对该表获得TM 锁。若在获取TM 锁的过程中发生争用，则等待enq:TM-contention 事件。（通俗理解为一个是很多session在dml同一row）
2. 在对父表进行update或delete时，子表上没有外键么有建立索引。如果外键上没有定义索引，对父表的主键执行 DML 操作时必须获得子表上的共享行排他表级锁（share row exclusive table lock）（也称为 share-subexclusive table lock，SSX）。此锁能够阻止其他事务对子表执行 DML 操作。SSX 锁在获得后立即释放。如果父表中有多个主键值被更新或删除，对每行执行 DML 操作时都需要对子表进行一次加锁及解锁操作。如果外键上定义了索引，则对父表的主键执行 DML 操作时只需获得子表上的行共享表级锁（row share table lock）（也称为 subshare table lock，SS）。此锁不允许其他事务排他地对子表加锁，但允许对父表及子表执行 DML 操作。
3. dml和ddl操作发生阻塞

实验：

CREATE TABLE supplier
    (     supplier_id     number(10)     not null,
        supplier_name     varchar2(50)     not null,
        contact_name     varchar2(50),    
        CONSTRAINT supplier_pk PRIMARY KEY (supplier_id)
    );

INSERT INTO supplier VALUES (1, 'Supplier 1', 'Contact 1');
INSERT INTO supplier VALUES (2, 'Supplier 2', 'Contact 2');
COMMIT;

CREATE TABLE product
    (     product_id     number(10)     not null,
        product_name    varchar2(50)    not null,
        supplier_id     number(10)     not null,
        CONSTRAINT fk_supplier
          FOREIGN KEY (supplier_id)
         REFERENCES supplier(supplier_id)
         ON DELETE CASCADE
    );

INSERT INTO product VALUES (1, 'Product 1', 1);
INSERT INTO product VALUES (2, 'Product 2', 1);
INSERT INTO product VALUES (3, 'Product 3', 2);
COMMIT;

然后去执行几条相关的语句
User 1: DELETE supplier WHERE supplier_id = 1;
User 2: DELETE supplier WHERE supplier_id = 2;(现象HANG住)
User 3: INSERT INTO supplier VALUES (5, 'Supplier 5', 'Contact 5');(现象HANG住)

现在检查锁的情况

col event format a20
col type format a10
col object_name a15
col object_type a15
SELECT l.sid, s.blocking_session blocker, s.event, l.type, l.lmode, l.request, o.object_name, o.object_type
FROM v$lock l, dba_objects o, v$session s
WHERE UPPER(s.username) = UPPER('&User')
AND   l.id1        = o.object_id (+)
AND   l.sid        = s.sid
ORDER BY sid, type;
输入 user 的值:  test
原值    3: WHERE UPPER(s.username) = UPPER('&User')
新值    3: WHERE UPPER(s.username) = UPPER('test')

SID    BLOCKER EVENT                TYPE            LMODE    REQUEST
---------- ---------- -------------------- ---------- ---------- ----------
OBJECT_NAME     OBJECT_TYPE
--------------- ---------------
       144        158 enq: TM - contention TM                  3          0
SUPPLIER        TABLE

144        158 enq: TM - contention TM                  0          2
PRODUCT         TABLE

153            SQL*Net message from TM                  3          0
                       client
SUPPLIER        TABLE

SID    BLOCKER EVENT                TYPE            LMODE    REQUEST
---------- ---------- -------------------- ---------- ---------- ----------
OBJECT_NAME     OBJECT_TYPE
--------------- ---------------

153            SQL*Net message from TM                  3          0
                       client
PRODUCT         TABLE

153            SQL*Net message from TX                  6          0
                       client

SID    BLOCKER EVENT                TYPE            LMODE    REQUEST
---------- ---------- -------------------- ---------- ---------- ----------
OBJECT_NAME     OBJECT_TYPE
--------------- ---------------
       158        153 enq: TM - contention TM                  0          5
PRODUCT         TABLE

158        153 enq: TM - contention TM                  3          0
SUPPLIER        TABLE

已选择7行。

SQL>

能看到enq: TM - contention TM  等待事件

检查没有索引的外键

SELECT * FROM (
   SELECT c.table_name, cc.column_name, cc.position column_position
   FROM   user_constraints c, user_cons_columns cc
   WHERE  c.constraint_name = cc.constraint_name
   AND    c.constraint_type = 'R'
   MINUS
   SELECT i.table_name, ic.column_name, ic.column_position
   FROM   user_indexes i, user_ind_columns ic
   WHERE  i.index_name = ic.index_name
   )
ORDER BY table_name, column_position;
TABLE_NAME
------------------------------
COLUMN_NAME
--------------------------------------------------------------------------------

COLUMN_POSITION
---------------
PRODUCT
SUPPLIER_ID
              1

建立索引后：

CREATE INDEX fk_supplier ON product (supplier_id);

INSERT INTO supplier VALUES (6, 'Supplier 6', 'Contact 6');

INSERT INTO supplier VALUES (7, 'Supplier 7', 'Contact 7');

User 1: DELETE supplier WHERE supplier_id = 6;
User 2: DELETE supplier WHERE supplier_id = 7;
User 3: INSERT INTO supplier VALUES (8, 'Supplier 8', 'Contact 8');

相关的等待事件消失。