[20170603]12c Top Frequency histogram.txt
---恢复内容开始---
[20170603]12c Top Frequency histogram.txt
--//个人对直方图了解很少,以前2种直方图类型对于目前的许多应用来讲已经足够,或者讲遇到的问题很少.
--//抽一点点时间,简单探究12c Top Frequency histogram.
--//以前的频率直方图Frequency histogram,受限bucket(桶的大小),如果有255个不同值,oracle分析后不会建立频率直方图,而是建立高
--//度直方图.这样的情况会导致一些流行值的统计在显示执行计划时差距很大.而12c引入了Top Frequency histogram,注意这里的top,
--//我的理解就是流行值(popular),也就是这样建立的直方图仅仅包括popular,其他non-popular不考虑,这样在sql语句的查询这些
--//popular时,显示的统计信息相对准确,从而有利于oracle选择正确的执行计划.
--//以下是我的学习笔记,也许会存在许多错误,仅仅做一个记录.我也看了许多别人的blog.^_^.而且我目前的环境只有12.0.0.1(版本太
--//低).
1.环境:
SCOTT@test01p> @ ver1
PORT_STRING VERSION BANNER CON_ID
------------------------------ -------------- -------------------------------------------------------------------------------- ----------
IBMPC/WIN_NT64-9.1.0 12.1.0.1.0 Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production 0
--//如果要建立Top Frequency histogram必须要满足几个条件:
--//链接 raajeshwaran.blogspot.co.id/2016/06/top-frequency-histogram-in-12c.html
The database creates a Top frequency histogram, when the following criteria are met.
NDV is greater than n, where n is the requested number of buckets (default 254)
The percentage of rows occupied by Top-frequent values is greater than or equal to the threshold p where p is (1-(1/n)*100).
The estimate_percent parameter in dbms_stats gathering procedure should be auto_sample_size (set to default)
--//翻译过来NDV(也就是字段的不同值)大于N(指bucket的数量).
--//流行值(popular)在Top-frequent中合计数量/总计数量之比要大于(1-(1/n)*100).如果建立10个桶,这样流行值的总计必须在90%以上
2.首先验证(1-(1/n)*100)比值是否正确:
SCOTT@test01p> create table t as select * from dba_objects;
Table created.
select column_name,num_distinct,density,histogram,SAMPLE_SIZE
from user_tab_col_statistics
where table_name ='T'
and column_name ='OWNER';
COLUMN_NAME NUM_DISTINCT DENSITY HISTOGRAM SAMPLE_SIZE
-------------------- ------------ ---------- --------------- -----------
OWNER 32 .03125 NONE 91695
--//12c ctas 建立统计信息,但是不会建立直方图.density 1/32=.03125.
SCOTT@test01p> select count(*) from t;
COUNT(*)
----------
91695
--//随手写的sql语句:
with a as (select distinct owner,count(*) over(partition by owner) n1 ,count(*) over () n2 from t order by 2 desc ),
b as (select owner,n1,n2,sum(n1) over (order by n1 desc) n3 from a order by n1 desc)
select rownum,owner,n1,n2,n3,round(n3/n2,5) x1,round(1-1/rownum,5) x2 from b;
ROWNUM OWNER N1 N2 N3 X1 X2
------ ----------------- ----- ---------- ---------- ---------- ----------
1 SYS 41942 91695 41942 .45741 0
2 PUBLIC 37142 91695 79084 .86247 .5
3 APEX_040200 3405 91695 82489 .8996 .66667
4 ORDSYS 3157 91695 85646 .93403 .75
5 MDSYS 1819 91695 87465 .95387 .8
6 XDB 985 91695 88450 .96461 .83333
7 SYSTEM 641 91695 89091 .9716 .85714
8 CTXSYS 405 91695 89496 .97602 .875
9 WMSYS 387 91695 89883 .98024 .88889
10 DVSYS 352 91695 90235 .98408 .9
11 SH 309 91695 90544 .98745 .90909
12 ORDDATA 292 91695 90836 .99063 .91667
13 LBACSYS 209 91695 91045 .99291 .92308
14 OE 142 91695 91187 .99446 .92857
15 SCOTT 96 91695 91283 .99551 .93333
16 GSMADMIN_INTERNAL 77 91695 91360 .99635 .9375
17 IX 58 91695 91418 .99698 .94118
18 DBSNMP 55 91695 91473 .99758 .94444
19 PM 44 91695 91517 .99806 .94737
20 HR 35 91695 91552 .99844 .95
21 OLAPSYS 25 91695 91577 .99871 .95238
22 OJVMSYS 23 91695 91600 .99896 .95455
23 DVF 19 91695 91619 .99917 .95652
24 FLOWS_FILES 13 91695 91632 .99931 .95833
25 AUDSYS 12 91695 91644 .99944 .96
26 ORDPLUGINS 10 91695 91664 .99966 .96154
27 OUTLN 10 91695 91664 .99966 .96296
28 BI 8 91695 91688 .99992 .96429
29 ORACLE_OCM 8 91695 91688 .99992 .96552
30 SI_INFORMTN_SCHEM 8 91695 91688 .99992 .96667
31 APPQOSSYS 5 91695 91693 .99998 .96774
32 TEST 2 91695 91695 1 .96875
--//如果加入条件where round(n3/n2,5) >round(1-1/rownum,5),全部输出.也就是这样如果桶小于32,大于1.建立的都是Top Frequency.
3.继续测试:
D:\temp>cat a1.sql
cat a1.sql
exec dbms_stats.gather_table_stats(ownname=>user,tabname=>'T',method_opt=>'for columns owner size &1');
select column_name,num_distinct,density,histogram,SAMPLE_SIZE from user_tab_col_statistics where table_name ='T' and column_name ='OWNER';
SCOTT@test01p> @ a1.sql 2
PL/SQL procedure successfully completed.
COLUMN_NAME NUM_DISTINCT DENSITY HISTOGRAM SAMPLE_SIZE
-------------------- ------------ ---------- --------------- -----------
OWNER 32 .03125 HYBRID 5500
SCOTT@test01p> @ a1.sql 3
PL/SQL procedure successfully completed.
COLUMN_NAME NUM_DISTINCT DENSITY HISTOGRAM SAMPLE_SIZE
-------------------- ------------ ---------- --------------- -----------
OWNER 32 5.4529E-06 TOP-FREQUENCY 91695
SCOTT@test01p> @ a1.sql 4
PL/SQL procedure successfully completed.
COLUMN_NAME NUM_DISTINCT DENSITY HISTOGRAM SAMPLE_SIZE
-------------------- ------------ ---------- --------------- -----------
OWNER 32 5.4529E-06 TOP-FREQUENCY 91695
SCOTT@test01p> @ a1.sql 31
PL/SQL procedure successfully completed.
COLUMN_NAME NUM_DISTINCT DENSITY HISTOGRAM SAMPLE_SIZE
-------------------- ------------ ---------- --------------- -----------
OWNER 32 5.4529E-06 TOP-FREQUENCY 91695
SCOTT@test01p> @ a1.sql 32
PL/SQL procedure successfully completed.
COLUMN_NAME NUM_DISTINCT DENSITY HISTOGRAM SAMPLE_SIZE
-------------------- ------------ ---------- --------------- -----------
OWNER 32 5.4529E-06 FREQUENCY 91695
--//除了bucket=2,32建立的直方图HYBRID,FREQUENCY外,建立的都是TOP-FREQUENCY.
--//以10个bucket为例.解方程式(90235-x)/(91695-x)=0.9 ,得到x=77095.也就是要减少77095.
--//delete t where owner='SYS' and rownum<=41000;
--//delete t where owner='PUBLIC' and rownum<=36095;
SCOTT@test01p> delete t where owner='SYS' and rownum<=41000;
41000 rows deleted.
SCOTT@test01p> delete t where owner='PUBLIC' and rownum<=36095;
36095 rows deleted.
SCOTT@test01p> commit ;
Commit complete.
with a as (select distinct owner,count(*) over(partition by owner) n1 ,count(*) over () n2 from t order by 2 desc ),
b as (select owner,n1,n2,sum(n1) over (order by n1 desc) n3 from a order by n1 desc)
select rownum,owner,n1,n2,n3,round(n3/n2,5) x1,round(1-1/rownum,5) x2 from b where rownum<=11;
ROWNUM OWNER N1 N2 N3 X1 X2
------ ----------- ---- ---------- ---------- ---------- ----------
1 APEX_040200 3405 14600 3405 .23322 0
2 ORDSYS 3157 14600 6562 .44945 .5
3 MDSYS 1819 14600 8381 .57404 .66667
4 PUBLIC 1047 14600 9428 .64575 .75
5 XDB 985 14600 10413 .71322 .8
6 SYS 942 14600 11355 .77774 .83333
7 SYSTEM 641 14600 11996 .82164 .85714
8 CTXSYS 405 14600 12401 .84938 .875
9 WMSYS 387 14600 12788 .87589 .88889
10 DVSYS 352 14600 13140 .9 .9
11 SH 309 14600 13449 .92116 .90909
11 rows selected.
--//backet=10,前面10个值占90%.
SCOTT@test01p> @ a1 10
PL/SQL procedure successfully completed.
COLUMN_NAME NUM_DISTINCT DENSITY HISTOGRAM SAMPLE_SIZE
-------------------- ------------ ---------- --------------- -----------
OWNER 32 .000034247 TOP-FREQUENCY 14600
--//再减少1条记录.
SCOTT@test01p> delete t where owner='SYS' and rownum<=1;
1 row deleted.
SCOTT@test01p> commit ;
Commit complete.
ROWNUM OWNER N1 N2 N3 X1 X2
------ ----------- ---- ---------- ---------- ---------- ----------
1 APEX_040200 3405 14599 3405 .23324 0
2 ORDSYS 3157 14599 6562 .44948 .5
3 MDSYS 1819 14599 8381 .57408 .66667
4 PUBLIC 1047 14599 9428 .6458 .75
5 XDB 985 14599 10413 .71327 .8
6 SYS 941 14599 11354 .77772 .83333
7 SYSTEM 641 14599 11995 .82163 .85714
8 CTXSYS 405 14599 12400 .84937 .875
9 WMSYS 387 14599 12787 .87588 .88889
10 DVSYS 352 14599 13139 .89999 .9
11 SH 309 14599 13448 .92116 .90909
11 rows selected.
--//现在前10占.89999.
SCOTT@test01p> @ a1 10
PL/SQL procedure successfully completed.
COLUMN_NAME NUM_DISTINCT DENSITY HISTOGRAM SAMPLE_SIZE
-------------------- ------------ ---------- --------------- -----------
OWNER 32 .018378 HYBRID 14599
--//可以发现建立的直方图不是TOP-FREQUENCY,而是HYBRID(混合型直方图).
4.转化成TOP-FREQUENCY.
SCOTT@test01p> insert into t select * from dba_objects where owner='SYS' and rownum=1;
1 row created.
SCOTT@test01p> commit ;
Commit complete.
SCOTT@test01p> @ a1 10
PL/SQL procedure successfully completed.
COLUMN_NAME NUM_DISTINCT DENSITY HISTOGRAM SAMPLE_SIZE
-------------------- ------------ ---------- --------------- -----------
OWNER 32 .000034247 TOP-FREQUENCY 14600
--//DENSITY=1/SAMPLE_SIZE/2, 1/14600/2=.00003424657534246575正好符合.
5.现在看看执行计划:
SCOTT@test01p> select count(*) from t where owner='DVSYS';
COUNT(*)
----------
352
SCOTT@test01p> @ dpc '' ''
PLAN_TABLE_OUTPUT
-------------------------------------
SQL_ID 2at34f0zaqhzj, child number 0
-------------------------------------
select count(*) from t where owner='DVSYS'
Plan hash value: 2966233522
---------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows |E-Bytes| Cost (%CPU)| E-Time | A-Rows | A-Time | Buffers |
---------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | | 428 (100)| | 1 |00:00:00.01 | 1544 |
| 1 | SORT AGGREGATE | | 1 | 1 | 8 | | | 1 |00:00:00.01 | 1544 |
|* 2 | TABLE ACCESS FULL| T | 1 | 352 | 2816 | 428 (0)| 00:00:01 | 352 |00:00:00.01 | 1544 |
---------------------------------------------------------------------------------------------------------------------
Query Block Name / Object Alias (identified by operation id):
-------------------------------------------------------------
1 - SEL$1
2 - SEL$1 / T@SEL$1
Predicate Information (identified by operation id):
---------------------------------------------------
2 - filter("OWNER"='DVSYS')
--//可以发现e_rows=A-Rows.可以发现非常准确.
--//查看直方图信息.
SCOTT@test01p> select endpoint_number,endpoint_actual_value from user_tab_histograms where table_name ='T' and column_name ='OWNER' order by 1;
ENDPOINT_NUMBER ENDPOINT_ACTUAL_VALU
--------------- --------------------
3405 APEX_040200
3810 CTXSYS
4162 DVSYS
5981 MDSYS
9138 ORDSYS
10185 PUBLIC
11127 SYS
11768 SYSTEM
12155 WMSYS
13140 XDB
10 rows selected.
--//4162-3810=352,可以发现正好符合.也就是popular值统计很正确.看看非popular值.
SCOTT@test01p> select count(*) from t where owner='DVSYS1';
COUNT(*)
----------
0
Plan hash value: 2966233522
---------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows |E-Bytes| Cost (%CPU)| E-Time | A-Rows | A-Time | Buffers |
---------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | | 428 (100)| | 1 |00:00:00.01 | 1544 |
| 1 | SORT AGGREGATE | | 1 | 1 | 8 | | | 1 |00:00:00.01 | 1544 |
|* 2 | TABLE ACCESS FULL| T | 1 | 66 | 528 | 428 (0)| 00:00:01 | 0 |00:00:00.01 | 1544 |
---------------------------------------------------------------------------------------------------------------------
SCOTT@test01p> select count(*) from t where owner='SCOTT';
COUNT(*)
----------
96
Plan hash value: 2966233522
---------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows |E-Bytes| Cost (%CPU)| E-Time | A-Rows | A-Time | Buffers |
---------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | | 428 (100)| | 1 |00:00:00.01 | 1544 |
| 1 | SORT AGGREGATE | | 1 | 1 | 8 | | | 1 |00:00:00.01 | 1544 |
|* 2 | TABLE ACCESS FULL| T | 1 | 66 | 528 | 428 (0)| 00:00:01 | 96 |00:00:00.01 | 1544 |
---------------------------------------------------------------------------------------------------------------------
--//可以发现估计E-Rows如何计算的呢?
6.做10053跟踪:
D:\tools\sqllaji>cat 10053x.sql
execute dbms_sqldiag.dump_trace(p_sql_id=>'&1',p_child_number=>&2,p_component=>'Compiler',p_file_id=>'&&1');
SCOTT@test01p> @ 10053x f31kz63ksu1tc 0
PL/SQL procedure successfully completed.
***************************************
SINGLE TABLE ACCESS PATH
Single Table Cardinality Estimation for T[T]
SPD: Return code in qosdDSDirSetup: NOCTX, estType = TABLE
Column (#1):
NewDensity:0.004545, OldDensity:0.000034 BktCnt:13140.000000, PopBktCnt:13140.000000, PopValCnt:10, NDV:32
Column (#1): OWNER(VARCHAR2)
AvgLen: 8 NDV: 32 Nulls: 0 Density: 0.000000
Histogram: Top-Freq #Bkts: 13140 UncompBkts: 13140 EndPtVals: 10 ActualVal: yes
Table: T Alias: T
Card: Original: 14600.000000 Rounded: 66 Computed: 66.36 Non Adjusted: 66.36
Access Path: TableScan
Cost: 428.36 Resp: 428.36 Degree: 0
Cost_io: 428.00 Cost_cpu: 14122025
Resp_io: 428.00 Resp_cpu: 14122025
Best:: AccessPath: TableScan
Cost: 428.36 Degree: 1 Resp: 428.36 Card: 66.36 Bytes: 0
check parallelism for statement[<unnamed>]
kkfdtParallel: parallel is possible (no statement type restrictions)
kkfdPaForcePrm: dop:1 ()
use dictionary DOP(1) on table
kkfdPaPrm:- The table : 106380
kkfdPaPrm:DOP = 1 (computed from hint/dictionary/autodop)
kkfdiPaPrm: dop:1 serial(?)
***************************************
--//使用 NewDensity:0.004545.
BktCnt:13140.000000, PopBktCnt:13140.000000 => 对应就是前10个流行值的总和.
--//非流行值的数量: 14600-13140=1460
--//非流行值的桶数量: 32-10=22
--//非流行值的数量/非流行值的桶数量 1460/22=66.36363636363636363636,四舍五入66,正好符合执行计划的推断.
--//NewDensity的计算 =1460/14600/22=.00454545454545454545,非常接近.
---恢复内容结束---
[20170603]12c Top Frequency histogram.txt的更多相关文章
- [20170604]12c Top Frequency histogram补充.txt
[20170604]12c Top Frequency histogram补充.txt 1.环境:SCOTT@test01p> @ ver1PORT_STRING ...
- [20181105]再论12c set feedback only.txt
[20181105]再论12c set feedback only.txt --//前一阵子的测试,链接:http://blog.itpub.net/267265/viewspace-2216290/ ...
- [20181015]12C SQL Translation Framework.txt
[20181015]12C SQL Translation Framework.txt --//12c提供一个dba改写sql语句的可能性,实际上10g,11g之前也有一个包DBMS_ADVANCED ...
- [20180914]oracle 12c 表 full_hash_value如何计算.txt
[20180914]oracle 12c 表 full_hash_value如何计算.txt --//昨天在12c下看表full_hash_value与11g的full_hash_value不同,不过 ...
- oracle 12c AUTO_SAMPLE_SIZE动态采用工作机制
The ESTIMATE_PERCENT parameter in DBMS_STATS.GATHER_*_STATS procedures controls the percentage of ro ...
- top 自动执行的shell脚本中，使用top -n 1 > log.txt, 上电自动执行，文件无输出
. 自动执行的shell脚本中,使用top -n > log.txt, 上电自动执行,文件无输出,使用一下命令解决: //usr/bin/top -d -n -b > log.txt 如果 ...
- oracle 12c直方图收集的增强
在oracle 12c之前,收集直方图信息是相对比较耗费资源的,因为要重复扫描几次:在oracle 12c中,则有较大的提升,具体可参考https://jonathanlewis.wordpress. ...
- 定制保存top输出信息的格式详解
top命令的重要性和使用方法不多说了,这里终点讨论如何保存top命令的输出信息. 保存top命令的输出到一个文件的方法是:top -n1b > topinfo.txt,这没什么好奇的,但 ...
- [20191127]表 full Hash Value的计算.txt
[20191127]表 full Hash Value的计算.txt --//曾经做过表full Hash Value的计算,当时我是通过建立简单的schema以及表名的形式,使用hashcat破解o ...
随机推荐
- 从session原理出发解决微信小程序的登陆问题
声明:本文为作者原创文章,转载请注明出处 https://www.cnblogs.com/MaMaNongNong/p/9127416.html 原理知识准备 对于已经熟悉了session原理的同 ...
- PowerDesigner使用方法
我们需要创建一个测试数据库,一步一步来学习使用PowerDesigner,为了简单,我们在这个数据库中只创建一个Student表和一个Major表.其表结构和关系如下所示. 看看怎样用PowerDes ...
- 关于Flutter初始化流程，我必须告诉你的是...
1. 引言 最近在做性能优化的时候发现,在混合栈开发中,第一次启动Flutter页面的耗时总会是第二次启动Flutter页面耗时的两倍左右,这样给人感觉很不好.分析发现第一次启动Flutter页面会做 ...
- Python从入门到精通系列文章总目录
Python最新全套课程(8月中旬开的课),共四个月.所有课件,项目源码,课后习题和答案都包括在内. 包括:Python实战项目引入.Python基础.爬虫基础.爬虫库.Scrapy爬虫框架.动态页面 ...
- flask中接收post传递数组方法
list = request.form.getlist("表单名")
- 痞子衡嵌入式：让你从此高效写作的轻量级标记语言(Markdown)
大家好,我是痞子衡,是正经搞技术的痞子.今天痞子衡给大家介绍的是轻量级标记语言Markdown. 1.假如你有过这样的烦恼? 想写出排版优雅简洁的文章,并且能够轻易地发表(连同文字和排版)到各大网站上 ...
- .Net Core 中间件之主机地址过滤(HostFiltering)源码解析
一.介绍 主机地址过滤中间件相当于一个白名单,标记哪些主机地址能访问接口. 二.使用 新建WebAPI项目,修改Startup中的代码段如下所示.下面表示允许主机名为“localhost”的主机访问( ...
- python可变对象和不可变对象的解释
数据类型分为可变.不可变.可变对象表示可以原处修改该数据对象,不可变对象表示必须创建新对象来保存修改后的数据. 在基础数据类型中: 数值.字符串.元组.frozenset是不可变对象 列表.set.d ...
- @property、@sythesize以及Ivar和@dynamic讲解（下）
下面仅仅是一些基本知识,可能有些知识用的比较少,不过知道怎么使用或者了解这个知识,还是不错的,毕竟技多不压身嘛!读完这篇文章大约需要5-10分钟左右!!! 一.@property 1.在头文件中: @ ...
- C#之WebApi权限认证_学习笔记1
自己并不懂,在此先记录下来,留待以后学习... 正文 前言:最近,讨论到数据库安全的问题,于是就引出了WebApi服务没有加任何验证的问题.也就是说,任何人只要知道了接口的url,都能够模拟http请 ...