再谈PG索引-存储架构
1.索引的基本架构
PG的索引是B+树，B+树是为磁盘或其他直接存取辅助设备而设计的一种平衡查找树，在B+树中，所有记录节点都是按键值的大小顺序存放在同一层的叶节点中，各叶节点指针进行连接：
meta page
|
root page（8kb，一个记录占32个bit，那么就能存256个branch page，超过了就需要扩充一级branch page来存储leaf page）
|
branch page …
| | |
branch page branch page branch page …
| | | | | | |
leaf page leaf page leaf page leaf page leaf page leaf page leaf page …
|
———————-------------
| key |
| (block,offset) | 一个leaf page存放多个索引值
———————-------------
其中meta page和root page是必须有的，meta page需要一个页来存储，表示指向root page的page id。
随着记录数的增加，一个root page可能存不下所有的heap item，就会有branch page，甚至多层的branch page。
leaf page存储具体的key和value。
一共有几层branch，就用btree page元数据的 level 来表示，如果level为0，则表示没有branch层，root page直接指向leaf page，最多记录256条记录（假如条指针占32bit）；level为1， 则表示有一层branch page，则root page存放branch page的指针，branch page指向leaf page，最多记录256*256条。依次类推，且bock size是可以设置的，当设置的更大，则一个级别存储的数据就更多。
2.看看具体的结构
PostgreSQL B-Tree是一种变种(high-concurrency B-tree management algorithm)，算法详情请参考 src/backend/access/nbtree/README。我们可以使用pageinspect插件，内窥B-Tree的结构：
apple=# create extension pageinspect;
CREATE EXTENSION
apple=# \dx
List of installed extensions
Name | Version | Schema | Description
-------------+---------+------------+-------------------------------------------------------
pageinspect | 1.7 | public | inspect the contents of database pages at a low level
plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language
(2 rows)
apple=# create table test(id int primary key, info text);
CREATE TABLE
apple=# insert into test select generate_series(1, 1000), md5(random()::text);
INSERT 0 1000
apple=# vacuum ANALYZE test;
VACUUM
apple=# \d test
Table "public.test"
Column | Type | Collation | Nullable | Default
--------+---------+-----------+----------+---------
id | integer | | not null |
info | text | | |
Indexes:
"test_pkey" PRIMARY KEY, btree (id)
apple=# select * from bt_metap('test_pkey');
magic | version | root | level | fastroot | fastlevel | oldest_xact | last_cleanup_num_tuples
--------+---------+------+-------+----------+-----------+-------------+-------------------------
340322 | 3 | 3 | 1 | 3 | 1 | 0 | 1000
(1 row)
apple=# select * from bt_page_stats('test_pkey',1);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
1 | l | 367 | 0 | 16 | 8192 | 808 | 0 | 2 | 0 | 1
(1 row)
apple=# select * from bt_page_stats('test_pkey',2);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
2 | l | 367 | 0 | 16 | 8192 | 808 | 1 | 4 | 0 | 1
(1 row)
apple=# select * from bt_page_stats('test_pkey',3);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
3 | r | 3 | 0 | 13 | 8192 | 8096 | 0 | 0 | 1 | 2
(1 row)
apple=# select * from bt_page_stats('test_pkey',0);
2019-05-29 11:10:47.567 CST [49885] ERROR: block 0 is a meta page
2019-05-29 11:10:47.567 CST [49885] STATEMENT: select * from bt_page_stats('test_pkey',0);
ERROR: block 0 is a meta page
apple=# select * from bt_page_stats('test_pkey',4);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
4 | l | 268 | 0 | 16 | 8192 | 2788 | 2 | 0 | 0 | 1
(1 row)
apple=# select * from bt_page_stats('test_pkey',5);
2019-05-29 11:11:18.181 CST [49885] ERROR: block number out of range
2019-05-29 11:11:18.181 CST [49885] STATEMENT: select * from bt_page_stats('test_pkey',5);
ERROR: block number out of range
meta page
root page # btpo_flags=2
branch page # btpo_flags=0
leaf page # btpo_flags=1
如果即是leaf又是root则 btpo_flags=3
根据btpo_flage可以看出结构应该是这样，索引占了5个block：
meta page （block 0，btpo_flags为2，上面可以看到最后一列）
|
root page （ block 3）
| | |
leaf page（ block 1） leaf page （ block 2）leaf page（ block 4）
|
———————-------------
| key |
| (block,offset) |
———————-------------
查看root page：
apple=# select * from bt_page_items('test_pkey',3);
itemoffset | ctid | itemlen | nulls | vars | data
------------+--------+---------+-------+------+-------------------------
1 | (1,0) | 8 | f | f |
2 | (2,7) | 16 | f | f | 6f 01 00 00 00 00 00 00
3 | (4,13) | 16 | f | f | dd 02 00 00 00 00 00 00
(3 rows)
查看leaf page：
apple=# select * from bt_page_items('test_pkey',1);
itemoffset | ctid | itemlen | nulls | vars | data
------------+---------+---------+-------+------+-------------------------
1 | (3,7) | 16 | f | f | 6f 01 00 00 00 00 00 00
2 | (0,1) | 16 | f | f | 01 00 00 00 00 00 00 00
3 | (0,2) | 16 | f | f | 02 00 00 00 00 00 00 00
4 | (0,3) | 16 | f | f | 03 00 00 00 00 00 00 00
5 | (0,4) | 16 | f | f | 04 00 00 00 00 00 00 00
6 | (0,5) | 16 | f | f | 05 00 00 00 00 00 00 00
7 | (0,6) | 16 | f | f | 06 00 00 00 00 00 00 00
8 | (0,7) | 16 | f | f | 07 00 00 00 00 00 00 00
….
查看一个item对应的记录：
apple=# select * from test where ctid = '(3,7)';
id | info
-----+----------------------------------
367 | 06818c090f9e5f63c95764342590a598
(1 row)
那么索引里面的key是怎么排序的？块为什么不是连续的，块2变为了root page。
3.查看level的变化
apple=# drop table test;
DROP TABLE
apple=# create table test(id int primary key, info text);
CREATE TABLE
apple=# insert into test select t.id, md5(random()::text) from generate_series(1, 20) as t(id);
INSERT 0 20
apple=# select * from bt_page_stats('test_pkey', 1);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
1 | l | 20 | 0 | 16 | 8192 | 7748 | 0 | 0 | 0 | 3
(1 row)
apple=# select * from bt_metap('test_pkey');
magic | version | root | level | fastroot | fastlevel | oldest_xact | last_cleanup_num_tuples
--------+---------+------+-------+----------+-----------+-------------+-------------------------
340322 | 3 | 1 | 0 | 1 | 0 | 0 | -1
(1 row)
apple=# select bt_page_items('test_pkey', 1);
bt_page_items
------------------------------------------------
(1,"(0,1)",16,f,f,"01 00 00 00 00 00 00 00")
(2,"(0,2)",16,f,f,"02 00 00 00 00 00 00 00")
(3,"(0,3)",16,f,f,"03 00 00 00 00 00 00 00")
(4,"(0,4)",16,f,f,"04 00 00 00 00 00 00 00")
(5,"(0,5)",16,f,f,"05 00 00 00 00 00 00 00")
(6,"(0,6)",16,f,f,"06 00 00 00 00 00 00 00")
(7,"(0,7)",16,f,f,"07 00 00 00 00 00 00 00")
(8,"(0,8)",16,f,f,"08 00 00 00 00 00 00 00")
(9,"(0,9)",16,f,f,"09 00 00 00 00 00 00 00")
(10,"(0,10)",16,f,f,"0a 00 00 00 00 00 00 00")
(11,"(0,11)",16,f,f,"0b 00 00 00 00 00 00 00")
(12,"(0,12)",16,f,f,"0c 00 00 00 00 00 00 00")
(13,"(0,13)",16,f,f,"0d 00 00 00 00 00 00 00")
(14,"(0,14)",16,f,f,"0e 00 00 00 00 00 00 00")
(15,"(0,15)",16,f,f,"0f 00 00 00 00 00 00 00")
(16,"(0,16)",16,f,f,"10 00 00 00 00 00 00 00")
(17,"(0,17)",16,f,f,"11 00 00 00 00 00 00 00")
(18,"(0,18)",16,f,f,"12 00 00 00 00 00 00 00")
(19,"(0,19)",16,f,f,"13 00 00 00 00 00 00 00")
(20,"(0,20)",16,f,f,"14 00 00 00 00 00 00 00")
(20 rows)
插入20条，一个root页面就能存放，那么就没有必要申请一个leaf page，root page就是leaf page，他们的btpo_flags就是3；由于没有branch page，因此level也就是0；
继续插入数据：
apple=# insert into test select t.id, md5(random()::text) from generate_series(21, 1000) as t(id);
INSERT 0 980
apple=# select * from bt_metap('test_pkey');
magic | version | root | level | fastroot | fastlevel | oldest_xact | last_cleanup_num_tuples
--------+---------+------+-------+----------+-----------+-------------+-------------------------
340322 | 3 | 3 | 1 | 3 | 1 | 0 | -1
(1 row)
apple=# select * from bt_page_stats('test_pkey', 0);
ERROR: block 0 is a meta page
apple=# select * from bt_page_stats('test_pkey', 1);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
1 | l | 367 | 0 | 16 | 8192 | 808 | 0 | 2 | 0 | 1
(1 row)
apple=# select * from bt_page_stats('test_pkey', 2);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
2 | l | 367 | 0 | 16 | 8192 | 808 | 1 | 4 | 0 | 1
(1 row)
apple=# select * from bt_page_stats('test_pkey', 3);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
3 | r | 3 | 0 | 13 | 8192 | 8096 | 0 | 0 | 1 | 2
(1 row)
apple=# select * from bt_page_stats('test_pkey', 4);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
4 | l | 268 | 0 | 16 | 8192 | 2788 | 2 | 0 | 0 | 1
(1 row)
apple=# select * from bt_page_stats('test_pkey', 5);
ERROR: block number out of range
可以看到leve由0变为了1，有了新的root page，从block 1变为了block 3，且加入了三个新的leaf page。
我们可以看到一个leaf页面大概在以int类型为索引时，大概可以存放367条记录，而一个root page中记录一个leaf page指针只需要13bit大小，那么我们继续增大多少条，可以出现branch page呢？
(8192/13) * 367 - 1000 = 230210条，那么我们就插入数据试试：
apple=# insert into test select t.id, md5(random()::text) from generate_series(1001, 230210) as t(id);
INSERT 0 229210
apple=# select * from bt_page_stats('test_pkey', 0);
ERROR: block 0 is a meta page
apple=# analyze ;
ANALYZE
apple=# select * from bt_metap('test_pkey');
magic | version | root | level | fastroot | fastlevel | oldest_xact | last_cleanup_num_tuples
--------+---------+------+-------+----------+-----------+-------------+-------------------------
340322 | 3 | 412 | 2 | 412 | 2 | 0 | -1
(1 row)
apple=# select * from bt_page_stats('test_pkey', 412);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
412 | r | 2 | 0 | 12 | 8192 | 8116 | 0 | 0 | 2 | 2
(1 row)
apple=# select * from bt_page_stats('test_pkey', 411);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
411 | i | 344 | 0 | 15 | 8192 | 1276 | 3 | 0 | 1 | 0
(1 row)
apple=# select * from bt_page_stats('test_pkey', 3);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
3 | i | 286 | 0 | 15 | 8192 | 2436 | 0 | 411 | 1 | 0
(1 row)
apple=# select * from bt_page_stats('test_pkey', 8);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
8 | l | 367 | 0 | 16 | 8192 | 808 | 7 | 9 | 0 | 1
(1 row)
apple=# select * from bt_page_stats('test_pkey', 1);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
1 | l | 367 | 0 | 16 | 8192 | 808 | 0 | 2 | 0 | 1
(1 row)
apple=# select * from bt_page_stats('test_pkey', 2);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
2 | l | 367 | 0 | 16 | 8192 | 808 | 1 | 4 | 0 | 1
(1 row)
apple=# select * from bt_page_stats('test_pkey', 4);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
4 | l | 367 | 0 | 16 | 8192 | 808 | 2 | 5 | 0 | 1
(1 row)
apple=# select * from bt_page_stats('test_pkey', 632);
blkno | type | live_items | dead_items | avg_item_size | page_size | free_size | btpo_prev | btpo_next | btpo | btpo_flags
-------+------+------------+------------+---------------+-----------+-----------+-----------+-----------+------+------------
632 | l | 362 | 0 | 16 | 8192 | 908 | 631 | 0 | 0 | 1
(1 row)
apple=# select * from bt_page_stats('test_pkey', 633);
ERROR: block number out of range
上面可以看到level由1变为了2，root page又由上面的block 3变为了block 412，而block 3蜕变为一个branch节点；增加了两个branch page分别是btpo_flags为0的block 411和 block 3；可以计算到最后一个page应该是(8192/13) + 2 = 632。
两个branch page是从第一个指向第二个3的下一个是411，在哪里看到3号block是第一个branch page？
所有的leaf page是从0->1->2->4->5 … ->632->0，怎么知道哪个leaf page是从哪个branch page指出来的呢？
答案是可以通过看root page和branch page里面的值来看：
从root page的值可以看到是存放了对应的branch page的块号和偏移量
apple=# select * from bt_page_items('test_pkey', 412);
itemoffset | ctid | itemlen | nulls | vars | data
------------+----------+---------+-------+------+-------------------------
1 | (3,0) | 8 | f | f |
2 | (411,31) | 16 | f | f | 77 97 01 00 00 00 00 00
(2 rows)
branch page存放了leaf page的block number和offset
apple=# select * from bt_page_items('test_pkey', 3);
itemoffset | ctid | itemlen | nulls | vars | data
------------+-----------+---------+-------+------+-------------------------
1 | (287,31) | 16 | f | f | 77 97 01 00 00 00 00 00
2 | (1,0) | 8 | f | f |
3 | (2,7) | 16 | f | f | 6f 01 00 00 00 00 00 00
4 | (4,13) | 16 | f | f | dd 02 00 00 00 00 00 00
5 | (5,19) | 16 | f | f | 4b 04 00 00 00 00 00 00
6 | (6,25) | 16 | f | f | b9 05 00 00 00 00 00 00
7 | (7,31) | 16 | f | f | 27 07 00 00 00 00 00 00
8 | (8,37) | 16 | f | f | 95 08 00 00 00 00 00 00
9 | (9,43) | 16 | f | f | 03 0a 00 00 00 00 00 00
那么root page和branch page的每个item会存放他们对应的块里面索引key按照B+树的方式进行组织，我们这里B+树高度为2，每页可存放630条记录，因此，可以大致的画出当前的拓扑图：
当然这些数据都是基于B+树进行按顺序排列的，B+树可以指定树的宽度，我们这边的宽度不指定，而是按实际大小计算的吗？例如这里：8192/13 = 630，即M=630。
4.扫描时时间消耗估算
apple=# explain (analyze, verbose, timing, costs, buffers) select id from test where id = 11;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------
Index Only Scan using test_pkey on public.test (cost=0.42..8.44 rows=1 width=4) (actual time=0.247..0.248 rows=1 loops=1)
Output: id
Index Cond: (test.id = 11)
Heap Fetches: 1
Buffers: shared hit=4
Planning Time: 0.151 ms
Execution Time: 0.287 ms
(7 rows)
costs = 1 meta page + root page + branch page + leaf page = 4
apple=# explain (analyze, verbose, timing, costs, buffers) select id from test where id < 11;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------
Index Only Scan using test_pkey on public.test (cost=0.42..8.61 rows=11 width=4) (actual time=0.006..0.009 rows=10 loops=1)
Output: id
Index Cond: (test.id < 11)
Heap Fetches: 10
Buffers: shared hit=4
Planning Time: 0.100 ms
Execution Time: 0.027 ms
(7 rows)
costs = 1 meta page + root page + branch page + leaf page = 4，一个块的读取消耗几乎忽略不计。
apple=# explain (analyze, verbose, timing, costs, buffers) select id from test where id in (1,3,1000, 222222, 111111111, 1232244,11);
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------
Index Only Scan using test_pkey on public.test (cost=0.42..35.06 rows=7 width=4) (actual time=0.035..0.088 rows=5 loops=1)
Output: id
Index Cond: (test.id = ANY ('{1,3,1000,222222,111111111,1232244,11}'::integer[]))
Heap Fetches: 5
Buffers: shared hit=27
Planning Time: 0.095 ms
Execution Time: 0.113 ms
(7 rows)
只有五条记录在leaf中有，因此：
costs = 1 meta page + 5*(root page + branch page 1 + branch page 2 + leaf page) + 2* (root page + branch page 1 + branch page 2) = 27
后续给出索引每个item对应的代码结构。
https://www.cnblogs.com/scu-cjx/p/9960483.html
再谈PG索引-存储架构的更多相关文章
- 再谈MySql索引
一.索引简介 MySQL索引的建立对于MySQL的高效运行是很重要的,索引可以大大提高MySQL的检索速度. 索引分单列索引(主键索引.唯一索引.普通索引)和组合索引.单列索引,即一个索引只包含单个列 ...
- 谈一下关于CQRS架构如何实现高性能
CQRS架构简介 前不久,看到博客园一位园友写了一篇文章,其中的观点是,要想高性能,需要尽量:避开网络开销(IO),避开海量数据,避开资源争夺.对于这3点,我觉得很有道理.所以也想谈一下,CQRS架构 ...
- 浅谈MySQL索引背后的数据结构及算法
摘要 本文以MySQL数据库为研究对象,讨论与数据库索引相关的一些话题.特别需要说明的是,MySQL支持诸多存储引擎,而各种存储引擎对索引的支持也各不相同,因此MySQL数据库支持多种索引类型,如BT ...
- 浅谈MySQL索引背后的数据结构及算法(转载)
转自:http://blogread.cn/it/article/4088?f=wb1 摘要 本文以MySQL数据库为研究对象,讨论与数据库索引相关的一些话题.特别需要说明的是,MySQL支持诸多存储 ...
- 浅谈MySQL索引背后的数据结构及算法【转】
摘要 本文以MySQL数据库为研究对象,讨论与数据库索引相关的一些话题.特别需要说明的是,MySQL支持诸多存储引擎,而各种存储引擎对索引的支持也各不相同,因此MySQL数据库支持多种索引类型,如BT ...
- 【原创】阿里三面:搞透Kafka的存储架构，看这篇就够了
阅读本文大约需要30分钟.这篇文章干货很多,希望你可以耐心读完. 你好, 我是华仔,在这个 1024 程序员特殊的节日里,又和大家见面了. 从这篇文章开始,我将对 Kafka 专项知识进行深度剖析, ...
- 深度长文：深入理解Ceph存储架构
点击上方"开源Linux",选择"设为星标" 回复"学习"获取独家整理的学习资料! 本文是一篇Ceph存储架构技术文章,内容深入到每个存储特 ...
- QQ 相册后台存储架构重构与跨 IDC 容灾实践
欢迎大家前往云加社区,获取更多腾讯海量技术实践干货哦~ 作者简介:xianmau,2015 年加入腾讯 TEG 架构平台部,一直负责 QQ 相册平台的维护和建设,主导相册上传架构重构和容灾优化等工作. ...
- [转载]再谈PostgreSQL的膨胀和vacuum机制及最佳实践
本文转载自 www.postgres.cn 下的文章: 再谈PostgreSQL的膨胀和vacuum机制及最佳实践http://www.postgres.cn/news/viewone/1/390 还 ...
随机推荐
- 关于C#的学习
长期以来对C#的认识一直停留在微软件开发的完全面向对象的语言的模糊印象上,对其工程也缺乏多文件以上级别的修改能力,而当前流行度的驱使下,想深入了解它并运用. 于是从git上下载了一个C#开源项目,打开 ...
- jquery timeStamp属性 语法
jquery timeStamp属性 语法 作用:timeStamp 属性包含从 1970 年 1 月 1 日到事件被触发时的毫秒数.直线模组 语法:event.timeStam 参数: 参数 描述 ...
- 计算机网络（四），TCP三次握手
目录 1.三次握手详情 2.为什么需要三次握手才能建立连接 3.首次握手的隐患---SYN超时的问题 4.建立连接之后,Client出现故障 四.TCP三次握手 1.三次握手详情 (1)一开始,客户端 ...
- 一些简单题(2)（Source : NOIP历年试题+杂题）
P3084 [USACO13OPEN]照片Photo 给出$m$个区间$[l_i,r_i]$覆盖$S=[1,n]$,试确定最大特殊点的数使得这每一个区间覆盖且仅覆盖一个特殊点. 如果无解,输出$-1$ ...
- classpath说明
概念解释: classpath : 即项目中WEB-INF下面的classes目录; 应用: [01] src路径下的文件在编译后会放到WEB-INF/classes路径下.默认的classpath是 ...
- ANR错误分析
链接1:https://www.cnblogs.com/xiyuan2016/p/6740623.html 链接2:https://www.jianshu.com/p/3959a601cea6
- spark MLlib 概念 6：ALS（Alternating Least Squares） or (ALS-WR)
Large-scale Parallel Collaborative Filtering for the Netflix Prize http://www.hpl.hp.com/personal/Ro ...
- spark 笔记 6: RDD
了解RDD之前,必读UCB的论文,个人认为这是最好的资料,没有之一. http://www.cs.berkeley.edu/~matei/papers/2012/nsdi_spark.pdf A Re ...
- 使用多块GPU进行训练 1.slim.arg_scope(对于同等类型使用相同操作) 2.tf.name_scope(定义名字的范围) 3.tf.get_variable_scope().reuse_variable(参数的复用) 4.tf.py_func(构造函数)
1. slim.arg_scope(函数, 传参) # 对于同类的函数操作,都传入相同的参数 from tensorflow.contrib import slim as slim import te ...
- Navicat1_介绍
https://study.163.com/course/courseMain.htm?courseId=1006383008&share=2&shareId=400000000398 ...