2. ClustrixDB 文件/参数说明
一、日志/data/clustrix/log/query.log
记录节点慢SQL/错误SQL/DDL 等信息，节点分开记录
Each entry in the query.log is categorized as one of these types. Specific logging for each query type is controlled by the global or session variable indicated.
|
Query Type |
Description |
|---|---|
| ALTER CLUSTER | Changes made to your cluster via the ALTER CLUSTER command are always logged to the query.log automatically. This logging is not controlled by a global variable. |
| BAD | The query reads more rows than necessary to return the expected results. This may indicate a bad plan or missing index. Logging of BAD queries is not enabled by default (session_log_bad_queries). |
| DDL | The query is DDL (i.e. schema change such as CREATE, DROP, ALTER), or a SET GLOBAL or SESSION command. All DDL queries are initially logged by default (session_log_ddl). |
| SLOW | Query execution time exceeded the threshold specified by the variable session_log_slow_threshold_ms. |
| SQLERR | These database errors are things such as syntax errors, timeout notifications, and permission issues. All SQLERR queries will be logged by default (session_log_error_queries). |
These are the variables that control query and user logging. The defaults shown are generally acceptable for most installations.
|
Name |
Description |
Default Value |
Session Variable |
|---|---|---|---|
| session_log_bad_queries | Log BAD queries to the query.log | false |
|
| session_log_ddl | Log DDL statements to query.log | true | |
| session_log_error_queries | Log ERROR statements to query.log | true | |
| session_log_slow_queries | Log SLOW statements to query.log | true | |
| session_log_slow_threshold_ms | Query duration threshold in milliseconds before logging this query | 10000 |
|
| session_log_users | Log LOGIN/LOGOUT to user.log | false |
MySQL [(none)]> show variables like '%session_log%';
+--------------------------------+--------+
| Variable_name | Value |
+--------------------------------+--------+
| session_log_bad_queries | false |
| session_log_bad_read_ratio | 100 |
| session_log_bad_read_threshold | 4000 |
| session_log_ddl | true |
| session_log_error_queries | true |
| session_log_slow_queries | true |
| session_log_slow_threshold_ms | 100000 |
| session_log_users | false |
+--------------------------------+--------+
8 rows in set (0.01 sec)
修改慢SQL时间，超过1s的SQL都被记录
MySQL [(none)]> set global session_log_slow_threshold_ms=1000;
Query OK, 0 rows affected (0.01 sec)MySQL [(none)]> show global variables like 'session_log_slow_threshold_ms';
+-------------------------------+-------+
| Variable_name | Value |
+-------------------------------+-------+
| session_log_slow_threshold_ms | 1000 |
+-------------------------------+-------+
查看慢SQL日志（集群所有的操作）：
MySQL [(none)]> select * from tail_query_log order by timestamp desc limit 10;
二、 日志 /data/clustrix/log/user.log
监控用户登录登出状态，单个节点文件记录单个节点信息，查看集群，system.tail_user_log表中查询。
默认关闭监控用户状态
MySQL [system]> show variables like 'session_log_users';
+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| session_log_users | false |
+-------------------+-------+
1 row in set (0.01 sec)MySQL [system]> set global session_log_users=1;
Query OK, 0 rows affected (0.51 sec)MySQL [system]> show variables like 'session_log_users';
+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| session_log_users | true |
+-------------------+-------+
查看集群用户登录登出状态
MySQL [system]> select * from system.tail_user_log;
+--------+----------------------------+------+----------+---------+---------------------------------------------------------------------------------------------------------------+----------+
| nodeid | timestamp | num | hostname | command | message | repeated |
+--------+----------------------------+------+----------+---------+---------------------------------------------------------------------------------------------------------------+----------+
| 1 | 2019-11-26 07:22:23.396171 | 0 | nid | NULL | 1 ip-10-1-3-88.cn-northwest-1.compute.internal clxnode: USER SID:16385 db="system" user=root@localhost LOGOUT | 0 |
| 1 | 2019-11-26 07:22:24.371812 | 1 | nid | NULL | 1 ip-10-1-3-88.cn-northwest-1.compute.internal clxnode: USER SID:17409 db=#undef user=root@localhost LOGIN | 0 |
| 1 | 2019-11-26 07:22:54.216004 | 2 | nid | NULL | 1 ip-10-1-3-88.cn-northwest-1.compute.internal clxnode: USER SID:18433 db=#undef user=root@localhost LOGIN | 0 |
| 1 | 2019-11-26 07:23:05.066316 | 3 | nid | NULL | 1 ip-10-1-3-88.cn-northwest-1.compute.internal clxnode: USER SID:18433 db=#undef user=root@localhost LOGOUT | 0 |
| 2 | 2019-11-26 07:23:07.523621 | 0 | nid | NULL | 2 ip-10-1-3-242.cn-northwest-1.compute.internal clxnode: USER SID:6146 db="system" user=root@localhost LOGOUT | 0 |
| 2 | 2019-11-26 07:23:08.049621 | 1 | nid | NULL | 2 ip-10-1-3-242.cn-northwest-1.compute.internal clxnode: USER SID:7170 db=#undef user=root@localhost LOGIN | 0 |
+--------+----------------------------+------+----------+---------+---------------------------------------------------------------------------------------------------------------+----------+
6 rows in set (0.00 sec)
三、Rebalance 配置
查看rebalance状态
sql> select * from system.rebalancer_activity_log order by started desc limit 10;
提高rebalance性能
sql> set global rebalancer_rebalance_task_limit = 8;
sql> set global rebalancer_vdev_task_limit = 4;
sql> set global task_rebalancer_rebalance_distribution_interval_ms = 5000;
sql> set global task_rebalancer_rebalance_interval_ms = 5000;
如果出现负载过高
sql> set global rebalancer_rebalance_task_limit = default;sql> set global rebalancer_vdev_task_limit = default;
sql> set global task_rebalancer_rebalance_distribution_interval_ms = default;
sql> set global task_rebalancer_rebalance_interval_ms = default;
四、查看那些配置不是默认值
sql> select * from system.global_variable_definitions where current_value != default_value;
五、监控SQL
列出消耗CPU最大的3个历史SQL
SELECT nodeid, exec_count, exec_ms, exec_ms/exec_count as avg_ms, left(statement,100)
FROM system.qpc_queries
ORDER BY exec_ms desc
LIMIT 3;
列出过去24小时内最频繁运行的100个SQL
sql> SELECT query_key,
min(rank),
max(rank),
database,
left(statement,100),
sum(exec_count) as calc_exec_count,
round(avg(avg_rows_read)) as calc_avg_rows_read,
round(avg(avg_exec_ms)) as calc_avg_exec_ms
FROM clustrix_statd.qpc_history
WHERE timestamp BETWEEN (now() - interval 24 hour) AND now()
AND database !='clustrix_statd'
GROUP BY query_key
ORDER BY calc_exec_count DESC,
calc_avg_rows_read DESC
LIMIT 100;
列出过去24小时内返回行数最多的100个SQL
sql> SELECT query_key,
min(rank),
max(rank),
database,
left(statement,100),
sum(exec_count) as calc_exec_count,
round(avg(avg_rows_read)) as calc_avg_rows_read,
round(avg(avg_exec_ms)) as calc_avg_exec_ms
FROM clustrix_statd.qpc_history
WHERE timestamp BETWEEN (now() - interval 24 hour) AND now()
AND database !='clustrix_statd'
GROUP BY query_key
ORDER BY calc_avg_rows_read DESC,
calc_exec_count DESC
LIMIT 100;
列出过去24小时内返回运行时间最长的100个SQL
sql> SELECT query_key,
min(rank),
max(rank),
database,
left(statement,100),
sum(exec_count) as calc_exec_count,
round(avg(avg_rows_read)) as calc_avg_rows_read,
round(avg(avg_exec_ms)) as calc_avg_exec_ms
FROM clustrix_statd.qpc_history
WHERE timestamp BETWEEN (now() - interval 24 hour) AND now()
AND database !='clustrix_statd'
GROUP BY query_key
ORDER BY calc_avg_exec_ms DESC,
calc_exec_count DESC
LIMIT 100;
六、system's tables
MySQL [(none)]> use system;
Database changed
MySQL [system]> show tables;
+-----------------------------------+
| Tables_in_system |
+-----------------------------------+
| activity |
| alerts_intervals |
| alerts_messages |
| alerts_parameters |
| alerts_subscriptions |
| alter_progress |
| autoinc_sequences |
| backups |
| backup_masters |
| backup_status |
| backup_tables |
| barriers |
| base_allocators |
| bigc_state |
| binlogs |
| binlog_commits |
| binlog_commits_segments |
| binlog_ignore_databases |
| binlog_ignore_tables |
| binlog_log_databases |
| binlog_log_tables |
| binlog_segments |
| bm_latch_waits |
| bm_stats |
| broadcast_nodes |
| check_constraints |
| cluster_session_stats |
| cluster_session_variables |
| columns |
| constraints |
| containers |
| container_stats |
| container_truncates |
| container_type_codes |
| cpm_history |
| cpm_info |
| cpu_activity |
| cpu_allocations |
| cpu_load |
| databases |
| debugpoints |
| deferred_foreign_keys |
| deferred_foreign_key_columns |
| definers |
| device_containers |
| device_containers2 |
| device_space_stats |
| disks |
| disk_activity |
| disk_paths |
| dlog_stats |
| dropped_binlogs |
| engines |
| error_codes |
| event_map |
| failpoints |
| fibers |
| flow_control_channels |
| flow_control_peers |
| foreign_keys |
| foreign_key_columns |
| global_stats |
| global_variables |
| global_variables_ignored |
| global_variable_definitions |
| groups |
| gtm_accepter |
| gtm_coord |
| gtm_coord_invocations |
| gtm_ddl |
| gtm_ddl_server |
| gtm_poisoned_invoc |
| gtm_poisoned_trx |
| gtm_repick_accepters |
| gtm_resolver |
| gtm_send_queues |
| hash_distribution_map |
| heaps |
| imported_index_pds |
| index_sizes |
| index_stats |
| init_graph_stats |
| internal_routines |
| internode_latency |
| invocations |
| irp_queues |
| key_caches |
| layercons |
| layers |
| layer_merges |
| license |
| load |
| lockman |
| lockman_holders |
| lockman_victims |
| lpdcache |
| lpds |
| ltm_transactions |
| mdstat |
| media_scanners |
| membership |
| memory_table_replicas |
| missing_pds |
| missing_pd_columns |
| missing_pd_details |
| mounts |
| mvcc_waiters |
| mysql_binlogs |
| mysql_binlog_index |
| mysql_binlog_segments |
| mysql_binlog_stats |
| mysql_binlog_trims |
| mysql_character_sets |
| mysql_collations |
| mysql_db_replication_policy |
| mysql_error_codes |
| mysql_indexed_binlogs |
| mysql_master_status |
| mysql_registered_slaves |
| mysql_repconfig |
| mysql_repslave_svars |
| mysql_repstate |
| mysql_repstate_until |
| mysql_sessions |
| mysql_slave_connection_status |
| mysql_slave_db_replication_policy |
| mysql_slave_driver_status |
| mysql_slave_log_updates |
| mysql_slave_rewrite_db |
| mysql_slave_skip_errors |
| mysql_slave_stats |
| mysql_slave_status |
| mysql_slave_variables |
| mysql_table_replication_policy |
| named_locks |
| networking |
| network_activity |
| nodeinfo |
| nodes |
| objects |
| partitioned_hash_distributions |
| partitions |
| partition_endpoints |
| partition_functions |
| pdcache_raw |
| pdms |
| pds |
| pd_groups |
| pd_group_columns |
| pending_invites |
| periodic_tasks |
| poisoned_barriers |
| privileges |
| problem_nodes |
| processlist |
| processlistfull |
| proc_cpu |
| proc_cpu_rates |
| proc_diskstats |
| proc_diskstat_rates |
| proc_interrupts |
| proc_interrupt_rates |
| proc_meminfo |
| proc_net_dev |
| proc_net_dev_rates |
| proc_softirqs |
| proc_softirq_rates |
| profiled_invocations |
| profiled_plans |
| profiled_statements |
| profiled_til |
| profiled_transactions |
| profiling |
| program_cache |
| protection_log |
| ps |
| public_keys |
| qpc_lru |
| qpc_plans |
| qpc_queries |
| queues |
| queue_readers |
| queue_replays |
| queue_replay_streams |
| queue_replay_waiters |
| queue_status |
| range_hash_distributions |
| rebalancer_activity_log |
| rebalancer_activity_targets |
| rebalancer_copies |
| rebalancer_copy_activity |
| rebalancer_copy_work_queue |
| rebalancer_hash_distributions |
| rebalancer_queued_activity |
| rebalancer_redistributes |
| rebalancer_replicas |
| rebalancer_representations |
| rebalancer_scheduled_replicas |
| rebalancer_slices |
| rebalancer_splits |
| rebalancer_started_activity |
| rebalancer_summary |
| rebalancer_vdevs |
| redistributes |
| relations |
| relation_alters |
| relation_alter_columns |
| relation_build_status |
| relation_sizes |
| replicas |
| replicated_containers |
| replication_checkpoint |
| replication_master_status |
| replica_copies |
| replica_sizes |
| replica_status_codes |
| representations |
| representation_builds |
| representation_columns |
| representation_sizes |
| representation_stats |
| restore_database_builds |
| restore_object_builds |
| ril_cache_p |
| ril_stats |
| routines |
| routine_parameters |
| sequences |
| sequence_state |
| sessions |
| session_call_stacks |
| session_containers |
| session_local_variables |
| session_pdcache |
| session_row_stats |
| sighandlers |
| sighandlers_json |
| signals |
| skiplists |
| slave_row_stats |
| slave_slices |
| slave_slice_readers |
| slave_slice_relations |
| slave_slice_writers |
| slices |
| slice_splits |
| softfailed_devices |
| softfailed_nodes |
| softfailing_containers |
| sqlstates |
| stats |
| strmaps |
| suspended_closures |
| system_users |
| sys_block_devs |
| tables_and_views |
| table_pdms |
| table_replicas |
| table_sizes |
| table_slices |
| tail_clustrix_log |
| tail_nanny_log |
| tail_query_log |
| tail_user_log |
| tail_webui_log |
| tasks |
| task_placement |
| tcp |
| temporary_tables |
| time_zones |
| tracepoints |
| transactions |
| triggers |
| trigger_event |
| trigger_orientation |
| trigger_timing |
| trxstate |
| trxstate_stats |
| underprotected_slices |
| users |
| user_accessible_databases |
| user_accessible_tables |
| user_accessible_triggers |
| user_accessible_users |
| user_acl |
| user_routine_acl |
| vdev_io |
| vdev_stat |
| version_history |
| views |
| virtual_relations |
| virtual_views |
| vmstats |
| wals |
| wal_windows |
+-----------------------------------+
295 rows in set (0.14 sec)
2. ClustrixDB 文件/参数说明的更多相关文章
- [转帖]/proc/sys目录下各文件参数说明
/proc/sys目录下各文件参数说明 https://blog.csdn.net/luteresa/article/details/68061881 一.前言 本文档针对OOP8生产环境,具体优 ...
- docker-compose模板文件参数说明
working_dir:一般这个参数用在应用程序Services下,我们指定应用程序所在的目录为当前目录,类似linux中的cd working_dir.其余的参数,例如command等就是基于此参数 ...
- /proc/sys目录下各文件参数说明
linux 其他知识目录 原文链接:https://blog.csdn.net/hshl1214/article/details/4596583 一.前言本文档针对OOP8生产环境,具体优化策略需要根 ...
- linux的ip文件参数说明
TYPE=Ethernet # 网卡类型:为以太网 PROXY_METHOD=none # 代理方式:关闭状态 BROWSER_ONLY=no # 只是浏览器:否 BOOTPROTO=dhcp # 网 ...
- 压缩和解压文件：tar gzip bzip2 compress(转)
tar[必要参数][选择参数][文件] 压缩:tar -czvf filename.tar.gz targetfile解压:tar -zxvf filename.tar.gz参数说明: -c 建立新的 ...
- 应用程序启动后修改自身EXE文件或自删除EXE文件（附VC++6.0源码）
在CSDN论坛看到这么一个问题:如何为第三方工具加上使用限制次数?问题的答案很简单,重新做一个应用程序,将第三方程序打包进这个应用程序,启动应用程序时可以检查第三方工具的使用次数,检查通过,可运行第三 ...
- 使用PHP文件锁写一个多个请求同时并发写入一个文件，要求不脏读、数据不丢失
使用PHP文件锁写一个多个请求同时并发写入一个文件,要求不脏读.数据不丢失. //并发文件操作 function filehandle($filename,$data){ $start = 0; $e ...
- Linux 文件内容查看（cat、tac、nl 、more 、less、head、tail ）
Linux系统中使用以下命令来查看文件的内容: cat: 由第一行开始显示文件内容tac :从最后一行开始显示,可以看出 tac 是 cat 的倒著写!nl: 显示的时候,顺道输出行号!more ...
- 深度学习原理与框架-Alexnet(迁移学习代码) 1.sys.argv[1:](控制台输入的参数获取第二个参数开始) 2.tf.split(对数据进行切分操作) 3.tf.concat(对数据进行合并操作) 4.tf.variable_scope(指定w的使用范围) 5.tf.get_variable(构造和获得参数) 6.np.load(加载.npy文件)
1. sys.argv[1:] # 在控制台进行参数的输入时,只使用第二个参数以后的数据 参数说明:控制台的输入:python test.py what, 使用sys.argv[1:],那么将获得w ...
随机推荐
- Series的idxmax和argmax
转载至:https://www.cnblogs.com/liulangmao/p/9211537.html pandas Series 的 argmax 方法和 idxmax 方法用于获取 Serie ...
- SCP,scp linux2台机器之间如何传输文件
关键词:scp 转自: http://blog.csdn.net/gatieme https://blog.csdn.net/gatieme/article/details/51673229 scp传 ...
- PostgreSQL通过解析日志，获取数据库增量变化，pg_recvlogical
1.首先用该工具来看我们的日志变化,需要先将test_decoding插件编译并安装(进入contrib,编译安装即可) 创建一个slot: SELECT * FROM pg_create_logic ...
- springboot - 应用实践（2）第一个springboot应用
1.使用maven创建一个快速启动项目 2.引入相关依赖 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:x ...
- [USACO13OPEN]照片Photo 题解
题面 这道题似乎可以用单调队列优化DP做,但这里讲的是一种差分约束的思路; 设s[i]表示1~i中选了多少个: s[b[i]]-s[a[i]-1]<=1; s[b[i]]-s[a[i]-1]&g ...
- Electric Charges CodeForces - 623C (二分答案)
大意: 平面上n个点每个点坐标为(x,0)或(0,y), 求任意两点距离平方最大值的最小值. 二分答案, 转化为判定最大值是否<=e, 按$x$排序后, 因为固定左端点, $y$绝对值的最大值是 ...
- 使用 tablib 来自动化管理测试用例，其他的工具都不用学了
你在学习 python 自动化测试吗?听过 requests 库吗?tablib 是 requests 库作者常年维护的一个可以操作 Excel 等多种文件格式,将他们变成一种通用数据集的第三方库. ...
- orm的设计思路
一,我们先搞懂什么是orm? ORM:对象关系映射(Object Relational Mapping,简称ORM),目的是想像操作对象一样操作数据库.因为数据库不是面向对象的,所以需要编程进行映射. ...
- numpy.random.uniform(记住文档网址)
http://docs.scipy.org/doc/numpy/reference/generated/numpy.random.uniform.html#numpy.random.uniform h ...
- GDAL支持中文路径和Shp文件中文属性写入
在使用GDAL的过程中,为了支持中文,比需手动进行中文路径的设置,同时特别是在对Shp的属性进行中文输入的时候,都必须进行必要的设定. 为了支持中文路径,在注册了驱动之后,加上第三句就可以了.必须设置 ...