摘抄自hbase ref guide 0.94

在写本文的时候,hbase ref guide已经更新到1.2及2.0了,但是个人感觉Thrift过滤语法部分写得都没有0.94的好,省掉了example,看起来很不方便,也不便于在hbase shell中进行调试,因而复制一份过来,以免以后找不到;

-------------

10.3. Thrift

Currently most of the documentation on Thrift exists in the Apache HBase Wiki on Thrift.

10.3.1. Filter Language

10.3.1.1. Use Case

Note: this feature was introduced in Apache HBase 0.92

This allows the user to perform server-side filtering when accessing HBase over Thrift. The user specifies a filter via a string. The string is parsed on the server to construct the filter

10.3.1.2. General Filter String Syntax

A simple filter expression is expressed as: “FilterName (argument, argument, ... , argument)”

You must specify the name of the filter followed by the argument list in parenthesis. Commas separate the individual arguments

If the argument represents a string, it should be enclosed in single quotes.

If it represents a boolean, an integer or a comparison operator like <, >, != etc. it should not be enclosed in quotes

The filter name must be one word. All ASCII characters are allowed except for whitespace, single quotes and parenthesis.

The filter’s arguments can contain any ASCII character. If single quotes are present in the argument, they must be escaped by a preceding single quote

10.3.1.3. Compound Filters and Operators

Currently, two binary operators – AND/OR and two unary operators – WHILE/SKIP are supported.

Note: the operators are all in uppercase

AND – as the name suggests, if this operator is used, the key-value must pass both the filters

OR – as the name suggests, if this operator is used, the key-value must pass at least one of the filters

SKIP – For a particular row, if any of the key-values don’t pass the filter condition, the entire row is skipped

WHILE - For a particular row, it continues to emit key-values until a key-value is reached that fails the filter condition

Compound Filters: Using these operators, a hierarchy of filters can be created. For example: “(Filter1 AND Filter2) OR (Filter3 AND Filter4)”

10.3.1.4. Order of Evaluation

Parenthesis have the highest precedence. The SKIP and WHILE operators are next and have the same precedence.The AND operator has the next highest precedence followed by the OR operator.

For example:

A filter string of the form:“Filter1 AND Filter2 OR Filter3” will be evaluated as:“(Filter1 AND Filter2) OR Filter3”

A filter string of the form:“Filter1 AND SKIP Filter2 OR Filter3” will be evaluated as:“(Filter1 AND (SKIP Filter2)) OR Filter3”

10.3.1.5. Compare Operator

A compare operator can be any of the following:

  1. LESS (<)

  2. LESS_OR_EQUAL (<=)

  3. EQUAL (=)

  4. NOT_EQUAL (!=)

  5. GREATER_OR_EQUAL (>=)

  6. GREATER (>)

  7. NO_OP (no operation)

The client should use the symbols (<, <=, =, !=, >, >=) to express compare operators.

10.3.1.6. Comparator

A comparator can be any of the following:

  1. BinaryComparator - This lexicographically compares against the specified byte array using Bytes.compareTo(byte[], byte[])

  2. BinaryPrefixComparator - This lexicographically compares against a specified byte array. It only compares up to the length of this byte array.

  3. RegexStringComparator - This compares against the specified byte array using the given regular expression. Only EQUAL and NOT_EQUAL comparisons are valid with this comparator

  4. SubStringComparator - This tests if the given substring appears in a specified byte array. The comparison is case insensitive. Only EQUAL and NOT_EQUAL comparisons are valid with this comparator

The general syntax of a comparator is: ComparatorType:ComparatorValue

The ComparatorType for the various comparators is as follows:

  1. BinaryComparator - binary

  2. BinaryPrefixComparator - binaryprefix

  3. RegexStringComparator - regexstring

  4. SubStringComparator - substring

The ComparatorValue can be any value.

Example1: >, 'binary:abc' will match everything that is lexicographically greater than "abc"

Example2: =, 'binaryprefix:abc' will match everything whose first 3 characters are lexicographically equal to "abc"

Example3: !=, 'regexstring:ab*yz' will match everything that doesn't begin with "ab" and ends with "yz"

Example4: =, 'substring:abc123' will match everything that begins with the substring "abc123"

10.3.1.7. Example PHP Client Program that uses the Filter Language

<? $_SERVER['PHP_ROOT'] = realpath(dirname(__FILE__).'/..');
require_once $_SERVER['PHP_ROOT'].'/flib/__flib.php';
flib_init(FLIB_CONTEXT_SCRIPT);
require_module('storage/hbase');
$hbase = new HBase('<server_name_running_thrift_server>', <port on which thrift server is running>);
$hbase->open();
$client = $hbase->getClient();
$result = $client->scannerOpenWithFilterString('table_name', "(PrefixFilter ('row2') AND (QualifierFilter (>=, 'binary:xyz'))) AND (TimestampsFilter ( 123, 456))");
$to_print = $client->scannerGetList($result,1);
while ($to_print) {
print_r($to_print);
$to_print = $client->scannerGetList($result,1);
}
$client->scannerClose($result);
?>

  

10.3.1.8. Example Filter Strings

  • “PrefixFilter (‘Row’) AND PageFilter (1) AND FirstKeyOnlyFilter ()” will return all key-value pairs that match the following conditions:

    1) The row containing the key-value should have prefix “Row”

    2) The key-value must be located in the first row of the table

    3) The key-value pair must be the first key-value in the row

  • “(RowFilter (=, ‘binary:Row 1’) AND TimeStampsFilter (74689, 89734)) OR ColumnRangeFilter (‘abc’, true, ‘xyz’, false))” will return all key-value pairs that match both the following conditions:

    1) The key-value is in a row having row key “Row 1”

    2) The key-value must have a timestamp of either 74689 or 89734.

    Or it must match the following condition:

    1) The key-value pair must be in a column that is lexicographically >= abc and < xyz

  • “SKIP ValueFilter (0)” will skip the entire row if any of the values in the row is not 0

10.3.1.9. Individual Filter Syntax

  1. KeyOnlyFilter

    Description: This filter doesn’t take any arguments. It returns only the key component of each key-value.

    Syntax: KeyOnlyFilter ()

    Example: "KeyOnlyFilter ()"

  2. FirstKeyOnlyFilter

    Description: This filter doesn’t take any arguments. It returns only the first key-value from each row.

    Syntax: FirstKeyOnlyFilter ()

    Example: "FirstKeyOnlyFilter ()"

  3. PrefixFilter

    Description: This filter takes one argument – a prefix of a row key. It returns only those key-values present in a row that starts with the specified row prefix

    Syntax: PrefixFilter (‘<row_prefix>’)

    Example: "PrefixFilter (‘Row’)"

  4. ColumnPrefixFilter

    Description: This filter takes one argument – a column prefix. It returns only those key-values present in a column that starts with the specified column prefix. The column prefix must be of the form: “qualifier”

    Syntax:ColumnPrefixFilter(‘<column_prefix>’)

    Example: "ColumnPrefixFilter(‘Col’)"

  5. MultipleColumnPrefixFilter

    Description: This filter takes a list of column prefixes. It returns key-values that are present in a column that starts with any of the specified column prefixes. Each of the column prefixes must be of the form: “qualifier”

    Syntax:MultipleColumnPrefixFilter(‘<column_prefix>’, ‘<column_prefix>’, …, ‘<column_prefix>’)

    Example: "MultipleColumnPrefixFilter(‘Col1’, ‘Col2’)"

  6. ColumnCountGetFilter

    Description: This filter takes one argument – a limit. It returns the first limit number of columns in the table

    Syntax: ColumnCountGetFilter (‘<limit>’)

    Example: "ColumnCountGetFilter (4)"

  7. PageFilter

    Description: This filter takes one argument – a page size. It returns page size number of rows from the table.

    Syntax: PageFilter (‘<page_size>’)

    Example: "PageFilter (2)"

  8. ColumnPaginationFilter

    Description: This filter takes two arguments – a limit and offset. It returns limit number of columns after offset number of columns. It does this for all the rows

    Syntax: ColumnPaginationFilter(‘<limit>’, ‘<offest>’)

    Example: "ColumnPaginationFilter (3, 5)"

  9. InclusiveStopFilter

    Description: This filter takes one argument – a row key on which to stop scanning. It returns all key-values present in rows up to and including the specified row

    Syntax: InclusiveStopFilter(‘<stop_row_key>’)

    Example: "InclusiveStopFilter ('Row2')"

  10. TimeStampsFilter

    Description: This filter takes a list of timestamps. It returns those key-values whose timestamps matches any of the specified timestamps

    Syntax: TimeStampsFilter (<timestamp>, <timestamp>, ... ,<timestamp>)

    Example: "TimeStampsFilter (5985489, 48895495, 58489845945)"

  11. RowFilter

    Description: This filter takes a compare operator and a comparator. It compares each row key with the comparator using the compare operator and if the comparison returns true, it returns all the key-values in that row

    Syntax: RowFilter (<compareOp>, ‘<row_comparator>’)

    Example: "RowFilter (<=, ‘xyz)"

  12. Family Filter

    Description: This filter takes a compare operator and a comparator. It compares each qualifier name with the comparator using the compare operator and if the comparison returns true, it returns all the key-values in that column

    Syntax: QualifierFilter (<compareOp>, ‘<qualifier_comparator>’)

    Example: "QualifierFilter (=, ‘Column1’)"

  13. QualifierFilter

    Description: This filter takes a compare operator and a comparator. It compares each qualifier name with the comparator using the compare operator and if the comparison returns true, it returns all the key-values in that column

    Syntax: QualifierFilter (<compareOp>,‘<qualifier_comparator>’)

    Example: "QualifierFilter (=,‘Column1’)"

  14. ValueFilter

    Description: This filter takes a compare operator and a comparator. It compares each value with the comparator using the compare operator and if the comparison returns true, it returns that key-value

    Syntax: ValueFilter (<compareOp>,‘<value_comparator>’)

    Example: "ValueFilter (!=, ‘Value’)"

  15. DependentColumnFilter

    Description: This filter takes two arguments – a family and a qualifier. It tries to locate this column in each row and returns all key-values in that row that have the same timestamp. If the row doesn’t contain the specified column – none of the key-values in that row will be returned.

    The filter can also take an optional boolean argument – dropDependentColumn. If set to true, the column we were depending on doesn’t get returned.

    The filter can also take two more additional optional arguments – a compare operator and a value comparator, which are further checks in addition to the family and qualifier. If the dependent column is found, its value should also pass the value check and then only is its timestamp taken into consideration

    Syntax: DependentColumnFilter (‘<family>’, ‘<qualifier>’, <boolean>, <compare operator>, ‘<value comparator’)

    Syntax: DependentColumnFilter (‘<family>’, ‘<qualifier>’, <boolean>)

    Syntax: DependentColumnFilter (‘<family>’, ‘<qualifier>’)

    Example: "DependentColumnFilter (‘conf’, ‘blacklist’, false, >=, ‘zebra’)"

    Example: "DependentColumnFilter (‘conf’, 'blacklist', true)"

    Example: "DependentColumnFilter (‘conf’, 'blacklist')"

  16. SingleColumnValueFilter

    Description: This filter takes a column family, a qualifier, a compare operator and a comparator. If the specified column is not found – all the columns of that row will be emitted. If the column is found and the comparison with the comparator returns true, all the columns of the row will be emitted. If the condition fails, the row will not be emitted.

    This filter also takes two additional optional boolean arguments – filterIfColumnMissing and setLatestVersionOnly

    If the filterIfColumnMissing flag is set to true the columns of the row will not be emitted if the specified column to check is not found in the row. The default value is false.

    If the setLatestVersionOnly flag is set to false, it will test previous versions (timestamps) too. The default value is true.

    These flags are optional and if you must set neither or both

    Syntax: SingleColumnValueFilter(<compare operator>, ‘<comparator>’, ‘<family>’, ‘<qualifier>’,<filterIfColumnMissing_boolean>, <latest_version_boolean>)

    Syntax: SingleColumnValueFilter(<compare operator>, ‘<comparator>’, ‘<family>’, ‘<qualifier>)

    Example: "SingleColumnValueFilter (<=, ‘abc’,‘FamilyA’, ‘Column1’, true, false)"

    Example: "SingleColumnValueFilter (<=, ‘abc’,‘FamilyA’, ‘Column1’)"

  17. SingleColumnValueExcludeFilter

    Description: This filter takes the same arguments and behaves same as SingleColumnValueFilter – however, if the column is found and the condition passes, all the columns of the row will be emitted except for the tested column value.

    Syntax: SingleColumnValueExcludeFilter(<compare operator>, '<comparator>', '<family>', '<qualifier>',<latest_version_boolean>, <filterIfColumnMissing_boolean>)

    Syntax: SingleColumnValueExcludeFilter(<compare operator>, '<comparator>', '<family>', '<qualifier>')

    Example: "SingleColumnValueExcludeFilter (‘<=’, ‘abc’,‘FamilyA’, ‘Column1’, ‘false’, ‘true’)"

    Example: "SingleColumnValueExcludeFilter (‘<=’, ‘abc’, ‘FamilyA’, ‘Column1’)"

  18. ColumnRangeFilter

    Description: This filter is used for selecting only those keys with columns that are between minColumn and maxColumn. It also takes two boolean variables to indicate whether to include the minColumn and maxColumn or not.

    If you don’t want to set the minColumn or the maxColumn – you can pass in an empty argument.

    Syntax: ColumnRangeFilter (‘<minColumn>’, <minColumnInclusive_bool>, ‘<maxColumn>’, <maxColumnInclusive_bool>)

    Example: "ColumnRangeFilter (‘abc’, true, ‘xyz’, false)"

---------------

注:

已知16.SingleColumnValueFilter和17.SingleColumnValueExcludeFilter的语法已经变掉,参数顺序调整为:

Syntax: SingleColumnValueFilter(‘<family>’, ‘<qualifier>', <compare operator>, ‘<comparator>’,)

HBase Thrift过滤语法的更多相关文章

  1. wireshark过滤语法总结

    抓包采用wireshark,提取特征时,要对session进行过滤,找到关键的stream,这里总结了wireshark过滤的基本语法,供自己以后参考.(脑子记不住东西) wireshark进行过滤时 ...

  2. wireshark过滤语法总结-重点偏移过滤

    http://chenjiji.com/post/3371.html 作者: CHAN | 发布: 2013 年 10 月 24 日 做应用识别这一块经常要对应用产生的数据流量进行分析. 抓包采用wi ...

  3. hbase thrift 定义

    /*  * Licensed to the Apache Software Foundation (ASF) under one  * or more contributor license agre ...

  4. 转: wireshark过滤语法总结

    from: http://blog.csdn.net/cumirror/article/details/7054496 wireshark过滤语法总结 原创 2011年12月09日 22:38:50 ...

  5. HBase & thrift & C++编程

    目录 目录 1 1. 前言 1 2. 启动和停止thrift2 1 2.1. 启动thrift2 1 2.2. 停止thrift2 1 2.3. 启动参数 2 3. hbase.thrift 2 3. ...

  6. wireshark过滤语法总结 (转载)

    做应用识别这一块经常要对应用产生的数据流量进行分析. 抓包采用wireshark,提取特征时,要对session进行过滤,找到关键的stream,这里总结了wireshark过滤的基本语法,供自己以后 ...

  7. HBase Filter 过滤器之RowFilter详解

    前言:本文详细介绍了HBase RowFilter过滤器Java&Shell API的使用,并贴出了相关示例代码以供参考.RowFilter 基于行键进行过滤,在工作中涉及到需要通过HBase ...

  8. HBase Filter 过滤器之FamilyFilter详解

    前言:本文详细介绍了 HBase FamilyFilter 过滤器 Java&Shell API 的使用,并贴出了相关示例代码以供参考.FamilyFilter 基于列族进行过滤,在工作中涉及 ...

  9. HBase Filter 过滤器之QualifierFilter详解

    前言:本文详细介绍了 HBase QualifierFilter 过滤器 Java&Shell API 的使用,并贴出了相关示例代码以供参考.QualifierFilter 基于列名进行过滤, ...

随机推荐

  1. 使用Java Low Level REST Client操作elasticsearch

    Java REST客户端有两种风格: Java低级别REST客户端(Java Low Level REST Client,以后都简称低级客户端算了,难得码字):Elasticsearch的官方low- ...

  2. SQL Server 查询某个字段值在哪张表的哪个字段

    我要查找值为‘WSCOL1525’的字段. declare @cloumns varchar(40)declare @tablename varchar(40)declare @str varchar ...

  3. page0902未完成

    /** * @author:(LiberHome) * @date:Created in 2019/3/1 23:14 * @description: * @version:$ */ import j ...

  4. 前台js根据当前时间生成订单号

    *********前台显示框**************** <input type="text" id="WIDout_trade_no" name=& ...

  5. Head First Python-python面向对象

    与大多数其他的编程语言一样,Python容许创建并定义面向对象的类,类可以将代码与代码处理的数据相关联. 对于更加复杂的数据,一般的列表已经不能满足需求了. 我们可以使用字典dict将数据值与键相关联 ...

  6. Exp2 后门原理与实践 20164320 王浩

    一.实践基本内容 1.实践目标 (1)使用netcat获取主机操作Shell,cron启动 (2)使用socat获取主机操作Shell, 任务计划启动 (3)使用MSF meterpreter(或其他 ...

  7. 在vue中使用lang="scss"出现报错解决思路

    最近在学习vue框架,使用lang="scss" 出现如下错误: This dependency was not found: * !!vue-style-loader!css-l ...

  8. Luogu4113 [HEOI2012]采花

    题目大意:给定一个长度为$n$的序列$a_i$,$m$次询问,每次询问$[l,r]$,求在区间内有多少个数出现了至少2次. 数据范围:$1\leq l\leq r\leq n\leq 2*10^6,1 ...

  9. SpringBoot打包不同配置profile

    1.application.properties添加变量 spring.profiles.active=@activatedProperties@ 2.pom中添加变量配置 <profiles& ...

  10. ADB——keyevent命令

    基本格式 adb shell input keyevent xxx # xxx为具体操作对应的数字 keycode 官方 KEYCODE 链接:戳这里 0 KEYCODE_UNKNOWN 未知按键 1 ...