Documentation for Built-In User-Defined Functions Related To XPath

UDFs

xpath, xpath_short, xpath_int, xpath_long, xpath_float, xpath_double, xpath_number, xpath_string

  • Functions for parsing XML data using XPath expressions.
  • Since version: 0.6.0

    Overview

The xpath family of UDFs are wrappers around the Java XPath library javax.xml.xpath provided by the JDK. The library is based on the XPath 1.0 specification. Please refer to http://java.sun.com/javase/6/docs/api/javax/xml/xpath/package-summary.html for detailed information on the Java XPath library.

All functions follow the form: xpath_*(xml_string, xpath_expression_string). The XPath expression string is compiled and cached. It is reused if the expression in the next input row matches the previous. Otherwise, it is recompiled. So, the xml string is always parsed for every input row, but the xpath expression is precompiled and reused for the vast majority of use cases.

Backward axes are supported. For example:

> select xpath ('<a><b id="1"><c/></b><b id="2"><c/></b></a>','/descendant::c/ancestor::b/@id') from t1 limit 1 ;
[1","2]

Each function returns a specific Hive type given the XPath expression:

  • xpath returns a Hive array of strings.
  • xpath_string returns a string.
  • xpath_boolean returns a boolean.
  • xpath_short returns a short integer.
  • xpath_int returns an integer.
  • xpath_long returns a long integer.
  • xpath_float returns a floating point number.
  • xpath_double,xpath_number returns a double-precision floating point number (xpath_number is an alias for xpath_double).

The UDFs are schema agnostic - no XML validation is performed. However, malformed xml (e.g., <a><b>1</b></aa>) will result in a runtime exception being thrown.

Following are specifics on each xpath UDF variant.

xpath

The xpath() function always returns a hive array of strings. If the expression results in a non-text value (e.g., another xml node) the function will return an empty array. There are 2 primary uses for this function: to get a list of node text values or to get a list of attribute values.

Examples:

Non-matching XPath expression:

> select xpath('<a><b>b1</b><b>b2</b></a>','a/*') from src limit 1 ;
[]

Get a list of node text values:

> select xpath('<a><b>b1</b><b>b2</b></a>','a/*/text()') from src limit 1 ;
[b1","b2]

Get a list of values for attribute 'id':

> select xpath('<a><b id="foo">b1</b><b id="bar">b2</b></a>','//@id') from src limit 1 ;
[foo","bar]

Get a list of node texts for nodes where the 'class' attribute equals 'bb':

> SELECT xpath ('<a><b class="bb">b1</b><b>b2</b><b>b3</b><c class="bb">c1</c><c>c2</c></a>''a/*[@class="bb"]/text()') FROM src LIMIT 1 ;
[b1","c1]

xpath_string

The xpath_string() function returns the text of the first matching node.

Get the text for node 'a/b':

> SELECT xpath_string ('<a><b>bb</b><c>cc</c></a>''a/b') FROM src LIMIT 1 ;
bb

Get the text for node 'a'. Because 'a' has children nodes with text, the result is a composite of text from the children.

> SELECT xpath_string ('<a><b>bb</b><c>cc</c></a>''a') FROM src LIMIT 1 ;
bbcc

Non-matching expression returns an empty string:

> SELECT xpath_string ('<a><b>bb</b><c>cc</c></a>''a/d') FROM src LIMIT 1 ;

Gets the text of the first node that matches '//b':

> SELECT xpath_string ('<a><b>b1</b><b>b2</b></a>''//b') FROM src LIMIT 1 ;
b1

Gets the second matching node:

> SELECT xpath_string ('<a><b>b1</b><b>b2</b></a>''a/b[2]') FROM src LIMIT 1 ;
b2

Gets the text from the first node that has an attribute 'id' with value 'b_2':

> SELECT xpath_string ('<a><b>b1</b><b id="b_2">b2</b></a>''a/b[@id="b_2"]') FROM src LIMIT 1 ;
b2

xpath_boolean

Returns true if the XPath expression evaluates to true, or if a matching node is found.

Match found:

> SELECT xpath_boolean ('<a><b>b</b></a>''a/b') FROM src LIMIT 1 ;
true

No match found:

> SELECT xpath_boolean ('<a><b>b</b></a>''a/c') FROM src LIMIT 1 ;
false

Match found:

> SELECT xpath_boolean ('<a><b>b</b></a>''a/b = "b"') FROM src LIMIT 1 ;
true

No match found:

> SELECT xpath_boolean ('<a><b>10</b></a>''a/b < 10') FROM src LIMIT 1 ;
false

xpath_short, xpath_int, xpath_long

These functions return an integer numeric value, or the value zero if no match is found, or a match is found but the value is non-numeric.
Mathematical operations are supported. In cases where the value overflows the return type, then the maximum value for the type is returned.

No match:

> SELECT xpath_int ('<a>b</a>''a = 10') FROM src LIMIT 1 ;
0

Non-numeric match:

> SELECT xpath_int ('<a>this is not a number</a>''a') FROM src LIMIT 1 ;
0
> SELECT xpath_int ('<a>this 2 is not a number</a>''a') FROM src LIMIT 1 ;
0

Adding values:

> SELECT xpath_int ('<a><b class="odd">1</b><b class="even">2</b><b class="odd">4</b><c>8</c></a>''sum(a/*)') FROM src LIMIT 1 ;
15
> SELECT xpath_int ('<a><b class="odd">1</b><b class="even">2</b><b class="odd">4</b><c>8</c></a>''sum(a/b)') FROM src LIMIT 1 ;
7
> SELECT xpath_int ('<a><b class="odd">1</b><b class="even">2</b><b class="odd">4</b><c>8</c></a>''sum(a/b[@class="odd"])') FROM src LIMIT 1 ;
5

Overflow:

> SELECT xpath_int ('<a><b>2000000000</b><c>40000000000</c></a>''a/b * a/c') FROM src LIMIT 1 ;
2147483647

xpath_float, xpath_double, xpath_number

Similar to xpath_short, xpath_int and xpath_long but with floating point semantics. Non-matches result in zero. However,
non-numeric matches result in NaN. Note that xpath_number() is an alias for xpath_double().

No match:

> SELECT xpath_double ('<a>b</a>''a = 10') FROM src LIMIT 1 ;
0.0

Non-numeric match:

> SELECT xpath_double ('<a>this is not a number</a>''a') FROM src LIMIT 1 ;
NaN

A very large number:

SELECT xpath_double ('<a><b>2000000000</b><c>40000000000</c></a>''a/b * a/c') FROM src LIMIT 1 ;
8.0E19

[HIve - LanguageManual] XPathUDF的更多相关文章

  1. [HIve - LanguageManual] Hive Operators and User-Defined Functions (UDFs)

    Hive Operators and User-Defined Functions (UDFs) Hive Operators and User-Defined Functions (UDFs) Bu ...

  2. [Hive - LanguageManual ] Windowing and Analytics Functions (待)

    LanguageManual WindowingAndAnalytics     Skip to end of metadata   Added by Lefty Leverenz, last edi ...

  3. [Hive - LanguageManual] Import/Export

    LanguageManual ImportExport     Skip to end of metadata   Added by Carl Steinbach, last edited by Le ...

  4. [Hive - LanguageManual] DML: Load, Insert, Update, Delete

    LanguageManual DML Hive Data Manipulation Language Hive Data Manipulation Language Loading files int ...

  5. [Hive - LanguageManual] Alter Table/Partition/Column

    Alter Table/Partition/Column Alter Table Rename Table Alter Table Properties Alter Table Comment Add ...

  6. Hive LanguageManual DDL

    hive语法规则LanguageManual DDL SQL DML 和 DDL 数据操作语言 (DML) 和 数据定义语言 (DDL) 一.数据库 增删改都在文档里说得也很明白,不重复造车轮 二.表 ...

  7. [Hive - LanguageManual ] ]SQL Standard Based Hive Authorization

    Status of Hive Authorization before Hive 0.13 SQL Standards Based Hive Authorization (New in Hive 0. ...

  8. [Hive - LanguageManual] Hive Concurrency Model (待)

    Hive Concurrency Model Hive Concurrency Model Use Cases Turn Off Concurrency Debugging Configuration ...

  9. [Hive - LanguageManual ] Explain (待)

    EXPLAIN Syntax EXPLAIN Syntax Hive provides an EXPLAIN command that shows the execution plan for a q ...

随机推荐

  1. Centos硬件信息查看命令

    [root@yan-001 ~] # uname -a # 查看内核/操作系统/CPU信息的linux系统信息命令 [root@yan-001 ~] # head -n 1 /etc/issue # ...

  2. LingPipe-TextClassification(文本分类)

    What is Text Classification? Text classification typically involves assigning a document to a catego ...

  3. C#实现GDI+基本图的缩放、拖拽、移动

    C#实现GDI+基本图的缩放.拖拽.移动示例代码如下: using System;using System.Collections.Generic;using System.ComponentMode ...

  4. 如何使java中double类型不以科学计数法表示

    在java中,把一个double或者BigDecimal的小数转换为字符串时,经常会用科学计数法表示,而我们一般不想使用科学计数法,可以通过:DecimalFormat a = new Decimal ...

  5. Codeforces 379D - New Year Letter

    原题地址:http://codeforces.com/contest/379/problem/D 题目大意:给出一种生成字符串的方法 s[k] = s[k - 2] + s[k - 1],其中加法意为 ...

  6. uva12034Race

    递推,组合. 考虑第一名有i个人,则f[n]=sum(C(n,i)*f[n-i]),递推即可.. #include<cstdio> #include<algorithm> #i ...

  7. Sass结合Modernizr的使用方法

    Modernizr在初始化的时候会首先找寻class=“no-js”的元素: <!DOCTYPE html> <html class="no-js"> &l ...

  8. css配合js模拟的select下拉框

    css配合js模拟的select下拉框 <!doctype html> <html> <head> <meta charset="utf-8&quo ...

  9. (转)c & c++内存分配

    一.预备知识—程序的内存分配 一个由c/C++编译的程序占用的内存分为以下几个部分1.栈区(stack)— 由编译器自动分配释放 ,存放函数的参数值,局部变量的值等.其操作方式类似于 数据结构中的栈. ...

  10. SharePoint 2013让页面显示错误

    转:http://blog.csdn.net/zmoneyz/article/details/20460263 1. 在网站端口下,如80端口下的Web.config修改 (1)将<custom ...