前言 列式文件,顾名思义就是按列存储到文件,和行式存储文件对应.保证了一列在一个文件中是连续的.下面从parquet常见术语,核心schema和文件结构来深入理解.最后通过java api完成write和read. 术语 block parquet层面和row group是一个意思 row group 逻辑概念,用于对row进行分区.由数据集中每个column的column chunk组成.是读写过程中的缓存单元,一般在hdfs上推荐一个block为1GB,一个HDFS文件1个bolock co

使用-z或-n对一个变量判空时,需要注意若直接使用[ -n ${ARG} ]这种形式,若${ARG}中有空格将会报错, #!/bin/bash ARG="sd dd" if [ -n ${ARG} ]; then echo 'ARG:' ${ARG} else echo 'ARG is empty.' fi 输出: line : [: sd: binary operator expected ARG is empty. 解决方法,使用[[ -n ${ARG} ]]或[ -n "

Given a binary tree, find all leaves and then remove those leaves. Then repeat the previous steps until the tree is empty. Example: Given binary tree 1 / \ 2 3 / \ 4 5 Returns [4, 5, 3], [2], [1]. Explanation: 1. Remove the leaves [4, 5, 3] from the

Given a binary tree, return all root-to-leaf paths. For example, given the following binary tree: 1 / \ 2 3 \ 5 All root-to-leaf paths are: ["1->2->5", "1->3"] 这道题给我们一个二叉树,让我们返回所有根到叶节点的路径,跟之前那道Path Sum II 二叉树路径之和之二很类似,比那道稍微简单一

Given a binary tree, find its minimum depth. The minimum depth is the number of nodes along the shortest path from the root node down to the nearest leaf node. 二叉树的经典问题之最小深度问题就是就最短路径的节点个数,还是用深度优先搜索DFS来完成,万能的递归啊...请看代码: /** * Definition for binary tre

Given two binary strings, return their sum (also a binary string). For example,a = "11"b = "1"Return "100". 二进制数想加,并且保存在string中,要注意的是如何将string和int之间互相转换,并且每位相加时,会有进位的可能,会影响之后相加的结果.而且两个输入string的长度也可能会不同.这时我们需要新建一个string,它的长度是两

1064. Complete Binary Search Tree (30) 时间限制 100 ms 内存限制 32000 kB 代码长度限制 16000 B 判题程序 Standard 作者 CHEN, Yue A Binary Search Tree (BST) is recursively defined as a binary tree which has the following properties: The left subtree of a node contains only

Given a non-empty binary search tree and a target value, find the value in the BST that is closest to the target. Note: Given target value is a floating point. You are guaranteed to have only one unique value in the BST that is closest to the target.

在Spark1.2.0版本中是用parquet存储类型时注意事项: sql语句: select * from order_created_dynamic_partition_parquet; 在spark-sql中执行结果: [B@4621484a [B@3311163e 2014-05 [B@70ab973a [B@11559aa0 2014-05 [B@b1a8744 [B@7aa6870d 2014-05 [B@765e2d02 [B@20dd1b04 2014-05 [B@1418b47

Problem D. GukiZ and Binary Operations Solution 一位一位考虑,就是求一个二进制序列有连续的1的种类数和没有连续的1的种类数. 没有连续的1的二进制序列的数目满足f[i]=f[i-1]+f[i-2],恰好是斐波那契数列. 数据范围在10^18,用矩阵加速计算,有连续的1的数目就用2^n-f[n+1] 最后枚举k的每一位,是1乘上2^n-f[n+1],是0乘上f[n+1] 注意以上需要满足 2^l>k.并且这里l的最大值为64,需要特判. #inclu

1043. Is It a Binary Search Tree (25) 时间限制 400 ms 内存限制 65536 kB 代码长度限制 16000 B 判题程序 Standard 作者 CHEN, Yue A Binary Search Tree (BST) is recursively defined as a binary tree which has the following properties: The left subtree of a node contains only

1110. Complete Binary Tree (25) 时间限制 400 ms 内存限制 65536 kB 代码长度限制 16000 B 判题程序 Standard 作者 CHEN, Yue Given a tree, you are supposed to tell if it is a complete binary tree. Input Specification: Each input file contains one test case. For each case, th

1099. Build A Binary Search Tree (30) 时间限制 100 ms 内存限制 65536 kB 代码长度限制 16000 B 判题程序 Standard 作者 CHEN, Yue A Binary Search Tree (BST) is recursively defined as a binary tree which has the following properties: The left subtree of a node contains only

1064. Complete Binary Search Tree (30) 时间限制 100 ms 内存限制 65536 kB 代码长度限制 16000 B 判题程序 Standard 作者 CHEN, Yue A Binary Search Tree (BST) is recursively defined as a binary tree which has the following properties: The left subtree of a node contains only

1102. Invert a Binary Tree (25) 时间限制 400 ms 内存限制 65536 kB 代码长度限制 16000 B 判题程序 Standard 作者 CHEN, Yue The following is from Max Howell @twitter: Google: 90% of our engineers use the software you wrote (Homebrew), but you can't invert a binary tree on a

Hive 导入 parquet 数据步骤如下: 查看 parquet 文件的格式 构造建表语句 倒入数据 一.查看 parquet 内容和结构 下载地址 社区工具 GitHub 地址 命令 查看结构: java -jar parquet-tools-1.6.0rc3-SNAPSHOT.jar schema -d activity.201711171437.0.parquet |head -n 30 查看内容: java -jar parquet-tools-1.6.0rc3-SNAPSHOT.j

spark 2.1.1 spark里执行sql报错 insert overwrite table test_parquet_table select * from dummy 报错如下: org.apache.spark.SparkException: Task failed while writing rows. at org.apache.spark.sql.hive.SparkHiveDynamicPartitionWriterContainer.writeToFile(hiveWrite

We are given the head node root of a binary tree, where additionally every node's value is either a 0 or a 1. Return the same tree where every subtree (of the given tree) not containing a 1 has been removed. (Recall that the subtree of a node X is X,

Given the root node of a binary search tree (BST) and a value to be inserted into the tree, insert the value into the BST. Return the root node of the BST after the insertion. It is guaranteed that the new value does not exist in the original BST. No

Given the root node of a binary search tree (BST) and a value. You need to find the node in the BST that the node's value equals the given value. Return the subtree rooted with that node. If such node doesn't exist, you should return NULL. For exampl