SupportKB

Problem Description:
After upgrading to HDP 3.0, the HiveServer2 fails to start and the following error is displayed in the HiveServer2 logs:

  1. WARN [main]: metastore.RetryingMetaStoreClient (:()) -
  2. MetaStoreClient lost connection. Attempting to reconnect (1 of 24) after 5s. getCurrentNotificationEventId
  3. org.apache.thrift.TApplicationException: Internal error processing get_current_notificationEventId
  4. at org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
  5. ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  6. at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
  7. ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  8. at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_current_notificationEventId
  9. (ThriftHiveMetastore.java:5575) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  10. at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_current_notificationEventId
  11. (ThriftHiveMetastore.java:5563) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  12. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getCurrentNotificationEventId
  13. (HiveMetaStoreClient.java:2723) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  14. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
  15. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112]
  16. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
  17. at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
  18. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
  19. ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  20. at com.sun.proxy.$Proxy40.getCurrentNotificationEventId(Unknown Source) ~[?:?]
  21. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
  22. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112]
  23. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
  24. at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
  25. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke
  26. (HiveMetaStoreClient.java:2773) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  27. at com.sun.proxy.$Proxy40.getCurrentNotificationEventId(Unknown Source) ~[?:?]
  28. at org.apache.hadoop.hive.metastore.messaging.EventUtils$MSClientNotificationFetcher.
  29. getCurrentNotificationEventId(EventUtils.java:73)
  30. ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  31. at org.apache.hadoop.hive.ql.metadata.events.NotificationEventPoll.<init>
  32. (NotificationEventPoll.java:103)
  33. ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  34. at org.apache.hadoop.hive.ql.metadata.events.NotificationEventPoll.initialize
  35. (NotificationEventPoll.java:59)
  36. ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  37. at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:275)
  38. ~[hive-service-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  39. at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1063)
  40. ~[hive-service-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  41. at org.apache.hive.service.server.HiveServer2.access$1600(HiveServer2.java:140)
  42. ~[hive-service-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  43. at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1332)
  44. ~[hive-service-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  45. at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1176)
  46. ~[hive-service-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  47. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
  48. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  49. ~[?:1.8.0_112]
  50. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
  51. at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
  52. at org.apache.hadoop.util.RunJar.run(RunJar.java:318) ~[hadoop-common-3.1.0.3.0.0.0-1634.jar:?]
  53. at org.apache.hadoop.util.RunJar.main(RunJar.java:232) ~[hadoop-common-3.1.0.3.0.0.0-1634.jar:?]

In hive-metastore.log, the following message is displayed at the same timeline:

  1. ERROR [pool-6-thread-200]: metastore.HiveMetaStore (HiveMetaStore.java:get_current_notificationEventId(7617))
  2. - Not authorized to make the get_current_notificationEventId call.
  3. You can try to disable metastore.metastore.event.db.notification.api.auth
  4. org.apache.hadoop.hive.metastore.api.MetaException: User hive is not allowed to perform this API call
  5. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.authorizeProxyPrivilege
  6. (HiveMetaStore.java:7655)
  7. ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  8. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_current_notificationEventId(HiveMetaStore.java:7615)
  9. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  10. at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) ~[?:?]
  11. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  12. ~[?:1.8.0_112]
  13. at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
  14. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
  15. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  16. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
  17. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  18. at com.sun.proxy.$Proxy34.get_current_notificationEventId(Unknown Source) [?:?]
  19. at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_current_notificationEventId.getResult
  20. (ThriftHiveMetastore.java:18364)
  21. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  22. at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_current_notificationEventId.getResult
  23. (ThriftHiveMetastore.java:18349) [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  24. at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
  25. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  26. at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
  27. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  28. at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
  29. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  30. at java.security.AccessController.doPrivileged(Native Method) [?:1.8.0_112]
  31. at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_112]
  32. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1688)
  33. [hadoop-common-3.1.0.3.0.0.0-1634.jar:?]
  34. at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
  35. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  36. at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
  37. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  38. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  39. [?:1.8.0_112]
  40. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  41. [?:1.8.0_112]
  42. at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]

Cause:
The connection to Metastore from HiveServer2 fails after upgrading the cluster when hive.metastore.event.db.notification.api.auth property is set to true.
Solution:
This is a known issue (HIVE-19740).

Workaround In HDFS > Configs > Custom core-site.xml, add the following:

  1. hadoop.proxyuser.hive.hosts=HS2_HOST
  2. hadoop.proxyuser.hive.groups=*

Or

In Hive >  Configs > Custom hive-site.xml, add the following:

  1. hive.metastore.event.db.notification.api.auth=false

Restart the required services.

For more details about this property, see HiveReplicationv2Development

About:
This article created by Hortonworks Support (Article: 000007703) on 2018-08-06 07:32
OS: All
Type: Cluster_Administration
Version: H3.0.0

Support ID: 000007703

Error:"MetaStoreClient lost connection. Attempting to reconnect (1 of 24) after 5s. getCurrentNotificationEventId" occurs as HiveServer2 fails to start as it cannot connect to Metastore in HDP 3.0的更多相关文章

  1. WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)

    [2017-05-19 13:32:14,933] INFO Waiting for keeper state SyncConnected (org.I0Itec.zkclient.ZkClient) ...

  2. WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) java.net.ConnectException: Connection refused

    1.启动kafka的脚本程序报如下所示的错误: [hadoop@slaver1 script_hadoop]$ kafka-start.sh start kafkaServer... [-- ::,] ...

  3. hadoop mapreduce 写入hbase报错 Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect

    现象:map任务构造数据正常,reduce任务,开始也正常,速度很快 ,在hbase 的管理界面,可以看到,5W以上的请求数 当reduce 执行到 70% 左右的时候,就堵住了,查看yarn的web ...

  4. Linux下启动tomcat报错，WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException:

    tomcat启动完了之后,一直不停的打印这种错误信息,看表面上,应该是zk节点下的数据是空的,连接不上服务,所以一直在尝试连接,然后一直又连不上: 完整的错误信息: 407662 [usf-ZooKe ...

  5. R语言保存文件 Error in save error writing to connection

    Error in save(filtered, file = paste(sampleName, "filtered", sep = "_")) :   err ...

  6. urllib2.URLError: <urlopen error [Errno 104] Connection reset by peer>

    http://www.dianping.com/shop/8010173 File "综合商场1.py", line 152, in <module>    httpC ...

  7. STM8学习 无法仿真原因Starting debug session... -> Emulator reset (usb://usb)... ** Connection error (usb://usb): swim error [30200]: st-link connection error

    刚调试程序时,STlink总是连不上,一直提示: Starting debug session...-> Emulator reset (usb://usb)...** Connection e ...

  8. mysqldump导出报错"mysqldump: Error 2013: Lost connection to MySQL server during query when dumping table `file_storage` at row: 29"

    今天mysql备份的crontab自动运行的时候,出现了报警,报警内容如下 mysqldump: Error 2013: Lost connection to MySQL server during ...

  9. Putty出现 Network error:Software caused connection abort

    使用centos7.5 用Putty连接使用没多久就会出现 Network error:Software caused connection abort #修改sshd配置文件.修改3项配置即可 vi ...

随机推荐

  1. iPhone越狱

    手机越狱 常用工具:PP越狱助手.盘古越狱.iTunes 如果手机第一次越狱使用pp越狱助手成功率比较高 iPhone固件下载地址http://iphone.91.com/fw/iphone5/ 具体 ...

  2. 聊聊Spring Cloud版本的那些事儿

    说说Spring Cloud版本的那些事儿. 版本命名 之前提到过,Spring Cloud是一个拥有诸多子项目的大型综合项目,原则上其子项目也都维护着自己的发布版本号.那么每一个Spring Clo ...

  3. 自动化测试用例getText()获取某一个元素的值返回null或空

    问题描述 页面上一个元素无法获取它的值. 用getText() 获取该元素的值返回null或空. 根本原因 ? 需要更多了解dom结构 解决办法 用getAttribute("value&q ...

  4. I/O----复制文本文件

    文件 "我的青春谁做主.txt" 位于 D 盘根目录下,要求将此文件的内容复制到 C:/myPrime.txt 中. package io.day03; import java.i ...

  5. [codeforces 901E] Cyclic Cipher 循环卷积-Bluestein's Algorithm

    题目大意: 传送门 给两个数列${B_i}.{C_i}$,长度均为$n$,且${B_i}$循环移位线性无关,即不存在一组系数${X_i}$使得对于所有的$k$均有$\sum_{i=0}^{n-1} X ...

  6. POJ_3342_Party at Hali-Bula_树形DP

    POJ_3342_Party at Hali-Bula_树形DP 题意:直接上司和本人不能同时参加,求最多参加人数,并回答是否唯一解. 分析:常规树形DP,需要再维护一下选/不选当前点取得最大值时是否 ...

  7. LVS+keepalived负载均衡

    背景:         随着你的网站业务量的增长你网站的服务器压力越来越大?需要负载均衡方案!商业的硬件如F5又太贵,你们又是创业型互联公司如何有效节约成本,节省不必要的浪费?同时实现商业硬件一样的高 ...

  8. java中八大基本数据类型详解

    1.基本数据类型的分类 java中的类型分为基本数据类型和引用类型,今天我们讨论的是java中的八大基本数据类型. 基本数据类型可以分为三类:1.数值类型.2.字符类型.3.布尔类型. 数值类型又分为 ...

  9. OKHttp源码学习同步请求和异步请求（二）

    OKHttp get private void doGet(String method, String s) throws IOException { String url = urlAddress ...

  10. sau交流学习社区--在element-ui中新建FormData对象组合上传图片和文件的文件对象，同时需要携带其他参数

    今天有一个坑,同时要上传图片和文件,而且图片要展示缩略图,文件要展示列表. 我的思路是: 首先,只上传附件照片,这个直接看ele的官方例子就行,不仅仅上传附件照片,还同时上传其他参数. 然后,再做上传 ...