SupportKB

Problem Description:
After upgrading to HDP 3.0, the HiveServer2 fails to start and the following error is displayed in the HiveServer2 logs:

  1. WARN [main]: metastore.RetryingMetaStoreClient (:()) -
  2. MetaStoreClient lost connection. Attempting to reconnect (1 of 24) after 5s. getCurrentNotificationEventId
  3. org.apache.thrift.TApplicationException: Internal error processing get_current_notificationEventId
  4. at org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
  5. ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  6. at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
  7. ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  8. at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_current_notificationEventId
  9. (ThriftHiveMetastore.java:5575) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  10. at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_current_notificationEventId
  11. (ThriftHiveMetastore.java:5563) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  12. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getCurrentNotificationEventId
  13. (HiveMetaStoreClient.java:2723) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  14. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
  15. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112]
  16. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
  17. at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
  18. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
  19. ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  20. at com.sun.proxy.$Proxy40.getCurrentNotificationEventId(Unknown Source) ~[?:?]
  21. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
  22. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112]
  23. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
  24. at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
  25. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke
  26. (HiveMetaStoreClient.java:2773) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  27. at com.sun.proxy.$Proxy40.getCurrentNotificationEventId(Unknown Source) ~[?:?]
  28. at org.apache.hadoop.hive.metastore.messaging.EventUtils$MSClientNotificationFetcher.
  29. getCurrentNotificationEventId(EventUtils.java:73)
  30. ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  31. at org.apache.hadoop.hive.ql.metadata.events.NotificationEventPoll.<init>
  32. (NotificationEventPoll.java:103)
  33. ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  34. at org.apache.hadoop.hive.ql.metadata.events.NotificationEventPoll.initialize
  35. (NotificationEventPoll.java:59)
  36. ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  37. at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:275)
  38. ~[hive-service-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  39. at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1063)
  40. ~[hive-service-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  41. at org.apache.hive.service.server.HiveServer2.access$1600(HiveServer2.java:140)
  42. ~[hive-service-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  43. at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1332)
  44. ~[hive-service-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  45. at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1176)
  46. ~[hive-service-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  47. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
  48. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  49. ~[?:1.8.0_112]
  50. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
  51. at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
  52. at org.apache.hadoop.util.RunJar.run(RunJar.java:318) ~[hadoop-common-3.1.0.3.0.0.0-1634.jar:?]
  53. at org.apache.hadoop.util.RunJar.main(RunJar.java:232) ~[hadoop-common-3.1.0.3.0.0.0-1634.jar:?]

In hive-metastore.log, the following message is displayed at the same timeline:

  1. ERROR [pool-6-thread-200]: metastore.HiveMetaStore (HiveMetaStore.java:get_current_notificationEventId(7617))
  2. - Not authorized to make the get_current_notificationEventId call.
  3. You can try to disable metastore.metastore.event.db.notification.api.auth
  4. org.apache.hadoop.hive.metastore.api.MetaException: User hive is not allowed to perform this API call
  5. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.authorizeProxyPrivilege
  6. (HiveMetaStore.java:7655)
  7. ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  8. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_current_notificationEventId(HiveMetaStore.java:7615)
  9. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  10. at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) ~[?:?]
  11. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  12. ~[?:1.8.0_112]
  13. at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
  14. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
  15. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  16. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
  17. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  18. at com.sun.proxy.$Proxy34.get_current_notificationEventId(Unknown Source) [?:?]
  19. at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_current_notificationEventId.getResult
  20. (ThriftHiveMetastore.java:18364)
  21. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  22. at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_current_notificationEventId.getResult
  23. (ThriftHiveMetastore.java:18349) [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  24. at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
  25. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  26. at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
  27. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  28. at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
  29. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  30. at java.security.AccessController.doPrivileged(Native Method) [?:1.8.0_112]
  31. at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_112]
  32. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1688)
  33. [hadoop-common-3.1.0.3.0.0.0-1634.jar:?]
  34. at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
  35. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  36. at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
  37. [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
  38. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  39. [?:1.8.0_112]
  40. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  41. [?:1.8.0_112]
  42. at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]

Cause:
The connection to Metastore from HiveServer2 fails after upgrading the cluster when hive.metastore.event.db.notification.api.auth property is set to true.
Solution:
This is a known issue (HIVE-19740).

Workaround In HDFS > Configs > Custom core-site.xml, add the following:

  1. hadoop.proxyuser.hive.hosts=HS2_HOST
  2. hadoop.proxyuser.hive.groups=*

Or

In Hive >  Configs > Custom hive-site.xml, add the following:

  1. hive.metastore.event.db.notification.api.auth=false

Restart the required services.

For more details about this property, see HiveReplicationv2Development

About:
This article created by Hortonworks Support (Article: 000007703) on 2018-08-06 07:32
OS: All
Type: Cluster_Administration
Version: H3.0.0

Support ID: 000007703

Error:"MetaStoreClient lost connection. Attempting to reconnect (1 of 24) after 5s. getCurrentNotificationEventId" occurs as HiveServer2 fails to start as it cannot connect to Metastore in HDP 3.0的更多相关文章

  1. WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)

    [2017-05-19 13:32:14,933] INFO Waiting for keeper state SyncConnected (org.I0Itec.zkclient.ZkClient) ...

  2. WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) java.net.ConnectException: Connection refused

    1.启动kafka的脚本程序报如下所示的错误: [hadoop@slaver1 script_hadoop]$ kafka-start.sh start kafkaServer... [-- ::,] ...

  3. hadoop mapreduce 写入hbase报错 Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect

    现象:map任务构造数据正常,reduce任务,开始也正常,速度很快 ,在hbase 的管理界面,可以看到,5W以上的请求数 当reduce 执行到 70% 左右的时候,就堵住了,查看yarn的web ...

  4. Linux下启动tomcat报错，WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException:

    tomcat启动完了之后,一直不停的打印这种错误信息,看表面上,应该是zk节点下的数据是空的,连接不上服务,所以一直在尝试连接,然后一直又连不上: 完整的错误信息: 407662 [usf-ZooKe ...

  5. R语言保存文件 Error in save error writing to connection

    Error in save(filtered, file = paste(sampleName, "filtered", sep = "_")) :   err ...

  6. urllib2.URLError: <urlopen error [Errno 104] Connection reset by peer>

    http://www.dianping.com/shop/8010173 File "综合商场1.py", line 152, in <module>    httpC ...

  7. STM8学习 无法仿真原因Starting debug session... -> Emulator reset (usb://usb)... ** Connection error (usb://usb): swim error [30200]: st-link connection error

    刚调试程序时,STlink总是连不上,一直提示: Starting debug session...-> Emulator reset (usb://usb)...** Connection e ...

  8. mysqldump导出报错"mysqldump: Error 2013: Lost connection to MySQL server during query when dumping table `file_storage` at row: 29"

    今天mysql备份的crontab自动运行的时候,出现了报警,报警内容如下 mysqldump: Error 2013: Lost connection to MySQL server during ...

  9. Putty出现 Network error:Software caused connection abort

    使用centos7.5 用Putty连接使用没多久就会出现 Network error:Software caused connection abort #修改sshd配置文件.修改3项配置即可 vi ...

随机推荐

  1. js判断浏览器是否支持flash的方法

    传统浏览器可以使用window.ActiveXObject检查浏览器是否启用相关的控件.检查浏览器是否启用flash控件,需要先检查浏览器是否支持ActiveXObject,可以使用typeof检查w ...

  2. this.state.menuList.toArray()[0].get('id'）

    用toArray()处理传过来的list用get(")获取里面的值 而我用的是attributes得方法

  3. W3C------JS

    ✄--------------------------------------------分割线--------------------------------------------✄ W3C:ht ...

  4. MD5加密算法（java及js）

    为了防止用户登陆过程中信息被拦截导致信息泄露,我们应该在客户端就对用户密码进行加密.浏览器提交给服务器的是加密后的信息,即使被恶意拦截,被拦截信息也已做了加密处理,现在比较安全的一种加密算法是MD5加 ...

  5. fixed元素随滚动条无抖动滚动

    页面上用fixed定位一个元素,随滚动条滚动位置不变,最开始我只用了css给元素身上写上fixed属性,发现滚动时元素会发生抖动,随后我就在网上找到解决办法,封装了个方法,如下: Css部分 此部分是 ...

  6. Flask开发微电影网站(三)

    页面完成后的最终布局 可以看到,页面共同的部分是顶部导航和底部导航 所以我们可以把页面顶部导航和底部导航部分单独定义一个文件home.html,然后让需要使用顶部导航和底部导航的页面都继承home.h ...

  7. Python常用算法（二）

    1.快速排序 过一趟排序将要排序的数据分割成独立的两部分,其中一部分的所有数据都比另外一部分的所有数据都要小 一般选取第一个数作为关键数据k,我们要把比k小的所有数据移到它的左面,从后往前找第一个比它 ...

  8. 【递推】Bzoj3612[Heoi2014]平衡

    Description 下课了,露露.花花和萱萱在课桌上用正三棱柱教具和尺子摆起了一个“跷跷板”.      这个“跷跷板”的结构是这样的:底部是一个侧面平行于地平面的正三棱柱教具, 上面 摆着一个尺 ...

  9. Hadoop权限管理

    1.Hadoop权限管理包括以下几个模块: (1) 用户分组管理.用于按组为单位组织管理,某个用户只能向固定分组中提交作业,只能使用固定分组中配置的资源:同时可以限制每个用户提交的作业数,使用的资源量 ...

  10. 【爆料】-《悉尼科技大学毕业证书》UTS一模一样原件

    ☞悉尼科技大学毕业证书[微/Q:2544033233◆WeChat:CC6669834]UC毕业证书/联系人Alice[查看点击百度快照查看][留信网学历认证&博士&硕士&海归 ...