本地运行spark报错

18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 ERROR SparkContext: Error initializing SparkContext.
java.net.BindException: Cannot assign requested address: bind: Service 'sparkDriver' failed after 16 retries (starting from 0)! Consider explicitly setting the appropriate port for the service 'sparkDriver' (for example spark.ui.port for SparkUI) to an available port or increasing spark.port.maxRetries.
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:127)
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:501)
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1218)
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:506)
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:491)
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:965)
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:210)
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:353)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:408)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:455)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
at java.lang.Thread.run(Thread.java:745)
18/12/18 12:56:55 INFO SparkContext: Successfully stopped SparkContext

看起来一直在尝试绑定端口0失败,默认起始端口应该是4040,设置spark.driver.port=4040还是报相同的错,后来发现增加环境变量即可

SPARK_LOCAL_IP=127.0.0.1

【原创】大叔经验分享(13)spark运行报错WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.的更多相关文章

  1. spark报错:warn util.utils::service 'sparkUI' can not bind on part 4040.Attempting port 4041.

    转载自:https://blog.csdn.net/weixin_41629917/article/details/83190258

  2. spark 运行报错:java.lang.AbstractMethodError

    报错日志如下: Caused by: java.lang.AbstractMethodError: sparkCore.JavaWordCount$2.call(Ljava/lang/Object;) ...

  3. Appium运行报错:No Chromedriver found that can automate Chrome '39.0.0'

    运行appium切换webview时候遇到报错:’No Chromedriver found that can automate Chrome 'xx.xx.xx' 此报错是因为Appium在运行过程 ...

  4. 【samtools】运行报错: error while loading shared libraries:libcrypto.so.1.0.0或libncurses.so.5或libtinfow.so.5

    samtools用conda安装后,总是出现共享库缺失的报错.即便你刚安装samtools时可以用,但后面在同一环境中安装其他相关软件,有可能产生了冲突,导致库替换,因而报错. 避免这种情况,可能最好 ...

  5. 【原创】经验分享:一个小小emoji尽然牵扯出来这么多东西?

    前言 之前也分享过很多工作中踩坑的经验: 一个线上问题的思考:Eureka注册中心集群如何实现客户端请求负载及故障转移? [原创]经验分享:一个Content-Length引发的血案(almost.. ...

  6. IDEA导出jar包后运行报错 找不到或无法加载主类

    开发工具:IDEA16 运行环境:ubuntu 问题:根据网上的Idea导出jar包的方法,将我的项目导出jar包后运行报错:找不到或无法加载主类.   为了找到这个原因,我重新搭建了一个测试例子,在 ...

  7. iOS微信支付demo运行报错解决如下

    要接入微信支付的小伙伴,首先要下载一份官方demo(APP微信支付官方Demo下载),然后打开工程,准备大干一场. 1.编译报错 编译的时候居然直接报错了(orz) 错误提示: APP微信支付官方De ...

  8. macaca运行报错之chrome-driver问题处理,关闭 Chrome 的自动更新

    由于chrome浏览器自动更新,导致 macaca运行报错,重新安装和更新chrome-driver 之后,还需要把chrome浏览器降级到50版本: 但是chrome会自动更新,所以需要禁止.找到这 ...

  9. Selenium Grid 运行报错 Exception thrown in Navigator.Start first time ->Error forwarding the new session Empty pool of VM for setup Capabilities

    Selenium Grid 运行报错 : Exception thrown in Navigator.Start first time ->Error forwarding the new se ...

随机推荐

  1. 从明面上学习ASP.NET Core

    一.前言     这篇文章就是从能看到地方去学习Core,没有很深奥,也没有很难懂,现在我们开始吧. 二.构建项目,引发思考     创建项目的步骤真的很简单,你要是不会,我真也没法了,我这是创建的M ...

  2. Jquery mobile动态生成ListView调用刷新方法报错

    错误:cannot call methods on listview prior to initialization... 示例: <div data-role="page" ...

  3. Linux使用百度云

    导读 百度云没有Linux客户端,于是有大神用Go语言写出来一个叫BaiduPCS-Go的命令行盘客户端,可以通过终端操作百度云盘,在Linux上实现上传下载.但是因为是命令行版本的,对没有命令行使用 ...

  4. Python exe2shellcode,shellcode2exe

    exe2shellcode #! /usr/bin/env python # -*- coding: utf-8 -*- import os import sys def payload(files) ...

  5. nodejs开发辅助工具nodemon

    前面的话 修改代码后,需要重新启动 Express 应用,所做的修改才能生效.若之后的每次代码修改都要重复这样的操作,势必会影响开发效率,本文将详细介绍Nodemon,它会监测项目中的所有文件,一旦发 ...

  6. kubernetes 将pod运行在某些特定的节点上,给节点打标签

    给节点打上标签: kubectl label node  <node_name> GPU=true   #打上标签 GPU=true 在创建pod的yaml文件时:  添加 nodeSel ...

  7. 5.15 pymysql 模块

    pymysql 模块 安装 pip3 install pymysql 链接,执行sql,关闭(游标) import pymysql user= input('用户名:>>').strip( ...

  8. LOJ #2731. 「JOISC 2016 Day 1」棋盘游戏(dp)

    题意 JOI 君有一个棋盘,棋盘上有 \(N\) 行 \(3\) 列 的格子.JOI 君有若干棋子,并想用它们来玩一个游戏.初始状态棋盘上至少有一个棋子,也至少有一个空位. 游戏的目标是:在还没有放棋 ...

  9. 使用Vlc.DotNet打开摄像头并截图 C#

      参考上一篇  使用vlc打开usb摄像头 理论上输入下面地址 "dshow:// :dshow-size=1600*1200:dshow-vdev=USB CAM2"C#就能打 ...

  10. 集成Tomcat环境到Eclipse中

    集成Tomcat环境到Eclipse中 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 一.安装Eclipse环境 1>.安装JDK环境 官方地址:https://www.or ...