Mac下安装Spark

1.Scala

官网下载scala安装包后解压，路径随意。

编辑/etc/bash_profile添加$SCALA_HOME并修改相应PATH

2.SSH无密码登陆

ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa

ssh-keygen表示生成秘钥；-t表示秘钥类型；-P用于提供密语；-f指定生成的秘钥文件。这个命令在”~/.ssh/“文件

结果显示：

Generating public/private rsa key pair.
Created directory '/var/root/.ssh'.
Your identification has been saved in /var/root/.ssh/id_rsa.
Your public key has been saved in /var/root/.ssh/id_rsa.pub.
The key fingerprint is:
5f:1c:f4:eb:ba:a5:59:73:41:9a:04:ca:a4:6b:0f:32 root@MAC.local
The key's randomart image is:
+--[ RSA 2048]----+
|         . ..    |
|        + ....   |
|       . o  .... |
|        .  ...+. |
|     E +S   oo.. |
|      + o. . .  .|
|         ..   = .|
|             * o |
|            =.   |
+-----------------+

秘钥生成成功。并在~/.ssh创建两个文件id_dsa和id_dsa.pub，是ssh的一对儿私钥和公钥。

接下来，将公钥追加到授权的key中去，输入

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

3.Spark安装

把prebuilt的Spark安装解压后，在/etc/bash_profile中设置好SPARK_HOME及相应PATH

在$SPARK_HOME/conf下配置spark-env.sh，export JAVA_HOME 及HADOOP_HOME，如果是local或者standalone，设置SPARK_MASTER_IP