第2节 azkaban调度：17、azkaban的两个服务模式的安装

2.3.3、azkaban两个服务模式安装

1、确认所需软件：

Azkaban Web服务安装包

azkaban-web-server-0.1.0-SNAPSHOT.tar.gz

Azkaban执行服务安装包

azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz

编译之后的sql脚本

create-all-sql-0.1.0-SNAPSHOT.sql

C程序文件脚本

execute-as-user.c程序

2、数据库准备

进入mysql的客户端执行以下命令

mysql  -uroot -p

执行以下命令：

CREATE DATABASE azkaban;

CREATE USER 'azkaban'@'%' IDENTIFIED BY 'azkaban';

GRANT all privileges ON azkaban.* to 'azkaban'@'%' identified by 'azkaban' WITH GRANT OPTION;

flush privileges;

use azkaban;

source /export/softwares/create-all-sql-0.1.0-SNAPSHOT.sql;

3、解压软件安装包

解压azkaban-web-server

cd /export/softwares

tar -zxvf azkaban-web-server-0.1.0-SNAPSHOT.tar.gz -C ../servers/

cd /export/servers

mv azkaban-web-server-0.1.0-SNAPSHOT/ azkaban-web-server-3.51.0

解压azkaban-exec-server

cd /export/softwares

tar -zxvf azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz -C ../servers/

cd /export/servers

mv azkaban-exec-server-0.1.0-SNAPSHOT/ azkaban-exec-server-3.51.0

4、安装SSL安全认证

安装ssl安全认证，允许我们使用https的方式访问我们的azkaban的web服务

密码一定要一个个的字母输入，或者粘贴也行

cd /export/servers/azkaban-web-server-3.51.0

keytool -keystore keystore -alias jetty -genkey -keyalg RSA

5、azkaban web server安装

修改azkaban-web-server的配置文件

cd /export/servers/azkaban-web-server-3.51.0/conf

vim azkaban.properties

# Azkaban Personalization Settings

azkaban.name=Azkaban

azkaban.label=My Azkaban

azkaban.color=#FF3601

azkaban.default.servlet.path=/index

web.resource.dir=web/

default.timezone.id=Asia/Shanghai

# Azkaban UserManager class

user.manager.class=azkaban.user.XmlUserManager

user.manager.xml.file=conf/azkaban-users.xml

# Loader for projects

executor.global.properties=conf/global.properties

azkaban.project.dir=projects

# Velocity dev mode

velocity.dev.mode=false

# Azkaban Jetty server properties.

jetty.use.ssl=true

jetty.maxThreads=25

jetty.port=8081

jetty.ssl.port=8443

jetty.keystore=/export/servers/azkaban-web-server-3.51.0/keystore

jetty.password=azkaban

jetty.keypassword=azkaban

jetty.truststore=/export/servers/azkaban-web-server-3.51.0/keystore

jetty.trustpassword=azkaban

# Azkaban Executor settings

# mail settings

mail.sender=

mail.host=

# User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users.

# enduser -> myazkabanhost:443 -> proxy -> localhost:8081

# when this parameters set then these parameters are used to generate email links.

# if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.

# azkaban.webserver.external_hostname=myazkabanhost.com

# azkaban.webserver.external_ssl_port=443

# azkaban.webserver.external_port=8081

job.failure.email=

job.success.email=

lockdown.create.projects=false

cache.directory=cache

# JMX stats

jetty.connector.stats=true

executor.connector.stats=true

# Azkaban mysql settings by default. Users should configure their own username and password.

database.type=mysql

mysql.port=3306

mysql.host=node03

mysql.database=azkaban

mysql.user=azkaban

mysql.password=azkaban

mysql.numconnections=100

#Multiple Executor

azkaban.use.multiple.executors=true

#azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus

azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1

azkaban.executorselector.comparator.Memory=1

azkaban.executorselector.comparator.LastDispatched=1

azkaban.executorselector.comparator.CpuUsage=1

azkaban.activeexecutor.refresh.milisecinterval=10000

azkaban.queueprocessing.enabled=true

azkaban.activeexecutor.refresh.flowinterval=10

azkaban.executorinfo.refresh.maxThreads=10

6、azkaban  executor server 安装

第一步：修改azkaban-exex-server配置文件

修改azkaban-exec-server的配置文件

cd /export/servers/azkaban-exec-server-3.51.0/conf

vim azkaban.properties

# Azkaban Personalization Settings

azkaban.name=Azkaban

azkaban.label=My Azkaban

azkaban.color=#FF3601

azkaban.default.servlet.path=/index

web.resource.dir=web/

default.timezone.id=Asia/Shanghai

# Azkaban UserManager class

user.manager.class=azkaban.user.XmlUserManager

user.manager.xml.file=conf/azkaban-users.xml

# Loader for projects

executor.global.properties=conf/global.properties

azkaban.project.dir=projects

# Velocity dev mode

velocity.dev.mode=false

# Azkaban Jetty server properties.

jetty.use.ssl=true

jetty.maxThreads=25

jetty.port=8081

jetty.keystore=/export/servers/azkaban-web-server-3.51.0/keystore

jetty.password=azkaban

jetty.keypassword=azkaban

jetty.truststore=/export/servers/azkaban-web-server-3.51.0/keystore

jetty.trustpassword=azkaban

# Where the Azkaban web server is located

azkaban.webserver.url=https://node03:8443

# mail settings

mail.sender=

mail.host=

# User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users.

# enduser -> myazkabanhost:443 -> proxy -> localhost:8081

# when this parameters set then these parameters are used to generate email links.

# if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.

# azkaban.webserver.external_hostname=myazkabanhost.com

# azkaban.webserver.external_ssl_port=443

# azkaban.webserver.external_port=8081

job.failure.email=

job.success.email=

lockdown.create.projects=false

cache.directory=cache

# JMX stats

jetty.connector.stats=true

executor.connector.stats=true

# Azkaban plugin settings

azkaban.jobtype.plugin.dir=plugins/jobtypes

# Azkaban mysql settings by default. Users should configure their own username and password.

database.type=mysql

mysql.port=3306

mysql.host=node03

mysql.database=azkaban

mysql.user=azkaban

mysql.password=azkaban

mysql.numconnections=100

# Azkaban Executor settings

executor.maxThreads=50

executor.flow.threads=30

第二步：添加插件

将我们编译后的C文件execute-as-user.c

上传到这个目录来/export/servers/azkaban-exec-server-3.51.0/plugins/jobtypes

或者直接将我们/export/softwares下面的文件拷贝过来也行

cp /export/softwares/execute-as-user.c /export/servers/azkaban-exec-server-3.51.0/plugins/jobtypes/

然后执行以下命令生成execute-as-user

yum -y install gcc-c++

cd /export/servers/azkaban-exec-server-3.51.0/plugins/jobtypes

gcc execute-as-user.c -o execute-as-user[a1]

chown root execute-as-user

chmod 6050 execute-as-user

第三步：修改配置文件

修改配置文件

cd  /export/servers/azkaban-exec-server-3.47.0/plugins/jobtypes

vim commonprivate.properties

execute.as.user=false

memCheck.enabled=false

azkaban.native.lib=/export/servers/azkaban-exec-server-3.51.0/plugins/jobtypes

最终生成如下

7、启动服务

第一步：启动azkaban exec server

cd /export/servers/azkaban-exec-server-3.51.0

bin/start-exec.sh

第二步：激活我们的exec-server

node03机器任意目录下执行以下命令

curl -G "node03:$(<./executor.port)/executor?action=activate" && echo

第三步：启动azkaban-web-server

cd /export/servers/azkaban-web-server-3.51.0/

bin/start-web.sh

访问地址：

https://node03:8443

修改linux的时区问题

由于先前做好了时钟同步，所以不用担心时区问题，不需要修改时区了

注：先配置好服务器节点上的时区

1、先生成时区配置文件Asia/Shanghai，用交互式命令 tzselect 即可

2、拷贝该时区文件，覆盖系统本地时区配置

cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

---

[a1]如果提示gcc  command  not  found  那么就在线安装gcc+c++即可。 任意目录执行命令 yum  -y     install  gcc-c++