集群规划

集群模式下，可配置多个 Master 及多个 Worker

通常可配置2~3个Master，若干个Worker

	cpu101	cpu102	cpu103
Master		√
Alert		√
ApiApplication		√
Worker	√	√	√
Logger	√	√	√

前置准备工作

三台节点均需部署JDK（1.8+），并配置相关环境变量

https://blog.csdn.net/qq_44226094/article/details/122685082

需部署数据库，支持MySQL（5.7+）或者PostgreSQL（8.2.15+）

https://blog.csdn.net/qq_44226094/article/details/123361850

需部署Zookeeper（3.4.6+）

https://blog.csdn.net/qq_44226094/article/details/123119682

安装 DolphinScheduler

解压 DolphinScheduler 安装包

上传 DolphinScheduler 安装包到 /opt/software 目录

解压安装包到当前目录 ( 注意 !!! )

tar -zxvf apache-dolphinscheduler-1.3.9-bin.tar.gz

修改软件名 dolphinscheduler-1.3.9

mv apache-dolphinscheduler-1.3.9-bin/ dolphinscheduler-1.3.9/

初始化数据库

DolphinScheduler 元数据存储在关系型数据库中

创建数据库

create database dolphinscheduler default character set utf8 default collate utf8_general_ci;

降低 MySQL 密码强度级别 :

set global validate_password_length = 4;

set global validate_password_policy = 0;

创建用户

create user 'dolphinscheduler'@'%' identified by 'dolphinscheduler';

赋予用户相应权限

GRANT ALL PRIVILEGES ON dolphinscheduler.* TO 'dolphinscheduler'@'%';

flush privileges;

修改数据源配置文件

进入 DolphinScheduler 解压目录 , 修改 conf 目录下的 datasource.properties 文件

vim datasource.properties

spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://cpu102:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8
spring.datasource.username=dolphinscheduler
spring.datasource.password=dolphinscheduler

拷贝 MySQL 驱动到 DolphinScheduler 的解压目录下的 lib 中

cp /opt/software/mysql-connector-java-5.1.49.jar /opt/software/dolphinscheduler-1.3.9/lib/

执行数据库初始化脚本

数据库初始化脚本位于 /opt/software/dolphinscheduler-1.3.9/script

./create-dolphinscheduler.sh

配置一键部署脚本

修改解压目录下的 conf/config 目录下的 install_config.conf 文件

vim install_config.conf

# postgresql or mysql
dbtype="mysql"# db config
# db address and port
dbhost="cpu102:3306"# db username
username="dolphinscheduler"# database name
dbname="dolphinscheduler"# db passwprd
# NOTICE: if there are special characters, please use the \ to escape, for example, `[` escape to `\[`
password="dolphinscheduler"# zk cluster
zkQuorum="cpu101:2181,cpu102:2181,cpu103:2181"# Note: the target installation path for dolphinscheduler, please not config as the same as the current path (pwd)
installPath="/opt/module/dolphinscheduler-1.3.9"# deployment user
# Note: the deployment user needs to have sudo privileges and permissions to operate hdfs. If hdfs is enabled, the root directory needs to be created by itself
deployUser="cpu"# resource storage type: HDFS, S3, NONE
resourceStorageType="HDFS"# resource store on HDFS/S3 path, resource file will store to this hadoop hdfs path, self configuration, please make sure the directory exists on hdfs and have read write permissions. "/dolphinscheduler" is recommended
resourceUploadPath="/dolphinscheduler"# if resourceStorageType is HDFS，defaultFS write namenode address，HA you need to put core-site.xml and hdfs-site.xml in the conf directory.
# if S3，write S3 address，HA，for example ：s3a://dolphinscheduler，
# Note，s3 be sure to create the root directory /dolphinscheduler
defaultFS="hdfs://mycluster:8020"# resourcemanager port, the default value is 8088 if not specified
resourceManagerHttpAddressPort="8088"# if resourcemanager HA is enabled, please set the HA IPs; if resourcemanager is single, keep this value empty
yarnHaIps="cpu101,cpu102,cpu103"# if resourcemanager HA is enabled or not use resourcemanager, please keep the default value; If resourcemanager is single, you only need to replace ds1 to actual resourcemanager hostname
#singleYarnIp="cpu101"# who have permissions to create directory under HDFS/S3 root path
# Note: if kerberos is enabled, please config hdfsRootUser=
hdfsRootUser="cpu"# api server port
apiServerPort="12345"# install hosts
# Note: install the scheduled hostname list. If it is pseudo-distributed, just write a pseudo-distributed hostname
ips="cpu101,cpu102,cpu103"# ssh port, default 22
# Note: if ssh port is not default, modify here
sshPort="22"# run master machine
# Note: list of hosts hostname for deploying master
masters="cpu102"# run worker machine
# note: need to write the worker group name of each worker, the default value is "default"
workers="cpu101:default,cpu102:default,cpu103:default"# run alert machine
# note: list of machine hostnames for deploying alert server
alertServer="cpu102"# run api machine
# note: list of machine hostnames for deploying api server
apiServers="cpu102"

一键部署 DolphinScheduler

启动Zookeeper集群

zk.sh start

一键部署并启动 DolphinScheduler

./install.sh 

查看进程

jpsall -l

访问DolphinScheduler UI

http://cpu102:12345/dolphinscheduler

初始用户的用户名为：admin ，密码为 dolphinscheduler123

DolphinScheduler启停命令

DolphinScheduler 的启停脚本均位于其安装目录的 bin 目录下

一键启停所有服务

./bin/start-all.sh

./bin/stop-all.sh

同 Hadoop 的启停脚本进行区分