irpas技术客

大数据Spark HA高可用一键启动_spark高可用启动_wuzhenyu1234

未知 7831

三台机器node4 node5 node6 node4为主节点 在node5的/root目录下放置下面两个文档 zk-status.py

import os import time b=os.popen('zkServer.sh status') a=b.read() if str(a).split(': ')[1].strip() =='leader': print('node5 是 leader') os.system('/export/server/spark/sbin/start-all.sh') time.sleep(3) os.system('ssh node4 <<remotessh \n /export/server/spark/sbin/start-master.sh \n exit \n remotessh')

zk-start.py

import os os.system('zkServer.sh start')

在node6的/root/下放置下面两个文档 zk-status.py

import os import time b=os.popen('zkServer.sh status') a=b.read() if str(a).split(': ')[1].strip() =='leader': print('node6 是 leader') os.system('/export/server/spark/sbin/start-all.sh') time.sleep(3) os.system('ssh node4 <<remotessh \n /export/server/spark/sbin/start-master.sh \n exit \n remotessh')

zk-start.py

import os os.system('zkServer.sh start')

在node4的/root/下放置下面文件 spark_ha_start.py

import os import time os.system('zkServer.sh start') os.system('ssh node5 <<remotessh \n python zk-start.py \n exit \n remotessh') os.system('ssh node6 <<remotessh \n python zk-start.py \n exit \n remotessh') time.sleep(3) os.system('ssh node5 <<remotessh \n python zk-status.py \n exit \n remotessh') os.system('ssh node6 <<remotessh \n python zk-status.py \n exit \n remotessh') b=os.popen('zkServer.sh status') a=b.read() if str(a).split(': ')[1].strip() =='leader': print('node4 是 leader') os.system('/export/server/spark/sbin/start-all.sh') time.sleep(3) os.system('ssh node5 <<remotessh \n /export/server/spark/sbin/start-master.sh \n exit \n remotessh')

最后在node4的~目录下运行如下命令 python spark_ha_start.py


1.本站遵循行业规范,任何转载的稿件都会明确标注作者和来源;2.本站的原创文章,会注明原创字样,如未注明都非原创,如有侵权请联系删除!;3.作者投稿可能会经我们编辑修改或补充;4.本站不提供任何储存功能只提供收集或者投稿人的网盘链接。

标签: #spark高可用启动 #Spark高可用一键启动