standalone模式

bin/spark-submit \ --class org.apache.spark.examples.SparkPi \ --master spark://harbor-n0:7077 \ ./examples/jars/spark-examples_2.11-2.4.4.jar \ 10

yarn client模式

bin/spark-submit \ --class org.apache.spark.examples.SparkPi \ --master yarn \ --deploy-mode client \ ./examples/jars/spark-examples_2.11-2.4.4.jar \ 10

yarn cluster模式

bin/spark-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode cluster ./examples/jars/spark-examples_2.11-2.4.4.jar bin\spark-submit.cmd --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode cluster .\examples\jars\spark-examples_2.11-2.4.4.jar hadoop fs -chmod -R 777 /user ./sbin/yarn-daemon.sh start timelineserver ../../sbin/start-yarn.sh ../../sbin/yarn-daemon.sh start resourcemanager ../../sbin/yarn-daemon.sh stop resourcemanager

總結

以上是生活随笔為你收集整理的spark on yarn相关脚本整理20210524的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。