完整報錯如下:

(Python3.6) appleyuchi@Desktop:target$ spark-submit --class WordCountLocal --master yarn --deploy-mode cluster ?--driver-memory 1g --executor-memory 1g --executor-cores 1 ?/home/appleyuchi/IdeaProjects/scala-learn/target/scala-learn-1.0-SNAPSHOT.jar
20/04/26 18:12:33 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
?

?

解決方案:
(Python3.6) appleyuchi@Desktop:~$ hdfs dfs -mkdir /spark
(Python3.6) appleyuchi@Desktop:~$ hdfs dfs -mkdir /spark/jars
(Python3.6) appleyuchi@Desktop:~$ hdfs?dfs -put ?$SPARK_HOME/jars/* hdfs://Desktop:9000/spark/jars/

?

上傳后查看：
hdfs dfs -ls hdfs://Desktop:9000/spark/jars

?

spark-defaults.conf中寫入

spark.yarn.jars hdfs://Desktop:9000/spark/jars

?

再次提交任務:

spark-submit --class WordCountLocal --master yarn --deploy-mode cluster ?--driver-memory 1g --executor-memory 1g --executor-cores 1 ?/home/appleyuchi/IdeaProjects/scala-learn/target/scala-learn-1.0-SNAPSHOT.jar

總結

以上是生活随笔為你收集整理的Neither spark.yarn.jars nor spark.yarn.archive is set的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。