完整報錯如下:

$ spark-submit --class WordCountLocal --master yarn --deploy-mode cluster ?--driver-memory 1g --executor-memory 1g --executor-cores 1 ?/home/appleyuchi/IdeaProjects/scala-learn/target/scala-learn-1.0-SNAPSHOT.jar
Exception in thread "main" java.lang.IllegalArgumentException: java.net.UnknownHostException: dbmtimehadoop
?? ?at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:378)
?? ?at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:310)
?? ?at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
?? ?at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:678)
?? ?at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
?? ?at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
?? ?at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
?? ?at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
?? ?at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
?? ?at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
?? ?at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
?? ?at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:528)
?? ?at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:525)
?? ?at scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:74)
?? ?at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:525)
?? ?at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:869)
?? ?at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:169)
?? ?at org.apache.spark.deploy.yarn.Client.run(Client.scala:1152)
?? ?at org.apache.spark.deploy.yarn.YarnClusterApplication.start(Client.scala:1520)
?? ?at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
?? ?at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
?? ?at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
?? ?at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
?? ?at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.net.UnknownHostException: dbmtimehadoop
?? ?... 24 more
?

摸索的思路:

①搜到[1]中發現這么個信息:

--conf spark.eventLog.dir=hdfs://dbmtimehadoop/tmp/spark2 \

②猜測是spark.eventLog.dir沒設置的關系

?

?

最終解決方案:

hdfs dfs -mkdir /logs
編輯conf/spark-defaults.conf內容如下:

spark.master ????????????????????spark://Desktop:9000 spark.eventLog.enabled ??????????true spark.eventLog.dir ??????????????hdfs://Desktop:9000/logs

Reference:

[1]spark任務提交到yarn上命令總結

總結

以上是生活随笔為你收集整理的java.net.UnknownHostException: dbmtimehadoop的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。