spark-cluster提交任務，提示 exit code :10 異常，具體需要到容器日志中去查看?

19/08/07 18:09:24 INFO yarn.Client: client token: N/Adiagnostics: Application application_1565171057576_0008 failed 2 times due to AM Container for appattempt_1565171057576_0008_000002 exited with exitCode: 10 For more detailed output, check application tracking page:http://xxx:8088/proxy/application_1565171057576_0008/Then, click on links to logs of each attempt. Diagnostics: Exception from container-launch. Container id: container_1565171057576_0008_02_000001 Exit code: 10 Stack trace: ExitCodeException exitCode=10: at org.apache.hadoop.util.Shell.runCommand(Shell.java:604)at org.apache.hadoop.util.Shell.run(Shell.java:507)at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:789)at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:373)at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)at java.util.concurrent.FutureTask.run(FutureTask.java:266)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:745)

發現spark 提交至yarn異常，超時?

19/08/07 17:48:39 INFO yarn.Client: Attempting to login to the Kerberos using principal: hdfs@IZHONGHONG.COM and keytab: hdfs-nb.keytab-77de1544-0cd0-4ef9-afa4-63f5e873faea 19/08/07 17:48:39 INFO client.RMProxy: Connecting to ResourceManager at xxx/xxx:8032 19/08/07 17:48:39 WARN security.UserGroupInformation: PriviledgedActionException as:hdfs (auth:SIMPLE) cause:org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 19/08/07 17:48:39 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 19/08/07 17:48:39 WARN security.UserGroupInformation: PriviledgedActionException as:hdfs (auth:SIMPLE) cause:java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 19/08/07 17:49:09 WARN security.UserGroupInformation: PriviledgedActionException as:hdfs (auth:SIMPLE) cause:org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 19/08/07 17:49:09 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 19/08/07 17:49:09 WARN security.UserGroupInformation: PriviledgedActionException as:hdfs (auth:SIMPLE) cause:java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 19/08/07 17:49:39 WARN security.UserGroupInformation: PriviledgedActionException as:hdfs (auth:SIMPLE) cause:org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 19/08/07 17:49:39 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 19/08/07 17:49:39 WARN security.UserGroupInformation: PriviledgedActionException as:hdfs (auth:SIMPLE) cause:java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 19/08/07 17:50:09 WARN security.UserGroupInformation: PriviledgedActionException as:hdfs (auth:SIMPLE) cause:org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 19/08/07 17:50:09 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 19/08/07 17:50:09 WARN security.UserGroupInformation: PriviledgedActionException as:hdfs (auth:SIMPLE) cause:java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 19/08/07 17:50:18 ERROR yarn.ApplicationMaster: Uncaught exception: java.util.concurrent.TimeoutException: Futures timed out after [100000 milliseconds]at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)at scala.concurrent.Await$.result(package.scala:107)at org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(ApplicationMaster.scala:342)at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:197)at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$main$1.apply$mcV$sp(ApplicationMaster.scala:680)at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:69)at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:68)at java.security.AccessController.doPrivileged(Native Method)at javax.security.auth.Subject.doAs(Subject.java:422)at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)at org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:68)at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:678)at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)

?說是客戶端無法通過kerberos驗證，找了半天發現我代碼中寫了提交 yarn-client 模式，spark-submit 提交指定了yarn-cluster 導致 ~~~~~~~ 日常才坑 ~

?

總結

以上是生活随笔為你收集整理的spark 提交至yarn异常超时 Client cannot authenticate via:[TOKEN, KERBEROS]的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。